CloudTurbine: Streaming Data via Cloud File Sharing, Phase II

We propose a novel technology to leverage rapidly evolving cloud based infrastructure to improve time constrained situational awareness for real-time decision making. Our "CloudTurbine" innovation eliminates the distinction between files and streams.
Streaming and static data have long been considered separately. Whereas streaming data protocols continue to fragment, a great unification of approach for static data has occurred. The paradigm for file-sharing services is simple: (1) put data in a local file folder, (2) it automatically shows up at other linked systems. Wouldn't it be nice if one could unify an approach for streaming data while leveraging the file-sharing cloud infrastructure? This is precisely what we propose. CloudTurbine is a streaming data interface to file-sharing and file-transport services such as Dropbox and FTP. It delegates much of the data transmittal, security, and server resources to the third-party service. It eliminates the distinction between files and streams, and enables a simple, cost effective new paradigm for streaming data middleware. Phase I demonstrated the features, utility, and performance of the prototype CloudTurbine to be very appropriate for a wide range of data-streaming applications. A significant performance issue is the inherent latency imposed by file bundling plus transfer time. Testing has proven this to be tractable, with low latency (2-5s) for several file sharing services, and very low latency (10-50ms) for protocols such as FTP and mapped drives. We propose an Open Source CloudTurbine web portal as a hub to provide this technology to NASA, scientific, and industrial communities. Following the legacy of the earlier-generation DataTurbine, we will vigorously move to establish a self-sustaining Open Source community which provides CloudTurbine access, development, services, spin-off products, and community support.

Data and Resources

Field Value
accessLevel public
bureauCode {026:00}
catalog_@context https://project-open-data.cio.gov/v1.1/schema/catalog.jsonld
catalog_@id https://data.nasa.gov/data.json
catalog_conformsTo https://project-open-data.cio.gov/v1.1/schema
catalog_describedBy https://project-open-data.cio.gov/v1.1/schema/catalog.json
identifier TECHPORT_90593
issued 2018-04-01
landingPage https://techport.nasa.gov/view/90593
modified 2020-01-29
programCode {026:027}
publisher Space Technology Mission Directorate
resource-type Dataset
source_datajson_identifier true
source_hash 6f69d1872e917917ef07c55cdfc18a08f224c165
source_schema_version 1.1
Groups
  • AmeriGEOSS
  • National Provider
  • North America
Tags
  • active
  • amerigeo
  • amerigeoss
  • armstrong-flight-research-center
  • ckan
  • geo
  • geoss
  • national
  • north-america
  • united-states
isopen False
license_id notspecified
license_title License not specified
maintainer TECHPORT SUPPORT
maintainer_email hq-techport@mail.nasa.gov
metadata_created 2025-11-19T17:49:44.878837
metadata_modified 2025-11-19T17:49:44.878843
notes We propose a novel technology to leverage rapidly evolving cloud based infrastructure to improve time constrained situational awareness for real-time decision making. Our "CloudTurbine" innovation eliminates the distinction between files and streams. Streaming and static data have long been considered separately. Whereas streaming data protocols continue to fragment, a great unification of approach for static data has occurred. The paradigm for file-sharing services is simple: (1) put data in a local file folder, (2) it automatically shows up at other linked systems. Wouldn't it be nice if one could unify an approach for streaming data while leveraging the file-sharing cloud infrastructure? This is precisely what we propose. CloudTurbine is a streaming data interface to file-sharing and file-transport services such as Dropbox and FTP. It delegates much of the data transmittal, security, and server resources to the third-party service. It eliminates the distinction between files and streams, and enables a simple, cost effective new paradigm for streaming data middleware. Phase I demonstrated the features, utility, and performance of the prototype CloudTurbine to be very appropriate for a wide range of data-streaming applications. A significant performance issue is the inherent latency imposed by file bundling plus transfer time. Testing has proven this to be tractable, with low latency (2-5s) for several file sharing services, and very low latency (10-50ms) for protocols such as FTP and mapped drives. We propose an Open Source CloudTurbine web portal as a hub to provide this technology to NASA, scientific, and industrial communities. Following the legacy of the earlier-generation DataTurbine, we will vigorously move to establish a self-sustaining Open Source community which provides CloudTurbine access, development, services, spin-off products, and community support.
num_resources 4
num_tags 10
title CloudTurbine: Streaming Data via Cloud File Sharing, Phase II