Introduction to Streams Concepts

Stream refers to a sequence of data elements or symbols made available over time .

• Data stream transmits data from a source and receives at the processing end in a network
  
• A continuous stream of data flows between the source and receiver ends, and which is processed in real time
  
• Stream also refers to communication of bytes or characters over sockets in a computer network

A program uses stream as an underlying data type in inter-process communication channels.

Examples of Data Stream Applications

1. Making data-driven marketing decisions in real time. It requires the use of data from trends analyses of real-time sales, and analysis of social media, and the sales distribution. 
2. Monitoring and detection of potential failures of system using network management tool
3. Monitoring of industrial or manufacturing machinery in real time 
4. A sensor network or IoT controlled by another entity, or a set of entities 
5. Watching online video lectures, and rewinding or forwarding them 

Data Stream Model

Stream is data in motion
Three approaches for updating the endpoints (sinks) are
      (i) non-overlapping,
      (ii) slow (batch processing) and
      (iii) fast (near real-time)

Different ways of modeling data stream, querying, processing and management.

Data Stream 

• An unbounded and time-ordered sequence of data items (relational tuples) in the data stream model
• The receiving software receives the sequences in order and sees the data items only once.
• Each tuple consists of a set of attributes, like a row in a database table.
• The tuples have a schema-like  traditional database. 
• One of the attributes in the tuple schema is a timestamp, usually represented by t.

Object-based data stream model 

• Data-flows modeled as objects Examples: Cougar and Tribeca object based data stream
• Cougar models sensors’ data as a stream of objects •
• Tribeca models the network monitoring data as a stream of objects

XML-based data stream model 

• Example: NiagaraCQ, an XML-based data stream model
• Scalable continuous query processing over XML documents
• Performs operations over millions of simultaneous queries by dynamically grouping them according to their structural similarities. 
  

Window-based data stream model 

• Stream data direction can be towards fixed window, sliding window or landmark window-sinks (end-points) [Window means a time window during which the data stream is looked at an instance.]