Page History

...

All queries defined in Odysseus are data pipelines, i.e. data is received from one operator, processed and send to the next operator. By this, complex processing pipelines can be built.

Pattern 1: Preprocessing

Filtering some events

This is one of the most and simple operation. Filtering in Odysseus can be done with the Select operator.

See Examples for Select here.

Reshaping a stream by removing, renaming, or adding new attributes to events in the stream

This can be done in different ways. If you just want to remove Attributes, you can use the Project operator.

If you want to rename attributes, use the Rename operator.

If you want to add attributes, use the Map operator. With this operator it is also possible to remove and rename attributes. For just removing or renaming you should use one of the above operators, because they provide a much lower footprint.

Examples: Selection, Projection and Map

Splitting and combining attributes in a stream

Splitting and combining of attributes in a single stream can be done with the Map operator. This operator allows many different  mathematical expressions over all attributes inside the input stream.

out = MAP({
			expressions = [
				['bid','renamed_bid'] /// Renaming of attribute
				/// Remark: The output only contains attributes/expressions that are given here, so removing an attribute is the by not using it here
			]}
		,nexmark:person)

/// Simple split name in forename and lastname by looking for the first blanc
out2 = MAP({
          expressions = [                
                ['Substring(name,0,indexOf(name," "))','forename'], 
                ['Substring(name,indexOf(name," ")+1)','lastname'],             
          ]            
        },nexmark:person
      )

/// Splitting of attribute into list 
/// Split string into substrings by " "
presplitted = MAP({
          expressions = [                
              ['split(name," ")','splittedName']
          ]            
        },nexmark:person
      )
/// access first and last element of list
out3 = MAP({
        EXPRESSIONS = [
            ['splittedName[0]','forename'],
            ['elementAt(splittedName,size(splittedName)-1)','lastname']    
        ]
        }, presplitted
    ) 

/// access first and last element of list with special function    
out4 = MAP({
        EXPRESSIONS = [
            ['first(splittedName)','forename'],
            ['last(splittedName)','lastname']    
        ]
        }, presplitted
    ) 

/// List functions
out5 = MAP({
        EXPRESSIONS = [
            ['sublist(splittedName,0,1)','nameAsList'],
            ['sublist(splittedName,1)','nameAsList2'],
            ['rest(splittedName)','allLastElements']    
        ]
        }, presplitted
    ) 

Transforming attributes

Transformation can also be done with the Map operator by applying mathematical functions (see Functions and Operators).

Pattern 2: Alerts and Thresholds

This pattern detects a condition and generates alerts based on a condition. (e.g., Alarm on high temperature). These alerts can be based on a simple value or more complex conditions such as rate of increase etc.

In Odysseus generating an altert is simply so create a data pipeline where an element reaches an final operator that sends out the value. E.g. if you use the Select operator to filter out every bid that has a lower price than 400 only bids with a value higher or equals 400 is send to the next operator. An altert can be sending an e-mail (in the following example without authentification at the SMTP server). See Transport handler for further ways to output values.

out = SELECT({
          predicate = 'price >= 400'
        },
        nexmark:bid
      )

mail = SENDER({
            sink='Sink',
            wrapper='GenericPush',
            transport='SMTP',
            protocol='CSV',
            datahandler='Tuple',
            options=[
              ['from','alert@test.de'],
              ['to','receiver@test.de'],
              ['subject','Price Alert'],
              ['host','smtp.server.de'],
              ['tls','false']
            ]
          },
          out
        )

Pattern 3: Simple Counting and Counting with Windows

For Counting (building sum, avg, finding min, max. etc.) the Aggregate (and Group) operator can be used. See Aggregation and Window for examples on this topic. 

Pattern 4: Joining Event Streams

Pattern 5: Data Correlation,Missing Events, and Erroneous Data

Pattern 6: Interacting with Databases

Pattern 7: Detecting Temporal Event Sequence Patterns

Pattern 8: Tracking

Pattern 9: Detecting Trends

Pattern 10: Running the Same Query in Batch and Realtime Pipelines

Pattern 11: Detecting and Switching to Detailed Analysis

Pattern 12: Using a Model

...