Versions Compared

Old Version 25

changes.mady.by.user Christian Kuka (库科)

Saved on

compared with

New Version 26

changes.mady.by.user Christian Kuka (库科)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The access operator can be used to integrate new sources into odysseusOdysseus. Further information can be found in the Documentation to the "Access Operator Framework".

...

  • source: The name of the access operator
  • wrapper: In Odysseus the default wrappers are GenericPush and GenericPull
  • transport: The transport defines the transport protocol to use.
  • protocol: The protocol parameter defines the application protocol to tansform transform the processing results.
  • datahandler: This parameter defines the transform of the single attributes of the processing results.
  • options: Transport protocol and application protocol depending options
  • schema: The output schema of the access operator (may depend on the protocol handler)

...

This operator assures that there will be periodically a heartbeat to avoid blocking because of missing information about time progress. The operator garanteesguarantees, that no element (heartbeat or streamobject) is send, that is older than the last send heartbeat (i.e. the generated heartbeats are in order and indicate time progress). Heartbeats can be send periodically (sendAlwaysHeartbeats = true) or only if no other stream elements indicate time progess progress (e.g. in out of order scenarios) independent if a new element has been received or not.

...

  • RealTimeDelay: How long should the operator wait in transaction time (real time) before it should send a punctuation
  • ApplicationTimeDelay: How long is the realTimeDelay in terms of application time (typically this should be the same, but for simulations this could be adapted)
  • timeUnit: What is the time unit (see Java TimeUnit). Minimum Time unit is milliseconds!
  • sendAlwaysHeartbeat: If true, a heartbeat is send periodically for every realTimeDelay. This is usefull useful for out of order processing
  • allowOutOfOrder: If set to true, the operator allows heartbeats to be send, that lie before the last send element. In other cases this is not allowed.

...

Performs a mapping of incoming attributes to outcoming out-coming attributes using map functions. Odysseus also provides a wide range of mapping functions.

...

  • expressions: A list of expressions to map multiple incoming atttribute attribute values to outcomping out-coming attributes

Example

 

Code Block
themeEclipse
languagejavascript
titleMap Operator
linenumberstrue
output = MAP({
              expressions = ['auction_id * 5','sqrt(auction_id)']
             }, input)

...

  • attributes: A list of attribute names to project on

...

Example

...

Code Block
themeEclipse
languagejavascript
titleMap Operator
linenumberstrue
output = PROJECT({
                  attributes = ['auction', 'bidder']
                 }, input)

...

  • aliases: The list new attribute names to use from now on. If the flag pairs is set, aliases will be interpretet interpreted as pairs of (old_name, new_name). See the examples below.
  • type: By default, each schema has the name of the source from which the data comes from (i.e. it is the name of the type that is processed by this operator). With this parameter the type (source) name can be changed. This is needed in the SASE operator. 
  • pairs: Optional boolean value that flags, if the given list of aliases should be interpretet interpreted as pairs of (old_name, new_name). Default value is false.

Example

...

Code Block
themeEclipse
languagejavascript
titleRename Operator
linenumberstrue
// Renames the first attribute to auction_id, the second to bidder_id and the last to another_id.
output = RENAME({
                  aliases = ['auction_id', 'bidder_id', 'another_id']
                 }, input)

// Due the set flag pairs, the rename operator renames the attribute auction_id to auction and bidder_id to bidder.
output = RENAME({
                 aliases = ['auction_id', 'auction', 'bidder_id', 'bidder'], 
                 pairs = 'true'
                }, input)

...

ROUTE
Description
This operator can be used to route the elements in the stream to different further processing operators, depending on the predicate.
...
 Code Block
themeEclipse
languagejavascript
titleSelect Operator
linenumberstrue
output = SELECT({ 
                 predicate=RelationalPredicate('price > 100') 
                }, input)
 
Sender
...
Description
This operator can be used to publish processing results to multiple endpoints using different transport and application protocols.
...
  • wrapper: In Odysseus the default wrappers are GenericPush and GenericPull
  • transport: The transport defines the transport protocol to use.
  • protocol: The protocol parameter defines the application protocol to tansform transform the processing results.
  • datahandler: This parameter defines the transform of the single attributes of the processing results.
  • options: Transport protocol and application protocol depending options
...
 Code Block
themeEclipse
languagejavascript
titleSender Operator
linenumberstrue
output = SENDER({
                 wrapper='GenericPush',
                 transport='TCPClient',
                 protocol='CSV',
                 dataHandler='Tuple',
                 options=[['host', 'example.com'],['port', '8081'],['read', '10240'],['write', '10240']]
                }, input)
SINK
Description
Parameter
Example
 
SOCKETSINK
Description
This operator can be used to send/provide data from Odysseus via a tcp socket connection. (Remark: This operator will potentially change in future)
...
socketsink({host='localhost', push=false, sinkport=4712, sinkType='bytebuffer', sinkName='driverChannel'}, timestampToPayload(person))
STORE
Description
Parameter
Example
 
TIMESTAMPTOPAYLOAD
Description
This operator is needed before data is send to another system (e.g. via a socket sink) to keep the time meta information (i.e. start and end time stamp). The input object gets two new fields with start and end timestamp. If this output is read again by (another) Odysseus instance, the following needs to be attached to the schema:
...
driverChannel = socketsink({sinkport=4712, sinkType='bytebuffer', sinkName='driverChannel'}, timestampToPayload(person))
UNION
Description
This operator calculates the union of two input sets
Parameter
none
Example
...
 Code Block
themeEclipse
languagejavascript
titleUnion Operator
output = UNION(left, right)
...
UNNEST
Description
The UnNest operator performs a split on the unpacks incoming tuple with a multi value attribute to create multiple tuples
Parameter
  • attribute: The attribute to perform the unnest onthat should be unpack.
Example
 
 Code Block
themeEclipse
languagejavascript
titleUnNest Operator
linenumberstrue
output = UNNEST({
                 attribute='myAttribute'
                },input)
 
UDO
Description
The UDO operator calls a user defined operator.
...
 Code Block
themeEclipse
languagejavascript
titleUDO Operator
linenumberstrue
output = UDO({
              class='MyUDO', 
              init='some parameter'
             }, input)
 
WINDOW
Description
Parameter
  • size: The size of the window
  • advance: The advance the window moves forward
  • slide: The slide of the window
  • type: The type of the window. The possible values are Time, Tuple, and Unbound
  • partition: The partition attribute of the window
  • startCondition: The start condition for a predicate window
  • endCondition: The end condition for a predicate window
...
 Code Block
themeEclipse
languagejavascript
titleWindow Operator
linenumberstrue
//sliding time window
 output = WINDOW({
                  size = 5, 
                  advance = 1, 
                  type = 'time'
                 }, input)

 //sliding tuple window partioniertpartitioned ueberover bidder
 output = WINDOW({
                  size = 5, 
                  advance = 1, 
                  type = 'tuple', 
                  partition=['bidder']
                 }, input) 

 //unbounded window
 output = WINDOW({
                  type = 'unbounded'
                 }, input) 

 //now window (size = 1, advance = 1)
 output = WINDOW({
                  type = 'time'
                 }, input)

 //sliding delta window, reduces time granularity to value of slide
 output = WINDOW({
                  size = 5, 
                  type = 'time', 
                  slide = 5
                 }, input)

 // Predicate window
 output = WINDOW({
                  startCondition = RelationalPredicate('a>10'), 
                  endCondition = RelationalPredicate('a<10')
                 }, input)
 

Benchmark
 
BATCHPRODUCER
Description
Parameter
Example
 
BENCHMARK
Description
Parameter
Example
 
PRIOIDJOIN
Description
Parameter
Example
 
TESTPRODUCER
Description
Parameter
Example
 
Data Mining
 
CLASSIFY
Description
Parameter
Example
 
HOEFFDINGTREE
Description
Parameter
Example
 
LEADER
Description
Parameter
Example
 
SIMPLESINGLEPASSKMEANS
Description
Parameter
Example
 
FREQUENT_ITEM
Description
Parameter
Example
 
Storing
 
DATABASESOURCE
Description
Parameter
Example
 
DATABASESINK
Description
Parameter
Example
 
Extension of PQL – Make New Logical Operators Available
...