Versions Compared

Old Version 19

changes.mady.by.user Michael Brand

Saved on

compared with

New Version 20

changes.mady.by.user Marco Grawunder

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
#CONFIG DISTRIBUTE true

 


This statement in Odysseus Script indicates that the queries which follows should be distributed in the network.  


Info

If you want to run several nodes on one machine, ensure that you set the configuration net.querydistribute.randomport=true. Otherwise, port conflicts may occur.

 

 


A better way would be to use docker-compose.



General process of query distribution

...

The strategies are explained in the next sections. Partition and Allocation allow only one selected strategy. The other phases can be used multiple times per query. If the user had not specified strategies for some phases, OdysseusNet will use specific strategies as default (configurable):

PhaseDefault strategy
Preprocess<none>
Partitionquerycloud
Modification<none>
Allocationquerycount
Postprocessmerge

The Transmission phase cannot be altered.

Preprocess

Strategy nameDescription
sourceStreamAOs are replaced with their logical operators.

Partition

Strategy nameDescription
querycloudThe entire query is one query part (no partitioning).
operatorcloudEach logical operator is its own query part (max. partitioning)

Modification

Strategy nameDescription
replication <degree>Query parts are replicated (with replicationdegree degree), executed multiple times. See Replication.

Allocation

Strategy nameDescription
direct

All query parts are assigned to the specified OdysseusNode.

Code Block
#NODE_ALLOCATE DIRECT MyNode1
querycountQuery parts are assigned to nodes with the least count of queries.
roundrobinQuery parts are assigned in order.
userQuery parts are assigned to user-specified nodes. For this, each logical operator has a DESTINATION-Parameter in PQL. Two operators which have the same DESTINATION-value are assigned to the same node.

Postprocess

Strategy nameDescription
localsinkLogical sink operators are staying local (on the distributing node) overriding allocation. Useful if the user wants to have the last operator to check the data stream results.
localsourceLogical source operators are staying local (on the distribution node) overriding allocation. Useful if the user do not want to share its source operators to other nodes.
mergeTwo adjacent query parts which are assigned to the same node are merged to omit unneeded network-transmission operators.
discardedreplicates

Post processor to insert a sender for each {@link ReplicationMergeAO}. The sender will be inserted for the output port 1 that is not used normally. All discarded replicates are sent to port 1. The sender writes the data in a CSV file (one file per merger). The only argument for this post processor is the path to the CSV files. The names of the files are determined by the {@link ReplicationMergeAO} (name and hashCode).

Used sender settings:
- transport handler "file"
- data hgandler "tuple"
- wrapper "GenericPush"
- protocol handler "csv"
- "createDir" "true"
- "filename", path_from_user + "/" + merger.getName() + merger.hashCode() + ".csv"
- "append", "true"

CalcLatency

Inserts a CalcLatencyAO before every real sink. Note that the ILatency metadata will be added to the sources automatically. This postprocessor needs the feature "Calculation Postprocessor Feature".

CalcDatarateInserts a DatarateAO after every source. Note that the IDatarate metadata will be added to the sources automatically. This postprocessor needs the feature "Calculation Postprocessor Feature".

 

...