This page describes Odysseus Script, a language that allows to run (sequences of) queries in different query languages like Continuous Query Language (CQL) or Procedural Query Language (PQL) and to configure the system.

Structure

The structure of an Odysseus Script may contain different things: commands, comments, variables, constants, macros or control flows.

Commands

Commands are normally those statements that are send to Odysseus, e.g. to install a query or to configure a setting. Each command in Odysseus Script begins with a hash/number sign (#) followed by its name and by some parameters (if the command needs some parameters).

#COMMAND parameter1 parameter2

Normally, one command is executed for its own and has no impact to other commands. However, the #QUERY command needs current settings like the parser, which is set by the command #PARSER or the transformation configuration, which is set by the command #TRANSCFG. Thus, you should normally always begin with the follwing stub (assuming that you use PQL and the standard transformation configuration): Remark: #TRANSCFG is no longer needed for standard processing.

#PARSER PQL
...

Comments

Comments mark lines that should be ignored by the parser. Useful for additional information for the reader. Comments are defined  by using three slashes per line. Currently, there is no way to comment multiple lines at once.

///this is ignored by the parser

Variables

Variables can be used to reuse certain values. Mostly, they are moved to the top of a file so that they become more clearly. A variable can be created by using #DEFINE and be removed by using #UNDEF. If a variable should be calculated out of other variables (or constants), #EVAL can be used. To access a varibale, the user has to write  ${....}. The existence of a variable (if it is created or not) can be checked with #IFDEF. The value of a variable can be printed by using #PRINT (into the console). The following example shows three variables: an integer called "currentid" that has the value "50", a variable named "path", which has the value "F:/odysseus/example/" and a calculated variable "maxid".

#DEFINE currentid 50
#DEFINE path F:/odysseus/example/
#EVAL maxid = currentid + 100
#RUNQUERY
SELECT * FROM example WHERE id >= ${currentid} AND id <= ${maxid}
#RUNQUERY
CREATE STREAM source (id Double, data STRING)
    WRAPPER 'GenericPush'
    PROTOCOL 'CSV'
    TRANSPORT 'File'
    DATAHANDLER 'Tuple'
    OPTIONS ( 'filename' '${path}input.csv')

The first and third variables "currentid" and "maxid" are used in the first query, so that it is equal to "SELECT * FROM example WHERE id >= 50 AND id <= 150". The second variable is used in the second query as a prefix for the filename. Note that variables are simply replaced by its values.

Constants

Constants are special varibales that exists without defining them explicitly. For example, a default variable is NOW so that ${NOW} can be used to get the current time in milliseconds. This is might be useful if the time of the script execution is needed (e.g. for filenames). The following table shows an excerpt of currently available constants. Developers can add application-specific constants, if needed (see Additional constant variables).

*Only available in Odysseus Studio

Additionally, system properties provided by System.getProperty(...) are also available. To avoid nameing collisions, each system property has "_" as prefix. Example: ${_user.name} will be replaced with the current system user name (NOT the username in Odysseus).

Control Flows

Control flows are statements that are used to define which commands are executed, which not and how often they are executed. There are simple control flows like a for-loop (#LOOP) or a if-then-else (#IFDEF). They are explained in more details below.

Procedures and Macros

Procedures and macros gives the user a possibility to reuse a certain snippet of the code. They can be distinguished between parameterizable procedures and simply reusable macros. Another advantage: Procedures (#PROCEDURE) are stored in the data dictionary so that their availablity is (according to the user's rights) system wide.

Control Flows

There are some control flows that allows to define how certain commands are executed.

#LOOP

This control flow allows a simple for-loop, which may be used to execute the same queries or commands two or more times.

Usage

The loop begins with a #LOOP command that needs some parameters and ends with #ENDLOOP. The LOOP command's syntax is as follows

<counter-name> <counter-start> UPTO <counter-end>

where <counter-name> is a string containing a variable. <counter-start> and <counter-end> are integers and defines that the loop starts with <counter-start> (inclusively) and runs until <counter-end> (exclusivly). Therefore, the loop repeats the commands between #LOOP and #ENDLOOP <counter-end> - <counter-start> times. Furthermore, the counter can be used like a variable that was set using #DEFINE.

Example

The examples shows a loop that repeats 10 times (i=0 until i<10) and executes the "#RUNQUERY SELECT ..." accordingly ten times. Furthermore, the counter "i" is used within the query, so that each ${i} is replaced by the current value of i from the loop.

#LOOP i 0 UPTO 10
    #RUNQUERY
    SELECT ${i} AS b, * FROM bid 
#ENDLOOP

You may also use ${i-1} or ${i+1} (only this two!). For example, this

#LOOP i 2 UPTO 4
    #RUNQUERY
    SELECT ${i-1} AS a, ${i+1} AS b, * FROM bid WHERE b>${i}
#ENDLOOP

is equal to

#RUNQUERY
SELECT 1 AS b, * FROM bid3 WHERE b>2
#RUNQUERY
SELECT 2 AS b, * FROM bid4 WHERE b>3

You may also use an additional offset variable. This offset variable adds a defined value to the current value of the actual loop variable. Following example uses an offset of x = 5:

#RUNQUERY
#LOOP i 2 UPTO 5 WITH x 5
    SELECT ${x} AS b, * FROM bid 
#ENDLOOP

This is equal to:

#RUNQUERY
SELECT 7 AS b, * FROM bid 
SELECT 8 AS b, * FROM bid 
SELECT 9 AS b, * FROM bid 

#IFDEF/#IFNDEF

With #IFDEF it is possible to check whether a variable exists and was set by #DEFINE or not. This is useful, for example, to run certain queries corresponding to the current setting. Use #IFNDEF for case where to check if a variable has no value

Usage

Like common if-statements in other programming languages, the structure of #IFDEF follows "if-then-else", where the "else" part is not necessary. Accordingly, the structure is as follows:

#IFDEF <variable-name>

<then-commands>

#ENDIF

or

#IFDEF <variable-name>

<then-commands>

#ELSE

<else-commands>

#ENDIF

So, if the variable named <variable-name> exists, the control flow runs the <then-commands>, if not and there is an else-part, the <else-commands> are executed. Look also add #DEFINE and #UNDEF to set or unset variables.

Example

The example defines a variable called latencyOn and uses the #IFDEF command to use either StandardLatency for the transformation configuration, if latencyOn is set or Standard if it is not set. Obviously, this example uses allways the <then-command>-part (since latencyOn is set), so you may switch to another transformation config by simply commenting the #DEFINE command out so that the <else-commands> are used.

#DEFINE latencyOn
....
#IFDEF latencyOn
	#TRANSCFG StandardLatency	
#ELSE
	#TRANSCFG Standard
#ENDIF

#IFSRCDEF/#IFSRCNDEF

Similar to #IFDEF but checking if a source with the given name is registered in the data dictionary. This can be helpful to define sources only if they are not already defined

#IFSRCNDEF basestream
#INCLUDE ${WORKSPACEPROJECT/}Source.qry
#ENDIF

#IF

Similar to #IFDEF and #IFSRCDEF but checking an arbitrary expression. This can be helpful to execute a block only if the expression evaluates to true.

#IF toInteger(CPU) > 1
#RUNQUERY
...
#ENDIF
 
...
 
#DEFINE a_string FOO
#IF a_string == "FOO"
...
#ENDIF

Stored Procedures and Macros

Stored procedures and macros allows to reuse written Odysseu Script.

#PROCEDURE

The #PROCEDURE allows to create stored procedures that are saved into the data dictionary and can be reused by any other scripts, see #EXECUTE how to run them.

Usage

Procedures can be compared to stored procedures in a database system or like methods in a programming language. It allows defining a snippet of Odysseus Script that is reusable. The structure for creating a new procedure is as follows:

#PROCEDURE <procedure-name>
<variable 1>
<variable 2>
...
<variable n>

BEGIN

    <arbitrary odysseus script commands>

END

First, the name, which must be after the #PROCEDURE in the same line, is defined. After that, there might be an ordered list of variables. The order is important! These variables will be used as the parameters. Finally, there can be any Odysseus-Script code including queries etc. between the BEGIN and END. The only exception are "global" variables. Between BEGIN and END, only variables are allowed that are defined before (after the procedure name). You can execute the procedure by using the #EXECUTE command.

There cannot be installed more than one procedure with the same name, so you may delete a procedure by calling #DROPPROCEDURE before.

Example

The example creates a procedure with name "setSomething" and has two parameters (varX and attribute). Remeber, order is important! The procedure calls a #RUNQUERY-Command including both parameters. See  #EXECUTE command how this example is used.

#PROCEDURE setSomething
varX
attribute
BEGIN
    #RUNQUERY
    SELECT 1 AS a, 2 AS ${attribute}, * FROM bid WHERE b>${varX}
END

#EXECUTE

The #EXECUTE command can be used for running installed procedures, which were created by using the #PROCEDURE command.

Usage

Procedures can be compared to stored procedures in a database system or like methods in a programming language. It allows defining a snippet of Odysseus Script that is reusable. Installed procedures can be executed at follows:

#EXECUTE <procedure-name>(<variable 1>, <variable 2>, ...,  <variable n>)

The usage is similar to function calls in programming languages like Java. After #EXECUTE the name of procedure with a comma-separated list of its parameters. The parameters must be according to the definition of the procedure. If there were two parameters defined by #PROCEDURE, here are also two parameters needed. Notice, the order is important!

You can create the procedure by using the #PROCEDURE command or delete a procedure by calling #DROPPROCEDURE

Example

This example concludes the one from #PROCEDURE. Therefore we have two parameters (varX and attribute). The following executes the "setSomething" procedure and sets varX=1 and attribute=b.

#EXECUTE setSomething(1, b)

The variables are replaced and the according Odysseus Script of the procedure is executed at this point. Therefore, the following is inserted insted of the #EXECUTE command:

    #RUNQUERY
    SELECT 1 AS a, 2 AS b, * FROM bid WHERE b>1

Since this snippet is simply inserted, all things of the surrounding Odysseus Script is taken. In our example, this is for example, the choosen #PARSER and #TRANSCFG that are necessary for #RUNQUERY here. Therefore, it is not guaranteed that a procedure is executable of its own.

#DROPPROCEDURE

The command can be used to remove stored procedures, which were created by using the #PROCEDURE (see for more information about procedures) command.

Usage

An installed procedure can be removed by the command followed by the procedure name:

#DROPPROCEDURE<procedure-name>

You can create the procedure by using the #PROCEDURE command or execute a procedure by calling #EXECUTE

Example

This example concludes the one from #PROCEDURE. Therefore, we want to remove the "setSomething" procedure:

#DROPPROCEDURE setSomething

Handling of Queries

Query can be modified via Odysseus script. For this, the queries must be named. A query can be named with #QNAME:

#PARSER CQL
#QNAME query1
#ADDQUERY
SELECT * FROM nexmark:person

#STARTQUERY

This command can be used to start a named query.

#STARTQUERY query1

#STOPQUERY

This command can be used to stop a nazmed query.

#STOPQUERY query1

#SUSPENDQUERY

This comand can be used to pause a query. Remark: received elements are stored, so this can lead to a memory problem.

#SUSPENDQUERY query1

#PARTIALQUERY

This comand can be used to reduce load of a query by throwing away some received elements. The factor is a number between 0 and 100 where 0 means keep every element and 100 throw away every element

#PARTIALQUERY query1 factor

#RESUMEQUERY

A suspended query can be resumed by this.

#RESUMEQUERY query1

#REMOVEQUERY

Remove a query from the system. Cannot be undone.

#REMOVEQUERY query1

#WAITFORQUERY

With this comand, the execution of a Odysseus Script can be paused until a query is stopped or removed. period in ms states in which time intervals the query should be checked (default is 1000). maxwaitingtime states how should be waited for the query at maxium (default wait forever).

#WAITFORQUERY query1 [period [maxwaitingtime]]

Commands

#ADDQUERY

This command executes a query in a certain langauge and is equal to #QUERY

#BUFFERPLACEMENT

This command is used to control how buffers are (automatically) placed within the query plan if a query is transformed (e.g. by #QUERY).

Parameters

The parameter is the name of a buffer placement strategy. Since the strategies are dynamically loaded, the availability of certain strategies depends on the current system setting (which features are installed and which not). Some possible stragies are shown in the examples.

Examples

No buffers:

#BUFFERPLACEMENT None

Adds a buffer before each operator:

#BUFFERPLACEMENT Standard Buffer Placement

Adds a buffer after each source:

#BUFFERPLACEMENT Source Buffer Placement

Adds a buffer for each query:

#BUFFERPLACEMENT Query Buffer Placement

#CONFIG

Allows to define additional processing informations, e.g. IsSecurtyAware.

Example

#CONFIG isSecurityAware true

#DEFINE

This command is used to define variables to reuse certain values. See also at Variables how to use a defined variable or at #IFDEF to see how to use defined variables within if-statements.

Parameters

The parameters are one or two values, which are separated by a blank or tab. Therefore, the parameters may not have any blanks or tabs. The first parameter is the name and the second paramter is the value that is assigned to the name. However, the second parameter (the value) is optional, because it a value for #IFDEF is not necessary, otherwise it cannot be used for replacement.

Example

The first variable is called "one" and has no value. The second variable is called "two" and has the value "1234". See at Variables or at #IFDEF for examples how to use a variable.

#DEFINE one
#DEFINE two 1234

#DOQUERYSHARING

This command switches the query sharing (which tries to optimize a query be reusing parts of already installed query plans) on or off.

Parameters

The parameter is a boolean: the parameter may be either "true" or "false".

Example

/// query sharing off
#DOQUERYSHARING false
/// query sharing on
#DOQUERYSHARING true

#DOREWRITE

This command switches the rewriting (tries to optimize a query plan by switching, deleting, splitting or merging operators without changing the query's semantics) on or off.

Parameters

The parameter is a boolean: the parameter may be either "true" or "false".

Example

/// query rewrite off
#DOREWRITE false
/// query rewrite on
#DOREWRITE true

#DROPALLQUERIES

This command drops all installed queries. It does not remove andy sources or sinks, but you can use #DROPALLSINKS or #DROPALLSOURCES for this.

Parameters

It has no parameters.

Example

#DROPALLQUERIES

#DROPALLSINKS

This command drops all installed sinks. It does not remove andy queries or sources, but you can use #DROPALLQUERIES or #DROPALLSOURCES

Parameters

It has no parameters.

Example

#DROPALLSINKS

#DROPALLSOURCES

This command drops all installed sources. It does not remove andy queries or sinks, but you can use #DROPALLQUERIES or #DROPALLSINKS

Parameters

It has no parameters.

Example

#DROPALLSOURCES

#EVAL

It behaves like #DEFINE, except it evaluates the expression to a result, which is then stored in the specified variable.

Parameters

The parameters are two values, which are separated by a blank, tab or "=". Therefore, the parameters may not have any blanks or tabs. The first parameter is the name and the second paramter is the expression, whose value is assigned to the name.

Example

The first variable is called "aNumber" and is a normal #DEFINE. "bNumber" gets the evaluated result of "aNumber + 2000": "3000". "cNumber" demonstrates an alternative way to specify new variables (omitting "=").

#DEFINE aNumber 1000
#EVAL bNumber = aNumber + 2000
#EVAL cNumber aNumber + 2000

#INPUT/#INCLUDE

This command copies the input from the source given into the current script file. The source can be a local file or a file on a web server (giving an URI).

#INPUT ${WORKSPACEPROJECT/}Source.qry
#INCLUDE http://odysseus.offis.uni-oldenburg.de/download/test/StreamSources.qry

#LOGIN

Changes the login that is used by other commands like #QUERY

Parameters

There are two parameters: the username and the password.

Example

This example changes the user to "System" with password "manager"

#LOGIN System manager

#LOGOUT

Logs the current used user out

Parameters

Has no parameters

Example

#LOGOUT

#METADATA

In the default processing scenario, all elements in Odysseus are tagged with time stamp meta data. This command can be used to define the meta data.

Hint: This flag overwrites the standard configuration so you must provide all metadata that should be used!

Example

#METADATA TimeInterval
#METADATA Latency
#METADATA Priority

Remark: There are some combined metadata elements available: e.g. IntervalLatency or IntervalLatencyPriority 

#ODYSSEUS_PARAM

Can be used to set internal Odysseus configuration params. This should be only

Parameters

The parameters are: The name of the Odysseus configuration param and the new value

Example

#ODYSSEUS_PARAM scheduler_TimeSlicePerStrategy 10

#OPTIMIZE_PREDICATES

Enable predicate optimization for the current query. The optimizer tries to simplify the predicates used in select operations and transforms the predicate into its conjunctive normal form to move clauses in the expression down to the sources.

Example

#OPTIMIZE_PREDICATES true

#PRINT

For debugging purpose the values of variables that are defined using #DEFINE or arbitrary expressions can be printed to the std output using #PRINT.

#DEFINE path F:/odysseus/example/
#PRINT path
#PRINT "Running on "+toString(OS.NAME) 

#PARSER

This command sets the current parser for following commands, e.g. by #QUERY or #ADDQUERY. The according parser is used until another parser is set.

Parameters

The parser: Which parsers are available strongly depends on the current system setting and installed features. Normally in the default product, there is "PQL" for Procedural Query Language (PQL) and "CQL" for Continuous Query Language (CQL).

Example

#PARSER PQL

#QName

Set the name of the following query.

#QNAME Query1

#QPRIORITY

Set the priority of the next following query

#QPRIORITY 10

#QUERY

This command executes a query in a certain language. This might be, for example Procedural Query Language (PQL) or Continuous Query Language (CQL). There are three different commands to execute such a query: #QUERY, #ADDQUERY and #RUNQUERY. While #QUERY and #ADDQUERY (they are one and the same) only passes the definied query to Odysseus, the #RUNQUERY additionally starts the query. This means, a query that was added with #QUERY or #ADDQUERY is inactive and not started until it is explicetely started. The #RUNQUERY in contrast immediatly starts a query after it is added, e.g. by using #STARTQUERIES .

Parameters

The query command is dependent on the current parser (which is set by #PARSER) and the current transformation configuration (which is set by #TRANSCFG). Therefore, it is necessary to run these two commands before. Furthermore, you can switch to other parsers / transformation within one script by using #PARSER or #TRANSCFG again. Thus, if you want to run a query in CQL that last #PARSER command before should set the parser to "CQL".

If #QName is defined before, the query will get this name.

Example

The example shows four queries after the parser is set to CQL and the transformation configuration is set to Standard. The first one uses #QUERY and it is executed as a CQL-Query, but not started. The second query is equal to the first one (it still uses CQL and is not started). The third query also uses CQL and the Standard transformation configuration, but is (in contrast to the first and second) started (it is directly running). Then, the parser is switched to PQL, so that the fourth query is parsed by the PQL-Parser and not  by the CQL-Parser anymore.

#PARSER CQL
#TRANSCFG Standard

#QUERY
SELECT * FROM bid

#ADDQUERY
SELECT * FROM bid

#RUNQUERY
SELECT * FROM bid

#PARSER PQL

#QUERY
result =  PROJECT({ATTRIBUTES=['id','name']}, person)

#RELOADFROMLOG

The reload log is a file that logs all queries that were sucessfully installed into the system. This command can be used to run these logged queries from the log, e.g. to recreat an old ystem state.

Parameters

This command has no parameters.

Example

#RELOADFROMLOG

#RUNQUERY

This command installs a query and starts it immediately. See #QUERY for parameters, examples and details.

#SCHEDULER

Sets the used scheduler and its scheduling strategy.

Parameters

It needs two parameters: The scheduler and the scheduling-strategy. The available schedulers and strategies depends on the current system setting (additional features could be necessary!), because they are dynamically bound.

Example

Uses the "Single Thread Scheduler" with a "Round Robin" scheduling strategy

#SCHEDULER "Single Thread Scheduler RR" "Round Robin"

Uses the "Single Thread Scheduler" with a "Aurora Min Cost" scheduling strategy

#SCHEDULER "Single Thread Scheduler RR" "Aurora Min Cost"

Uses the "Single Thread Scheduler" with a "Aurora Min Latency" scheduling strategy

#SCHEDULER "Single Thread Scheduler RR" "Aurora Min Latency"

Uses the "Single Thread Scheduler" with a "Chain" scheduling strategy

#SCHEDULER "Single Thread Scheduler RR" "Chain"

Uses the "Single Thread Scheduler" with a "Biggest Queue" scheduling strategy

#SCHEDULER "Single Thread Scheduler RR" "Biggest Queue"

 Uses the "Simple Dynamic Priority  Scheduler" with a "Round Robin" scheduling strategy

#SCHEDULER "Simple Dynamic Priority  Scheduler" "Round Robin"

#SLEEP

This command can be used to wait a certain time before executing the next command

Parameters

The parameter is a number. It defines the time in milliseconds for which the script execution should sleep.

Example

Waiting 2 seconds (2000 milliseconds) until the next command is invoked.

#SLEEP 2000

#STARTQUERIES

This command starts all installed queries that are not running at the moment.

Parameters

This command has no parameters.

Example

#STARTQUERIES

#STARTSCHEDULER

This command starts the scheduling.Notice that the scheduling strongly influences the processing and should be carefully used. The scheduler is running by default. You can stop it by using #STOPSCHEDULER

Parameters

This command has no parameters.

Example

#STARTSCHEDULER

#STOPSCHEDULER

This command stops the scheduling.Notice that the scheduling strongly influences the processing and should be carefully used. The scheduler is running by default. You can start it by using #STARTSCHEDULER

Parameters

This command has no parameters.

Example

#STOPSCHEDULER

#TRANSCFG

This command sets the transformation configuration for following commands. The transformation configuration defines how a query is transformed into an executable plan. The transformation configuration that was set is used until another configuration is explicitly set.

Parameters

The configuration: Which configruations are available strongly depends on the current system setting and installed features. Normally in the default product, there is "Standard" for the default transformation configuration. If the latency feature is installed, there is also "StandardLatency" available, which invokes specific transformations so that the latency is automatically calculated.

Remark: For most cases, this parameter is not needed anymore. Use #METADATA if you want to attach additional metadata. Use #CONFIG to define distinct aspects.

Example

#TRANSCFG Standard

#UNDEF

This command sets the transformation configuration for following commands. The transformation configuration defines how a query is transformed into an executable plan. The transformation configuration that was set is used until another configuration is explicitly set.

Parameters

The configuration: Which configruations are available strongly depends on the current system setting and installed features. Normally in the default product, there is "Standard" for the default transformation configuration. If the latency feature is installed, there is also "StandardLatency" available, which invokes specific transformations so that the latency is automatically calculated.

Example

Features and Updates

You may use Odysseus Script to modify your installation or assert some plugin to be installed, e.g. like a certain wrapper when it used in a subsequent query.

#REQUIRED

This command allows you to check and install the existence of a certain feature.

Remeber that your installation must be connected to the internet to install new plugins, because they are automatically loaded from the Odysseus Updatesite.

Remark: After an installation, the system will automatically restart!

Parameters

The first parameter is the ID of the feature.

The second parameter is optional and allows you to automatically and must be "true" or "false"

Example

Check and install mining feature, if it is not already installed

#REQUIRED de.uniol.inf.is.odyssseus.mining

Check and install mining feature, if it is not already installed (some as above)

#REQUIRED de.uniol.inf.is.odyssseus.mining true

Only check if mining feature is already installed, but do not install it automatically

#REQUIRED de.uniol.inf.is.odyssseus.mining false

#UPDATE

This command allows you to check for updates and installs them, if there are any.

Remeber that your installation must be connected to the internet to install new plugins, because they are automatically loaded from the Odysseus Updatesite.

Remark: After the update, the system will automatically force a restart!

Parameters

Has no parameters

Example

#UPDATE