This page describes Odysseus Script, a language that allows to run (sequences of) queries in different query languages like Continuous Query Language (CQL) or Procedural Query Language (PQL) and to configure the system.
The structure of an Odysseus Script may contain different things: commands, comments, variables, constants, macros or control flows.
Commands are normally those statements that are send to Odysseus, e.g. to install a query or to configure a setting. Each command in Odysseus Script begins with a hash/number sign (#) followed by its name and, finally, by some parameters (if the command needs some parameters).
#COMMAND parameter1 parameter2 |
Normally, one command is executed for its own and has no impact to other commands. However, the #QUERY command needs current settings like the parser, which is set by the command #PARSER or the transformation configuration, which is set by the command #TRANSCFG. Thus, you should normally always begin with the follwing stub (assuming that you use PQL and the standard transformation configuration): Remark: #TRANSCFG is no longer needed for standard processing.
#PARSER PQL ... |
Comments mark lines that should be ignored by the parser. They are defined per line by using three slashes.
///this is ignored by the parserg |
Variables can be used to reuse certain values or can be used for changeable parameters that are moved to the top of a file so that it becomes more clearly. A variable can be set by using #DEFINE and be unset using #UNDEF. The access to a varibale is possible with ${....}. The existence of a variable (if it is defined or not) can be used and checked with the #IFDEF control flow. The value of a variable can be printed by using #PRINT. The following example shows two variables: an integer called "currentid" that has the value "1234" and a variable named "path", which has the value "F:/odysseus/example/".
#DEFINE currentid 1234 #DEFINE path F:/odysseus/example/ #RUNQUERY SELECT * FROM example WHERE id = ${currentid} #RUNQUERY CREATE STREAM source (id Double, data STRING) WRAPPER 'GenericPush' PROTOCOL 'CSV' TRANSPORT 'File' DATAHANDLER 'Tuple' OPTIONS ( 'filename' '${path}input.csv') |
The first variable "currentid" is used in the first query, so that it is equal to "SELECT * FROM example WHERE id = 1234". The second variable is used in the second query as a prefix for the filename. Notice, that variables are simply replaced by its values (there is no string-concatenation like you may see in the path-example).
Control flows are statements that are used to define which of the commands are executed, which not or how often they are executed. There are simple control flows like a for-loop (#LOOP) or a if-then-else (#IFDEF)
Constants are varibales that exists without defining them explicitly. For example, a default variable is NOW so that ${NOW} can be used to get the current time in millis. This is might be useful if the time of the script execution is needed (e.g. for filenames).
Symbol | Value |
---|---|
NOW | Current timestamp |
WORKSPACE* | The absolute path to the workspace |
PROJECT* | The project name |
PROJECTPATH* | The absolute path to the project |
WORKSPACEPROJECT* | The absolute path to the workspace extended by the project name |
ROOT* | The absolute path to the current file |
OS.ARCH | The operating system architecture |
OS.VERSION | The operating system version |
OS.NAME | The operating system name |
CPU | The amount of available processors |
MEM | The total amount of memory |
VM.NAME | The name of the Java VM |
VM.VENDOR | The vendor of the Java VM |
VM.VERSION | The version of the Java VM |
*Only available in Odysseus Studio
Procedures and macros gives the user a possibility to reuse a certain snippet of the code. They can be distinguished between parameterizable procedures and simply reusable macros. Another advantage: Procedures (#PROCEDURE) are stored in the data dictionary so that their availablity is (according to the user's rights) system wide.
There are some control flows that allows to define how certain commands are executed.
This control flow allows a simple for-loop, which may be used to execute the same queries or commands two or more times.
The loop begins with a #LOOP command that needs some parameters and ends with #ENDLOOP. The LOOP command's syntax is as follows
<counter-name> <counter-start> UPTO <counter-end>
where <counter-name> is a string containing a variable. <counter-start> and <counter-end> are integers and defines that the loop starts with <counter-start> (inclusively) and runs until <counter-end> (exclusivly). Therefore, the loop repeats the commands between #LOOP and #ENDLOOP <counter-end> - <counter-start> times. Furthermore, the counter can be used like a variable that was set using #DEFINE.
The examples shows a loop that repeats 10 times (i=0 until i<10) and executes the "#RUNQUERY SELECT ..." accordingly ten times. Furthermore, the counter "i" is used within the query, so that each ${i} is replaced by the current value of i from the loop.
#LOOP i 0 UPTO 10 #RUNQUERY SELECT ${i} AS b, * FROM bid #ENDLOOP |
You may also use ${i-1} or ${i+1} (only this two!). For example, this
#LOOP i 2 UPTO 4 #RUNQUERY SELECT ${i-1} AS a, ${i+1} AS b, * FROM bid WHERE b>${i} #ENDLOOP |
is equal to
#RUNQUERY SELECT 1 AS b, * FROM bid3 WHERE b>2 #RUNQUERY SELECT 2 AS b, * FROM bid4 WHERE b>3 |
You may also use an additional offset variable. This offset variable adds a defined value to the current value of the actual loop variable. Following example uses an offset of x = 5:
#RUNQUERY #LOOP i 2 UPTO 5 WITH x 5 SELECT ${x} AS b, * FROM bid #ENDLOOP |
This is equal to:
#RUNQUERY SELECT 7 AS b, * FROM bid SELECT 8 AS b, * FROM bid SELECT 9 AS b, * FROM bid |
With #IFDEF it is possible to check whether a variable exists and was set by #DEFINE or not. This is useful, for example, to run certain queries corresponding to the current setting. Use #IFNDEF for case where to check if a variable has no value
Like common if-statements in other programming languages, the structure of #IFDEF follows "if-then-else", where the "else" part is not necessary. Accordingly, the structure is as follows:
#IFDEF <variable-name>
<then-commands>
#ENDIF
or
#IFDEF <variable-name>
<then-commands>
#ELSE
<else-commands>
#ENDIF
So, if the variable named <variable-name> exists, the control flow runs the <then-commands>, if not and there is an else-part, the <else-commands> are executed. Look also add #DEFINE and #UNDEF to set or unset variables.
The example defines a variable called latencyOn and uses the #IFDEF command to use either StandardLatency for the transformation configuration, if latencyOn is set or Standard if it is not set. Obviously, this example uses allways the <then-command>-part (since latencyOn is set), so you may switch to another transformation config by simply commenting the #DEFINE command out so that the <else-commands> are used.
#DEFINE latencyOn .... #IFDEF latencyOn #TRANSCFG StandardLatency #ELSE #TRANSCFG Standard #ENDIF |
Similar to #IFDEF but checking if a source with the given name is registered in the data dictionary. This can be helpful to define sources only if they are not already defined
#IFSRCNDEF basestream #INCLUDE ${WORKSPACEPROJECT/}Source.qry #ENDIF |
Similar to #IFDEF and #IFSRCDEF but checking an arbitrary expression. This can be helpful to execute a block only if the expression evaluates to true.
#IF toInteger(CPU) > 1 #RUNQUERY ... #ENDIF |
Stored procedures and macros allows to reuse written Odysseu Script.
The #PROCEDURE allows to create stored procedures that are saved into the data dictionary and can be reused by any other scripts, see #EXECUTE how to run them.
Procedures can be compared to stored procedures in a database system or like methods in a programming language. It allows defining a snippet of Odysseus Script that is reusable. The structure for creating a new procedure is as follows:
#PROCEDURE <procedure-name>
<variable 1>
<variable 2>
...
<variable n>
BEGIN
<arbitrary odysseus script commands>
END
First, the name, which must be after the #PROCEDURE in the same line, is defined. After that, there might be an ordered list of variables. The order is important! These variables will be used as the parameters. Finally, there can be any Odysseus-Script code including queries etc. between the BEGIN and END. The only exception are "global" variables. Between BEGIN and END, only variables are allowed that are defined before (after the procedure name). You can execute the procedure by using the #EXECUTE command.
There cannot be installed more than one procedure with the same name, so you may delete a procedure by calling #DROPPROCEDURE before.
The example creates a procedure with name "setSomething" and has two parameters (varX and attribute). Remeber, order is important! The procedure calls a #RUNQUERY-Command including both parameters. See #EXECUTE command how this example is used.
#PROCEDURE setSomething varX attribute BEGIN #RUNQUERY SELECT 1 AS a, 2 AS ${attribute}, * FROM bid WHERE b>${varX} END |
The #EXECUTE command can be used for running installed procedures, which were created by using the #PROCEDURE command.
Procedures can be compared to stored procedures in a database system or like methods in a programming language. It allows defining a snippet of Odysseus Script that is reusable. Installed procedures can be executed at follows:
#EXECUTE <procedure-name>(<variable 1>, <variable 2>, ..., <variable n>)
The usage is similar to function calls in programming languages like Java. After #EXECUTE the name of procedure with a comma-separated list of its parameters. The parameters must be according to the definition of the procedure. If there were two parameters defined by #PROCEDURE, here are also two parameters needed. Notice, the order is important!
You can create the procedure by using the #PROCEDURE command or delete a procedure by calling #DROPPROCEDURE
This example concludes the one from #PROCEDURE. Therefore we have two parameters (varX and attribute). The following executes the "setSomething" procedure and sets varX=1 and attribute=b.
#EXECUTE setSomething(1, b) |
The variables are replaced and the according Odysseus Script of the procedure is executed at this point. Therefore, the following is inserted insted of the #EXECUTE command:
#RUNQUERY SELECT 1 AS a, 2 AS b, * FROM bid WHERE b>1 |
Since this snippet is simply inserted, all things of the surrounding Odysseus Script is taken. In our example, this is for example, the choosen #PARSER and #TRANSCFG that are necessary for #RUNQUERY here. Therefore, it is not guaranteed that a procedure is executable of its own.
The command can be used to remove stored procedures, which were created by using the #PROCEDURE (see for more information about procedures) command.
An installed procedure can be removed by the command followed by the procedure name:
#DROPPROCEDURE<procedure-name>
You can create the procedure by using the #PROCEDURE command or execute a procedure by calling #EXECUTE
This example concludes the one from #PROCEDURE. Therefore, we want to remove the "setSomething" procedure:
#DROPPROCEDURE setSomething |
This command executes a query in a certain langauge and is equal to #QUERY
This command is used to control how buffers are (automatically) placed within the query plan if a query is transformed (e.g. by #QUERY).
The parameter is the name of a buffer placement strategy. Since the strategies are dynamically loaded, the availability of certain strategies depends on the current system setting (which features are installed and which not). Some possible stragies are shown in the examples.
No buffers:
#BUFFERPLACEMENT None |
Adds a buffer before each operator:
#BUFFERPLACEMENT Standard Buffer Placement |
Adds a buffer after each source:
#BUFFERPLACEMENT Source Buffer Placement |
Adds a buffer for each query:
#BUFFERPLACEMENT Query Buffer Placement |
Allows to define additional processing informations, e.g. IsSecurtyAware.
Example
#CONFIG isSecurityAware true |
This command is used to define variables to reuse certain values. See also at Variables how to use a defined variable or at #IFDEF to see how to use defined variables within if-statements.
The parameters are one or two values, which are separated by a blank or tab. Therefore, the parameters may not have any blanks or tabs. The first parameter is the name and the second paramter is the value that is assigned to the name. However, the second parameter (the value) is optional, because it a value for #IFDEF is not necessary, otherwise it cannot be used for replacement.
The first variable is called "one" and has no value. The second variable is called "two" and has the value "1234". See at Variables or at #IFDEF for examples how to use a variable.
#DEFINE one #DEFINE two 1234 |
This command switches the query sharing (which tries to optimize a query be reusing parts of already installed query plans) on or off.
The parameter is a boolean: the parameter may be either "true" or "false".
/// query sharing off #DOQUERYSHARING false /// query sharing on #DOQUERYSHARING true |
This command switches the rewriting (tries to optimize a query plan by switching, deleting, splitting or merging operators without changing the query's semantics) on or off.
The parameter is a boolean: the parameter may be either "true" or "false".
/// query rewrite off #DOREWRITE false /// query rewrite on #DOREWRITE true |
This command drops all installed queries. It does not remove andy sources or sinks, but you can use #DROPALLSINKS or #DROPALLSOURCES for this.
It has no parameters.
#DROPALLQUERIES |
This command drops all installed sinks. It does not remove andy queries or sources, but you can use #DROPALLQUERIES or #DROPALLSOURCES
It has no parameters.
#DROPALLSINKS |
This command drops all installed sources. It does not remove andy queries or sinks, but you can use #DROPALLQUERIES or #DROPALLSINKS
It has no parameters.
#DROPALLSOURCES |
This command copies the input from the file given into the current script file.
#INPUT ${WORKSPACEPROJECT/}Source.qry |
This command executes the input from the given file a sub script (i.e. it must be a valid odysseus script file)
#INCLUDE ${WORKSPACEPROJECT/}Source.qry |
Changes the login that is used by other commands like #QUERY
There are two parameters: the username and the password.
This example changes the user to "System" with password "manager"
#LOGIN System manager |
Logs the current used user out
Has no parameters
#LOGOUT |
In the default processing scenario, all elements in Odysseus are tagged with time stamp meta data. This command can be used to define the meta data.
Hint: This flag overwrites the standard configuration so you must provide all metadata that should be used!
Example
#METADATA TimeInterval #METADATA Latency #METADATA Priority |
Remark: There are some combined metadata elements available: e.g. IntervalLatency or IntervalLatencyPriority
Can be used to set internal Odysseus configuration params. This should be only
The parameters are: The name of the Odysseus configuration param and the new value
#ODYSSEUS_PARAM scheduler_TimeSlicePerStrategy 10 |
For debugging purpose the values of variables that are defined using #DEFINE or arbitrary expressions can be printed to the std output using #PRINT.
#DEFINE path F:/odysseus/example/ #PRINT path #PRINT "Running on "+toString(OS.NAME) |
This command sets the current parser for following commands, e.g. by #QUERY or #ADDQUERY. The according parser is used until another parser is set.
The parser: Which parsers are available strongly depends on the current system setting and installed features. Normally in the default product, there is "PQL" for Procedural Query Language (PQL) and "CQL" for Continuous Query Language (CQL).
#PARSER PQL |
Set the name of the following queries.
#QNAME Query1 |
This command executes a query in a certain language. This might be, for example Procedural Query Language (PQL) or Continuous Query Language (CQL). There are three different commands to execute such a query: #QUERY, #ADDQUERY and #RUNQUERY. While #QUERY and #ADDQUERY (they are one and the same) only passes the definied query to Odysseus, the #RUNQUERY additionally starts the query. This means, a query that was added with #QUERY or #ADDQUERY is inactive and not started until it is explicetely started. The #RUNQUERY in contrast immediatly starts a query after it is added, e.g. by using #STARTQUERIES .
The query command is dependent on the current parser (which is set by #PARSER) and the current transformation configuration (which is set by #TRANSCFG). Therefore, it is necessary to run these two commands before. Furthermore, you can switch to other parsers / transformation within one script by using #PARSER or #TRANSCFG again. Thus, if you want to run a query in CQL that last #PARSER command before should set the parser to "CQL".
If #QName is defined before, the query will get this name.
The example shows four queries after the parser is set to CQL and the transformation configuration is set to Standard. The first one uses #QUERY and it is executed as a CQL-Query, but not started. The second query is equal to the first one (it still uses CQL and is not started). The third query also uses CQL and the Standard transformation configuration, but is (in contrast to the first and second) started (it is directly running). Then, the parser is switched to PQL, so that the fourth query is parsed by the PQL-Parser and not by the CQL-Parser anymore.
#PARSER CQL #TRANSCFG Standard #QUERY SELECT * FROM bid #ADDQUERY SELECT * FROM bid #RUNQUERY SELECT * FROM bid #PARSER PQL #QUERY result = PROJECT({ATTRIBUTES=['id','name']}, person) |
The reload log is a file that logs all queries that were sucessfully installed into the system. This command can be used to run these logged queries from the log, e.g. to recreat an old ystem state.
This command has no parameters.
#RELOADFROMLOG |
This command installs a query and starts it immediately. See #QUERY for parameters, examples and details.
Sets the used scheduler and its scheduling strategy.
It needs two parameters: The scheduler and the scheduling-strategy. The available schedulers and strategies depends on the current system setting (additional features could be necessary!), because they are dynamically bound.
Uses the "Single Thread Scheduler" with a "Round Robin" scheduling strategy
#SCHEDULER "Single Thread Scheduler RR" "Round Robin" |
Uses the "Single Thread Scheduler" with a "Aurora Min Cost" scheduling strategy
#SCHEDULER "Single Thread Scheduler RR" "Aurora Min Cost" |
Uses the "Single Thread Scheduler" with a "Aurora Min Latency" scheduling strategy
#SCHEDULER "Single Thread Scheduler RR" "Aurora Min Latency" |
Uses the "Single Thread Scheduler" with a "Chain" scheduling strategy
#SCHEDULER "Single Thread Scheduler RR" "Chain" |
Uses the "Single Thread Scheduler" with a "Biggest Queue" scheduling strategy
#SCHEDULER "Single Thread Scheduler RR" "Biggest Queue" |
Uses the "Simple Dynamic Priority Scheduler" with a "Round Robin" scheduling strategy
#SCHEDULER "Simple Dynamic Priority Scheduler" "Round Robin" |
This command can be used to wait a certain time before executing the next command
The parameter is a number. It defines the time in milliseconds for which the script execution should sleep.
Waiting 2 seconds (2000 milliseconds) until the next command is invoked.
#SLEEP 2000 |
This command starts all installed queries that are not running at the moment.
This command has no parameters.
#STARTQUERIES |
This command starts the scheduling.Notice that the scheduling strongly influences the processing and should be carefully used. The scheduler is running by default. You can stop it by using #STOPSCHEDULER
This command has no parameters.
#STARTSCHEDULER |
This command stops the scheduling.Notice that the scheduling strongly influences the processing and should be carefully used. The scheduler is running by default. You can start it by using #STARTSCHEDULER
This command has no parameters.
#STOPSCHEDULER |
This command sets the transformation configuration for following commands. The transformation configuration defines how a query is transformed into an executable plan. The transformation configuration that was set is used until another configuration is explicitly set.
The configuration: Which configruations are available strongly depends on the current system setting and installed features. Normally in the default product, there is "Standard" for the default transformation configuration. If the latency feature is installed, there is also "StandardLatency" available, which invokes specific transformations so that the latency is automatically calculated.
Remark: For most cases, this parameter is not needed anymore. Use #METADATA if you want to attach additional metadata. Use #CONFIG to define distinct aspects.
#TRANSCFG Standard |
This command sets the transformation configuration for following commands. The transformation configuration defines how a query is transformed into an executable plan. The transformation configuration that was set is used until another configuration is explicitly set.
The configuration: Which configruations are available strongly depends on the current system setting and installed features. Normally in the default product, there is "Standard" for the default transformation configuration. If the latency feature is installed, there is also "StandardLatency" available, which invokes specific transformations so that the latency is automatically calculated.
#TRANSCFG Standard |