Odysseus allow to move some query parts to a subsctructe that can be included with the SubQuery-Operator

#ADDQUERY
in = TIMER({PERIOD = 1000,TIMEFROMSTART = true, SOURCE = 'source1'})

in2 = TIMER({PERIOD = 700, TIMEFROMSTART = true, SOURCE = 'source2'})

out = SUBQUERY({NAME='COMPLEX', QUERYFILE = '${PROJECTPATH}/innerQuery.qry'}, in, in2)

out0 = OUTPUTCONNECTOR({PORT = 0, name="Conn1"}, 0:out)
out1 = OUTPUTCONNECTOR({PORT = 1, name="Conn2"}, 1:out)

join = JOIN(in, out1)

Above is a simple example, how this operator can be used.

The subquery operator behaves like any other operator. It can have a name (here complex) and must provide the source of the subquery. This can be a file, contain the querytext (QUERYFILE) or this can be a running query (queryID or queryName), or the query text can be given inside a string (query). In the last case, the parser must be set (if other than Odysseus Script). Hte outputconnectors in the ouside query are not necessary. It is possible just to connect to out in the example above, if only the output from port 0 is used

The subquery can have 0 to n inputs. If (like in the above example) an input from the outside query is given, connectors need to be defined inside the subquery.

#PARSER PQL
#ADDQUERY
in = CONNECTOR({port = 0, SOURCE = 'connector0', SCHEMA = [['test', 'integer']]})
in2 = CONNECTOR({port = 1, SOURCE = 'connector1', SCHEMA = [['test', 'integer']]})

merged = MERGE(in,in2)

out = FILTER({NAME='Test',PREDICATE = 'test % 2 == 0'},merged)

out0 = OUTPUTCONNECTOR({PORT = 0, name="Conn01"}, 0:out)

out1Input = MAP({EXPRESSIONS = ['counter()','test']},1:out)

out1 = OUTPUTCONNECTOR({PORT = 1, name="Conn02"}, out1Input)

These CONNECTORS behave as sources inside the subquery and allows to connect to the responding element from the outer query. Other queries can connect to this operator via OUTPUTCONNECTOR.

As the CONNECTOR is just a source, is can have all parameters like the Access Operators, i.e. is must provide a name and should provide a schema. With the parameter type, the object type can be set (default is tuple). The port parameter is mapped to the position of the subquery operator.

The end of the subquery is defined by the outputconnectors. If, as in the example above, multiple output ports should be available, each port must be defined with this connector.

Hint: Each value defined outside in the query are available inside the query. So there is something like this possible:

Outside-Query

#DEFINE sm_schema [['i', 'STARTTIMESTAMP'],['v','Double'],['c','Double'],['batch','Integer']]
[...]
features = SUBQUERY({
                queryfile = '${preprocessing_subquery}',
                name = 'P and Q calculation',
                schema = ${feature_schema},
                options = [['OUTOFORDER','${OUTOFORDER}']]                                                                                                                                                                                                                                                                                                                                                                                                                                                                
              },
              sm_data_ooo
            ) 
[...]

Part of subquery

sm_data = CONNECTOR({
              port = 0,
              source = 'preprocessing_source',
              schema = ${sm_schema}            
            }          
          )

[...]
calc_w1 = PREDICATEWINDOW({
              start = 'true',
#IF OUTOFORDER="TRUE"
			  end = "!isNull(__last_1.i) && floor(__last_1.i/${w1_size}) != floor(i/${w1_size})",
#ELSE
              end = "size(__all) == ${w1_size}-1",
#ENDIF
              nesting = true,
              keependingelement = true,
              USEELEMENTONLYFORSTARTOREND = true,
              allowsamestartandendts = true,
              keepTimeOrder = false             
            },
            w1_preprocess
          )

Additional: The options given are available in the query query as parameters in Odysseus Script format (i.e. ${OUTOFORDER} in the above example)


For a complete and complex example see our solution for the debs grand challenge 2020:

https://git.swl.informatik.uni-oldenburg.de/projects/ODR/repos/gc2020/browse 

Or directly the workspace:

https://git.swl.informatik.uni-oldenburg.de/projects/ODR/repos/gc2020/browse/workspace/DEBS_GC_20

  • No labels