Tutorial: Simple Query Processing
In this tutorial you will learn elementary steps in using Odysseus.
Step 1: Install/Start Nexmark
As sources we will use the Nexmark examples source. See Getting Started with Nexmark for the installation of the nexmark server.
Step 2: Install/Start Odysseus
For this example you should use the standard Odysseus Studio (Odysseus Server + Odysseus Studio Client) (See How to install Odysseus).
Step 3: Create a new project
Create a new project: FirstSteps
...
The new project is shown in the project explorer (hint if you do not see the project explorer, use Window/Show View/Other ... and choose General/Project Explorer).
Step 4: Describe sources
The queries will access sources that deliver events. In our example we will connect to the already started nexmark server. In this examples we will use PQL to create the sources. For detailed information about source definitions see The Odysseus Access Operator Frameworkframework
There are different ways to created sources/queries in Odysseus. We will use the Odysseus Script way.
...
Give the script a name, e.g. NexmarkSources, and choose Next
There a different templates that can be used: Choose PQL Basic, and click on Finish.
A new editor window will open as following:
Now we have to tell Odysseus, how to access the different sources (person, bid, auction and category).
...
Code Block |
---|
nexmark:person ::= ACCESS({source='nexmark:person',
wrapper='GenericPush',
transport='TCPClient',
protocol='SizeByteBuffer',
dataHandler='Tuple',
options=[
['host', 'localhost'],
['port', '65440'],
['ByteOrder', 'LittleEndian']
],
schema=[
['timestamp', 'STARTTIMESTAMP'],
['id', 'INTEGER'],
['name', 'STRING'],
['email', 'STRING'],
['creditcard', 'STRING'],
['city', 'STRING'],
['state', 'STRING']
]
}) |
...
There are two types of parameters here. The first part is the physical connection to the server (We will not describe all parameters here (see The Odysseus Access Operator Frameworkframework for further information).If you have started the nexmark server on another machine change 'host' and if you started the person server on another port change 'port'.) and the seconds part describes the information this souces delivers.
...
There are different build in datatypes available in Odysseus (see The Odysseus Access Operator Frameworkframework for further information). If the sources provides application time (as the nexmark server does), the datatype STARTTIMESTAMP can be used to state that a LONG value should be used as timestamp for this event.
Now save the editor input (e.g. with CTRL-S) and run the query by using Execute Script (e.g. by right clicking on the file in the project explorer, on the editor it self or by using the button
in the toolbar)
If all is correct, there will be a new source and an access operator in the source-view.
You can use the arrow to show more information about the sources.
Now remove the sources again by clicking on the symbol. It will remove all sources. will remove the current marked source. A source can be removed by right clicking on that source and choosing: Drop Source, too.
The other sources are created the same way. To go further, add the following to the NexmarkSource editor (after the definition of the person source) and execute the script.
Code Block |
---|
#RUNQUERY
nexmark:bid := ACCESS({source='nexmark:bid',
wrapper='GenericPush',
transport='TCPClient',
protocol='SizeByteBuffer',
dataHandler='Tuple',
options=[
['host', 'localhost'],
['port', '65442'],
['ByteOrder', 'LittleEndian']
],
schema=[
['nexmark:bid','timestamp', 'STARTTIMESTAMP'],
['nexmark:bid','auction', 'INTEGER'],
['nexmark:bid','bidder', 'INTEGER'],
['nexmark:bid','datetime', 'LONG'],
['nexmark:bid','price', 'DOUBLE']
]
})
#RUNQUERY
nexmark:auction := ACCESS({source='nexmark:auction',
wrapper='GenericPush',
transport='TCPClient',
protocol='SizeByteBuffer',
dataHandler='Tuple',
options=[
['host', 'localhost'],
['port', '65441'],
['ByteOrder', 'LittleEndian']
],
schema=[
['timestamp', 'STARTTIMESTAMP'],
['id', 'INTEGER'],
['itemname', 'STRING'],
['description', 'STRING'],
['initialbid', 'INTEGER'],
['reserve', 'INTEGER'],
['expires', 'LONG'],
['seller', 'INTEGER'],
['category', 'INTEGER']
]
})
#RUNQUERY
nexmark:category := ACCESS({source='nexmark:category',
wrapper='GenericPush',
transport='TCPClient',
protocol='SizeByteBuffer',
dataHandler='Tuple',
options=[
['host', 'localhost'],
['port', '65443'],
['ByteOrder', 'LittleEndian']
],
schema=[
['id', 'INTEGER'],
['name', 'STRING'],
['description', 'STRING'],
['parentid', 'INTEGER']
]
}) |
After that you see a list of four sources and access operators in the source view:
To test, if everthings is connected correctly, right click on System.nexmark:person and choose "Query Source"
A new window should with person events should open.Your window should look somehow like this:
Now close the window by clicking on .
If you choose show data again, the source will be beginning from start(but only if no other connection to the nexmark-server is open, e.g. via bids).
Step 5: Formulating Queries
Now we are ready to formulate queries over the nexmark sources. Queries in Odysseus are data pipelines. Every operator is responsible for a single subtask in the processing. It receives data from the operator nearer to the source and sends its output to all connected output operators (subscriber). In the following you will see simple examples of processing pipelines.
For this, create a new Odysseus Script file named query1 the same way as above.
We will use Procedural Query Language (PQL) in this example.
The simplest query is just to access all elements of the source. For this, write:
Code Block |
---|
out = nexmark:person |
save and execute the script with .
A new query should be seen in the Queries-view:
Typically, the queries will run in the background. To see, what the query delivers, you can right right on the query in the Queries-View, select Show Stream Elements, and choose e.g. Table - Show last 50 elements:
Depending on the time, you click on the query, the current elements are shown. Hint: You have no chance to see older elements, because they are already processed.
The ouput should look somehow like:
A query can be removed by selecting the query in the Queries view and choosing in the toolbar above the query. With all queries are removed.
You can also start and stop queries with the elements in the toolbar by choosing the seconds and the fourth button. Restaring (third button) is the same, as stopping and starting a query.
There are also buttons to do same same for all queries (the fifth, sixth and seventh button).
To see the current execution plan for a query double click on the query or choose from the context menu Show Query Plan.
Hint: The button in the toolbar, shows the query plan for all current running queries (of this user).
The current query should look like in the following:
You will only see, a single operator, representing the access to the nexmark:person source.
You can right click on (each) operator in the view to see the data that this operator is currently processing (similar to the query source).
In the Outline-View you can see further information about each operator:
In the Operator Detail Info View further operator specific information can be found
This was a simple step through. More complex examples can be found in the Tutorials section.