In this tutorial you will learn elementary steps in using Odysseus.
Step 1: Install/Start Nexmark
As sources we will use the Nexmark examples source. See Getting Started with Nexmark for the installation of the nexmark server.
Step 2: Install/Start Odysseus
For this example you should use the standard Odysseus Studio (Odysseus Server + Odysseus Studio Client) (See How to install Odysseus).
Step 3: Create a new project
Create a new project: FirstSteps
Select File/New... and choose Odysseus/Odysseus Project
Name it e.g.FirstSteps:
The new project is shown in the project explorer (hint if you do not see the project explorer, use Window/Show View/Other ... and choose General/Project Explorer).
Step 4: Describe sources
The queries will access sources that deliver events. In our example we will connect to the already started nexmark server. In this examples we will use PQL to create the sources. For detailed information about source definitions see Access framework
There are different ways to created sources/queries in Odysseus. We will use the Odysseus Script way.
Right click on the new created project and choose New/Odysseus Script
Give the script a name, e.g. NexmarkSources, and choose Next
There a different templates that can be used: Choose PQL Basic, and click on Finish.
A new editor window will open as following:
Now we have to tell Odysseus, how to access the different sources (person, bid, auction and category).
Start with person by typing the following in to the new editor (below #RUNQUERY):
nexmark:person ::= ACCESS({source='nexmark:person', wrapper='GenericPush', transport='TCPClient', protocol='SizeByteBuffer', dataHandler='Tuple', options=[ ['host', 'localhost'], ['port', '65440'], ['ByteOrder', 'LittleEndian'] ], schema=[ ['timestamp', 'STARTTIMESTAMP'], ['id', 'INTEGER'], ['name', 'STRING'], ['email', 'STRING'], ['creditcard', 'STRING'], ['city', 'STRING'], ['state', 'STRING'] ] })
This statement describes a connection to the nexmark person server, delivered with Odysseus.
We create a new source with the name nexmark:person (and the type nexmark:person).
There are two types of parameters here. The first part is the physical connection to the server (We will not describe all parameters here (see Access framework for further information).If you have started the nexmark server on another machine change 'host' and if you started the person server on another port change 'port'.) and the seconds part describes the information this souces delivers.
The most important part is the schema description. Here (somehow similar to a create table statement in SQL) the names and the types of the attributes are defined.
There are different build in datatypes available in Odysseus (see Access framework for further information). If the sources provides application time (as the nexmark server does), the datatype STARTTIMESTAMP can be used to state that a LONG value should be used as timestamp for this event.
Now save the editor input (e.g. with CTRL-S) and run the query by using Execute Script (e.g. by right clicking on the file in the project explorer, on the editor it self or by using the button
in the toolbar)If all is correct, there will be a new source in the source-view.
Now remove the source again by clicking on the
. It will remove all current sources. A single source can be removed by right clicking on that source and choosing: Drop Source.The other sources are created the same way. To go further, add the following to the NexmarkSource editor (after the definition of the person source) and execute the script.
#RUNQUERY nexmark:bid ::= ACCESS({source='nexmark:bid', wrapper='GenericPush', transport='TCPClient', protocol='SizeByteBuffer', dataHandler='Tuple', options=[ ['host', 'localhost'], ['port', '65442'], ['ByteOrder', 'LittleEndian'] ], schema=[ ['nexmark:bid','timestamp', 'STARTTIMESTAMP'], ['nexmark:bid','auction', 'INTEGER'], ['nexmark:bid','bidder', 'INTEGER'], ['nexmark:bid','datetime', 'LONG'], ['nexmark:bid','price', 'DOUBLE'] ] }) #RUNQUERY nexmark:auction ::= ACCESS({source='nexmark:auction', wrapper='GenericPush', transport='TCPClient', protocol='SizeByteBuffer', dataHandler='Tuple', options=[ ['host', 'localhost'], ['port', '65441'], ['ByteOrder', 'LittleEndian'] ], schema=[ ['timestamp', 'STARTTIMESTAMP'], ['id', 'INTEGER'], ['itemname', 'STRING'], ['description', 'STRING'], ['initialbid', 'INTEGER'], ['reserve', 'INTEGER'], ['expires', 'LONG'], ['seller', 'INTEGER'], ['category', 'INTEGER'] ] }) #RUNQUERY nexmark:category ::= ACCESS({source='nexmark:category', wrapper='GenericPush', transport='TCPClient', protocol='SizeByteBuffer', dataHandler='Tuple', options=[ ['host', 'localhost'], ['port', '65443'], ['ByteOrder', 'LittleEndian'] ], schema=[ ['id', 'INTEGER'], ['name', 'STRING'], ['description', 'STRING'], ['parentid', 'INTEGER'] ] })
After that you see a list of four sources in the source view:
To test, if everthings is connected correctly, right click on System.nexmark:person and choose Show data (New "Query source"):
A new window should with person events should open. If your using the standard configuration of nexmark, it takes 5 seconds before the first event occures. Then every 5 second a new person will be shown.
Your window should look somehow like this:
Now close the window by clicking on
.If you choose show data again, the source will be beginning from start(but only if no other connection to the nexmark-server is open, e.g. via bids).
Step 5: Formulating Queries
Now we are ready to formulate queries over the nexmark sources. Queries in Odysseus are data pipelines. Every operator is responsible for a single subtask in the processing. It receives data from the operator nearer to the source and sends its output to all connected output operators (subscriber). In the following you will see simple examples of processing pipelines.
For this, create a new Odysseus Script file named query1 the same way as above.
We will use Procedural Query Language (PQL) in this example.
The simplest query is just to access all elements of the source. For this, write:
out = nexmark:person
save and execute the script with
.A new query should be seen in the Queries-view:
Typically, the queries will run in the background. To see, what the query delivers, you can right right on the query in the Queries-View, select Show Stream Elements, and choose e.g. Table - Show last 50 elements:
Depending on the time, you click on the query, the current elements are shown. Hint: You have no chance to see older elements, because they are already processed.
The ouput should look somehow like:
A query can be removed by selecting the query in the Queries view and choosing
in the toolbar above the query. With all queries are removed.You can also start and stop queries with the elements in the toolbar by choosing the seconds and the fourth button. Restaring (third button) is the same, as stopping and starting a query.
There are also buttons to do same same for all queries (the fifth, sixth and seventh button).
To see the current execution plan for a query double click on the query or choose from the context menu
Show Query Plan.Hint: The button in the toolbar, shows the query plan for all current running queries (of this user).
The current query should look like in the following:
Remark: The image above is only true for older versions. In the current release, only nexmark:person is diplayed.
Here you can see, that there is a difference between the logical query (in PQL) and the real physical query. The logical has only one operator, the phyiscal three.
- nexmark:person: Is the access operator that is connected to the source.
MetadataCreatonPO: appends to each event the empty metadata object that is necessary for processing (e.g. timestamps)ApplicationTime: uses the information from the schema description. In this case you can see (if you look at the table above) that it updates the starttimestamp of each element. the first column contains the timestamp that is delivered from the nexmark-server and the last column contains the updated interval (the end timestamp is still on infinity)
You can right click on each operator in the view to see the data that this operator is currently processing. If you choose the values from MetadataCreationPO the output will be similar to the following, where you can see, that no metadata is set.
If you choose the output of the nexmark:person, metadata will be <null>
In the Outline-View you can see further information about each operator. Choose the ApplicationTime operator and you will see somehing like:
In the Operator Detail Info View further operator specific information can be found
This was a simple step through. More complex examples can be found in the Tutorials section.