You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

ALPHA!

You could use the LineSender program to provide sample input from a file to Odysseus.

The source code can be found here: https://git.swl.informatik.uni-oldenburg.de/projects/ODYDATGEN/repos/linesender/browse

A runnable jar can be downloaded here: https://odysseus.informatik.uni-oldenburg.de/download/linesender/linesender-0.0.1-SNAPSHOT-jar-with-dependencies.jar

Use 

java -jar linesender-0.0.1-SNAPSHOT-jar-with-dependencies.jar

to see a list of parameters:

Usage: LineSender [-hrV] [-d=<delay>] [-p=<port>] <path>
Sends a csv file to a tcp socket connection
      <path>            The file whose checksum to calculate.
  -d, --delay=<delay>   Delay ms between each line sent.
  -h, --help            Show this help message and exit.
  -p, --port=<port>     The server port.
  -r, --repeat          Repeat the file when ended.
  -V, --version         Print version information and exit.

This program reads a csv file and sends each csv file via tcp to a connected tcp client. For each new connected client, the reading starts from the beginning of the file. If you set the repeat flag, the content of the csv file is sent again and again to simulate a never ending input stream. You could limit the datarate with the delay flag.

Example

If you have the following csv file:

a;1;1.0
b;2;2.0
c;3;3.0
d;4;4.0
e;5;5.0

and you have started the sender on port 9876. You could use the following Odysseus query to access the content:

#PARSER PQL
#ADDQUERY
in = ACCESS({
          transport = 'tcpclient',
          source = 'source_test',
          datahandler = 'tuple',
          wrapper = 'GenericPush',
          protocol = 'SimpleCSV',
          options = [['host','localhost'],['port','9876'],['delimiter',';']],
          schema=[
            ['A', 'String'],
            ['B', 'Integer'],
            ['C', 'Double']
          ] 
        }      
      )

Remark: If you use repeat, you need to assure at the client side, that the timestamps are increasing. In the above example this is simply done by using system time but when using application time (i.e. providing timestamp datatypes) you will need to increase the timestamps manually,

in_prep = STATEMAP({
              KEEPINPUT = true,
              expressions = [
                ['!isNull(__last_1.B) && __last_1.B > B','nextRound']
              ]
            },
            in
          )



  • No labels