...
To process external data streams they need to be registered in Odysseus. This is typically done with one of the query languages Odysseus provides. Although it is possible to use CQL to attach data streams, the PQL approach is much more flexible. In the following we will concentrate on the integration approach with PQL (some corresponding approaches with CQL can be found in the appendix)
PQL - ACCESS
To integrate new streams with PQL the ACCESS-Operator is needed. Because of compatibility issues, there are a lot of more deprecated parameters, which can be set. In the following we will only describe the preferred parameters. The deprecated parameters will be removed in a future version.
The following parameters can be used in the ACCESS-Operator:
...
- Line: This simple handler just reads one line from the input and sends the Text to the data handler.
- Delay [in ms] To reduce the data rate
- SimpleCSV: This handler is similar to line. Additionally, it splits the line based on a delimiter that needs to be set in the options. This handler does not treat escaping of the delimiter (e.g. by quotes or backslash). A string array is send to the data handler.
- Delimiter: The delimiter that separates each element from another.
- CSV: Same as SimpleCSV but treats quotes. Because this version is slower used SimpleCSV if no quoted elements are contained in the source.
- Text: This handler can be used to identify elements in a character stream where a distinct delimiter is used to separate the objects. The whole object is send to the data handler.
- Delimiter: The delimiter that should be used to separate the object
- KeepDelimiter: A flag that indicates if the delimiter should be part of the result send to the data handler or not
- Charset: The java char set that should be used to decode the input (e.g. "utf-8").
GenericPush
- SizeByteBuffer: This handler initially reads 4 byte and interpreting them as input size. The byte order is the java byte order (BIG_ENDIAN) or can be set with the parameter ByteOrder to "LITTLE_ENDIAN". After reading size bytes the Buffer is send to the data handler.
- MarkerByteBuffer: This handler reads elements between a start and an end byte.
- Start: The byte that marks the start token (token is not part of the object)
- End: The byte that marks the end token (token is not part of the object)
- ByteOrder: The Order of the Bytes (default BIG_EDIAN)
...
worldBoundaries := ACCESS({Source='WorldBoundaries', Wrapper='GenericPull',
Schema=[['geometry','SpatialGeometry'],
['geometry_vertex_count','Integer'],
['OBJECTID','Integer'],
['ISO_2DIGIT','String'],
['Shape_Leng','Double'],
['Shape_Area','Double'],
['Name','String'],
['import_notes','String'],
['Google requests','String']
],
InputSchema=['SpatialKML','Integer','Integer','String','Double','Double','String','String','String'],
transport='File',
protocol='csv',
dataHandler='Tuple',
Options=[['filename','C:/Users/Marco Grawunder/Documents/My Dropbox/OdysseusQuickShare/Daten/Geo/World Country Boundaries.csv'],
['Delimiter',',']]})
Extending the framework
AccessAO, AccessAOBuilder ¿ PQL Documentation
GenericPush and GenericPull
New Wrapper
...