The directory watcher transport handler allows to watch a directory and process modification and creation of files
Options
directortdirectory:
The directoryfilter
: The file filter (optional)cachefiles
: If true, changed file handles get cached so that only new content gets processed instead of the whole file on each change (default. false)cachesize
: The size of the cache. File handles get removed in a Least Recently Used fashion (default: 10)
Example
PQL
Code Block |
---|
language | pql |
---|
title | Example |
---|
linenumbers | true |
---|
|
input = ACCESS({source='Source',
wrapper='GenericPush',
transport='Directory',
protocol='CSV',
dataHandler='Tuple',
options=[['directory', '/src/odysseus'], ['filter', '.txt$']],
schema=[
['id', 'Double'],
['data', 'String']]
})
|
...
Code Block |
---|
language | sql |
---|
title | Example |
---|
linenumbers | true |
---|
|
CREATE STREAM source (id Double, data STRINGString)
WRAPPER 'GenericPush'
PROTOCOL 'CSV'
TRANSPORT 'Directory'
DATAHANDLER 'Tuple'
OPTIONS ( 'filename' '/src/odysseus')
|