The directory watcher transport handler allows to watch a directory and process modification and creation of files

Options

  • directory: The directory
  • filter: The file filter (optional)
  • cachefiles: If true, changed file handles get cached so that only new content gets processed instead of the whole file on each change (default. false)
  • cachesize: The size of the cache. File handles get removed in a Least Recently Used fashion (default: 10)

Example

PQL

Example
input = ACCESS({source='Source',
wrapper='GenericPush',
transport='Directory',
protocol='CSV',
dataHandler='Tuple',
options=[['directory', '/src/odysseus'], ['filter', '.txt$']],
schema=[
['id', 'Double'],
['data', 'String']]
})

CQL

Example
CREATE STREAM source (id Double, data String)
    WRAPPER 'GenericPush'
    PROTOCOL 'CSV'
    TRANSPORT 'Directory'
    DATAHANDLER 'Tuple'
    OPTIONS ( 'filename' '/src/odysseus')
  • No labels