The HTTP transport handler allows the communication over the HTTP standard.

Options

  • uri: The URI
  • method: The HTTP method, one of: GET, POST, PUT, DELETE, HEAD (default: GET) (optional)
  • body: The content for POST and PUT requests when using this transport handler for a source (optional)
  • chunked: Controls the 'chunked' header (see Chunked transfer encoding). (default: true) (optional)
  • scheduler.delay: If you want to query in a given time-interval. The unit are milliseconds.
  • oneDocPerCall: If set to false, the transport handler assumes an endless stream. If set to false, assumes whole documents per call
  • contenttype: Set the Content-Type int he HTTP header
  • closeOn404: closes the stream when a 404 status code is returned by the server

Example

PQL

HTTP Transport Handler
input = ACCESS({source='Source',
wrapper='GenericPull',
transport='HTTP',
protocol='CSV',
dataHandler='Tuple',
options=[['uri', 'http://www.example.com'], ['method', 'get']],
schema=[
['id', 'Double'],
['data', 'String']]
})

output = SENDER({sink='Sink',
wrapper='GenericPush',
transport='HTTP',
protocol='CSV',
dataHandler='Tuple',
options=[['uri', 'http://www.example.com'], ['method', 'post']]
}, input)


/// Example for the scheduler.delay option
#PARSER PQL
#ADDQUERY
web =  ACCESS({
                  source='Google',
                  wrapper='GenericPull',
                  transport='HTTP',
                  protocol='Document',
                  datahandler='Document',
                  options=[
                    ['uri', 'http://google.com'],
                    ['scheduler.delay','1000'],
                    ['oneDocPerCall','true']
                  ]                                                               
                }                                                     
              )

CQL

HTTP Transport Handler
CREATE STREAM source (id Double, data STRING)
    WRAPPER 'GenericPush'
    PROTOCOL 'CSV'
    TRANSPORT 'HTTP'
    DATAHANDLER 'Tuple'
    OPTIONS ( 'uri' 'http://www.example.com' 'method' 'get')


CREATE SINK sink (id Double, data STRING)
    WRAPPER 'GenericPush'
    PROTOCOL 'CSV'
    TRANSPORT 'HTTP'
    DATAHANDLER 'Tuple'
    OPTIONS ( 'uri' 'http://www.example.com' 'method' 'post')
  • No labels