Page History

Versions Compared

Key

This line was added.
This line was removed.
Formatting was changed.

...

fs.default.name: The name of the Hadoop filesystem
Filename: The path to the file
append: append values to end of file. If set to false (default) the values are overwritten.

Code Block

#PARSER PQL
#RUNQUERY
input = ACCESS({source='Source',
wrapper='GenericPull',
transport='hdfs',
protocol='csv',
dataHandler='Tuple',
Internship 2015
options=[['fs.default.name','hdfs://master:9000'],
['username','hduser'],['Filename', '/user/sorted_data.csv'],
['append','false']],
schema=[
['1', 'String'],
['2', 'String'],})

Code Block

output = SENDER({sink='Sink',
wrapper='GenericPush',
transport='hdfs',
protocol='CSV',
dataHandler='Tuple',
options=[['fs.default.name','hdfs://master:9000'],
['username','hduser'],['Filename', '/user/sorted_data.csv'],
['append','false']]
}, input)