0 votes
in Hadoop by
Explain Hadoop streaming?

1 Answer

0 votes
by

Explain Hadoop streaming?

Hadoop distribution provides generic application programming interface (API). This allows writing Map and Reduce jobs in any desired programming language. The utility allows creating/running jobs with any executable as Mapper/Reducer.

For example:

hadoop jar hadoop-streaming-3.0.jar \

-input myInputDirs \

-output myOutputDir \

-mapper /bin/cat \

-reducer /usr/bin/wc

In the example, both the Mapper and reducer are executables. That read the input from stdin (line by line) and emit the output to stdout. The utility allows creating/submitting Map/Reduce job, to an appropriate cluster. It also monitors the progress of the job until it completes. Hadoop Streaming uses both streaming command options as well as generic command options. Be sure to place the generic options before the streaming. Otherwise, the command will fail.

The general line syntax shown below:

[php]Hadoop command [genericOptions] [streamingOptions][/php]

Related questions

0 votes
asked Oct 23, 2022 in Hadoop by DavidAnderson
0 votes
asked Nov 8, 2020 in Hadoop by rahuljain1
...