How to debug Hadoop code?
First, check the list of MapReduce jobs currently running. Then, check whether orphaned jobs is running or not; if yes, you need to determine the location of RM logs.
First of all, Run: “ps –ef| grep –I ResourceManager” and then, look for log directory in the displayed result. Find out the job-id from the displayed list. Then check whether error message associated with that job or not.
Now, on the basis of RM logs, identify the worker node which involves in the execution of the task.
Now, login to that node and run- “ps –ef| grep –I NodeManager”
Examine the NodeManager log.
The majority of errors come from user level logs for each amp-reduce job.
