Nov 24, 2020 in Hadoop
Q: What is speculative execution in Hadoop?

1 Answer

0 votes
Nov 24, 2020

If a DataNode is executing any task slowly, the master node can redundantly execute another instance of the same task on another node. The task that finishes first will be accepted, and the other task would be killed. Therefore, speculative execution is useful if you are working in an intensive workload kind of environment.

The following image depicts the speculative execution:

From the above example, you can see that node A has a slower task. A scheduler maintains the resources available, and with speculative execution turned on, a copy of the slower task runs on node B. If node A task is slower, then the output is accepted from node B.

Click here to read more about Loan/Mortgage
Click here to read more about Insurance

Related questions

0 votes
Nov 8, 2020 in Hadoop