0 votes
in Hadoop by
Explain the process of spilling in MapReduce.

1 Answer

0 votes
by

Spilling is a process of copying the data from memory buffer to disk when the buffer usage reaches a specific threshold size. This happens when there is not enough memory to fit all of the mapper output. By default, a background thread starts spilling the content from memory to disk after 80 percent of the buffer size is filled. 

For a 100 MB size buffer, the spilling will start after the content of the buffer reaches a size of 80 MB. 

Related questions

+1 vote
asked Jun 21, 2023 in Hadoop by sharadyadav1986
0 votes
asked Feb 17, 2023 in Hadoop by sharadyadav1986
...