Categories
5G Network
Agile
Amazon EC2
Android
Angular
Ansible
Arduino
Artificial Intelligence
Augmented Reality
AWS
Azure
Big Data
Blockchain
BootStrap
Cache Teachniques
Cassandra
Commercial Insurance
C#
C++
Cloud
CD
CI
Cyber Security
Data Handling
Data using R
Data Science
DBMS
Design-Pattern
DevOps
ECMAScript
Fortify
Ethical Hacking
Framework
GIT
GIT Slack
Gradle
Hadoop
HBase
HDFS
Hibernate
Hive
HTML
Image Processing
IOT
JavaScript
Java
Jenkins
Jira
JUnit
Kibana
Linux
Machine Learning
MangoDB
MVC
NGINX
Onsen UI
Oracle
PHP
Python
QTP
R Language
Regression Analysis
React JS
Robotic
Salesforce
SAP
Selenium
Service Discovery
Service Now
SOAP UI
Spark SQL
Testing
TOGAF
Research Method
Virtual Reality
Vue.js
Home
Recent Q&A
Feedback
Ask a Question
Why Mapper runs in heavy weight process and not in a thread in MapReduce?
Home
>
Big Data | Hadoop
>
Why Mapper runs in heavy weight process and not in a thread in MapReduce?
Jan 8, 2020
in
Big Data | Hadoop
Q:
Why Mapper runs in heavy weight process and not in a thread in MapReduce?
1
Answer
0
votes
Jan 8, 2020
Each task is launches as a separate process instead of thread because :
Mappers are run across Hadoop clusters in distributed manner, in distributed processing environment the task is split and are run in parallel.
Threads are multiple tasks of a single process which shares the same memory area and the data and usually threads are within the boundary of a single system, but each mapper uses different data for processing(since its distributed).
Each of the Mapper task in Hadoop runs as a different JVM process, this is because MapReduce programs are long running processes and they can be killed due to usage of commodity hardware. If Map reduce were implemented as thread, one error in a single mapper could kill the entire process and hence you would have to re run all the process(since as stated above, it’s sub-task under same process).
Managing threads is relatively more complex, if a thread execution hangs then it needs to be killed and the task would have to start from where it left.
Click here to read more about Loan/Mortgage
Click here to read more about Insurance
Facebook
Twitter
LinkedIn
Related questions
0
votes
Q: What is Mapper in Hadoop MapReduce?
Jan 8, 2020
in
Big Data | Hadoop
0
votes
Q: Why MapReduce uses the key-value pair to process the data?
Nov 8, 2020
in
Hadoop
#mapreducuse
0
votes
Q: For which of the following type of data it is not possible to store in big data environment and then process/parse it?
Jan 7, 2020
in
Big Data | Hadoop
#environment-process
0
votes
Q: Why ‘Reading‘ is done in parallel and ‘Writing‘ is not in HDFS?
Feb 23, 2020
in
Big Data | Hadoop
#hdfs-reading
0
votes
Q: What is the default maximum dynamic partition that can be created by a mapper/reducer?
Jan 10, 2020
in
Big Data | Hadoop
#hive-mapper
0
votes
Q: Why is MongoDB not chosen for a 32-bit system in MongoDB ?
Mar 22, 2020
in
Big Data | Hadoop
#mongodb-system
...