SparkCore is a general execution engine for the Spark platform, including all the functionalities. It offers in-memory computing capabilities to deliver a good speed, a generalized execution model to support various applications, and Java, Scala, and Python APIs that make the development easy.
The main responsibility of SparkCore is to perform all the basic I/O functions, scheduling, monitoring, etc. It is also responsible for fault recovery and effective memory management.
The key functions of SparkCore are:
- Perform all the basic I/O functions
- Job scheduling
- Monitoring jobs
- Memory management
- Fault-tolerance
- Interaction with storage systems
