0 votes
in Apache Spark by
Explain how Spark runs applications with the help of its architecture.

1 Answer

0 votes
by

This is one of the most frequently asked spark interview questions, and the interviewer will expect you to give a thorough answer to it.

worker-node

Spark applications run as independent processes that are coordinated by the SparkSession object in the driver program. The resource manager or cluster manager assigns tasks to the worker nodes with one task per partition. Iterative algorithms apply operations repeatedly to the data so they can benefit from caching datasets across iterations. A task applies its unit of work to the dataset in its partition and outputs a new partition dataset. Finally, the results are sent back to the driver application or can be saved to the disk.

Related questions

0 votes
asked Mar 29, 2022 in Apache Spark by sharadyadav1986
0 votes
asked Mar 29, 2022 in Apache Spark by sharadyadav1986
...