0 votes
in Spark Preliminaries by (19.8k points)
How can you set up automatic cleanups in Spark to deal with metadata that has built up?

1 Answer

0 votes
by (19.8k points)

You can start the cleanups by splitting long-running jobs into batches and writing the intermediate results to disc.

Related questions