in Spark Preliminaries by (20.8k points)
How can you set up automatic cleanups in Spark to deal with metadata that has built up?

1 Answer

0 votes
by (20.8k points)

You can start the cleanups by splitting long-running jobs into batches and writing the intermediate results to disc.

Related questions

+1 vote
asked Feb 2, 2022 in Apache Pig by sharadyadav1986 (31.6k points)
0 votes
asked Aug 24 in Apache Superset by Robin (14.6k points)
0 votes
asked Feb 16 in Spark Preliminaries by Robindeniel (20.8k points)