Mar 14, 2020 in Spark Sql
Q: How to minimize data transfers working with Spark in Spark?

1 Answer

0 votes
Mar 14, 2020
The data transfer minimization and shuffling removal can be very much helpful in writing Spark programs that can run reliably. There are different ways of minimizing data transfers with Spark:

They are:

Using Accumulators

Using Broadcast Variable

Avoiding operations by key

Related questions

+1 vote
Mar 9, 2020 in Spark Sql
+1 vote
Mar 9, 2020 in Spark Sql