0 votes
in Azure Databricks by
Do You Select All Columns of a CSV File When Using Schema With Spark .read?

1 Answer

0 votes
by

CSV cannot identify a vertical slice of data. Therefore, it has to read the full file. For columnar files like Parquet, you may avoid reading each column.

Related questions

0 votes
asked Nov 19, 2022 in Azure Databricks by SakshiSharma
0 votes
asked Feb 15, 2023 in Spark Preliminaries by Robindeniel
...