0 votes
in Azure Databricks by
Do You Select All Columns of a CSV File When Using Schema With Spark .read?

1 Answer

0 votes
by

CSV cannot identify a vertical slice of data. Therefore, it has to read the full file. For columnar files like Parquet, you may avoid reading each column.

...