It is the Azure implementation of Apache Spark that is an open-source big data processing platform. In the data lifecycle, Azure Databricks lies in the data preparation or processing stage. First of all, data is ingested in Azure using Data Factory and stored in permanent storage (such as ADLS Gen2 or Blob Storage). Further, data is processed using Machine Learning (ML) in Databricks and then extracted insights are loaded into the Analysis Services in Azure like Azure Synapse Analytics or Cosmos DB.
Finally, insights are visualised and presented to the end-users with the help of Analytical reporting tools like Power BI.
