Unlock your data’s potential with Azure SQL Data Warehouse and Azure Databricks
Getting the most out of your data is critical for any business in a competitive environment. Businesses need the ability to get the right data into the right hands at the right time. Azure Databricks and Azure SQL Data Warehouse can help you do just that through a Modern Data Warehouse.
Azure SQL Data Warehouse is an elastic, globally available, cloud data warehouse that leverages Massively Parallel Processing (MPP) to quickly run complex queries across petabytes of data. Azure SQL Data Warehouse provides a familiar interface for your analysts who know SQL and want to drive action in your business.
Azure Databricks combines the best of Databricks and Azure to help customers accelerate innovation with one-click set up, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts powered by Apache Spark.
With the general availability of the Azure Databricks Service comes built-in support for Azure SQL Data Warehouse. This enables any data scientist or data engineer to have a seamless experience connecting their Azure Databricks Cluster and their Azure SQL Data Warehouse when building advanced ETL (extract, transform, and load data) for Modern Data Warehouse Architectures or accessing relational data for Machine Learning and AI. We are seeing that the combination of these services can provide a truly transformative analytics platform for businesses.
Advanced ETL in the Modern Data Warehouse
In a cloud first ecosystem, data is generated in a wide variety of formats based on the source system. It can be structured, semi-structured, nested, and repeated. To run analytics over this data, many customers are choosing to prepare data outside of their warehouse to leverage new skills and tooling that are emerging in their organization. With Azure Databricks, your data engineers can use the full power of the Databricks Runtime with a variety of language choices to develop ETL processes that can scale as you grow and write the data directly into SQL Data Warehouse. With this data in SQL Data Warehouse, your analysts can use SQL and leading BI tools like Power BI to perform complex, interactive analysis of your organizations most valuable data.
Accessing all your data for Machine Learning and AI
In the Machine Learning and Artificial Intelligence space, data wrangling is one of the most time-consuming aspects of a data scientists job. They need to be able to connect and read massive amounts of data to train their models. With built-in support for Azure SQL Data Warehouse, we have made it not only possible, but easy for data scientists to gain access to petabytes of Data Warehousing data. Now they can spend more time on modeling and less time moving data. This all leads to more efficient use of your data scientists precious time.
We are extremely excited to see how the combination of Azure Databricks and Azure SQL Data Warehouse will transform analytics inside your organization. You can get started today by trying out the ETL into Azure SQL Data Warehouse tutorial.
Next steps
This functionality is generally available today in Databricks Runtime 4.0 and connects to any version of Azure SQL DataWarehouse. To get started:
- Create an Azure Databricks workspace
- Create an Azure SQL Data Warehouse
- Reference Connector documentation
Source: Azure Blog Feed