From Storage to Analysis: Accessing Azure Data Lake with Databricks

The article demonstrates how to access an Azure Data Lake Gen2 account using a Databricks Notebook. This is a common use case for data engineers in their daily activities.

Photo by xyzcharlize on Unsplash

Azure Databricks has several ways to mount storage and manage data more efficiently.

Access Keys

• Requirements: Storage account name and access key.
• Steps:

1. Retrieve the storage account access key from Azure.
2. Use the access key to mount Blob Storage to Databricks:

configs = {
    "fs.azure.account.key.<storage-account-name>.blob.core.windows.net": "<access-key>"
}

dbutils.fs.mount(
    source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/",
    mount_point = "/mnt/<mount-name>",
    extra_configs = configs
)