An Apache Spark-based analytics platform optimized for Azure.
Hi Chetan,
Since the error is specifically:
Request for user delegation key is not authorized
I would focus on the user delegation key permission path, not only cluster-to-storage private endpoint connectivity.
A few things to check:
- Make sure Storage Blob Delegator is assigned to the Access Connector managed identity at the storage account, resource group, or subscription scope. Container-level scope is not enough for
Get User Delegation Key.
Confirm the role is assigned to the Access Connector’s managed identity principal ID, not just to the Access Connector resource.
Keep Storage Blob Data Contributor on the storage account/container for data access.
If public network access is disabled on the storage account, enable Allow Azure services on the trusted services list to access this storage account. Databricks documentation calls this out for Unity Catalog external locations when public network access is disabled.
Azure CLI example:
az storage account update \
--name <storage-account-name> \
--resource-group <resource-group-name> \
--bypass AzureServices
Wait for Azure RBAC propagation, then re-validate the storage credential / external location.
Also check ADLS Gen2 ACLs on the target path if you are using ACLs in addition to RBAC.
Docs:
Unity Catalog external locations on ADLS Gen2: https://learn.microsoft.com/en-us/azure/databricks/connect/unity-catalog/cloud-storage/external-locations-adls
Azure managed identities for Unity Catalog storage: https://learn.microsoft.com/en-us/azure/databricks/connect/unity-catalog/cloud-storage/azure-managed-identities
Get User Delegation Key authorization: https://learn.microsoft.com/en-us/rest/api/storageservices/get-user-delegation-key
In short, private endpoint connectivity from the cluster proves the data plane path is reachable, but Unity Catalog also needs to generate temporary credentials. For that, the Access Connector managed identity must be allowed to request a user delegation key, and the storage firewall must allow the required Databricks/Azure service path.