Share via

Use Microsoft Fabric to read data that is registered in Unity Catalog

  • Article

This article gives an overview of how to use Microsoft Fabric to read data that is registered in Unity Catalog.

Overview of the Fabric integration with Unity Catalog

Azure Databricks uses Azure Data Lake Storage Gen2 for open, accessible, and low-cost storage. Unity Catalog manages and governs that data based on user-defined policies. Microsoft Fabric lets users read some of these tables registered in Unity Catalog using a file shortcut.

To enable users to read data registered in Unity Catalog from Fabric, a Fabric catalog admin creates OneLake shortcuts that point to the data registered in Unity Catalog. In the Fabric sharing feature, an initial handshake is made with Unity Catalog’s open APIs to vend credentials that give access to the underlying storage paths for supported tables. When the handshake is confirmed, Fabric creates OneLake shortcuts to data stored in Azure Data Lake Storage Gen2. The credentials that are provided by Unity Catalog are short-lived, refreshed every hour, and can be revoked using Unity Catalog to deny Fabric user access.

Note

When you use Fabric to read data that is registered in Unity Catalog, it is important to understand the following:

  • “Mirrored Azure Databricks Catalog” items in Fabric do not replicate data. Credentials to access data are fetched on-demand from Fabric engines using Unity Catalog APIs. Databricks provides an open platform based on the lakehouse architecture: data copies are not required or advised.
  • Fabric engines perform authorization with Unity Catalog using short-lived credentials tied to the identity of the user who configured the connection, not the user who is querying the actual data. Fabric engines do not apply Unity Catalog governance and security policies to Fabric users. In other words, when a table is exposed in Fabric, no Unity Catalog-governed access controls on that table apply to Fabric users. This can lead to potential violations of corporate information security policies.

Before you begin

To access Unity Catalog data using Fabric, the user who configures the connection to Databricks from Fabric must have permission to get temporary credentials using Unity Catalog open API credential vending.

This requires that the configuring user have the EXTERNAL USE SCHEMA privilege on the schema in Unity Catalog that contains the tables that will be accessed from Fabric. For instructions, see Control external access to data in Unity Catalog.

Note

Once the connection is made, no downstream Fabric users who have access to the connection are required to have this privilege, effectively bypassing Unity Catalog governance.

Limitations

Using Fabric to read data that is registered in Unity Catalog does not support the following:

  • Existing Unity Catalog security policies on downstream users in Fabric.
  • Views, materialized views, and streaming tables.
  • Delta Sharing catalogs.
  • Lakehouse Federation catalogs.
  • Tables with row-level filters or column masks enabled.
  • Tables that do not use Delta Lake as the format.
  • Unity Catalog lineage for operations performed in Fabric.
  • Azure Databricks workspaces that use private endpoints or IP access lists.
  • Tables whose underlying Azure Data Lake Storage is behind a firewall.

Using Fabric also requires a running Fabric capacity to perform metadata scans and refreshes, which introduces additional cost.

Note

These limitations do not apply when you use Power BI Direct Query with Unity Catalog registered data. See Alternatives for accessing data registered in Unity Catalog.

How to read Unity Catalog data using Fabric

For complete instructions, see the Microsoft documentation on configuring Fabric shortcuts.

Alternatives for accessing data registered in Unity Catalog

Azure Databricks uses Azure Data Lake Storage Gen2 for open, accessible, and low-cost storage. Learn more in Best practices for cloud storage with Unity Catalog. If you are uncomfortable with the risk exposure and limitations associated with the Fabric sharing feature, we recommend the following paths for accessing data registered in Unity Catalog: