Share via

Import databricks data using tabular editor

George Matlhogela 20 Reputation points
2024-10-02T10:31:51.0633333+00:00

I imported data from databricks to AAS using the Simba ODBC driver but after deploying the model, i can't seem to make the refresh work.
I get this error "Failed to save modifications to the server. Error returned: 'Expression in partition '<oii>my_partion_name</oii>' in table '<oii>my_table_name</oii>' references unknown variable or function 'Databricks.Catalogs'"

Azure Analysis Services
Azure Analysis Services
An Azure service that provides an enterprise-grade analytics engine.
454 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Vinodh247 21,881 Reputation points
    2024-10-02T11:23:42.71+00:00

    Hi George Matlhogela,

    Thanks for reaching out to Microsoft Q&A.

    The error indicates that the partition is referencing a function or object (Databricks.Catalogs) that AAS could not recognize.

    Here are a few steps you can follow to troubleshoot and resolve the issue:

    1. Check ODBC Connection:

    Ensure that the Simba ODBC driver is properly configured, and the connection string used in the model is correct. Verify that the connection works as expected outside of AAS (ex., in PBI or SSMS).

    1. Review M Query:

    The error suggests an issue with the M query or data source expression. If you are using Tabular Editor or any custom M script, review the query inside the partition and make sure that it doesn't reference unsupported functions or objects. AAS might not fully support all functions available in Databricks, so ensure the query only uses supported constructs.

    1. Use Power Query for Databricks:

    Consider using Power Query to connect to Databricks, as it might handle the connection and refresh better within AAS.

    You can do this by:

    • Opening your model in Tabular Editor.
    • Modifying the data source connection to use Power Query to pull data from Databricks.
    1. Test Direct Query:

    If you're using an Import mode, consider switching to DirectQuery mode in Azure Analysis Services and test if the data refreshes properly. Databricks typically performs well with DirectQuery due to the distributed nature of its data processing.

    1. Check Data Source Permissions:

    Ensure that the AAS service principal or user has proper permissions to access the Databricks data. Sometimes, permission issues can cause refresh failures.

    1. ODBC Driver Compatibility:

    Ensure that the Simba ODBC driver is the latest version and that it is fully compatible with AAS. You might need to update the driver or verify that all necessary drivers are installed on the AAS server.

    1. Deploy with Partition Expressions:

    If you are deploying the model using Tabular Editor, ensure that the partition expressions are correctly specified and valid. Incorrect expressions can cause issues when the model tries to process or refresh data.

    1. Logs and Diagnostics:

    Check the AAS logs for more detailed error messages, which might provide more insights into why the Databricks.Catalogs reference is failing. You can access AAS diagnostics logs in Azure portal under the "Diagnostics Settings" of the AAS instance.

    If these steps do not resolve the issue, you might want to consider reevaluating the query approach or using a different method for importing data from Databricks into AAS.

    Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.