2,284 questions with Azure Databricks tags

Sort by: Updated
2 answers

Latency in API call

how to fix the api latency where parallel apis sending to microsoft crm .. how to fix this latency api is taking time to send the data

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-23T14:34:46.37+00:00
Vineet S 1,350 Reputation points
answered 2024-12-24T19:06:34.0033333+00:00
phemanth 12,575 Reputation points Microsoft Vendor
1 answer

How can you see the user that is assigned the Role in an ARM (Bicep) deployment.

I have this Bicep code which creates an Azure Databricks access connector and then gives Reader Role to some Service Principal that have been given as an input in the parameters. resource accessConnector…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-20T13:09:42.3733333+00:00
Aris 20 Reputation points
commented 2024-12-24T18:39:52.4766667+00:00
phemanth 12,575 Reputation points Microsoft Vendor
1 answer

Read data from Databricks Delta Sharing to Azure Synapse

Hi, I am trying to read the data from Databricks Delta Sharing and write the data to Azure Synapse by referring this documentation https://docs.databricks.com/en/connect/external-systems/synapse-analytics.html#language-python I have constructed below…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,093 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-18T15:05:16.13+00:00
Cheng, Jack 0 Reputation points
edited a comment 2024-12-24T16:13:46.5666667+00:00
Ganesh Gurram 2,305 Reputation points Microsoft Vendor
1 answer

Need to create a policy which allows specific users for cluster creation in azure databricks

Need help to create a policy which allows specific users for cluster creation in azure databricks

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
Azure Policy
Azure Policy
An Azure service that is used to implement corporate governance and standards at scale for Azure resources.
939 questions
asked 2024-11-28T12:47:02.7133333+00:00
Sahitya Nigam 0 Reputation points
commented 2024-12-24T16:13:31.39+00:00
Pranay Reddy Madireddy 1,230 Reputation points Microsoft Vendor
2 answers One of the answers was accepted by the question author.

How to run SQL stored procedure in Azure Database for SQL Server using Azure Databricks Notebook

We have Stored Procedure available in Azure Database for SQL Server and we want to call or run or execute the SQL stored procedures in Azure Databricks through Notebook We are attempting to run SQL stored procedures, through Azure Databricks notebook and…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-09T14:27:49.43+00:00
AnujSinghCognizant-4026 90 Reputation points
answered 2024-12-24T13:34:29+00:00
Rahul Ahuja 0 Reputation points
1 answer

can the path of source files be the same as the volume's location path?

hi friend, I currently have multiple source files stored on SA (Storage Account) in a directory called /raw/, which also contains a subdirectory /raw/sub1/. When I create a volume, the location is also pointing to SA. Can this location path point to…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,292 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-20T03:33:25.1933333+00:00
Huang, Winston-HR 300 Reputation points
commented 2024-12-24T05:35:19.4766667+00:00
Keshavulu Dasari 2,420 Reputation points Microsoft Vendor
1 answer

I couldnt create Access tokens. I dont see the option of "user setting" in the profile

I couldnt create Access tokens. I dont see the option of "user setting".  I have checked the "roles" under "Access control"  where I have the "owner" role. I have checked the Workspace admin settings,I could see I…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-18T10:01:48.6+00:00
JANARDH Y 0 Reputation points
commented 2024-12-23T07:05:20.9066667+00:00
Smaran Thoomu 18,710 Reputation points Microsoft Vendor
1 answer

I could see that In the 'Roles' in Access Control, Im the owner of my account. But I was not able to create accesss control. I think I dont have the permissions of a Databricks workspace admin

In azure databricks, I wanted to create Access Token. But Im not able to. I may not have the permissions of a Databricks workspace admin. How do i do that

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-18T05:11:23.9333333+00:00
JANARDH Y 0 Reputation points
commented 2024-12-23T06:58:07.98+00:00
Smaran Thoomu 18,710 Reputation points Microsoft Vendor
2 answers One of the answers was accepted by the question author.

Azure Databricks Usage Cost

Hi Team, We have an Azure Databricks subscription with reservations, and multiple workspaces for different projects. We want to monitor our cost usage in the best way possible. I've explored two options: Creating a consumption dashboard in the Account…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-18T16:43:33.6566667+00:00
Mukesh Thakor 40 Reputation points
commented 2024-12-22T02:14:45.2833333+00:00
Mukesh Thakor 40 Reputation points
1 answer One of the answers was accepted by the question author.

Unable to Enable Serverless Clusters in Azure Databricks Workspace

I've been trying to activate serverless clusters in Azure Databricks, but I can't find the feature enabled in the workspace console settings. My understanding is that this feature should be available in all regions; I'm currently working in US East.…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-20T15:08:30.3533333+00:00
Alejandro Montenegro Ortiz 40 Reputation points
commented 2024-12-20T17:53:00.8566667+00:00
Alejandro Montenegro Ortiz 40 Reputation points
2 answers

Not Able To Access DataBricks Manage Account Console. Logging through global admin account. Can anyone help what to be done to resolve this.

Getting above error when try to click on manage account in databricks. Can anyone guide me to resolve this issue.

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-18T19:09:28.6366667+00:00
Rahul Goel 0 Reputation points
commented 2024-12-20T16:06:45.87+00:00
Ganesh Gurram 2,305 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

not being able to create azure free account or even to upgrade to pay as u go

how am i suppose to learn the certification if am not being able to create account. i am really frustrated that am not being able to create account. provide me an efficient way to create account to practice for my certification

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,250 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,093 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,047 questions
asked 2024-12-18T20:10:24.3266667+00:00
ashfa 25 Reputation points
edited the question 2024-12-20T13:40:32.9233333+00:00
simo-k 1,350 Reputation points
1 answer One of the answers was accepted by the question author.

How to get utilization(cpu,memory) report of Adb cluster

I have three clusters in adb i need its utilization report how can i get that

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-19T16:01:18.6566667+00:00
Aetemaad Hussain 20 Reputation points
edited a comment 2024-12-19T17:33:17.9+00:00
Ganesh Gurram 2,305 Reputation points Microsoft Vendor
1 answer

How to configure default Lakehouse in notebook in Fabric

I want to configure defualt laekhouse in notebook. i can do it using %%configure command. but i am note able to use variables here to be used in %%configure command. As configure should be the first cell of the notebook and if you put it in middle of the…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
38 questions
asked 2024-12-16T06:53:15.4766667+00:00
Ketan Bharat | MAQ Software 60 Reputation points
commented 2024-12-18T02:01:09.1+00:00
Chandra Boorla 5,755 Reputation points Microsoft Vendor
1 answer

Concurrency issue in partitioned table while updating same table from different spark notebook

Created partitioned table with below schema CREATE TABLE Landing.TargetTable ( Id INT, Name STRING, SourceSystem INT, SourceId INT, ETLId INT ) USING DELTA PARTITIONED BY (SourceId); Two source tables are there from where data are…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,093 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-10T11:23:16.31+00:00
heta desai 357 Reputation points
commented 2024-12-17T01:16:02.7466667+00:00
Chandra Boorla 5,755 Reputation points Microsoft Vendor
1 answer

Can a Databricks workspace use an access connector to retrieve secrets from Key Vault?

faced error when trying to access secrets by adding the access connector object ID as key Vault Administrator on Key Vault.

Azure Key Vault
Azure Key Vault
An Azure service that is used to manage and protect cryptographic keys and other secrets used by cloud apps and services.
1,342 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-10T15:32:28+00:00
Mitasha Dutta (LTIMINDTREE LIMITED) 0 Reputation points Microsoft Vendor
commented 2024-12-16T13:32:53.8866667+00:00
Harshitha Eligeti 1,210 Reputation points Microsoft Vendor
2 answers One of the answers was accepted by the question author.

Issue with logging MLflow run in Databricks - BAD_REQUEST: Unable to connect to the linked AzureML workspace

I am encountering an issue while trying to log MLflow runs to an experiment and register a model in Azure Databricks. While executing the mlflow.start_run(), I continually receive a RestException: BAD_REQUEST error with the message "Unable to…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2023-11-07T08:20:43.3633333+00:00
Joe 25 Reputation points
answered 2024-12-16T09:00:56.7833333+00:00
Narsingh Maurya 0 Reputation points
1 answer

How to configure IP access list to databricks workspace

How to configure the ip access list to databricks workspace. I have seen the microsoft link for this : https://zcusa.951200.xyz/en-us/azure/databricks/security/network/front-end/ip-access-list-workspace but not found (databricks workspace-conf…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-11T15:51:42.62+00:00
rkreddy 0 Reputation points
commented 2024-12-16T01:54:13.32+00:00
Chandra Boorla 5,755 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

Analysis of Cost Saved

Hi, Current Solution: Delta table from HiveMetastore -> Staging Table with Business Logic in Databricks -> PowerBI Current Solution in Detailed: Main responsibilities for me is to consume the data the delta tables from hive_metastore in…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,047 questions
asked 2024-12-11T05:47:30.34+00:00
Jayaswaroop Adabala 20 Reputation points
accepted 2024-12-13T06:50:15.1566667+00:00
Jayaswaroop Adabala 20 Reputation points
3 answers

Selected user account does not exist in tenant 'Microsoft Services' and cannot access the application

Following Establish your first account admin bellow: https://zcusa.951200.xyz/en-us/azure/databricks/admin/#account-admins I login with owner, global admin account in azure portal, and Go to accounts.azuredatabricks.net and sign in with my global…

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,284 questions
asked 2024-12-09T21:33:07.3366667+00:00
Xuanbin Zhang 0 Reputation points
commented 2024-12-13T03:58:23.1366667+00:00
Chandra Boorla 5,755 Reputation points Microsoft Vendor