Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,285 questions
2 answers
Latency in API call
how to fix the api latency where parallel apis sending to microsoft crm .. how to fix this latency api is taking time to send the data
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
can the path of source files be the same as the volume's location path?
hi friend, I currently have multiple source files stored on SA (Storage Account) in a directory called /raw/, which also contains a subdirectory /raw/sub1/. When I create a volume, the location is also pointing to SA. Can this location path point to…
asked 2024-12-20T03:33:25.1933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 46%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHW%3C/text%3E%3C/svg%3E)
Huang, Winston-HR
320
Reputation points
edited a comment 2024-12-26T07:53:10.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 6%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Keshavulu Dasari
2,570
Reputation points Microsoft Vendor
1 answer
Unable to create compute in Databricks with Azure Free Trial even after several trials.
Hello Team, I have taken up azure free trial on the sole purpose to learn Databricks. Right after creating the account, I went up straight to launch databricks workspace and tried creating a cluster but was unsuccessful. I have went through most of the…
asked 2024-12-26T07:00:57.0166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 80%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
kkv_manikanta
0
Reputation points
answered 2024-12-26T07:46:56.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
18,710
Reputation points Microsoft Vendor
1 answer
How can you see the user that is assigned the Role in an ARM (Bicep) deployment.
I have this Bicep code which creates an Azure Databricks access connector and then gives Reader Role to some Service Principal that have been given as an input in the parameters. resource accessConnector…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 83%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
Read data from Databricks Delta Sharing to Azure Synapse
Hi, I am trying to read the data from Databricks Delta Sharing and write the data to Azure Synapse by referring this documentation https://docs.databricks.com/en/connect/external-systems/synapse-analytics.html#language-python I have constructed below…
asked 2024-12-18T15:05:16.13+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 55.00000000000001%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECJ%3C/text%3E%3C/svg%3E)
Cheng, Jack
0
Reputation points
edited a comment 2024-12-24T16:13:46.5666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
2,305
Reputation points Microsoft Vendor
1 answer
Need to create a policy which allows specific users for cluster creation in azure databricks
Need help to create a policy which allows specific users for cluster creation in azure databricks
asked 2024-11-28T12:47:02.7133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 11%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESN%3C/text%3E%3C/svg%3E)
Sahitya Nigam
0
Reputation points
commented 2024-12-24T16:13:31.39+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 20%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Pranay Reddy Madireddy
1,310
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
How to run SQL stored procedure in Azure Database for SQL Server using Azure Databricks Notebook
We have Stored Procedure available in Azure Database for SQL Server and we want to call or run or execute the SQL stored procedures in Azure Databricks through Notebook We are attempting to run SQL stored procedures, through Azure Databricks notebook and…
asked 2024-12-09T14:27:49.43+00:00
AnujSinghCognizant-4026
90
Reputation points
answered 2024-12-24T13:34:29+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 55.00000000000001%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Rahul Ahuja
0
Reputation points
1 answer
I couldnt create Access tokens. I dont see the option of "user setting" in the profile
I couldnt create Access tokens. I dont see the option of "user setting". I have checked the "roles" under "Access control" where I have the "owner" role. I have checked the Workspace admin settings,I could see I…
asked 2024-12-18T10:01:48.6+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 65%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJY%3C/text%3E%3C/svg%3E)
JANARDH Y
0
Reputation points
commented 2024-12-23T07:05:20.9066667+00:00
Smaran Thoomu
18,710
Reputation points Microsoft Vendor
1 answer
I could see that In the 'Roles' in Access Control, Im the owner of my account. But I was not able to create accesss control. I think I dont have the permissions of a Databricks workspace admin
In azure databricks, I wanted to create Access Token. But Im not able to. I may not have the permissions of a Databricks workspace admin. How do i do that
asked 2024-12-18T05:11:23.9333333+00:00
JANARDH Y
0
Reputation points
commented 2024-12-23T06:58:07.98+00:00
Smaran Thoomu
18,710
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Azure Databricks Usage Cost
Hi Team, We have an Azure Databricks subscription with reservations, and multiple workspaces for different projects. We want to monitor our cost usage in the best way possible. I've explored two options: Creating a consumption dashboard in the Account…
asked 2024-12-18T16:43:33.6566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 15%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Mukesh Thakor
40
Reputation points
commented 2024-12-22T02:14:45.2833333+00:00
Mukesh Thakor
40
Reputation points
1 answer
One of the answers was accepted by the question author.
Unable to Enable Serverless Clusters in Azure Databricks Workspace
I've been trying to activate serverless clusters in Azure Databricks, but I can't find the feature enabled in the workspace console settings. My understanding is that this feature should be available in all regions; I'm currently working in US East.…
asked 2024-12-20T15:08:30.3533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 45%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
Alejandro Montenegro Ortiz
40
Reputation points
commented 2024-12-20T17:53:00.8566667+00:00
Alejandro Montenegro Ortiz
40
Reputation points
2 answers
Not Able To Access DataBricks Manage Account Console. Logging through global admin account. Can anyone help what to be done to resolve this.
Getting above error when try to click on manage account in databricks. Can anyone guide me to resolve this issue.
asked 2024-12-18T19:09:28.6366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 70%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rahul Goel
0
Reputation points
commented 2024-12-20T16:06:45.87+00:00
Ganesh Gurram
2,305
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
not being able to create azure free account or even to upgrade to pay as u go
how am i suppose to learn the certification if am not being able to create account. i am really frustrated that am not being able to create account. provide me an efficient way to create account to practice for my certification
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 81%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 19%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
How to get utilization(cpu,memory) report of Adb cluster
I have three clusters in adb i need its utilization report how can i get that
asked 2024-12-19T16:01:18.6566667+00:00
Aetemaad Hussain
20
Reputation points
edited a comment 2024-12-19T17:33:17.9+00:00
Ganesh Gurram
2,305
Reputation points Microsoft Vendor
1 answer
How to configure default Lakehouse in notebook in Fabric
I want to configure defualt laekhouse in notebook. i can do it using %%configure command. but i am note able to use variables here to be used in %%configure command. As configure should be the first cell of the notebook and if you put it in middle of the…
asked 2024-12-16T06:53:15.4766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 15%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKB%3C/text%3E%3C/svg%3E)
Ketan Bharat | MAQ Software
60
Reputation points
commented 2024-12-18T02:01:09.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
5,755
Reputation points Microsoft Vendor
1 answer
Concurrency issue in partitioned table while updating same table from different spark notebook
Created partitioned table with below schema CREATE TABLE Landing.TargetTable ( Id INT, Name STRING, SourceSystem INT, SourceId INT, ETLId INT ) USING DELTA PARTITIONED BY (SourceId); Two source tables are there from where data are…
asked 2024-12-10T11:23:16.31+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 52%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHD%3C/text%3E%3C/svg%3E)
heta desai
357
Reputation points
commented 2024-12-17T01:16:02.7466667+00:00
Chandra Boorla
5,755
Reputation points Microsoft Vendor
1 answer
Can a Databricks workspace use an access connector to retrieve secrets from Key Vault?
faced error when trying to access secrets by adding the access connector object ID as key Vault Administrator on Key Vault.
asked 2024-12-10T15:32:28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 17%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMD%3C/text%3E%3C/svg%3E)
Mitasha Dutta (LTIMINDTREE LIMITED)
0
Reputation points Microsoft Vendor
commented 2024-12-16T13:32:53.8866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 89%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHE%3C/text%3E%3C/svg%3E)
Harshitha Eligeti
1,210
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Issue with logging MLflow run in Databricks - BAD_REQUEST: Unable to connect to the linked AzureML workspace
I am encountering an issue while trying to log MLflow runs to an experiment and register a model in Azure Databricks. While executing the mlflow.start_run(), I continually receive a RestException: BAD_REQUEST error with the message "Unable to…
asked 2023-11-07T08:20:43.3633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 84%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Joe
25
Reputation points
answered 2024-12-16T09:00:56.7833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 32%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENM%3C/text%3E%3C/svg%3E)
Narsingh Maurya
0
Reputation points
1 answer
How to configure IP access list to databricks workspace
How to configure the ip access list to databricks workspace. I have seen the microsoft link for this : https://zcusa.951200.xyz/en-us/azure/databricks/security/network/front-end/ip-access-list-workspace but not found (databricks workspace-conf…
asked 2024-12-11T15:51:42.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 96%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
rkreddy
0
Reputation points
commented 2024-12-16T01:54:13.32+00:00
Chandra Boorla
5,755
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Analysis of Cost Saved
Hi, Current Solution: Delta table from HiveMetastore -> Staging Table with Business Logic in Databricks -> PowerBI Current Solution in Detailed: Main responsibilities for me is to consume the data the delta tables from hive_metastore in…
asked 2024-12-11T05:47:30.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 43%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJA%3C/text%3E%3C/svg%3E)
Jayaswaroop Adabala
20
Reputation points
accepted 2024-12-13T06:50:15.1566667+00:00
Jayaswaroop Adabala
20
Reputation points