Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,513 questions
0 answers
I am getting the below error when I try to insert into a azure data lake table. "Type=System.IO.FileNotFoundException,Message=Could not load file or assembly 'System.Text.Json"
I am getting the below error when I try to insert into a azure data lake table from adf pipeline . "The type initializer for 'Azure.Core.Serialization.JsonObjectSerializer' threw an…
asked 2024-12-27T08:56:47.04+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 87%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBT%3C/text%3E%3C/svg%3E)
Beebi Thasleem M
20
Reputation points
commented 2024-12-27T17:06:25.06+00:00
AnnuKumari-MSFT
33,976
Reputation points Microsoft Employee
0 answers
How to Automate Large .SAV File to Parquet Conversion in Azure?
I'm using Azure ADLS as our primary storage, Azure Data Factory (ADF) for data transformations, and Power BI for reporting and visualization. I have a large .SAV file (200-300 MB, containing 2-4 million rows) stored in Azure Data Lake Storage (ADLS). To…
asked 2024-12-27T10:31:15.69+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 16%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Akshay Patel
70
Reputation points
commented 2024-12-27T16:48:28.0333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 2%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHB%3C/text%3E%3C/svg%3E)
Hari Babu Vattepally
1,195
Reputation points Microsoft Vendor
2 answers
Getting TokenNotFoundInConfigurationException error when uploading a Python wheel as workspace package in Synpase
Hello Everyone I'm using Azure Synapse to run my Spark pipelines. I've build my pipeline to use a custom library code from utils package of the same project to perform some operation. To access the utils.pipeline file, have packaged my entire project in…
asked 2024-12-25T05:25:33.01+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 22%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
manigandan
0
Reputation points
commented 2024-12-27T08:52:58.9433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
18,790
Reputation points Microsoft Vendor
1 answer
issues in synapse pipelines which accessing the checkpoints key with input parameters
hello team, i am getting synapse analytics pipelines failures with error referring the checkpoint mistaching with the checkpoint in input parameters. tried to check the input as the respective dataflow is fetching data from cosmosDb with CDC approach…
asked 2024-12-20T14:24:05.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 98%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
ANKIT GHENGE
40
Reputation points
edited a comment 2024-12-27T07:55:57.2866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
2,305
Reputation points Microsoft Vendor
3 answers
Troubleshooting Power BI and Azure Storage Explorer Connection Issues to Data Lake Gen2 from Azure VM
I rented a virtual machine (VM) and had no issues using it until I needed to integrate it with the Data Lake to optimize my processes. I subscribed to the Data Lake, but I cannot connect to it through Power BI when using the VM. My account is a Gmail…
asked 2024-12-10T11:54:22.8833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 73%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERC%3C/text%3E%3C/svg%3E)
Rafael CST
80
Reputation points
commented 2024-12-26T10:21:02.1166667+00:00
Hari Babu Vattepally
1,195
Reputation points Microsoft Vendor
1 answer
Getting "storage_root does not specify a URI scheme." while setting ADLS path in default metastore.
Hi Team, I am facing while setting the ADLS path for the existing default metastore in account console page in azure. Below is the image showing i have already created the container and have enabled Hierarchical namespace as well. But getting below…
asked 2024-11-18T11:01:44.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 53%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Akancha
0
Reputation points
commented 2024-12-23T07:34:12.8333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 6%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Keshavulu Dasari
2,570
Reputation points Microsoft Vendor
1 answer
When generating a SAS token, specifying the path causes an authorization failure.
I use following code to generate SAS token, when path is null or empty string, it's working. String sasToken = new DataLakeSasImplUtil(signatureValues, container, path, true) .generateUserDelegationSas(userDelegationKey, accountName,…
asked 2024-12-12T14:52:46.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 73%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Jun
0
Reputation points
commented 2024-12-23T07:27:42.72+00:00
Keshavulu Dasari
2,570
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
ADLS Gen2 operation failed for: Operation returned an invalid status code 'Conflict'. Account: 'abc'. ErrorCode: 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete.
I am new to ADF. while creating linked service for data lake storage Gen2 I am getting a error ADLS Gen2 operation failed. ADLS Gen2 operation failed for: Operation returned an invalid status code 'Conflict'. Account: 'abc'. ErrorCode:…
asked 2024-12-21T17:12:26.5366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 6%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
manoj reddy
20
Reputation points
commented 2024-12-22T14:50:01.7933333+00:00
manoj reddy
20
Reputation points
2 answers
One of the answers was accepted by the question author.
How to monitor data lake sas token expired?
Hi there, I have store sas token value in key vault, but I can't know when token expires. So I will find this issue after the token expires. Is there any way to monitor when the token expires? Thanks zmsoft
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 53%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZ%3C/text%3E%3C/svg%3E)
1 answer
How cn i restrict someone from not downloading any data from Fabric Lakehouse or Fabric warehouse
As a data admin, i want to control data access for a user in your Microsoft Fabric Warehouse. The goal is to allow this user, who has a Contributor role, to view data directly in the workspace without being able to download it as a file. This scenario…
asked 2024-11-05T05:14:23.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 11%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Jain, Shreyans
0
Reputation points
edited the question 2024-12-18T06:34:30.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 17%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
RNareddy
1,430
Reputation points Microsoft Vendor
1 answer
In MS fabric users able to view data but cannot download it
In MS Fabric, I want users to be able to view data in the workspace but not be able to download it. Please provide clear steps, along with links to verify the authenticity of the solution provided.
asked 2024-11-11T11:51:04.92+00:00
Jain, Shreyans
0
Reputation points
edited the question 2024-12-18T06:33:29.7366667+00:00
RNareddy
1,430
Reputation points Microsoft Vendor
2 answers
Append vs update in Azure Data Lake Storage Gen2 for a csv file.
I have a CSV file in a Data Lake Storage Gen2. I am referring to the ACL permissions below.https://zcusa.951200.xyz/en-us/azure/storage/blobs/data-lake-storage-access-control#common-scenarios-related-to-acl-permissionsThe table gives different ACL…
asked 2024-12-10T03:23:07.9433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 70%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Ravikiran Srinivasulu
10
Reputation points
commented 2024-12-16T03:52:40.77+00:00
Keshavulu Dasari
2,570
Reputation points Microsoft Vendor
1 answer
Trying to connect to the Fabric Lakehouse from the SSIS Package. However it is failing when executing the package
I have Fabric Lakehouse as a Source , I am able to connect to the Fabric Lakehouse via Authentication as "ActiveDirectoryPassword" from SSMS without any issue. I am creating the SSIS package and created the ADO.NET connection Manager and…
asked 2024-12-12T22:39:50.1766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 30%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVT%3C/text%3E%3C/svg%3E)
Vineeta Trivedi
0
Reputation points
commented 2024-12-16T01:42:10.08+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
5,835
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Azure Data Lake Storage NFSv3 Protocol support for nconnect for Redhat
Hi, I have a ADLS Storage Account configured for HNS and NFSv3 protocol support. When attempting to mount the ADLS container on a Azure RedHat 9.5 NFS Client instance with the nconnect mount option set it fails with. $ sudo mount /mntfs/adls01c01…
asked 2024-12-05T00:32:40.59+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 75%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMG%3C/text%3E%3C/svg%3E)
Malcolm Gibbs
20
Reputation points
commented 2024-12-12T17:36:06.5866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 86%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amrinder Singh
5,555
Reputation points Microsoft Employee
1 answer
Possible ways to extract data from SAP Reports (Transaction Data) to ADLS Gen2
Hello, I want to extract SAP ECC transaction data into ADLS Gen2. I have multiple transaction codes (T-codes) for generating reports, such as ME5A and ME2N, and I want to fetch the data from these reports into ADLS. However, it seems that Azure Data…
asked 2024-12-12T06:08:53.2566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 31%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Sulagno Roy
40
Reputation points
answered 2024-12-12T11:50:55.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
8,166
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
How to read the last ingested date partitioned file in ADLS
Hi All, How to read the last ingested file in ADLS. ADLS folder structure: XXX/year=yyyy/month=MM/day=dd Now in the day=dd I will have two different files on a given day. I want to read the last ingested file based on the timestamp as a source in my…
asked 2023-05-26T05:37:49.1633333+00:00
Sulagno Roy
40
Reputation points
accepted 2024-12-12T05:46:22.51+00:00
Sulagno Roy
40
Reputation points
1 answer
One of the answers was accepted by the question author.
How to read multiple files from different folders in azure data flow?
I have multiple folders from different source systems in my landing area of ADLS GEN2. Let's say I have folders named as X,Y,Z. Inside X, I again have subfolders based on the tables from where the data has been extracted. For example X has 3 subfolders…
asked 2023-05-26T05:30:51.0533333+00:00
Sulagno Roy
40
Reputation points
accepted 2024-12-12T05:45:18.67+00:00
Sulagno Roy
40
Reputation points
2 answers
One of the answers was accepted by the question author.
Data lake name sequence
Hi friends, when creating Azure Data Lake the containers are sorted by alphabetical order, it looks a bit incorrect order : bronze, landing zone gold silver I want it in the landing, bronze, silver, and gold orders. How can I do that? Is there a…
asked 2024-12-09T06:29:24.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
525
Reputation points
accepted 2024-12-11T08:44:10.44+00:00
azure_learner
525
Reputation points
1 answer
how to fix this error The request content was invalid and could not be deserialized: Required property 'name' not found in 'sku'
getting error while synapse workspace deployment
asked 2024-12-09T14:46:47.8+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 53%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Narendra Singh
5
Reputation points
commented 2024-12-11T08:04:11.14+00:00
Chandra Boorla
5,835
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
ADF pipeline to read the data from UC table to adls gen2 account
Hello Team, We have a requirement to create Azure Datafactory pipeline to read the data from UC table, access on the table is provided ( to Azure Datafactory Managed Identity) and copy the data into adls gen2. Is there a way or article to implement this?…
asked 2024-04-11T19:05:05.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 77%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
Ashwini Gaikwad
130
Reputation points
answered 2024-12-06T05:09:57.8666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 21%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
AMIT KUMAR PANDA
0
Reputation points