Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,714 questions
2 answers
trigger issue
Hi Getting below error message in main branch after push the code unable to trigger the pipeline
asked 2024-10-12T01:10:24.9133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
910
Reputation points
answered 2024-10-14T14:03:45.7766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
16,005
Reputation points Microsoft Vendor
1 answer
Resultsets returned by script activity after switching to new SNF connector for ADF are not backward compatible - breaking pipelines
Hi MS support team, I am trying to upgrade Snowflake connector to V2 in my ADF pipelines. I have script activity which has multiple SQL commands (eg. set query_tag, call a stored procedure). Although these get executed successfully now, my subsequent IF…
asked 2024-09-24T11:31:00.4666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 78%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMW%3C/text%3E%3C/svg%3E)
Manjari Walia
0
Reputation points
answered 2024-10-14T12:40:20.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 33%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EWT%3C/text%3E%3C/svg%3E)
Willi, Tobias
0
Reputation points
1 answer
Could not find a path of file, In ADF SHIR VM error
I have encountered this problem on alternate days. One day it runs fine, but the next day it shows a message saying the file path is not available. How can I troubleshoot this? Please help us.
asked 2024-10-13T17:35:49.8433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 55.00000000000001%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Balwant Singh Chouhan
50
Reputation points
commented 2024-10-14T11:28:40.8666667+00:00
Balwant Singh Chouhan
50
Reputation points
1 answer
How to scale up limit of the length of execution output(around 4MB currently)
How to scale up limit of the length of execution output(around 4MB currently) ? Error code : 2001 HTTP GET Activity
asked 2024-10-14T08:36:18.3433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 53%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZ%3C/text%3E%3C/svg%3E)
zmsoft
0
Reputation points
answered 2024-10-14T11:23:45.8833333+00:00
Amira Bedhiafi
24,711
Reputation points
2 answers
Backup up branch
Hi, How to backup your branch in adf pipeline.. Like download and store it somewhere
asked 2024-10-14T03:32:13.31+00:00
Vineet S
910
Reputation points
answered 2024-10-14T10:36:44.96+00:00
Pinaki Ghatak
4,295
Reputation points Microsoft Employee
1 answer
Adding headers to a txt file in ADF
Hi Team, I am having a .txt file where I have data without headers, which is separated by delimiter. how do I add headers to txt file using ADF Dataflow? Suggestions Please!
asked 2024-10-14T07:11:26.99+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 73%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Anandhakumar Cholendran
45
Reputation points
answered 2024-10-14T09:48:25.1833333+00:00
PRADEEPCHEEKATLA-MSFT
89,646
Reputation points Microsoft Employee
1 answer
Parameter passing differences between Web activity and Copy activity with REST dataset. Web activity success but Copy Activity error 401
I have a project that requires data to be imported into Azure SQL DB using Azure Data Factory from Sage 300 REST API. The REST API requires a Cookie as part of the Header of the POST method. Data is successfully returned from SAGE using the Web activity.…
asked 2024-10-12T08:51:52.1033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 75%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGV%3C/text%3E%3C/svg%3E)
Gauchet van Antwerpen
0
Reputation points
edited the question 2024-10-14T09:39:19.1066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 60%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMK%3C/text%3E%3C/svg%3E)
Mahesh Kurva
245
Reputation points Microsoft Vendor
1 answer
Azure Data Factory v2. PowerQuery integration issue for Azure data lake Storage Gen2
In Azure Data Factory v2, I’m working on creating a Power Query (ADF) using Azure Gen2 to access the dataset. I can successfully preview the data within the dataset. However, I’m encountering an error in Power Query that states, "We could not…
asked 2024-10-13T13:16:48.1466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 33%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGS%3C/text%3E%3C/svg%3E)
Gilbert Staline
0
Reputation points
answered 2024-10-14T08:44:01.29+00:00
PRADEEPCHEEKATLA-MSFT
89,646
Reputation points Microsoft Employee
1 answer
Preview data of Azure Database for PostgreSQL
I have created an Azure Database for PostgreSQL in Azure Portal. I created a database, a schema and a table. I also created a linked service and a dataset in Azure Datafactory. If I go on this dataset, enter schema name and table name, and then I click…
asked 2024-10-11T18:35:13.4033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 44%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
Dirk Sachse
106
Reputation points
edited a comment 2024-10-14T08:38:41.58+00:00
Dirk Sachse
106
Reputation points
1 answer
Pull request
How to add 2 approver in pull request.. Please share screenshot
asked 2024-10-14T04:06:55.1+00:00
Vineet S
910
Reputation points
answered 2024-10-14T08:33:35.5066667+00:00
PRADEEPCHEEKATLA-MSFT
89,646
Reputation points Microsoft Employee
6 answers
SAP CDC and SAP Tables connectors regarding SAP Note 3255746 ?
Dear Team, Is there any one who when through the SAP Note 3255746 ? What impact shroud we expect regarding the SAP CDC and SAP Tables connectors already implemented for our customers? Thanks for you input. Tarik
asked 2024-04-23T11:45:42.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 52%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETZ%3C/text%3E%3C/svg%3E)
Tarik Zaaboul
5
Reputation points
commented 2024-10-14T08:31:31.17+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 19%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Christopher Brunner
0
Reputation points
1 answer
ADF REST API Auth header parameters not showing up in other environments which are not connected to Git
We have two ADF instances - dev and production. Our dev environment is connected to Git repository, however when we move code to production, we only move ARM template files. We are observing that auth header parameters in REST API are not showing up in…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 68%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
0 answers
Adf Linked Service Salesforce Marketing Cloud Linked Services
I am using the Salesforce Marketing Cloud linked services. I have tried using the Authentication Type OAuth 2.0 and Enhanced STS OAuth 2.0, but it returns a 401 response. The client ID, secret, account ID (with or without), and domain are the same ones…
asked 2024-10-03T21:08:53.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 17%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECL%3C/text%3E%3C/svg%3E)
Cristiano Lopes
0
Reputation points
commented 2024-10-14T05:53:21.0633333+00:00
PRADEEPCHEEKATLA-MSFT
89,646
Reputation points Microsoft Employee
2 answers
How can I trigger Fabric Notebook or Fabric data pipeline from Azure Data Factory?
I am trying to trigger fabric notebook via ADF via API calls but it is throwing an error. Details: https://zcusa.951200.xyz/en-us/fabric/data-engineering/notebook-public-api#run-a-notebook-on-demand Using above article's API Error:…
asked 2024-10-11T09:43:46.26+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 44%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Sunay Pistulkar
5
Reputation points
commented 2024-10-14T04:33:44.86+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
10,555
Reputation points Microsoft Vendor
1 answer
the specified network name is no longer available. Facing Error in ADF pipeline
I am having this error in my ADF pipeline, My SHIR VM showing the logs that data factory endpoint is not getting resolved. Please look into this and i want to troubleshoot this i have confused if everything looks and run fine, how can i get this error…
asked 2024-10-11T17:23:37.6866667+00:00
Balwant Singh Chouhan
50
Reputation points
commented 2024-10-14T04:18:02.0433333+00:00
phemanth
10,555
Reputation points Microsoft Vendor
1 answer
Upgrade to new Snowflake Linked Service, buggy with Source=mscorlib
Hi, we recently upgraded our snowflake linked service to the newer version and once we released it to our production environment half of our activities started throwing Failure happened on 'Source' side. 'Type=System.Exception,Message=[Snowflake]…
asked 2024-09-26T18:34:03.21+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 20%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMF%3C/text%3E%3C/svg%3E)
Mike Fatal
0
Reputation points
commented 2024-10-14T04:09:12.47+00:00
phemanth
10,555
Reputation points Microsoft Vendor
0 answers
Failing to copy large dataset from Salesforce using new salesforce V2 connector in Azure data factory
I am trying to migrate the salesforce legacy linked service to new salesforce V2 linked service in Azure data factory. The test connection is successful and I am able to copy the smaller datasets using new salesforce V2 linked service. But when i am…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 2%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EL%3C/text%3E%3C/svg%3E)
1 answer
Time Zone Parameter Ignored in Azure Data Factory PostgreSQL Connector After Migration
I'm experiencing an issue with Azure Data Factory (ADF) when using the PostgreSQL connector. Problem: The "Time Zone" parameter in the Linked Service appears to be ignored, and timestamps are always retrieved in UTC, despite attempts to set…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 55.00000000000001%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EI%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 43%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Azure datalake and data consistency
Hi Experts, Azure Data Lake Storage (ADLS) does not natively provide full ACID (Atomicity, Consistency, Isolation, Durability) transaction support unlike traditional relational databases designed to support ACID transactions. This raises the following…
asked 2024-10-12T09:44:01.3866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
280
Reputation points
accepted 2024-10-13T11:24:21.8266667+00:00
azure_learner
280
Reputation points
1 answer
Automating execution of ".bat" file using ADF / LogicApp
Hello Team, Currently we are manually executing a .bat file in one of the client VMs and we need to automate the same using ADF / Logic App. The bat file is inside a folder and it executes a .jar file which resides within the same folder. The .jar file…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 53%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENC%3C/text%3E%3C/svg%3E)
