Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,726 questions
0 answers
automate devops
How to automate easily ci cd in devops in ADF .. please share screenshot
0 answers
Main Repo in ADF
How to set up main repo between Dev and Accptance envionment in ADF. Genrally it should be same repo or diffrerent ...please share screenshot any
2 answers
One of the answers was accepted by the question author.
trigger issue
Hi Getting below error message in main branch after push the code unable to trigger the pipeline
1 answer
One of the answers was accepted by the question author.
Data Flow Activities delayed in queued status after billing issue
I'm experiencing an issue with my ADF pipeline, where the Dataflow activity is stuck in a queued state for about 5 minutes before acquiring computing resources and starting. This has increased the total runtime from 2-3 minutes to 10-15 minutes, even…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 64%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EW%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
Extract status of azure function activity in adf
How to extract status of azure function activity in adf using dynamic expression? @activity('AzureFunction').output.executionDetails[0].status is not working as it is not able to find executionDetails in its output.
asked 2024-10-16T07:39:36.0566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 39%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Aniesha Razdan
20
Reputation points
accepted 2024-10-16T15:09:44.4066667+00:00
Aniesha Razdan
20
Reputation points
1 answer
ServiceNow Incident table extraction with V2 Connectors in ADF
Using the new ServiceNowV2 connectors in Azure Data Factory, we are trying to extract the incident table. When setting up a copy activity, we receive the error message below. I have tried to limit the number of records by providing a filter to small…
asked 2024-10-10T13:01:02.9233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 61%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERM%3C/text%3E%3C/svg%3E)
Ryan McCrady
5
Reputation points
commented 2024-10-16T15:04:55.0166667+00:00
Ryan McCrady
5
Reputation points
1 answer
Adding headers to a txt file in ADF
Hi Team, I am having a .txt file where I have data without headers, which is separated by delimiter. how do I add headers to txt file using ADF Dataflow? Suggestions Please!
asked 2024-10-14T07:11:26.99+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 73%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Anandhakumar Cholendran
45
Reputation points
commented 2024-10-16T12:17:15.4533333+00:00
PRADEEPCHEEKATLA-MSFT
89,731
Reputation points Microsoft Employee
1 answer
Pull request
How to add 2 approver in pull request.. Please share screenshot
asked 2024-10-14T04:06:55.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
930
Reputation points
commented 2024-10-16T12:02:10.94+00:00
PRADEEPCHEEKATLA-MSFT
89,731
Reputation points Microsoft Employee
2 answers
Backup up branch
Hi, How to backup your branch in adf pipeline.. Like download and store it somewhere
asked 2024-10-14T03:32:13.31+00:00
Vineet S
930
Reputation points
commented 2024-10-16T11:56:18.82+00:00
PRADEEPCHEEKATLA-MSFT
89,731
Reputation points Microsoft Employee
1 answer
Azure Data Factory v2. PowerQuery integration issue for Azure data lake Storage Gen2
In Azure Data Factory v2, I’m working on creating a Power Query (ADF) using Azure Gen2 to access the dataset. I can successfully preview the data within the dataset. However, I’m encountering an error in Power Query that states, "We could not…
asked 2024-10-13T13:16:48.1466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 33%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGS%3C/text%3E%3C/svg%3E)
Gilbert Staline
0
Reputation points
commented 2024-10-16T11:55:28.81+00:00
PRADEEPCHEEKATLA-MSFT
89,731
Reputation points Microsoft Employee
1 answer
Time Zone Parameter Ignored in Azure Data Factory PostgreSQL Connector After Migration
I'm experiencing an issue with Azure Data Factory (ADF) when using the PostgreSQL connector. Problem: The "Time Zone" parameter in the Linked Service appears to be ignored, and timestamps are always retrieved in UTC, despite attempts to set…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 55.00000000000001%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EI%3C/text%3E%3C/svg%3E)
2 answers
Data factory storage event trigger support rename event
Is it possible to add storage event trigger to support blob rename? Currently it only support create and delete activity. But we have another platform called SAP DataSphere, it's using BlobRenamed event type and RenameFile api to push data into storage…
asked 2024-10-16T05:23:00.34+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 8%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJZ%3C/text%3E%3C/svg%3E)
Joe Zeng
6
Reputation points
answered 2024-10-16T10:36:06.88+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 4%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sina Salam
10,811
Reputation points
0 answers
Configuration for Transferring JSON from Azure Cosmos DB to Snowflake Using Azure Data Factory
We need to transfer an entire JSON object from Azure Cosmos DB to a Snowflake table. We have created a table in Snowflake with a single column named emp_details of type VARIANT to store the JSON data. How can we configure the mapping in Azure Data…
asked 2024-10-15T09:40:05.8+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 91%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sowndar Kumaresan
0
Reputation points
edited a comment 2024-10-16T06:06:09.9633333+00:00
Sowndar Kumaresan
0
Reputation points
1 answer
How to fix ADF error 2200 Failure happened on 'Sink' side.
I have 22 pipelines running in parallel, each copying data from 22 Salesforce objects into Parquet files in Azure. The total data size is approximately 5GB. All copy activities have their maximum concurrent connections set to 1 on the sink side. While…
asked 2024-10-01T00:32:29.7566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 95%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFB%3C/text%3E%3C/svg%3E)
FERNANDO BASAEZ
20
Reputation points
commented 2024-10-16T05:38:18.95+00:00
PRADEEPCHEEKATLA-MSFT
89,731
Reputation points Microsoft Employee
0 answers
Microsoft Integration runtime (Self hosted) node has encountered an error during registration
I am looking for support for the Microsoft Integration runtime (Self hosted) node to function. It's failing to register with the valid key. I checked and found that the Integration runtime (Self hosted) host service is running under services. Not sure…
asked 2024-10-15T09:56:38.2633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 75%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
RATHOR, Abhishek
0
Reputation points
commented 2024-10-16T03:24:13.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
710
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Unable to use a variable with a SOQL query in ADF
We are trying to switch our ADF pipeline from using the old SFDC dataset which used Query to the new dataset which uses SOQL. In the process, we found out that it does not support field aliasing which had made our life easier. But now, to go around that…
asked 2024-10-15T04:03:40.6933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 28.999999999999996%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nimesh Raj Manandhar
21
Reputation points
accepted 2024-10-16T02:59:06.6533333+00:00
Nimesh Raj Manandhar
21
Reputation points
1 answer
Could not find a path of file, In ADF SHIR VM error
I have encountered this problem on alternate days. One day it runs fine, but the next day it shows a message saying the file path is not available. How can I troubleshoot this? Please help us.
asked 2024-10-13T17:35:49.8433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 55.00000000000001%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Balwant Singh Chouhan
50
Reputation points
commented 2024-10-15T21:11:56.2633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
2,125
Reputation points Microsoft Vendor
0 answers
How can I add keyExchangeAlgorithm values in Azure Data Factory SFTP linked services?
How to set key exchange algorithms in ADF SFTP linked service
asked 2024-10-15T08:41:21.6133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 13%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHK%3C/text%3E%3C/svg%3E)
Hampannavar, Karthik SBOBNG-PTIY/FBD
0
Reputation points
commented 2024-10-15T20:00:34.62+00:00
Chandra Boorla
2,125
Reputation points Microsoft Vendor
1 answer
partitioning of Data lake
I have just the below link on data partitioning of Azure data lake : https://zcusa.951200.xyz/en-us/azure/architecture/best-practices/data-partitioning But it is too brief and does not cover this important topic extensively. I request you to please…
asked 2024-10-08T10:09:02.7433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
280
Reputation points
commented 2024-10-15T18:56:04.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 6%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Keshavulu Dasari
765
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
import csv text file with fixed length columns
I am trying to import into a sql table a fixed column length csv file and contains different types of records and I only need to import some of them Is it posible using datafactory? thanks in advanced
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 18%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJZ%3C/text%3E%3C/svg%3E)