Pipeline Runs - Query By Factory

Reference

Service:: Data Factory

API Version:: 2018-06-01

Query pipeline runs in the factory based on input filter conditions.

POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/queryPipelineRuns?api-version=2018-06-01

URI Parameters

Name	In	Required	Type	Description
factoryName	path	True	string	The factory name. Regex pattern: `^[A-Za-z0-9]+(?:-[A-Za-z0-9]+)*$`
resourceGroupName	path	True	string	The resource group name. Regex pattern: `^[-\w\._\(\)]+$`
subscriptionId	path	True	string	The subscription identifier.
api-version	query	True	string	The API version.

Request Body

Name	Required	Type	Description
lastUpdatedAfter	True	string	The time at or after which the run event was updated in 'ISO 8601' format.
lastUpdatedBefore	True	string	The time at or before which the run event was updated in 'ISO 8601' format.
continuationToken		string	The continuation token for getting the next page of results. Null for first page.
filters		RunQueryFilter[]	List of filters.
orderBy		RunQueryOrderBy[]	List of OrderBy option.

Responses

Name	Type	Description
200 OK	PipelineRunsQueryResponse	OK.
Other Status Codes	CloudError	An error response received from the Azure Data Factory service.

Security

azure_auth

Azure Active Directory OAuth2 Flow.

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize

Scopes

Name	Description
user_impersonation	impersonate your user account

Examples

PipelineRuns_QueryByFactory

Sample request

POST https://management.azure.com/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/queryPipelineRuns?api-version=2018-06-01

{
  "lastUpdatedAfter": "2018-06-16T00:36:44.3345758Z",
  "lastUpdatedBefore": "2018-06-16T00:49:48.3686473Z",
  "filters": [
    {
      "operand": "PipelineName",
      "operator": "Equals",
      "values": [
        "examplePipeline"
      ]
    }
  ]
}


import com.azure.resourcemanager.datafactory.models.RunFilterParameters;
import com.azure.resourcemanager.datafactory.models.RunQueryFilter;
import com.azure.resourcemanager.datafactory.models.RunQueryFilterOperand;
import com.azure.resourcemanager.datafactory.models.RunQueryFilterOperator;
import java.time.OffsetDateTime;
import java.util.Arrays;

/**
 * Samples for PipelineRuns QueryByFactory.
 */
public final class Main {
    /*
     * x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/
     * PipelineRuns_QueryByFactory.json
     */
    /**
     * Sample code: PipelineRuns_QueryByFactory.
     * 
     * @param manager Entry point to DataFactoryManager.
     */
    public static void pipelineRunsQueryByFactory(com.azure.resourcemanager.datafactory.DataFactoryManager manager) {
        manager.pipelineRuns().queryByFactoryWithResponse("exampleResourceGroup", "exampleFactoryName",
            new RunFilterParameters().withLastUpdatedAfter(OffsetDateTime.parse("2018-06-16T00:36:44.3345758Z"))
                .withLastUpdatedBefore(OffsetDateTime.parse("2018-06-16T00:49:48.3686473Z"))
                .withFilters(Arrays.asList(new RunQueryFilter().withOperand(RunQueryFilterOperand.PIPELINE_NAME)
                    .withOperator(RunQueryFilterOperator.EQUALS).withValues(Arrays.asList("examplePipeline")))),
            com.azure.core.util.Context.NONE);
    }
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

from typing import Any, IO, Union

from azure.identity import DefaultAzureCredential

from azure.mgmt.datafactory import DataFactoryManagementClient

"""
# PREREQUISITES
    pip install azure-identity
    pip install azure-mgmt-datafactory
# USAGE
    python pipeline_runs_query_by_factory.py

    Before run the sample, please set the values of the client ID, tenant ID and client secret
    of the AAD application as environment variables: AZURE_CLIENT_ID, AZURE_TENANT_ID,
    AZURE_CLIENT_SECRET. For more info about how to get the value, please see:
    https://docs.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal
"""


def main():
    client = DataFactoryManagementClient(
        credential=DefaultAzureCredential(),
        subscription_id="12345678-1234-1234-1234-12345678abc",
    )

    response = client.pipeline_runs.query_by_factory(
        resource_group_name="exampleResourceGroup",
        factory_name="exampleFactoryName",
        filter_parameters={
            "filters": [{"operand": "PipelineName", "operator": "Equals", "values": ["examplePipeline"]}],
            "lastUpdatedAfter": "2018-06-16T00:36:44.3345758Z",
            "lastUpdatedBefore": "2018-06-16T00:49:48.3686473Z",
        },
    )
    print(response)


# x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/PipelineRuns_QueryByFactory.json
if __name__ == "__main__":
    main()

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

package armdatafactory_test

import (
	"context"
	"log"

	"time"

	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/datafactory/armdatafactory/v9"
)

// Generated from example definition: https://github.com/Azure/azure-rest-api-specs/blob/edb7904bfead536c7aa9716d44dba15bdabd0b00/specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/PipelineRuns_QueryByFactory.json
func ExamplePipelineRunsClient_QueryByFactory() {
	cred, err := azidentity.NewDefaultAzureCredential(nil)
	if err != nil {
		log.Fatalf("failed to obtain a credential: %v", err)
	}
	ctx := context.Background()
	clientFactory, err := armdatafactory.NewClientFactory("<subscription-id>", cred, nil)
	if err != nil {
		log.Fatalf("failed to create client: %v", err)
	}
	res, err := clientFactory.NewPipelineRunsClient().QueryByFactory(ctx, "exampleResourceGroup", "exampleFactoryName", armdatafactory.RunFilterParameters{
		Filters: []*armdatafactory.RunQueryFilter{
			{
				Operand:  to.Ptr(armdatafactory.RunQueryFilterOperandPipelineName),
				Operator: to.Ptr(armdatafactory.RunQueryFilterOperatorEquals),
				Values: []*string{
					to.Ptr("examplePipeline")},
			}},
		LastUpdatedAfter:  to.Ptr(func() time.Time { t, _ := time.Parse(time.RFC3339Nano, "2018-06-16T00:36:44.334Z"); return t }()),
		LastUpdatedBefore: to.Ptr(func() time.Time { t, _ := time.Parse(time.RFC3339Nano, "2018-06-16T00:49:48.368Z"); return t }()),
	}, nil)
	if err != nil {
		log.Fatalf("failed to finish the request: %v", err)
	}
	// You could use response here. We use blank identifier for just demo purposes.
	_ = res
	// If the HTTP response code is 200 as defined in example definition, your response structure would look as follows. Please pay attention that all the values in the output are fake values for just demo purposes.
	// res.PipelineRunsQueryResponse = armdatafactory.PipelineRunsQueryResponse{
	// 	Value: []*armdatafactory.PipelineRun{
	// 		{
	// 			AdditionalProperties: map[string]any{
	// 				"annotations": []any{
	// 				},
	// 				"runDimension": map[string]any{
	// 					"JobId": "79c1cc52-265f-41a5-9553-be65e736fbd3",
	// 				},
	// 			},
	// 			DurationInMs: to.Ptr[int32](28105),
	// 			InvokedBy: &armdatafactory.PipelineRunInvokedBy{
	// 				Name: to.Ptr("Manual"),
	// 				ID: to.Ptr("80a01654a9d34ad18b3fcac5d5d76b67"),
	// 			},
	// 			LastUpdated: to.Ptr(func() time.Time { t, _ := time.Parse(time.RFC3339Nano, "2018-06-16T00:38:12.731Z"); return t}()),
	// 			Message: to.Ptr(""),
	// 			Parameters: map[string]*string{
	// 				"OutputBlobNameList": to.Ptr("[\"exampleoutput.csv\"]"),
	// 			},
	// 			PipelineName: to.Ptr("examplePipeline"),
	// 			RunEnd: to.Ptr(func() time.Time { t, _ := time.Parse(time.RFC3339Nano, "2018-06-16T00:38:12.731Z"); return t}()),
	// 			RunID: to.Ptr("2f7fdb90-5df1-4b8e-ac2f-064cfa58202b"),
	// 			RunStart: to.Ptr(func() time.Time { t, _ := time.Parse(time.RFC3339Nano, "2018-06-16T00:37:44.625Z"); return t}()),
	// 			Status: to.Ptr("Succeeded"),
	// 		},
	// 		{
	// 			AdditionalProperties: map[string]any{
	// 				"annotations": []any{
	// 				},
	// 				"runDimension": map[string]any{
	// 					"JobId": "84a3c493-0628-4b44-852f-ef5b3a11bdab",
	// 				},
	// 			},
	// 			InvokedBy: &armdatafactory.PipelineRunInvokedBy{
	// 				Name: to.Ptr("Manual"),
	// 				ID: to.Ptr("7c5fd7ef7e8a464b98b931cf15fcac66"),
	// 			},
	// 			LastUpdated: to.Ptr(func() time.Time { t, _ := time.Parse(time.RFC3339Nano, "2018-06-16T00:39:51.216Z"); return t}()),
	// 			Message: to.Ptr(""),
	// 			Parameters: map[string]*string{
	// 				"OutputBlobNameList": to.Ptr("[\"exampleoutput.csv\"]"),
	// 			},
	// 			PipelineName: to.Ptr("examplePipeline"),
	// 			RunID: to.Ptr("16ac5348-ff82-4f95-a80d-638c1d47b721"),
	// 			RunStart: to.Ptr(func() time.Time { t, _ := time.Parse(time.RFC3339Nano, "2018-06-16T00:39:49.274Z"); return t}()),
	// 			Status: to.Ptr("Cancelled"),
	// 	}},
	// }
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

const { DataFactoryManagementClient } = require("@azure/arm-datafactory");
const { DefaultAzureCredential } = require("@azure/identity");

/**
 * This sample demonstrates how to Query pipeline runs in the factory based on input filter conditions.
 *
 * @summary Query pipeline runs in the factory based on input filter conditions.
 * x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/PipelineRuns_QueryByFactory.json
 */
async function pipelineRunsQueryByFactory() {
  const subscriptionId =
    process.env["DATAFACTORY_SUBSCRIPTION_ID"] || "12345678-1234-1234-1234-12345678abc";
  const resourceGroupName = process.env["DATAFACTORY_RESOURCE_GROUP"] || "exampleResourceGroup";
  const factoryName = "exampleFactoryName";
  const filterParameters = {
    filters: [
      {
        operand: "PipelineName",
        operator: "Equals",
        values: ["examplePipeline"],
      },
    ],
    lastUpdatedAfter: new Date("2018-06-16T00:36:44.3345758Z"),
    lastUpdatedBefore: new Date("2018-06-16T00:49:48.3686473Z"),
  };
  const credential = new DefaultAzureCredential();
  const client = new DataFactoryManagementClient(credential, subscriptionId);
  const result = await client.pipelineRuns.queryByFactory(
    resourceGroupName,
    factoryName,
    filterParameters,
  );
  console.log(result);
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

using Azure;
using Azure.ResourceManager;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Azure.Core;
using Azure.Core.Expressions.DataFactory;
using Azure.Identity;
using Azure.ResourceManager.DataFactory.Models;
using Azure.ResourceManager.Resources;
using Azure.ResourceManager.DataFactory;

// Generated from example definition: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/PipelineRuns_QueryByFactory.json
// this example is just showing the usage of "PipelineRuns_QueryByFactory" operation, for the dependent resources, they will have to be created separately.

// get your azure access token, for more details of how Azure SDK get your access token, please refer to https://learn.microsoft.com/en-us/dotnet/azure/sdk/authentication?tabs=command-line
TokenCredential cred = new DefaultAzureCredential();
// authenticate your client
ArmClient client = new ArmClient(cred);

// this example assumes you already have this DataFactoryResource created on azure
// for more information of creating DataFactoryResource, please refer to the document of DataFactoryResource
string subscriptionId = "12345678-1234-1234-1234-12345678abc";
string resourceGroupName = "exampleResourceGroup";
string factoryName = "exampleFactoryName";
ResourceIdentifier dataFactoryResourceId = DataFactoryResource.CreateResourceIdentifier(subscriptionId, resourceGroupName, factoryName);
DataFactoryResource dataFactory = client.GetDataFactoryResource(dataFactoryResourceId);

// invoke the operation and iterate over the result
RunFilterContent content = new RunFilterContent(DateTimeOffset.Parse("2018-06-16T00:36:44.3345758Z"), DateTimeOffset.Parse("2018-06-16T00:49:48.3686473Z"))
{
    Filters =
    {
    new RunQueryFilter(RunQueryFilterOperand.PipelineName,RunQueryFilterOperator.EqualsValue,new string[]
    {
    "examplePipeline"
    })
    },
};
await foreach (DataFactoryPipelineRunInfo item in dataFactory.GetPipelineRunsAsync(content))
{
    Console.WriteLine($"Succeeded: {item}");
}

Console.WriteLine($"Succeeded");

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

az datafactory pipeline-run query-by-factory --factory-name "exampleFactoryName" --filters operand="PipelineName" operator="Equals" values="examplePipeline" --last-updated-after "2018-06-16T00:36:44.3345758Z" --last-updated-before "2018-06-16T00:49:48.3686473Z" --resource-group "exampleResourceGroup"

Click here to learn more about the Azure CLI command.

Sample response

Status code:: 200

Date: Sat, 16 Jun 2018 00:40:01 GMT
X-Content-Type-Options: nosniff
x-ms-ratelimit-remaining-subscription-reads: 14989
x-ms-request-id: 2f955e10-c6df-45a7-97d4-81acdb8540cd
x-ms-correlation-request-id: 2f955e10-c6df-45a7-97d4-81acdb8540cd

{
  "value": [
    {
      "runId": "2f7fdb90-5df1-4b8e-ac2f-064cfa58202b",
      "pipelineName": "examplePipeline",
      "parameters": {
        "OutputBlobNameList": "[\"exampleoutput.csv\"]"
      },
      "invokedBy": {
        "id": "80a01654a9d34ad18b3fcac5d5d76b67",
        "name": "Manual"
      },
      "runStart": "2018-06-16T00:37:44.6257014Z",
      "runEnd": "2018-06-16T00:38:12.7314495Z",
      "durationInMs": 28105,
      "status": "Succeeded",
      "message": "",
      "lastUpdated": "2018-06-16T00:38:12.7314495Z",
      "annotations": [],
      "runDimension": {
        "JobId": "79c1cc52-265f-41a5-9553-be65e736fbd3"
      }
    },
    {
      "runId": "16ac5348-ff82-4f95-a80d-638c1d47b721",
      "pipelineName": "examplePipeline",
      "parameters": {
        "OutputBlobNameList": "[\"exampleoutput.csv\"]"
      },
      "invokedBy": {
        "id": "7c5fd7ef7e8a464b98b931cf15fcac66",
        "name": "Manual"
      },
      "runStart": "2018-06-16T00:39:49.2745128Z",
      "runEnd": null,
      "durationInMs": null,
      "status": "Cancelled",
      "message": "",
      "lastUpdated": "2018-06-16T00:39:51.216097Z",
      "annotations": [],
      "runDimension": {
        "JobId": "84a3c493-0628-4b44-852f-ef5b3a11bdab"
      }
    }
  ]
}

Definitions

Name	Description
CloudError	The object that defines the structure of an Azure Data Factory error response.
PipelineRun	Information about a pipeline run.
PipelineRunInvokedBy	Provides entity name and id that started the pipeline run.
PipelineRunsQueryResponse	A list pipeline runs.
RunFilterParameters	Query parameters for listing runs.
RunQueryFilter	Query filter option for listing runs.
RunQueryFilterOperand	Parameter name to be used for filter. The allowed operands to query pipeline runs are PipelineName, RunStart, RunEnd and Status; to query activity runs are ActivityName, ActivityRunStart, ActivityRunEnd, ActivityType and Status, and to query trigger runs are TriggerName, TriggerRunTimestamp and Status.
RunQueryFilterOperator	Operator to be used for filter.
RunQueryOrder	Sorting order of the parameter.
RunQueryOrderBy	An object to provide order by options for listing runs.
RunQueryOrderByField	Parameter name to be used for order by. The allowed parameters to order by for pipeline runs are PipelineName, RunStart, RunEnd and Status; for activity runs are ActivityName, ActivityRunStart, ActivityRunEnd and Status; for trigger runs are TriggerName, TriggerRunTimestamp and Status.

CloudError

The object that defines the structure of an Azure Data Factory error response.

Name	Type	Description
error.code	string	Error code.
error.details	CloudError[]	Array with additional error details.
error.message	string	Error message.
error.target	string	Property name/path in request associated with error.

PipelineRun

Information about a pipeline run.

Name	Type	Description
durationInMs	integer	The duration of a pipeline run.
invokedBy	PipelineRunInvokedBy	Entity that started the pipeline run.
isLatest	boolean	Indicates if the recovered pipeline run is the latest in its group.
lastUpdated	string	The last updated timestamp for the pipeline run event in ISO8601 format.
message	string	The message from a pipeline run.
parameters	object	The full or partial list of parameter name, value pair used in the pipeline run.
pipelineName	string	The pipeline name.
runDimensions	object	Run dimensions emitted by Pipeline run.
runEnd	string	The end time of a pipeline run in ISO8601 format.
runGroupId	string	Identifier that correlates all the recovery runs of a pipeline run.
runId	string	Identifier of a run.
runStart	string	The start time of a pipeline run in ISO8601 format.
status	string	The status of a pipeline run. Possible values: Queued, InProgress, Succeeded, Failed, Canceling, Cancelled

PipelineRunInvokedBy

Provides entity name and id that started the pipeline run.

Name	Type	Description
id	string	The ID of the entity that started the run.
invokedByType	string	The type of the entity that started the run.
name	string	Name of the entity that started the pipeline run.
pipelineName	string	The name of the pipeline that triggered the run, if any.
pipelineRunId	string	The run id of the pipeline that triggered the run, if any.

PipelineRunsQueryResponse

A list pipeline runs.

Name	Type	Description
continuationToken	string	The continuation token for getting the next page of results, if any remaining results exist, null otherwise.
value	PipelineRun[]	List of pipeline runs.

RunFilterParameters

Query parameters for listing runs.

Name	Type	Description
continuationToken	string	The continuation token for getting the next page of results. Null for first page.
filters	RunQueryFilter[]	List of filters.
lastUpdatedAfter	string	The time at or after which the run event was updated in 'ISO 8601' format.
lastUpdatedBefore	string	The time at or before which the run event was updated in 'ISO 8601' format.
orderBy	RunQueryOrderBy[]	List of OrderBy option.

RunQueryFilter

Query filter option for listing runs.

Name	Type	Description
operand	RunQueryFilterOperand	Parameter name to be used for filter. The allowed operands to query pipeline runs are PipelineName, RunStart, RunEnd and Status; to query activity runs are ActivityName, ActivityRunStart, ActivityRunEnd, ActivityType and Status, and to query trigger runs are TriggerName, TriggerRunTimestamp and Status.
operator	RunQueryFilterOperator	Operator to be used for filter.
values	string[]	List of filter values.

RunQueryFilterOperand

Parameter name to be used for filter. The allowed operands to query pipeline runs are PipelineName, RunStart, RunEnd and Status; to query activity runs are ActivityName, ActivityRunStart, ActivityRunEnd, ActivityType and Status, and to query trigger runs are TriggerName, TriggerRunTimestamp and Status.

Name	Type	Description
ActivityName	string
ActivityRunEnd	string
ActivityRunStart	string
ActivityType	string
LatestOnly	string
PipelineName	string
RunEnd	string
RunGroupId	string
RunStart	string
Status	string
TriggerName	string
TriggerRunTimestamp	string

RunQueryFilterOperator

Operator to be used for filter.

Name	Type	Description
Equals	string
In	string
NotEquals	string
NotIn	string

RunQueryOrder

Sorting order of the parameter.

Name	Type	Description
ASC	string
DESC	string

RunQueryOrderBy

An object to provide order by options for listing runs.

Name	Type	Description
order	RunQueryOrder	Sorting order of the parameter.
orderBy	RunQueryOrderByField	Parameter name to be used for order by. The allowed parameters to order by for pipeline runs are PipelineName, RunStart, RunEnd and Status; for activity runs are ActivityName, ActivityRunStart, ActivityRunEnd and Status; for trigger runs are TriggerName, TriggerRunTimestamp and Status.

RunQueryOrderByField

Parameter name to be used for order by. The allowed parameters to order by for pipeline runs are PipelineName, RunStart, RunEnd and Status; for activity runs are ActivityName, ActivityRunStart, ActivityRunEnd and Status; for trigger runs are TriggerName, TriggerRunTimestamp and Status.

Name	Type	Description
ActivityName	string
ActivityRunEnd	string
ActivityRunStart	string
PipelineName	string
RunEnd	string
RunStart	string
Status	string
TriggerName	string
TriggerRunTimestamp	string