How to cancel current pipeline job "from within" in Azure ML sdk v2

Celso Araujo 0

I am porting a sdk v1 machine learning pipeline to SDK v2. It runs on a daily basis on a scheduler. We have a step which, under certain conditions, cancels the whole pipeline job (ie, the other steps won't run). Its code is like this:

from azureml.core import Run
from azureml.pipeline.core import PipelineRun
run = Run.get_context()
ws = run.experiment.workspace
pipeline_run = PipelineRun(run.experiment, run.parent.id)
if condition:
    pipeline_run.cancel()

I can't find a way to do something similar using Python SDK v2. And I don't want to mix v1 and v2 code, since it is a bad practice. How could I do it? Any ideas?

YutongTie-MSFT 53,211 Reputation points

2024-11-15T00:03:04.2166667+00:00

Hello @Celso Araujo

Thanks for reaching out to us, you may consider begin_cancel method of Jobs in Azure Machine Learning V2 as below document - https://zcusa.951200.xyz/en-us/python/api/azure-ai-ml/azure.ai.ml.operations.joboperations?view=azure-python#azure-ai-ml-operations-joboperations-begin-cancel

Or you may also consider (Cancel/Disable)Schedule machine learning pipeline jobs if you are working on daily basis as below document - https://zcusa.951200.xyz/en-us/azure/machine-learning/how-to-schedule-pipeline-job?view=azureml-api-2&tabs=cliv2

I understand that different scenarios may need different solution, please do have a try and see how it works.

If you need more help during migrating your project from V1 to V2, we are happy to enable you a free support ticket to solve this issue with support team.

Regards,

Yutong
YutongTie-MSFT 53,211 Reputation points

2024-11-17T01:55:31.53+00:00

Hello Celso,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Regards,

Yutong
Celso Araujo 0 Reputation points

2024-11-18T12:21:13.79+00:00
Hi, @YutongTie-MSFT , thanks for your answer. Unfortunately, none of both solutions apply here:

I could theoretically use ml_client.jobs.begin_cancel method, provided I had the pipeline job's name at hand. This name must be retrieved from within this child job - which "does something", then checks whether the pipeline job should continue (ie, fire the other steps) or canceled. It is very easy to do it using sdk v1. How could I do it with sdk v2?

The pipeline should be run daily, so I'm not able to cancel/disable its scheduler.
YutongTie-MSFT 53,211 Reputation points

2024-11-18T16:16:56.3233333+00:00

@Celso Araujo Thanks for your response, I would highly recommend you raise a support ticket for your scenario as you want to run your pipeline daily and you may want to cancel the timeline in V2 so that you can discuss this scenario with support team, happy to enable you a free ticket for this case, please let me know if you want to do it.

Regards,

Yutong

Share via

How to cancel current pipeline job "from within" in Azure ML sdk v2

Your answer