Why is my AutoML Job environment version different than my notebook environment ?

Question

Hi
I'm new in AzureML and I'm trying to run the "automl-image-object-detection-task-fridge-items" example.
Everything works but I've noticed that although I'm working with the Jupiter notebook Python 3.10 - SDK v2 env, the Job that is created in the example has in its definition.json file "BaseDockerfile" with the param python=3.9.
Also in the notebook i get 2 python version when i run these 2 command:

import sys
print(f"Python version: {sys.version}")
!python --version

Python version: 3.10.2 | packaged by conda-forge | (main, Feb 1 2022, 19:29:00) [GCC 9.4.0]

Python 3.9.19

Answer

Hello Oren Koler,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are having issues with your AutoML Job environment version different than my notebook environment.

The reason for the discrepancy in Python versions you saw was because the AzureML job uses a predefined Docker image with Python 3.9, while your local Jupyter notebook environment is set to Python 3.10. This is why you see different Python versions when running the commands. To resolve the version mismatch and ensure consistency across your development and deployment environments.

Option1:

You can either:

Modify the Dockerfile by updating the BaseDockerfile in the definition.json file to use Python 3.10.

or

Use a Custom Docker Image by creating a custom Docker image with Python 3.10 and specify it in your job configuration.

For example Dockerfile:

FROM mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:20230110.v1
RUN conda install -y python=3.10

Then, update your definition.json to point to this custom Dockerfile.

Alternatively, in your local environment ensure it matches the job's environment by downgrading your local Python version to 3.9. This can be done using conda:

For example Bash:

conda install python=3.9

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam

Share via

Why is my AutoML Job environment version different than my notebook environment ?

1 answer

Your answer