Tutorial: Treinar um modelo de detecção de objetos com o AutoML e o Python

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

Crie um arquivo .yml com a configuração a seguir.

$schema: https://azuremlschemas.azureedge.net/latest/amlCompute.schema.json 
name: gpu-cluster
type: amlcompute
size: Standard_NC24s_v3
min_instances: 0
max_instances: 4
idle_time_before_scale_down: 120

Para criar a computação, execute o seguinte comando da CLI v2 com o caminho para o arquivo .yml, o nome do workspace, o grupo de recursos e a ID da assinatura.

az ml compute create -f [PATH_TO_YML_FILE] --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

from azure.ai.ml.entities import AmlCompute
compute_name = "gpu-cluster"
cluster_basic = AmlCompute(
    name=compute_name,
    type="amlcompute",
    size="Standard_NC24s_v3",
    min_instances=0,
    max_instances=4,
    idle_time_before_scale_down=120,
)
ml_client.begin_create_or_update(cluster_basic)

Essa computação é usada posteriormente durante a criação do trabalho automl específico da tarefa.

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

O nome do experimento pode ser fornecido usando a chave experiment_name da seguinte maneira:

experiment_name: dpv2-cli-automl-image-object-detection-experiment

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

O nome do experimento é usado posteriormente durante a criação do trabalho automl específico da tarefa.

exp_name = "dpv2-image-object-detection-experiment"

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

Crie um arquivo .yml com a configuração a seguir.

$schema: https://azuremlschemas.azureedge.net/latest/data.schema.json
name: fridge-items-images-object-detection
description: Fridge-items images Object detection
path: ./data/odFridgeObjects
type: uri_folder

Para carregar as imagens como um ativo de dados, execute o seguinte comando da CLI v2 com o caminho para o arquivo .yml, o nome do workspace, o grupo de recursos e a ID da assinatura.

az ml data create -f [PATH_TO_YML_FILE] --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# Uploading image files by creating a 'data asset URI FOLDER':

from azure.ai.ml.entities import Data
from azure.ai.ml.constants import AssetTypes, InputOutputModes
from azure.ai.ml import Input

my_data = Data(
    path=dataset_dir,
    type=AssetTypes.URI_FOLDER,
    description="Fridge-items images Object detection",
    name="fridge-items-images-object-detection",
)

uri_folder_data_asset = ml_client.data.create_or_update(my_data)

print(uri_folder_data_asset)
print("")
print("Path to folder in Blob Storage:")
print(uri_folder_data_asset.path)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

A configuração a seguir cria dados de treinamento e validação com base na MLTable.

target_column_name: label
training_data:
  path: data/training-mltable-folder
  type: mltable
validation_data:
  path: data/validation-mltable-folder
  type: mltable

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

Você pode criar entradas de dados com base no treinamento e validação da MLTable com o seguinte código:

from azure.ai.ml import Input
from azure.ai.ml.constants import AssetTypes

# Training MLTable defined locally, with local data to be uploaded
my_training_data_input = Input(type=AssetTypes.MLTABLE, path=training_mltable_path)

# Validation MLTable defined locally, with local data to be uploaded
my_validation_data_input = Input(type=AssetTypes.MLTABLE, path=validation_mltable_path)

# WITH REMOTE PATH: If available already in the cloud/workspace-blob-store
# my_training_data_input = Input(type=AssetTypes.MLTABLE, path="azureml://datastores/workspaceblobstore/paths/vision-classification/train")
# my_validation_data_input = Input(type=AssetTypes.MLTABLE, path="azureml://datastores/workspaceblobstore/paths/vision-classification/valid")

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

Para usar a computação sem servidor (versão prévia), substitua a linha compute: azureml:gpu-cluster por neste código:

resources:
 instance_type: Standard_NC24s_v3
 instance_count: 4

task: image_object_detection
primary_metric: mean_average_precision
compute: azureml:gpu-cluster

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# Create the AutoML job with the related factory-function.

image_object_detection_job = automl.image_object_detection(
    compute=compute_name,
    experiment_name=exp_name,
    training_data=my_training_data_input,
    validation_data=my_validation_data_input,
    target_column_name="label",
    primary_metric=ObjectDetectionPrimaryMetrics.MEAN_AVERAGE_PRECISION,
    tags={"my_custom_tag": "My custom value"},
)

image_object_detection_job.resources = ResourceConfiguration(instance_type="Standard_NC24rs_v3",instance_count =4)

image_object_detection_job.set_limits(
    max_trials=10,
    max_concurrent_trials=2,
)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

limits:
  max_trials: 10
  max_concurrent_trials: 2

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# Trigger AutoMode
image_object_detection_job.set_limits(max_trials=10, max_concurrent_trials=2)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

Para enviar seu trabalho do AutoML, execute o seguinte comando da CLI v2 com o caminho para o arquivo .yml, o nome do workspace, o grupo de recursos e a ID da assinatura.

az ml job create --file ./hello-automl-job-basic.yml --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

Quando você tiver configurado o Trabalho do AutoML com as configurações desejadas, envie o trabalho.

# Submit the AutoML job
returned_job = ml_client.jobs.create_or_update(
    image_object_detection_job
)  # submit the job to the backend

print(f"Created job: {returned_job}")

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

limits:
  timeout_minutes: 60
  max_trials: 10
  max_concurrent_trials: 2

# Set limits
image_object_detection_job.set_limits(
    timeout_minutes=60,
    max_trials=10,
    max_concurrent_trials=2,
)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

sweep:
  sampling_algorithm: random
  early_termination:
    type: bandit
    evaluation_interval: 2
    slack_factor: 0.2
    delay_evaluation: 6

search_space:
  - model_name:
      type: choice
      values: [yolov5]
    learning_rate:
      type: uniform
      min_value: 0.0001
      max_value: 0.01
    model_size:
      type: choice
      values: [small, medium]

  - model_name:
      type: choice
      values: [fasterrcnn_resnet50_fpn]
    learning_rate:
      type: uniform
      min_value: 0.0001
      max_value: 0.001
    optimizer:
      type: choice
      values: [sgd, adam, adamw]
    min_size:
      type: choice
      values: [600, 800]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# Configure sweep settings
image_object_detection_job.set_sweep(
    sampling_algorithm="random",
    early_termination=BanditPolicy(
        evaluation_interval=2, slack_factor=0.2, delay_evaluation=6
    ),
)

# Define search space
image_object_detection_job.extend_search_space(
    [
        SearchSpace(
            model_name=Choice(["yolov5"]),
            learning_rate=Uniform(0.0001, 0.01),
            model_size=Choice(["small", "medium"]),  # model-specific
            # image_size=Choice([640, 704, 768]),  # model-specific; might need GPU with large memory
        ),
        SearchSpace(
            model_name=Choice(["fasterrcnn_resnet50_fpn"]),
            learning_rate=Uniform(0.0001, 0.001),
            optimizer=Choice(["sgd", "adam", "adamw"]),
            min_size=Choice([600, 800]),  # model-specific
            # warmup_cosine_lr_warmup_epochs=Choice([0, 3]),
        ),
    ]
)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

Para enviar seu trabalho do AutoML, execute o seguinte comando da CLI v2 com o caminho para o arquivo .yml, o nome do workspace, o grupo de recursos e a ID da assinatura.

az ml job create --file ./hello-automl-job-basic.yml --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

Quando você tiver configurado o Trabalho do AutoML com as configurações desejadas, envie o trabalho.

# Submit the AutoML job
returned_job = ml_client.jobs.create_or_update(
    image_object_detection_job
)  # submit the job to the backend

print(f"Created job: {returned_job}")

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

CLI example not available, please use Python SDK.

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

hd_job = ml_client.jobs.get(returned_job.name + '_HD')
hd_job

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

CLI example not available, please use Python SDK.

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# Get the best model's child run

best_child_run_id = mlflow_parent_run.data.tags["automl_best_child_run_id"]
print(f"Found best child run id: {best_child_run_id}")

best_run = mlflow_client.get_run(best_child_run_id)

print("Best child run: ")
print(best_run)

# Create local folder
local_dir = "./artifact_downloads"
if not os.path.exists(local_dir):
    os.mkdir(local_dir)

# Download run's artifacts/outputs
local_path = mlflow_client.download_artifacts(
    best_run.info.run_id, "outputs", local_dir
)
print(f"Artifacts downloaded in: {local_path}")
print(f"Artifacts: {os.listdir(local_path)}")

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

 az ml model create --name od-fridge-items-mlflow-model --version 1 --path azureml://jobs/$best_run/outputs/artifacts/outputs/mlflow-model/ --type mlflow_model --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

model_name = "od-fridge-items-mlflow-model"
model = Model(
    path=f"azureml://jobs/{best_run.info.run_id}/outputs/artifacts/outputs/mlflow-model/",
    name=model_name,
    description="my sample object detection model",
    type=AssetTypes.MLFLOW_MODEL,
)

# for downloaded file
# model = Model(
#     path=mlflow_model_dir,
#     name=model_name,
#     description="my sample object detection model",
#     type=AssetTypes.MLFLOW_MODEL,
# )

registered_model = ml_client.models.create_or_update(model)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: od-fridge-items-endpoint
auth_mode: key

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# Creating a unique endpoint name with current datetime to avoid conflicts
import datetime

online_endpoint_name = "od-fridge-items-" + datetime.datetime.now().strftime(
    "%m%d%H%M%f"
)

# create an online endpoint
endpoint = ManagedOnlineEndpoint(
    name=online_endpoint_name,
    description="this is a sample online endpoint for deploying model",
    auth_mode="key",
    tags={"foo": "bar"},
)
print(online_endpoint_name)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

az ml online-endpoint create --file .\create_endpoint.yml --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

ml_client.begin_create_or_update(endpoint).result()

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

name: od-fridge-items-mlflow-deploy
endpoint_name: od-fridge-items-endpoint
model: azureml:od-fridge-items-mlflow-model@latest
instance_type: Standard_DS3_v2
instance_count: 1
liveness_probe:
    failure_threshold: 30
    success_threshold: 1
    timeout: 2
    period: 10
    initial_delay: 2000
readiness_probe:
    failure_threshold: 10
    success_threshold: 1
    timeout: 10
    period: 10
    initial_delay: 2000 

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

deployment = ManagedOnlineDeployment(
    name="od-fridge-items-mlflow-deploy",
    endpoint_name=online_endpoint_name,
    model=registered_model.id,
    instance_type="Standard_DS4_V2",
    instance_count=1,
    request_settings=req_timeout,
    liveness_probe=ProbeSettings(
        failure_threshold=30,
        success_threshold=1,
        timeout=2,
        period=10,
        initial_delay=2000,
    ),
    readiness_probe=ProbeSettings(
        failure_threshold=10,
        success_threshold=1,
        timeout=10,
        period=10,
        initial_delay=2000,
    ),
)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

az ml online-deployment create --file .\create_deployment.yml --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

ml_client.online_deployments.begin_create_or_update(deployment).result()

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

az ml online-endpoint update --name 'od-fridge-items-endpoint' --traffic 'od-fridge-items-mlflow-deploy=100' --workspace-name [YOUR_AZURE_WORKSPACE] --resource-group [YOUR_AZURE_RESOURCE_GROUP] --subscription [YOUR_AZURE_SUBSCRIPTION]

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# od fridge items deployment to take 100% traffic
endpoint.traffic = {"od-fridge-items-mlflow-deploy": 100}
ml_client.begin_create_or_update(endpoint).result()

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

CLI example not available, please use Python SDK.

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

# Create request json
import base64

sample_image = os.path.join(dataset_dir, "images", "1.jpg")


def read_image(image_path):
    with open(image_path, "rb") as f:
        return f.read()


request_json = {
    "input_data": {
        "columns": ["image"],
        "data": [base64.encodebytes(read_image(sample_image)).decode("utf-8")],
    }
}

import json

request_file_name = "sample_request_data.json"

with open(request_file_name, "w") as request_file:
    json.dump(request_json, request_file)

resp = ml_client.online_endpoints.invoke(
    endpoint_name=online_endpoint_name,
    deployment_name=deployment.name,
    request_file=request_file_name,
)

APLICA-SE A: Extensão de ML da CLI do Azurev2 (atual)

CLI example not available, please use Python SDK.

APLICA-SE A: SDK do Python azure-ai-ml v2 (atual)

%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import matplotlib.patches as patches
from PIL import Image
import numpy as np
import json

IMAGE_SIZE = (18, 12)
plt.figure(figsize=IMAGE_SIZE)
img_np = mpimg.imread(sample_image)
img = Image.fromarray(img_np.astype("uint8"), "RGB")
x, y = img.size

fig, ax = plt.subplots(1, figsize=(15, 15))
# Display the image
ax.imshow(img_np)

# draw box and label for each detection
detections = json.loads(resp)
for detect in detections[0]["boxes"]:
    label = detect["label"]
    box = detect["box"]
    conf_score = detect["score"]
    if conf_score > 0.6:
        ymin, xmin, ymax, xmax = (
            box["topY"],
            box["topX"],
            box["bottomY"],
            box["bottomX"],
        )
        topleft_x, topleft_y = x * xmin, y * ymin
        width, height = x * (xmax - xmin), y * (ymax - ymin)
        print(
            f"{detect['label']}: [{round(topleft_x, 3)}, {round(topleft_y, 3)}, "
            f"{round(width, 3)}, {round(height, 3)}], {round(conf_score, 3)}"
        )

        color = np.random.rand(3)  #'red'
        rect = patches.Rectangle(
            (topleft_x, topleft_y),
            width,
            height,
            linewidth=3,
            edgecolor=color,
            facecolor="none",
        )
        ax.add_patch(rect)
        plt.text(topleft_x, topleft_y - 10, label, color=color, fontsize=20)
plt.show()