Error InvalidAuthenticationTokenTenant on az ml online-deployment create from BICEP

The error message (InvalidAuthenticationTokenTenant) you're encountering typically occurs when the access token being used to authenticate against Azure services is issued from a different tenant than the one associated with the subscription you are trying to operate in. Based on your logs, it looks like the tenant associated with your subscription (ending with 5e33d) is different from the tenant issuing the token (which is tied to your service connection).

Here are some steps you can take to troubleshoot and resolve this issue:

1. Check Azure Service Connection Configuration

• Ensure that the Azure service connection (defined by $(azureServiceConnection) in your YAML) is configured to use the correct tenant. If the service principal used in the pipeline is from a different tenant than the one linked to your subscription, you will see this error.
  • Go to Azure DevOps > Project Settings > Service Connections > Edit your service connection, and confirm that the correct tenant ID and subscription ID are used.
  • Make sure that the service principal has access to both the subscription and the resource group where you're deploying the model.

2. Switch to the Correct Tenant in the Pipeline

• Use az account tenant set to switch the tenant to the correct one before the deployment. You can add this step in your pipeline YAML script:

  
       - task: AzureCLI@2
  
         name: SwitchTenant
  
         inputs:
  
           azureSubscription: $(azureServiceConnection)
  
           scriptType: 'bash'
  
           scriptLocation: 'inlineScript'
  
           inlineScript: |
  
             echo "Switching to the correct tenant..."
  
             az account tenant set --tenant <correct_tenant_id>
  
  

  • This ensures that all subsequent operations in the pipeline use the correct tenant.

3. Use az login with the Correct Tenant

If the service connection cannot be reconfigured, you can force the az ml commands to authenticate with the correct tenant using az login or by specifying the --tenant option explicitly when creating the deployment. For example:

   az ml online-deployment create --file ./deploy_model_to_endpoint.yml --name $(deploymentName) --endpoint-name $(endPointName) --resource-group $(resourceGroupName) --workspace-name $(workspaceName) --subscription *my_subscription* --tenant <correct_tenant_id>

4. Ensure Correct Role Assignment

• Verify that the service principal used in the service connection has the necessary roles (Owner, Azure AI Inference Deployment Operator) on the correct tenant and subscription. If the service principal does not have access to the correct tenant, it will fail during the deployment step.
  • If needed, assign the correct role on the subscription in the target tenant by navigating to Azure Portal > Subscriptions > Access Control (IAM) > Add role assignment.

5. Check for Private Endpoint Configuration

If your deployment involves private endpoints, ensure that your network settings are correct. Sometimes, private endpoints require different configurations for accessing resources, and the wrong tenant error can be a symptom of an underlying network misconfiguration. Check Azure Private Link settings to ensure that you're routing correctly through the correct tenant and subscription.

By following these steps, you should be able to resolve the InvalidAuthenticationTokenTenant error. Let me know if you need further clarification or assistance!

Hello,

Thank you very much for your replies.

I tested the different suggestions and still have no success.

Here are the logs of my tests :

Setting AZURE_CONFIG_DIR env variable to: /home/vsts/work/_temp/.azclitask
Setting active cloud to: AzureCloud
/usr/bin/az cloud set -n AzureCloud
/usr/bin/az login --service-principal -u *** --password=*** --tenant ***471c --allow-no-subscriptions
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "***471c",
    "id": "***0fde",
    "isDefault": true,
    "managedByTenants": [],
    "name": "***",
    "state": "Enabled",
    "tenantId": "***471c",
    "user": {
      "name": "***",
      "type": "servicePrincipal"
    }
  }
]
>az account tenant list
[
  {
    "id": "/tenants/***471c",
    "tenantId": "***471c"
  }
]
>az account clear
>az account set --name ***e086
ERROR: The subscription of '***' doesn't exist in cloud 'AzureCloud'
>az account tenant set --tenant ***471c
ERROR: 'tenant' is misspelled or not recognized by the system.
>az account show
{
  "environmentName": "AzureCloud",
  "homeTenantId": "***471c",
  "id": "***0fde",
  "isDefault": true,
  "managedByTenants": [],
  "name": "***",
  "state": "Enabled",
  "tenantId": "***471c",
  "user": {
    "name": "***",
    "type": "servicePrincipal"
  }
}

>az ml online-deployment create --file ./deploy_model_to_endpoint.yml --name $(deploymentName) --endpoint-name $(endPointName) --resource-group $(resourceGroupName) --workspace-name $(workspaceName) --subscription ***0fde --tenant ***471c
ERROR: unrecognized arguments: --tenant ***471c

I am not sure about what is 'AzureCloud', it is not something I called for but it seems to me that it

by default.

I added tested previously the Tenant options and already had those error messages.

This leaves your comment arond privage endpoint, I don't know much about this, I was requested to provide an URL in our resources group that enables us to provide a sentence, the URL must interrogate an AI Model and return a translation of the sentence, is a private endpoint really what I need ? In that case, would I need a VNET and specific network configurations ?

Thanks a lot for your guidance.