Get started with the Azure OpenAI security building block

• Microsoft AI Chat Protocol provides standardized API contracts across AI solutions and languages. The chat app conforms to the Microsoft AI Chat Protocol, which allows the evaluations app to run against any chat app that conforms to the protocol.
• A Python Quart that uses the openai package to generate responses to user messages.
• A basic HTML/JavaScript frontend that streams responses from the backend using JSON Lines over a ReadableStream.

• A Blazor web app that uses the Azure.AI.OpenAI NuGet package to generate responses to user messages.

• A TypeScript web app that uses the openai npm package to generate responses to user messages.

Learn more about cost in the sample repo.

Learn more about cost in the sample repo.

Learn more about cost in the sample repo.

Exploring the sample code

While OpenAI and Azure OpenAI Service rely on a common Python client library, small code changes are needed when using Azure OpenAI endpoints. Let's see how this sample configures keyless authentication with Microsoft Entra ID and communicates with Azure OpenAI.

Configure authentication with managed identity

In this sample, the src\quartapp\chat.py file begins with configuring keyless authentication.

The following snippet uses the azure.identity.aio module to create an asynchronous Microsoft Entra authentication flow.

The following code snippet uses the AZURE_CLIENT_ID azd environment variable to create a ManagedIdentityCredential instance capable of authenticating via user-assigned managed identity.

user_assigned_managed_identity_credential = ManagedIdentityCredential(client_id=os.getenv("AZURE_CLIENT_ID")) 

The following code snippet uses AZURE_TENANT_ID azd resource environment variable to create an AzureDeveloperCliCredential instance capable of authenticating with the current Microsoft Entra tenant.

azure_dev_cli_credential = AzureDeveloperCliCredential(tenant_id=os.getenv("AZURE_TENANT_ID"), process_timeout=60)  

The Azure Identity client library provides credentials—public classes that implement the Azure Core library's TokenCredential protocol. A credential represents a distinct authentication flow for acquiring an access token from Microsoft Entra ID. These credentials can be chained together to form an ordered sequence of authentication mechanisms to be attempted.

The following snippet creates a ChainedTokenCredential using a ManagedIdentityCredential and an AzureDeveloperCliCredential:

• The ManagedIdentityCredential is used for Azure Functions and Azure App Service. A user-assigned managed identity is supported by passing the client_id to ManagedIdentityCredential.
• The AzureDeveloperCliCredential is used for local development. It was set previously based on the Microsoft Entra tenant to use.

azure_credential = ChainedTokenCredential(
    user_assigned_managed_identity_credential,
    azure_dev_cli_credential
)

The following code snippet gets the Azure OpenAI token provider based on the selected Azure credential. This value is obtained by calling the azure.identity.aio.get_bearer_token_provider with two arguments:

• azure_credential: The ChainedTokenCredential instance created earlier to authenticate the request.

• https://cognitiveservices.azure.com/.default: Required one or more bearer token scopes. In this case, the Azure Cognitive Services endpoint.

token_provider = get_bearer_token_provider(
    azure_credential, "https://cognitiveservices.azure.com/.default"
)

The following lines check for the required AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_CHATGPT_DEPLOYMENT azd resource environment variables, which are provisioned during azd app deployment. An error is thrown if a value isn't present.

if not os.getenv("AZURE_OPENAI_ENDPOINT"):
    raise ValueError("AZURE_OPENAI_ENDPOINT is required for Azure OpenAI")
if not os.getenv("AZURE_OPENAI_CHATGPT_DEPLOYMENT"):
    raise ValueError("AZURE_OPENAI_CHATGPT_DEPLOYMENT is required for Azure OpenAI")

This snippet initializes the Azure OpenAI client, setting the api_version, azure_endpoint, and azure_ad_token_provider (client_args) parameters:

bp.openai_client = AsyncAzureOpenAI(
    api_version=os.getenv("AZURE_OPENAI_API_VERSION") or "2024-02-15-preview",
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    azure_ad_token_provider=token_provider,
)  

The following line sets the Azure OpenAI model deployment name for use in API calls:

bp.openai_model = os.getenv("AZURE_OPENAI_CHATGPT_DEPLOYMENT")

Once this function completes, the client is properly configured and ready to interact with Azure OpenAI services.

Response stream using the OpenAI Client and model

The response_stream handles the chat completion call in the route. The following code snippet shows how openai_client and model are used.

async def response_stream():
    # This sends all messages, so API request may exceed token limits
    all_messages = [
        {"role": "system", "content": "You are a helpful assistant."},
    ] + request_messages

    chat_coroutine = bp.openai_client.chat.completions.create(
        # Azure Open AI takes the deployment name as the model name
        model=bp.openai_model,
        messages=all_messages,
        stream=True,
    )

Explore the sample code

.NET applications rely on the Azure.AI.OpenAI client library to communicate with Azure OpenAI services, which takes a dependency on the OpenAI library. The sample app configures keyless authentication using Microsoft Entra ID to communicate with Azure OpenAI.

Configure authentication and service registration

In this sample, keyless authentication is configured in the program.cs file. The following code snippet uses the AZURE_CLIENT_ID environment variable set by azd to create a ManagedIdentityCredential instance capable of authenticating via user-assigned managed identity.

var userAssignedIdentityCredential = 
    new ManagedIdentityCredential(builder.Configuration.GetValue<string>("AZURE_CLIENT_ID"));

The following code snippet uses the AZURE_TENANT_ID environment variable set by azd to create an AzureDeveloperCliCredential instance capable of authenticating locally using the account signed-in to azd.

var azureDevCliCredential = new AzureDeveloperCliCredential(
    new AzureDeveloperCliCredentialOptions()
    { 
        TenantId = builder.Configuration.GetValue<string>("AZURE_TENANT_ID") 
    });

The Azure Identity client library provides credential classes that implement the Azure Core library's TokenCredential protocol. A credential represents a distinct authentication flow for acquiring an access token from Microsoft Entra ID. These credentials can be chained together using ChainedTokenCredential to form an ordered sequence of authentication mechanisms to be attempted.

The following snippet registers the AzureOpenAIClient for dependency injection and creates a ChainedTokenCredential using a ManagedIdentityCredential and an AzureDeveloperCliCredential:

• The ManagedIdentityCredential is used for Azure Functions and Azure App Service. A user-assigned managed identity is supported using the AZURE_CLIENT_ID that was provided to the ManagedIdentityCredential.
• The AzureDeveloperCliCredential is used for local development. It was set previously based on the Microsoft Entra tenant to use.

builder.Services.AddAzureClients(
    clientBuilder => {
        clientBuilder.AddClient<AzureOpenAIClient, AzureOpenAIClientOptions>((options, _, _)
            => new AzureOpenAIClient(
                new Uri(endpoint),
                new ChainedTokenCredential(
                    userAssignedIdentityCredential, azureDevCliCredential), options));
    });

Get chat completions using the Azure OpenAI client

The Blazor web app injects the registered AzureOpenAIClient at the top of the Home.Razor component:

@inject AzureOpenAIClient azureOpenAIClient

When the user submits the form, the AzureOpenAIClient sends their prompt to the OpenAI model to generate a completion:

ChatClient chatClient = azureOpenAIClient.GetChatClient("gpt-4o-mini");

messages.Add(new UserChatMessage(model.UserMessage));

ChatCompletion completion = await chatClient.CompleteChatAsync(messages);
    messages.Add(new SystemChatMessage(completion.Content[0].Text));

Explore the sample code

While OpenAI and Azure OpenAI Service rely on a openai (common JavaScript client library), small code changes are needed when using Azure OpenAI endpoints. Let's see how this sample configures keyless authentication with Microsoft Entra ID and communicates with Azure OpenAI.

Keyless authentication for each environment

The Azure Identity client library provides credential classes that implement the Azure Core library's TokenCredential protocol. A credential represents a distinct authentication flow for acquiring an access token from Microsoft Entra ID. These credentials can be chained together using ChainedTokenCredential to form an ordered sequence of authentication mechanisms to be attempted. This allows you to deploy the same code in both production and local development environments.

Configure authentication with managed identity

In this sample, the ./src/azure-authentication.ts provides several functions to provide keyless authentication to Azure OpenAI.

The first function, getChainedCredential(), returns the first valid Azure credential found in the chain.

function getChainedCredential() {

    return new ChainedTokenCredential(
        new ManagedIdentityCredential(process.env.AZURE_CLIENT_ID!), 
        new AzureDeveloperCliCredential({
            tenantId: process.env.AZURE_TENANT_ID! ? process.env.AZURE_TENANT_ID! : undefined
          })
    );
}

• ManagedIdentityCredential is attempted first. It's set up with the AZURE_CLIENT_ID environment variable in the production runtime and is capable of authenticating via user-assigned managed identity.
• AzureDeveloperCliCredential is attempted second. It's set up when a develop signs in with Azure CLI az login.

Get bearer token for OpenAI

The second function in ./src/azure-authentication.ts is getTokenProvider(), which returns a callback that provides a bearer token scoped to the Azure Cognitive Services endpoint.

function getTokenProvider(): () => Promise<string> {
    const credential  = getChainedCredential();
    const scope = "https://cognitiveservices.azure.com/.default";
    return getBearerTokenProvider(credential, scope);
}

The preceding code snippet uses getBearerTokenProvider to take the credential and the scope, then returns a callback that provides a bearer token.

Create authenticated Azure OpenAI client

The third function in ./src/azure-authentication.ts is getOpenAiClient(), which returns the Azure OpenAI client.

export function getOpenAiClient(): AzureOpenAI | undefined{
    try {

        if (!process.env.AZURE_OPENAI_ENDPOINT) {
            throw new Error("AZURE_OPENAI_ENDPOINT is required for Azure OpenAI");
        }
        if (!process.env.AZURE_OPENAI_CHAT_DEPLOYMENT) {
            throw new Error("AZURE_OPENAI_CHAT_DEPLOYMENT is required for Azure OpenAI");
        }

        const options = { 
            azureADTokenProvider: getTokenProvider(), 
            deployment: process.env.AZURE_OPENAI_CHAT_DEPLOYMENT!, 
            apiVersion: process.env.AZURE_OPENAI_API_VERSION! || "2024-02-15-preview",
            endpoint: process.env.AZURE_OPENAI_ENDPOINT!
        }

        // Create the Asynchronous Azure OpenAI client
        return new AzureOpenAI (options);

    } catch (error) {
        console.error('Error getting Azure OpenAI client: ', error);
    }
}

This code takes the options, including the correctly scoped token, and creates the AzureOpenAI client

Stream chat answer with Azure OpenAI

Use the following Fastify route handler in ./src/openai-chat-api.ts to send a message to Azure OpenAI and stream the response.

import { FastifyReply, FastifyRequest } from 'fastify';
import { AzureOpenAI } from "openai";
import { getOpenAiClient } from './azure-authentication.js';
import { ChatCompletionChunk, ChatCompletionMessageParam } from 'openai/resources/chat/completions';

interface ChatRequestBody {
    messages: ChatCompletionMessageParam [];
  }

export async function chatRoute (request: FastifyRequest<{ Body: ChatRequestBody }>, reply: FastifyReply) {

    const requestMessages: ChatCompletionMessageParam[] = request?.body?.messages;
    const openaiClient: AzureOpenAI | undefined = getOpenAiClient();

    if (!openaiClient) {
      throw new Error("Azure OpenAI client is not configured");
    }

    const allMessages = [
      { role: "system", content: "You are a helpful assistant."},
      ...requestMessages
    ] as ChatCompletionMessageParam [];

    const chatCompletionChunks = await openaiClient.chat.completions.create({
      // Azure Open AI takes the deployment name as the model name
      model: process.env.AZURE_OPENAI_CHAT_DEPLOYMENT_MODEL || "gpt-4o-mini",
      messages: allMessages,
      stream: true

    })
    reply.raw.setHeader('Content-Type', 'text/html; charset=utf-8');
    reply.raw.setHeader('Cache-Control', 'no-cache');
    reply.raw.setHeader('Connection', 'keep-alive');
    reply.raw.flushHeaders();

    for await (const chunk of chatCompletionChunks as AsyncIterable<ChatCompletionChunk>) {
      for (const choice of chunk.choices) {
        reply.raw.write(JSON.stringify(choice) + "\n")
      }
    }

    reply.raw.end()

}

The function gets the chat conversation, including any previous messages, and sends them to Azure OpenAI. As the stream chunks are returned from Azure OpenAI, the are sent to the client.

If your issue isn't addressed, log your issue to the repository's Issues.

Next steps

If your issue isn't addressed, log your issue to the repository's Issues.

If your issue isn't addressed, log your issue to the repository's Issues.