openai.RateLimitError - Unable to use Azure Open AI for multi modal input

Tried with different max tokens. Receiving same response. Even for single page input as attached, it is returning the rate limit error.

Input File: testinput.png

Harish I just tried with same image and able to see the results.

AFAIK, free trial has very limited access to the features. I would suggest you, first try with any other image and see if that works.

Did you check if you have exceeded the quota limit for your Azure OpenAI resources?

If that doesn't help, please increase the quota.

Tried with another simple image. Receiving same ratelimit error. Providing the code which I tried. Max tokens Increased to 1000, which also provides same error output.

Response is properly returned when I uncomment the text input. Only the image input is not working.

Sample input file I tried test1.png

Received below error

-------begin--------

Image Path: C:\Projects\LibreOfficeFiles\test1.png

-------------entered the block - markdown from images------------

Error in OpenAI callError code: 429 - {'error': {'code': '429', 'message': 'Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-08-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 86400 seconds. Please contact Azure support service if you would like to further increase the default rate limit.'}}

-------end--------

import os
import base64
from openai import AzureOpenAI

client = AzureOpenAI(
  azure_endpoint = "<endpoint>", 
  api_key="<key>",  
  api_version="2024-08-01-preview"
)

# Base64 encode images
def encode_image(image_path):
    with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode("utf-8")
        

def extract_markdown_from_image(image_path):
    try:
        base64_image = encode_image(image_path)
        print ("Image Path: ", image_path)
        print("-------------entered the block - markdown from images------------")
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                { "role": "system", "content": "You are a helpful assistant." },
                { "role": "user", "content": [  
                    { 
                        "type": "text", 
                        "text": """Extract everything you see in this image to markdown. 
                            Convert all charts such as line, pie and bar charts to markdown tables and include a note that the numbers are approximate.
                        """ 
                    },
                    {
                        "type": "image_url", 
                        "image_url": {"url": f"data:image/png;base64,{base64_image}"}
                    }
                ] 
                }
            ],
            # messages=[
            #     {"role": "system", "content": "You are a helpful assistant."},
            #     {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
            #     {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
            #     {"role": "user", "content": "Do other Azure AI services support this too?"}
            # ],
            max_tokens=256 
        )
        print("response:-----------------------",response.choices[0].message.content)
        return response.choices[0].message.content
    except Exception as ex:
        print(f"Error in OpenAI call" + str(ex))
        return ""


extract_markdown_from_image(r"<Image Path>")

This is the sample image I tried test1.png