Why does GPT-4-O in azure openAI randomly return inappropriate content errors when the content is appropriate?

@AMROUN Lysa Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

.

1. Please let me know how are you accessing the Azure OpenAi service through REST API / SDK or Open Ai Studio ?
2. Please clarify what does inappropriate content mean.
3. Are you getting any error / exception message ?
4. Please share the screenshot with the returned content VS expected content.
5. Is this something which started recently ? or was it working earlier and returning the right content ?
6. Is the issue intermittent or occurs all the time ?
7. May I know which region is the resource deployed to ?
8. Are you using Add your own data feature ?

.

Awaiting your reply.

hi @navba-MSFT

thanks for your response,

1.      Please let me know how are you accessing the Azure OpenAi service through REST API / SDK or Open Ai Studio ?

I access azure OpenAI in REST API/SDK of azure in python , like this :

        response = openai.chat.completions.create(
            model=os.getenv("AZURE_OPENAI_DEPLOYMENT_ID_US"),
            messages=self.conversation
        )
        assistant_response = response.choices[0].message.content
        self.conversation.append({"role": "assistant", "content": assistant_response})

2.      Please clarify what does inappropriate content mean.

      **=>  inapproriated content is any content prohibited by azure openai such as sexual content, violent content, ...**

3.      Please share the screenshot with the returned content VS expected content.

   **=> This is the chat response :**

Error code: 400 - {'error': {'message': "The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766", 'type': None, 'param': 'prompt', 'code': 'content_filter', 'status': 400, 'innererror': {'code': 'ResponsibleAIPolicyViolation', 'content_filter_result': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': True, 'severity': 'high'}, 'violence': {'filtered': False, 'severity': 'safe'}}}}}

   For this request in french : « Aide moi a rédiger un mail pour un client afin d'obtenir un rendez vous client TECUMSEF > in english it’s : « Help me to write a mail for a customer in order to get a customer appointment TECUMSEF »

4.      Is this something which started recently ? or was it working earlier and returning the right content ?

=> I only started using gpt4-o from azure oenai recently (1 week) and it happened to me from the first application, but with gpt3.5-turbo or gpt4-turbo it never happened to me with normal content, unlike with gpt4-o.

5.      Is the issue intermittent or occurs all the time ?

=> It’s intermittent and random

6.      Are you using Add your own data feature ?

=> No , i use it only to generate text with a system instruction and user prompt without my own data.

thank's

Lysa

@AMROUN Lysa Thanks for getting back and sharing the details.

.

While I used the same text from chat playground for my GPT4o model it worked just fine. See below:

Could you please check if you are encountering the same issue from Azure Open AI Studio chat playground ?

The default content filtering configuration is set to filter at the medium severity threshold for all four content harms categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low or safe is not filtered by the content filters.

Could you please check the Content Filters and its severity set for the sexual filter ? Could you please set it to low and check ?

Awaiting your reply.

This also happens via the 'direct' OpenAI API and the ChatGPT web app. I suspect it is a bug/issue related to the GPT-4o model and not actually moderation.

I do not have access to GPT-4o via Azure, but would not be surprised if this is the same underlying issue, based on the lack of moderation-relevant content, which is also the observation I have made in the other environments that use GPT-4o. Perhaps it is triggered when the model responds with a message in a modality that aren't currently available? (ie audio/image output?)

To reproduce it, one can include a number of image messages (3-4) and then increase the (text) message count gradually in the request.

@AMROUN Lysa I have sent you a private message asking for more details about your resource, so that I can look at the backend logs for the cause of this issue.

Awaiting your reply.