Azure AI Content Safety Frequently Asked Questions

For references on how to start submitting text and images to Azure AI Content Safety, and view model responses and results, visit the Azure AI Foundry portal page. You can use the sidebar to navigate through the Safety+ Security page.

Our content harm classification models currently support the moderation of text, images, and multimodal (images with text + OCR). The protected material, prompt shields, and groundedness detection models work with text content.

A: We generally charge by volume. For example, the Image API is priced based on the number of images submitted. The Text API is billed for the number of text records submitted to the service. However, each model has its own corresponding rate. See the Azure pricing page for more information about pricing tiers.

Azure Content Moderator is deprecated as of March 2024. Microsoft recommends that Azure Content Moderator users migrate to Azure AI Content Safety because:

• Azure AI Content Safety offers more accurate and granular detection of harmful content in text and images using state-of-the-art AI models.
• Azure AI Content Safety supports multilingual content moderation (see Language support).
• Azure AI Content Safety enables responsible AI practices by monitoring both user-generated and AI-generated content.

The main differences between the two services are:

• Azure Content Moderator uses a binary classification for each content flag (such as profanity or adult), while Azure AI Content Safety uses multiple severity levels with the different classes (sexual, violent, hate, and self-harm).
• Azure AI Content Safety supports multilingual content moderation (see Language support), while Azure Content Moderator's AI classifiers only support English.
• Azure Content Moderator has a built-in term list and a custom term list feature, while Azure AI Content Safety doesn't have a built-in term list but relies on advanced language and vision models to detect harmful content. It also provides a custom term list feature for customization.
• Azure AI Content Safety has an interactive studio for exploring and testing the service capabilities, while Azure Content Moderator does not.

No. The Azure AI Content Safety API returns classification metadata based on model outputs. Our results tell users whether material across various classes (sexual, violence, hate, self-harm) is present in input content, via either a returned severity level (such as in the Text API) or binary results (such as in Prompt Shields API).

As a user, you use those results to inform appropriate enforcement actions - such as automatically tagging or removing certain content - based on your own policies and practices.

Service usage is throttled if you reach the transaction limit on the Free tier.

The Content Safety API is optimized for real-time moderation needs. Our model results are returned directly in the API response message.

Refer to the overview for the current RPS limits for each API. To request an increase in RPS, email us with justification and an estimated traffic forecast.

To decide whether to filter at severity level 0, 2, 4 or 6, we strongly encourage you to consult the descriptive examples of each severity level listed in Harm Categories. Generally, text that scores at severity level 0 might reference a sensitive topic but may be benign. Deciding whether to moderate at level 2 or 4 will depend on your community guidelines and sensitivity around controversial topics.

Overall, it depends on your datasets and your own policies and risk tolerance. We recommend starting a severity level of 4 for new projects. You can then adjust your severity level based on model performance.

We're also available to run evaluations with your data in order to recommend thresholds, and you can use our Content Safety Studio batch evaluation for more information on best practices.

No input texts or images are stored by the model during detection (except for customer-supplied blocklists, and user inputs are not used to train, retrain, or improve the Azure AI Content Safety models.

To learn more about Microsoft's privacy and security commitments visit Data, privacy, and security for Azure AI Content Safety.

No. Microsoft hosts the Azure AI Content Safety models within our Azure infrastructure. All customer data sent to Azure AI Content Safety remains within Azure AI Content Safety during processing and in the region you chose and is not transmitted to other regions.

Content Safety APIs don't support batch processing. Currently, Content Safety APIs achieve high concurrency in processing with generous default rate limits of 1000 per minute, depending on the model. While this is sufficient for most users, we're happy to increase rate limits for users that process or are looking to process higher volumes.

Currently, we can combine several classifications into a single API endpoint, enabling users to access both model outputs with a single task, but this API is in private preview, email us for allowlisting your subscription ID.

In our standard DSAT process, users need to create a ticket to submit cases to us, as emails won't initiate the triage process. Users can submit cases through the Azure portal, and customer support engineer can assist them in creating an ICM ticket.

Create a support request to submit a ticket to us:

• For Issue type, select Technical.
• For Subscription, select your subscription.
• For Service, select All services, then select Content Safety.
• For Summary, add a description of your issue.

The model requires time to iterate and retrain. If you create a support ticket through the Azure portal, a customer support engineer will assist you in tracking the status and updating you on progress. However, the product team encourages you to mitigate issues more quickly on your own using our recommended tools, such as the blocklist. Refer to Mitigate false results in Azure AI Content Safety.

No. Currently, text moderation tasks are limited to 10k-character submissions. You can, however, split longer text content into segments (for example, based on punctuation or spacing) and submit each segment as related tasks to the Content Safety API.

Yes, you can use the Azure AI Content Safety custom categories (standard) API to create categories, upload examples, and train the service to detect similar content. Custom categories are available for both text and image moderation.

The training process for custom categories begins with uploading data to your dedicated blob storage container. You then grant blob access to the Azure AI Content Safety resource. When training starts, your data is temporarily stored under strict control and is automatically deleted 48 hours later.

Trained models are secured to remain exclusive to your organization and the selected region, ensuring that they are not shared with other organizations, thus maintaining confidentiality and data integrity. Models are stored in Microsoft’s storage with multi-layer encryption. Customer-managed keys (CMK) are employed to enhance security.

To improve security and control over access, especially with concerns around API key usage, you could take a multi-layered approach such as disabling local authentication for key-based access: Disabling local authentication methods that rely on API keys can prevent unauthorized access and reduce the risk of credential exposure. This ensures that users and applications authenticate through more secure, managed identity solutions like Microsoft Entra ID.

You can also explore more granular access controls to support departmental or user-specific configurations, allowing customized permissions and secure access for different groups within the organization. This would ensure that each department or user group can securely manage and access their custom categories without compromising overall security.

Yes, users can request an increase in their custom categories quota for specific service deployments. Simply email us with the desired quota amount, and our team will review your request to accommodate your needs where possible.