Customize and suggest image search queries

Use this article to learn how to customize queries and suggest search terms to send to the Bing Image Search API.

Suggest search terms

If your app has a search box where search terms are entered, you can use the Bing Autosuggest API to improve the experience. The API can display suggested search terms in real time. The API returns suggested query strings based on partial search terms and Azure AI services.

Pivot the query

If Bing can segment the original search query, the returned Images object contains pivotSuggestions. Pivot suggestions can be displayed as optional search terms to the user. For example, if the original query was Microsoft Surface, Bing might segment the query into Microsoft and Surface and provide suggested pivots for each. These suggestions can be displayed as optional query terms to the user.

The following example shows the pivot suggestions for Microsoft Surface:

{
    "_type": "Images",
    "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=microsoft%20surface&FORM=OIIARP",
    "totalEstimatedMatches": 1000,
    "value": [...],
    "queryExpansions": [...],
    "pivotSuggestions": [{
        "pivot": "microsoft",
        "suggestions": [{
            "text": "Contoso Surface",
            "displayText": "Contoso",
            "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=OtterBox+Surface&FORM=IRQBPS",
            "searchLink": "https:\/\/api.cognitive.microsoft.com\/api\/v7\/images\/search?q=Contoso...",
                    "searchLink": "https:\/\/api.cognitive.microsoft.com\/api...",
            "thumbnail": {
                "thumbnailUrl": "https:\/\/tse3.mm.bing.net\/th?q=Contoso+Surface..."
            }
        },
        {
            "text": "Adatum Surface",
            "displayText": "Adatum",
            "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=Adatum+Surface&FORM=IRQBPS",
            "searchLink": "https:\/\/api.cognitive.microsoft.com\/api\/v7\/images\/search?q=...",
            "thumbnail": {
                "thumbnailUrl": "https:\/\/tse3.mm.bing.net\/th?q=Adatum+Surface&pid=Ap..."
            }
        },
        ...
        ]
    },
    {
        "pivot": "surface",
        "suggestions": [{
            "text": "Microsoft Surface4",
            "displayText": "Surface4",
            "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=Microsoft+Surface...",
            "searchLink": "https:\/\/api.cognitive.microsoft.com\/api\/v7\/images\/search?...",
            "thumbnail": {
                "thumbnailUrl": "https:\/\/tse4.mm.bing.net\/th?q=Microsoft..."
            }
        },
        {
            "text": "Microsoft Tablet",
            "displayText": "Tablet",
            "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=Microsoft+Tablet&FORM=IRQBPS",
            "searchLink": "https:\/\/api.cognitive.microsoft.com\/api\/v7\/images\/search?...",
            "thumbnail": {
                "thumbnailUrl": "https:\/\/tse3.mm.bing.net\/th?q=Microsoft+Tablet..."
            }
        },
        ...
    ],
    "nextOffsetAddCount": 0
}

The pivotSuggestions field contains the list of segments (pivots) that the original query was broken into. For each pivot, the response contains a list of Query objects that contain suggested queries. The text field contains the suggested query. The displayText field contains the term that replaces the pivot in the original query. An example is Release Date of Surface.

If the pivot query string is what the user is looking for, use the text and thumbnail fields to display the pivot query strings. Make the thumbnail and text clickable by using the webSearchUrl URL or the searchLink URL. Use webSearchUrl to send the user to the Bing search results. If you provide your own results page, use searchLink.

Expand the query

If Bing can expand the query to narrow the original search, the Images object contains the queryExpansions field. For example, if the query was Microsoft Surface, the expanded queries might be:

• Microsoft Surface Pro 3.
• Microsoft Surface RT.
• Microsoft Surface Phone.
• Microsoft Surface Hub.

The following example shows the expanded queries for Microsoft Surface.

{
    "_type": "Images",
    "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=microsoft%20surface...",
    "totalEstimatedMatches": 1000,
    "value": [...],
    "queryExpansions":  [{
        "text": "Microsoft Surface Pro 3",
        "displayText": "Pro 3",
        "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=Microsoft+Surface+Pro+3...",
        "searchLink": "https:\/\/api.cognitive.microsoft.com\/api\/v7\/images\/search?q=Microsoft...",
        "thumbnail": {
            "thumbnailUrl": "https:\/\/tse4.mm.bing.net\/th?q=Microsoft+Surface+Pro+3..."
        }
    },
    {
        "text": "Microsoft Surface RT",
        "displayText": "RT",
        "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=Microsoft+Surface+RT...",
        "searchLink": "https:\/\/api.cognitive.microsoft.com\/api\/v7\/images\/search?q=...",
        "thumbnail": {
            "thumbnailUrl": "https:\/\/tse4.mm.bing.net\/th?q=Microsoft+Surface+RT..."
        }
    },
    {
        "text": "Microsoft Surface Phone",
        "displayText": "Phone",
        "webSearchUrl": "https:\/\/www.bing.com\/images\/search?q=Microsoft+Surface+Phone",
        "searchLink": "https:\/\/api.cognitive.microsoft.com\/api\/v7\/images\/search?q=...",
        "thumbnail": {
            "thumbnailUrl": "https:\/\/tse4.mm.bing.net\/th?q=Microsoft+Surface+Phone..."
        }
    }],
    "pivotSuggestions": [...],
    "nextOffsetAddCount": 0
}

The queryExpansions field contains a list of Query objects. The text field contains the expanded query. The displayText field contains the expansion term. If the expanded query string is what the user is looking for, use the text and thumbnail fields to display the expanded query strings. Make the thumbnail and text clickable by using the webSearchUrl URL or the searchLink URL. Use webSearchUrl to send the user to the Bing search results. if you provide your own results page, use searchLink.

Throttling requests

The service and your subscription type determine the number of queries per second (QPS) that you can make. Make sure your application includes the logic to stay within your quota. If the QPS limit is met or exceeded, the request fails and an HTTP 429 status code is returned. The response includes the Retry-After header, which indicates how long you must wait before sending another request.

Denial-of-service versus throttling

The service makes a differentiation between a denial-of-service (DoS) attack and a QPS violation. If the service suspects a DoS attack, the request succeeds (HTTP status code is 200 OK). However, the body of the response is empty.

Next steps

If you haven't tried the Bing Image Search API before, try a quickstart. If you're looking for something more complex, try the tutorial to create a single-page web app.