message delivery failed with errorCode=RequestEntityTooLarge although only 100KB

Hobday, Colin 45 Reputation points
2024-11-04T08:22:44.6066667+00:00

I have a trigger that subscribes to changes in Cosmos DB and pushes the document to Azure Event Grid. I am getting FailedEvents.

From the diagnostic log, it is clear that the reason is PayloadTooLarge, although the message is only about 150KB. (Event Grid is supposed to support messages up to 1MB).

outcome=PayloadTooLarge, latencyInMs=86, id=8e10d942-42f5-435c-a416-49af181c936e, outputEventSystemId=e549e75e-dc5d-484f-a839-67ef9fd3210a, state=Filtered, deliveryTime=11/4/2024 7:51:11 AM, deliveryCount=0, probationCount=0, deliverySchema=EventGridEvent, trackedSystemTopicState=, eventSubscriptionDeliverySchema=EventGridEvent, outputEventFields=InputEvent| EventSubscriptionId| DeliveryTime| State| Id| LastHttpStatusCode| DeliverySchema| LastDeliveryAttemptTime| SystemId, outputEventFieldCount=, dedicatedQueueDeliveryQueueId=, requestExpiration=1/1/0001 12:00:00 AM, delivered=False id=eqpt-1111, inputEventSystemId=5d72a537-62ec-4bcd-b24a-3433a53cec68 publishTime=11/4/2024 7:51:11 AM, eventTime=11/4/2024 7:51:11 AM, eventType=update, deliveryTime=11/4/2024 7:51:11 AM, filteringState=FilteredWithRpc, inputSchema=EventGridEvent, publisher=XXX-EVENT-TOPIC.SOUTHEASTASIA-1.EVENTGRID.AZURE.NET, size=131852, subject=cosmos/update, inputEventFields=Id| PublishTime| SerializedBody| EventType| Topic| Subject| FilteringHashCode| SystemId| Publisher| FilteringTopic| TopicCategory| DataVersion| MetadataVersion| InputSchema| EventTime| FilteringPolicy, inputEventFieldCount=16, type=HttpPost, subType=AzureDataFactory, supportsBatching=False, aadIntegration=False, managedIdentityType=None, urlPath=https://xxx.svc.datafactory.azure.com:4443/triggerevent/XXTrigger/triggerCosmosDbChange, deliveryResponse=PayloadTooLarge, errorCode=RequestEntityTooLarge, HttpRequestMessage: httpVersion=1.1, HttpResponseMessage: HttpVersion=1.1, StatusCode=RequestEntityTooLarge(RequestEntityTooLarge), StatusDescription=Payload Too Large, IsRedirected=False, RedirectUrl=,

The code is in Azure Function and uses output binding to send messages to Azure Event Grid.

@app.event_grid_output(
    arg_name="outputEvent",
    topic_endpoint_uri="EventGridTopicEndpointUri",  # in application settings
    topic_key_setting="EventGridTopicKeySetting",  # in application settings
)                     
def cosmosdb_trigger1(
    docs: func.DocumentList, 
    outputEvent: func.Out[func.EventGridOutputEvent]
):
    logging.info('Python CosmosDB triggered.')
    if docs:
        for document in docs:
            json_obj = document.to_json()
            logging.info(f"Content: {json_obj[:1000]}")
            logging.info(f"len:{len(json_obj)}")
            event_message = func.EventGridOutputEvent(
                id= document['id'],
                data= json_obj,
                subject="cosmos/update",
                event_type="update",
                event_time=datetime.datetime.utcnow(),
                data_version="1.0"
            )
            outputEvent.set(event_message)
            logging.info(f'sent {event_message}')
    else:
        logging.info("No documents found in the trigger.")

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,257 questions
Azure Event Grid
Azure Event Grid
An Azure event routing service designed for high availability, consistent performance, and dynamic scale.
408 questions
{count} votes

1 answer

Sort by: Most helpful
  1. LeelaRajeshSayana-MSFT 16,786 Reputation points
    2024-11-07T00:54:05.69+00:00

    Hi @Hobday, Colin I have tested the behavior locally and I could not replicate this issue. I could successfully send an event of size ~68kb from my Azure function to the Event Grid topic. As you pointed out, the default limit of Event size is 1MB. The only limitation I see on the documentation that has a 32kb limit is for an MQTT topic which says MQTTv5 total size of all user properties has to be 32Kb.

    Unless your custom topic is routing the events to an MQTT topic end point, I don't see why you would see this error for a data size greater than 32Kb.

    One alternate approach I would recommend to try is to use the EventGridPublisherClient in the Function App instead of output binding. You can find the reference code in the following GitHub repository

    If you continue to see the same behavior, please reach out to me through the comments below for a deeper investigation on this issue.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.