3.1.5.6 Handling a Chunked Response
To facilitate a positive connection between the server and client, the server uses the Transfer-Encoding header, as specified in section 2.2.3.2.5, with "chunked" transfer encoding, as specified in [RFC2616]. By using "chunked" transfer encoding, the server is able to return data to the client while the request is still being processed. This gives the client the expectation of getting an immediate acknowledgment of the request. If the client doesn't get this immediate acknowledgment, it can abandon the request and recover if the server's initial response is not received within a reasonable time.
After the initial immediate response, the client periodically receives a keep-alive response indicating that the server is still processing the request. The keep-alive response contains the PENDING meta-tag. As with the client's ability to quickly detect whether a request was received by the server, the client can abandon a request if no periodic keep-alive response is received within a reasonable amount of time. A secondary advantage of receiving the periodic keep-alive data is that the underlying HTTP connection remains open. This is particularly useful when there are devices in the middle that might be prone to timing out long-running requests.
The immediate response includes an X-PendingPeriod header, specified in section 2.2.3.3.5, to tell the client the number of milliseconds to be expected between keep-alive PENDING meta-tags in the response stream during the time a request is currently being processed on the server.
The client MUST be able to receive a "chunked" response for any request and know how to parse the chunked transfer encoding and work with the inner response stream (the response body) as if the chunked transfer encoding wasn't present. The initial response, plus the intermediate keep-alive transmissions, and the final response body are all part of the inner response stream. The use of "chunked" transfer encoding is a means to return an unknown amount of data to the client. The client MUST isolate the receiving of response "chunks" from the parsing and interpreting of the inner response stream. For more details about the "chunked" response, see section 3.2.5.2.
The inner response stream contains response meta-tags, as specified in section 2.2.7, to be used by the client in interpreting where the server is in the process of completing the request, the keep-alive messages, and finally the response body. For more details about the use of meta-tags, see section 3.2.5.2.