StreamingChatCompletionUpdate Class

Definition

public ref class StreamingChatCompletionUpdate

public class StreamingChatCompletionUpdate

type StreamingChatCompletionUpdate = class

Public Class StreamingChatCompletionUpdate

Inheritance

Object

StreamingChatCompletionUpdate

Remarks

Conceptually, this combines the roles of ChatCompletion and ChatMessage in streaming output. For ease of consumption, it also flattens the nested structure you see on streaming chunks in some AI service, so instead of a dictionary of choices, each update represents a single choice (and hence has its own role, choice ID, etc.).

StreamingChatCompletionUpdate is so named because it represents streaming updates to a single chat completion. As such, it is considered erroneous for multiple updates that are part of the same completion to contain competing values. For example, some updates that are part of the same completion may have a nullRole value, and others may have a non-null value, but all of those with a non-null value must have the same value (e.g. Assistant. It should never be the case, for example, that one StreamingChatCompletionUpdate in a completion has a role of Assistant while another has a role of "AI".

The relationship between ChatCompletion and StreamingChatCompletionUpdate is codified in the ToChatCompletionAsync(IAsyncEnumerable<StreamingChatCompletionUpdate>, Boolean, CancellationToken) and ToStreamingChatCompletionUpdates(), which enable bidirectional conversions between the two. Note, however, that the conversion may be slightly lossy, for example if multiple updates all have different RawRepresentation objects whereas there's only one slot for such an object available in RawRepresentation.

Constructors

Properties

Methods

Applies to