DistributedCachingChatClient Class

Reference

Definition

Namespace:: Microsoft.Extensions.AI

Assembly:: Microsoft.Extensions.AI.dll

Package:: Microsoft.Extensions.AI v9.0.0-preview.9.24556.5

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

A delegating chat client that caches the results of completion calls, storing them as JSON in an IDistributedCache.

public ref class DistributedCachingChatClient : Microsoft::Extensions::AI::CachingChatClient

public class DistributedCachingChatClient : Microsoft.Extensions.AI.CachingChatClient

type DistributedCachingChatClient = class
    inherit CachingChatClient

Public Class DistributedCachingChatClient
Inherits CachingChatClient

Inheritance: Object

DelegatingChatClient

CachingChatClient
DistributedCachingChatClient

Remarks

The provided implementation of IChatClient is thread-safe for concurrent use so long as the employed IDistributedCache is similarly thread-safe for concurrent use.

Constructors

DistributedCachingChatClient(IChatClient, IDistributedCache)

Initializes a new instance of the DistributedCachingChatClient class.

Properties

CoalesceStreamingUpdates	Gets or sets a value indicating whether to coalesce streaming updates. (Inherited from CachingChatClient)
InnerClient	Gets the inner IChatClient. (Inherited from DelegatingChatClient)
JsonSerializerOptions	Gets or sets JSON serialization options to use when serializing cache data.
Metadata	Gets metadata that describes the IChatClient. (Inherited from DelegatingChatClient)

Methods

CompleteAsync(IList<ChatMessage>, ChatOptions, CancellationToken)	(Inherited from CachingChatClient)
CompleteStreamingAsync(IList<ChatMessage>, ChatOptions, CancellationToken)	(Inherited from CachingChatClient)
Dispose()	Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. (Inherited from DelegatingChatClient)
Dispose(Boolean)	Provides a mechanism for releasing unmanaged resources. (Inherited from DelegatingChatClient)
Equals(Object)	Determines whether the specified object is equal to the current object. (Inherited from Object)
GetCacheKey(Boolean, IList<ChatMessage>, ChatOptions)	Computes a cache key for the specified call parameters.
GetHashCode()	Serves as the default hash function. (Inherited from Object)
GetService<TService>(Object)	Asks the IChatClient for an object of type `TService`. (Inherited from DelegatingChatClient)
GetType()	Gets the Type of the current instance. (Inherited from Object)
MemberwiseClone()	Creates a shallow copy of the current Object. (Inherited from Object)
ReadCacheAsync(String, CancellationToken)	Returns a previously cached ChatCompletion, if available. This is used when there is a call to CompleteAsync(IList<ChatMessage>, ChatOptions, CancellationToken).
ReadCacheStreamingAsync(String, CancellationToken)	Returns a previously cached list of StreamingChatCompletionUpdate values, if available. This is used when there is a call to CompleteStreamingAsync(IList<ChatMessage>, ChatOptions, CancellationToken).
ToString()	Returns a string that represents the current object. (Inherited from Object)
WriteCacheAsync(String, ChatCompletion, CancellationToken)	Stores a ChatCompletion in the underlying cache. This is used when there is a call to CompleteAsync(IList<ChatMessage>, ChatOptions, CancellationToken).
WriteCacheStreamingAsync(String, IReadOnlyList<StreamingChatCompletionUpdate>, CancellationToken)	Stores a list of StreamingChatCompletionUpdate values in the underlying cache. This is used when there is a call to CompleteStreamingAsync(IList<ChatMessage>, ChatOptions, CancellationToken).

Extension Methods

CompleteAsync(IChatClient, String, ChatOptions, CancellationToken)	Sends a user chat text message to the model and returns the response messages.
CompleteStreamingAsync(IChatClient, String, ChatOptions, CancellationToken)	Sends a user chat text message to the model and streams the response messages.
CompleteAsync<T>(IChatClient, IList<ChatMessage>, ChatOptions, Nullable<Boolean>, CancellationToken)	Sends chat messages to the model, requesting a response matching the type `T`.
CompleteAsync<T>(IChatClient, IList<ChatMessage>, JsonSerializerOptions, ChatOptions, Nullable<Boolean>, CancellationToken)	Sends chat messages to the model, requesting a response matching the type `T`.
CompleteAsync<T>(IChatClient, String, ChatOptions, Nullable<Boolean>, CancellationToken)	Sends a user chat text message to the model, requesting a response matching the type `T`.
CompleteAsync<T>(IChatClient, String, JsonSerializerOptions, ChatOptions, Nullable<Boolean>, CancellationToken)	Sends a user chat text message to the model, requesting a response matching the type `T`.

Applies to

Share via