IResponseCacheProvider Interface
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Provides a way to get the IDistributedCache that caches the AI responses associated with a particular ScenarioRun.
public interface class IResponseCacheProviderpublic interface IResponseCacheProvidertype IResponseCacheProvider = interfacePublic Interface IResponseCacheProvider- Derived
Remarks
IResponseCacheProvider can be used to set up caching of AI-generated responses (both the AI responses under evaluation as well as the AI responses for the evaluations themselves). When caching is enabled, the AI responses associated with each ScenarioRun are stored in the IDistributedCache that is returned from this IResponseCacheProvider. So long as the inputs (such as the content included in the requests, the AI model being invoked etc.) remain unchanged, subsequent evaluations of the same ScenarioRun use the cached responses instead of invoking the AI model to generate new ones. Bypassing the AI model when the inputs remain unchanged results in faster execution at a lower cost.
Methods
| DeleteExpiredCacheEntriesAsync(CancellationToken) |
Deletes expired cache entries for all ScenarioRuns. |
| GetCacheAsync(String, String, CancellationToken) |
Returns an IDistributedCache that caches the AI responses associated with a particular ScenarioRun. |
| ResetAsync(CancellationToken) |
Deletes cached AI responses for all ScenarioRuns. |
