Using the Azure AI Search Vector Store connector (Preview)
Warning
The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release.
Overview
The Azure AI Search Vector Store connector can be used to access and manage data in Azure AI Search. The connector has the following characteristics.
Feature Area | Support |
---|---|
Collection maps to | Azure AI Search Index |
Supported key property types | string |
Supported data property types |
|
Supported vector property types | ReadOnlyMemory<float> |
Supported index types |
|
Supported distance functions |
|
Supports multiple vectors in a record | Yes |
IsFilterable supported? | Yes |
IsFullTextSearchable supported? | Yes |
StoragePropertyName supported? | No, use JsonSerializerOptions and JsonPropertyNameAttribute instead. See here for more info. |
Limitations
Notable Azure AI Search connector functionality limitations.
Feature Area | Workaround |
---|---|
Configuring full text search analyzers during collection creation is not supported. | Use the Azure AI Search Client SDK directly for collection creation |
Getting started
Add the Azure AI Search Vector Store connector NuGet package to your project.
dotnet add package Microsoft.SemanticKernel.Connectors.AzureAISearch --prerelease
You can add the vector store to the dependency injection container available on the KernelBuilder
or to the IServiceCollection
dependency injection container using extension methods provided by Semantic Kernel.
using Azure;
using Microsoft.SemanticKernel;
// Using Kernel Builder.
var kernelBuilder = Kernel
.CreateBuilder()
.AddAzureAISearchVectorStore(new Uri(azureAISearchUri), new AzureKeyCredential(secret));
using Azure;
using Microsoft.SemanticKernel;
// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddAzureAISearchVectorStore(new Uri(azureAISearchUri), new AzureKeyCredential(secret));
Extension methods that take no parameters are also provided. These require an instance of the Azure AI Search SearchIndexClient
to be separately registered with the dependency injection container.
using Azure;
using Azure.Search.Documents.Indexes;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
// Using Kernel Builder.
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.Services.AddSingleton<SearchIndexClient>(
sp => new SearchIndexClient(
new Uri(azureAISearchUri),
new AzureKeyCredential(secret)));
kernelBuilder.AddAzureAISearchVectorStore();
using Azure;
using Azure.Search.Documents.Indexes;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton<SearchIndexClient>(
sp => new SearchIndexClient(
new Uri(azureAISearchUri),
new AzureKeyCredential(secret)));
builder.Services.AddAzureAISearchVectorStore();
You can construct an Azure AI Search Vector Store instance directly.
using Azure;
using Azure.Search.Documents.Indexes;
using Microsoft.SemanticKernel.Connectors.AzureAISearch;
var vectorStore = new AzureAISearchVectorStore(
new SearchIndexClient(
new Uri(azureAISearchUri),
new AzureKeyCredential(secret)));
It is possible to construct a direct reference to a named collection.
using Azure;
using Azure.Search.Documents.Indexes;
using Microsoft.SemanticKernel.Connectors.AzureAISearch;
var collection = new AzureAISearchVectorStoreRecordCollection<Hotel>(
new SearchIndexClient(new Uri(azureAISearchUri), new AzureKeyCredential(secret)),
"skhotels");
Getting started
Install semantic kernel with the azure extras, which includes the Azure AI Search SDK.
pip install semantic-kernel[azure]
You can then create a vector store instance using the AzureAISearchStore
class, this will use the environment variables AZURE_AI_SEARCH_ENDPOINT
and AZURE_AI_SEARCH_API_KEY
to connect to the Azure AI Search instance, those values can also be supplied directly. You can also supply azure credentials or token credentials instead of a API key.
from semantic_kernel.connectors.memory.azure_ai_search import AzureAISearchStore
vector_store = AzureAISearchStore()
You can also create the vector store with your own instance of the azure search client.
from azure.search.documents.indexes import SearchIndexClient
from semantic_kernel.connectors.memory.azure_ai_search import AzureAISearchStore
search_client = SearchIndexClient(endpoint="https://<your-search-service-name>.search.windows.net", credential="<your-search-service-key>")
vector_store = AzureAISearchStore(search_index_client=search_client)
You can also create a collection directly.
from semantic_kernel.connectors.memory.azure_ai_search import AzureAISearchCollection
collection = AzureAISearchCollection(collection_name="skhotels", data_model_type=hotel)
Serialization
Since the Azure AI Search connector needs a simple dict with the fields corresponding to the index as the input, the serialization is quite easy, just return a dict with the values with the keys corresponding to the index fields, the built-in step from dict to the the store model is a straight passthrough of the created dict.
For more details on this concept see the serialization documentation.
Getting started
Include the latest version of the Semantic Kernel Azure AI Search data connector in your Maven project by adding the following dependency to your pom.xml
:
<dependency>
<groupId>com.microsoft.semantic-kernel</groupId>
<artifactId>semantickernel-data-azureaisearch</artifactId>
<version>[LATEST]</version>
</dependency>
You can then create a vector store instance using the AzureAISearchVectorStore
class, having the AzureAISearch client as a parameter.
import com.azure.core.credential.AzureKeyCredential;
import com.azure.search.documents.indexes.SearchIndexClientBuilder;
import com.microsoft.semantickernel.data.azureaisearch.AzureAISearchVectorStore;
import com.microsoft.semantickernel.data.azureaisearch.AzureAISearchVectorStoreOptions;
import com.microsoft.semantickernel.data.azureaisearch.AzureAISearchVectorStoreRecordCollection;
import com.microsoft.semantickernel.data.azureaisearch.AzureAISearchVectorStoreRecordCollectionOptions;
public class Main {
public static void main(String[] args) {
// Build the Azure AI Search client
var searchClient = new SearchIndexClientBuilder()
.endpoint("https://<your-search-service-name>.search.windows.net")
.credential(new AzureKeyCredential("<your-search-service-key>"))
.buildAsyncClient();
// Build an Azure AI Search Vector Store
var vectorStore = AzureAISearchVectorStore.builder()
.withSearchIndexAsyncClient(searchClient)
.withOptions(new AzureAISearchVectorStoreOptions())
.build();
}
}
You can also create a collection directly.
var collection = new AzureAISearchVectorStoreRecordCollection<>(searchClient, "skhotels",
AzureAISearchVectorStoreRecordCollectionOptions.<Hotel>builder()
.withRecordClass(Hotel.class)
.build());
Data mapping
The default mapper used by the Azure AI Search connector when mapping data from the data model to storage is the one provided by the Azure AI Search SDK.
This mapper does a direct conversion of the list of properties on the data model to the fields in Azure AI Search and uses System.Text.Json.JsonSerializer
to convert to the storage schema. This means that usage of the JsonPropertyNameAttribute
is supported if a different storage name to the
data model property name is required.
It is also possible to use a custom JsonSerializerOptions
instance with a customized property naming policy. To enable this, the JsonSerializerOptions
must be passed to both the SearchIndexClient
and the AzureAISearchVectorStoreRecordCollection
on construction.
var jsonSerializerOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseUpper };
var collection = new AzureAISearchVectorStoreRecordCollection<Hotel>(
new SearchIndexClient(
new Uri(azureAISearchUri),
new AzureKeyCredential(secret),
new() { Serializer = new JsonObjectSerializer(jsonSerializerOptions) }),
"skhotels",
new() { JsonSerializerOptions = jsonSerializerOptions });