Microsoft Azure AI Cloud Developer Associate (AI-200) Training Course

Cosmos DB for NoSQL can store vectors alongside the documents they describe and search them with the same query language you use for everything else. That co-location is the argument for using it: your chunk text, its metadata, its tenant, and its embedding live in one item, so a similarity search returns everything the application needs in one round trip and filters apply naturally.

Enabling and declaring vectors

Vector search is enabled at the account level as a capability, and then configured per container in two places.

The vector embedding policy declares, for each vector property, three things: its path in the document, its dataType, its dimensions, and its distanceFunction. Dimensions must match the embedding model's output exactly — if the model produces 1,536-dimensional vectors and the policy says 768, inserts fail or results are meaningless. The distance function is typically cosine for text embeddings, with dotproduct and euclidean also available.

The indexing policy then declares a vector index on that path with a type. Both must be in place: the embedding policy describes the data, the index makes searching it efficient. A container with the embedding policy but no vector index will still answer similarity queries — by scanning every vector, at a cost that grows linearly and quickly becomes unacceptable.

Both policies are set at container creation for vector purposes. Treat them, like the partition key, as a design decision rather than something to adjust later.

Index types

Cosmos DB offers several vector index types and the choice is a scale decision.

flat stores vectors without an approximate structure and compares exhaustively. Exact results, no recall loss, and cost proportional to the number of vectors examined. It is appropriate for small collections, and it has a dimension ceiling.

quantizedFlat compresses vectors before the exhaustive comparison. It reduces storage and speeds the scan while introducing a small approximation. It sits between flat and diskANN and suits mid-sized collections.

diskANN builds a graph-based approximate index designed to live largely on disk, with a compressed representation in memory guiding the search. This is what makes very large vector collections practical without provisioning enormous memory. It is the default recommendation for production-scale vector workloads in Cosmos DB, and the type the exam associates with the service.

The property to understand about diskANN rather than memorize: it is designed for the case where the full index does not fit in memory, which is exactly the case that defeats a purely in-memory graph index like HNSW. That is why the same exam can recommend HNSW in PostgreSQL and diskANN in Cosmos DB without contradiction — they target different scale points and different storage assumptions.

Querying

Similarity search uses the VectorDistance system function, which takes two vectors and returns their distance under the container's configured function. The canonical query computes the distance between a stored vector and a query vector, orders by it, and takes the top results.

SELECT TOP 10 c.id, c.text, VectorDistance(c.embedding, @queryVector) AS score FROM c WHERE c.tenantId = @tenant ORDER BY VectorDistance(c.embedding, @queryVector)

Three things about that query deserve attention.

First, the ORDER BY VectorDistance(...) is what allows the engine to use the vector index. Computing the distance in a projection but ordering by something else does not.

Second, the WHERE c.tenantId = @tenant clause is doing enormous work if tenantId is the partition key. It scopes the search to one partition. Without it, the query fans out across every physical partition and performs a vector search in each, which is the single most common cause of runaway RU consumption on a vector workload.

Third, TOP matters. Vector search cost scales with how many results you ask for as well as how many vectors exist. Retrieving 100 candidates when the application uses 5 is waste that compounds on every request.

Hybrid search

Pure vector similarity misses exact-match cases: product codes, names, acronyms, and rare terms that embeddings blur. Hybrid search combines vector similarity with full-text relevance scoring and merges the rankings, commonly with reciprocal rank fusion.

Cosmos DB supports full-text indexing and scoring functions alongside vector search, so a hybrid query can rank by both signals. When a scenario complains that semantic search returns plausible but wrong results for queries containing specific identifiers, hybrid search is the intended fix rather than a different embedding model.

Cost control

Vector operations are expensive relative to document reads, and several levers reduce that cost.

• Scope by partition key. The largest single lever, as above.
• Exclude the vector path from the standard index. Cosmos DB indexes every property by default. A 1,536-element array indexed as an ordinary array is pure waste — it inflates write RU cost and storage without helping any query. Add the vector path to excludedPaths in the indexing policy while keeping the vector index on it. This is one of the highest-value optimizations available and appears in questions about unexpectedly high write costs on a chunk container.
• Store the minimum. Keep chunk text and embeddings in a container separate from full documents so vector queries do not drag large payloads through the engine.
• Consider dimension reduction. Some embedding models support shortened output dimensions with modest quality loss. Fewer dimensions means less storage, cheaper distance computation, and smaller indexes.
• Cache repeated queries. A semantic cache in Redis in front of the vector store removes duplicate work entirely.

Keeping embeddings current

When a source document changes, its chunks and their embeddings are stale. The natural mechanism is the change feed: a Function reads changes on the documents container, re-chunks and re-embeds, and writes to the chunks container. Because the change feed guarantees at-least-once delivery, the re-embedding step must be idempotent — write the chunk under a deterministic ID derived from the document and chunk index, so a repeated delivery overwrites rather than duplicates.

Also plan for model changes. If you switch embedding models, every existing vector is in a different space and cannot be compared against new query vectors. The migration is a full re-embed, and the practical approach is to write to a new container with the new policy and cut over, rather than mixing spaces in one collection.

What to verify

Supported index types, maximum dimensions per type, availability of hybrid scoring functions, and preview status of individual capabilities all change. Confirm the current state in the Azure Cosmos DB documentation before committing to a design that depends on a specific limit.