Vector Databases

---

name: vector-databases description: Deep vector database workflow—embedding choice, index algorithms, recall/latency trade-offs, hybrid search, filtering, operational tuning, and cost. Use when selecting or optimizing Pinecone, Milvus, Qdrant, Weaviate, pgvector, OpenSearch kNN, etc.

---

Vector Databases (Deep Workflow)

Vector search is approximate nearest neighbor (ANN) at scale—not magic semantic understanding. Success requires embedding model alignment, index parameters, metadata filters, and evaluation against real queries.

When to Offer This Workflow

Trigger conditions:

Building RAG, similarity search, dedup, recommendations, anomaly clustering

Comparing managed vector DB vs pgvector vs search engine kNN

Recall issues, stale vectors, slow queries, or cost explosions

Initial offer:

Use six stages: (1) problem & metrics, (2) embeddings & schema, (3) index & parameters, (4) hybrid & filtering, (5) operations & cost, (6) evaluation & iteration. Confirm scale (vectors, QPS, dimension) and latency SLO.

---

Stage 1: Problem & Metrics

Goal: Define what “similar” means for the product—not only cosine similarity.

Questions

1. Query types: short keyword vs long paragraph? multilingual? 2. Precision vs recall preference: legal/medical may need high precision 3. Freshness: how often do vectors change? Real-time upserts? 4 Ground truth: any labeled relevant pairs for eval?

Metrics

Recall@k, MRR, nDCG when judgments exist; otherwise human spot checks + proxy tasks

Exit condition: Success metric and minimum acceptable recall/latency stated.

---

Stage 2: Embeddings & Schema

Goal: Stable embedding pipeline with versioning and metadata design.

Embeddings

Model choice: domain fit (code vs general text); dimension; distance metric (cosine vs dot vs L2)—match DB defaults

Chunking strategy upstream—bad chunks → bad retrieval regardless of DB

Schema

Payload/metadata per vector: doc_id, tenant_id, acl, source, timestamps

Multi-vector per doc (passages) vs single centroid—trade-offs

Versioning

Re-embed all on model change—plan downtime or dual-write period

Exit condition: ID strategy + metadata filter needs documented.

---

Stage 3: Index & Parameters

Goal: Pick index type and build params for data size and recall.

Common families (vendor-specific names)

HNSW: strong latency/recall; memory hungry; tunable efConstruction, M

IVF: better memory; needs training nlist; probe tuning

PQ/OPQ: compression—recall hit; good for huge scale

Tuning loop

Start defaults; sweep parameters with benchmark queries

Watch insert throughput during index build on large backfills

Exit condition: Benchmark results: p95 latency vs recall at fixed k.

---

Stage 4: Hybrid Search & Filtering

Goal: Combine vector similarity with structured constraints—most production needs this.

Patterns

Pre-filter metadata (tenant, date) before ANN when supported—verify filter selectivity

Hybrid: BM25 + vector with weighted fusion or rerank stage

Reranking: cross-encoder on top-k candidates—quality boost, latency cost

Pitfalls

Filtering that leaves too few candidates—empty results despite “similar” existing in other tenants

Exit condition: Query plan documented: ANN → filter → rerank (as applicable).

---

Stage 5: Operations & Cost

Goal: Reliable ingestion, monitoring, and predictable bills.

Ops

Upsert idempotency; delete tombstones for compliance

Backups, multi-region if needed—eventual consistency semantics per vendor

Capacity: memory per node vs sharding; replication factor

Cost

Managed per dimension × count; egress; query units—estimate from peak QPS

Exit condition: Runbook for reindex, scaling, and incident “search degraded.”

---

Stage 6: Evaluation & Iteration

Goal: Continuous improvement with labeled or proxy eval.

Loop

Golden query set updated when product changes

A/B embedding models or rerankers with guardrails on latency

Monitor click-through, thumbs, or human grading in RAG

Debugging bad retrieval

Chunk inspection, metadata leaks, wrong tenant filter, stale index

---

Final Review Checklist

[ ] Metrics and embedding/model versioning plan

[ ] Index family chosen with benchmark evidence

[ ] Hybrid/filter strategy matches product needs

[ ] Ops: upsert, delete, scaling, backup understood

[ ] Eval set and iteration process in place

Tips for Effective Guidance

Never promise “semantic search understands intent”—ground with eval.

pgvector vs specialized: trade-offs on scale, ops, features—state honestly.

Warn: high-cardinality filters + ANN can be slow—design metadata carefully.

Handling Deviations

Tiny corpus: brute force or simple index may suffice—avoid over-engineering.

Multimodal: separate embedding spaces or unified model—fusion strategy required.