You Probably Don't Need a Vector Database Yet: pgvector in Practice

Every AI product-decision doc starts with the same slide, and someone on the call always asks the same thing: "which vector database are we picking?" It's a reasonable question and usually the wrong first one. For most teams shipping retrieval-augmented generation or semantic search, the honest answer in 2026 is that you don't need a dedicated vector database at all — you need a column on the Postgres you already run.

If you're picking infra for a RAG feature while every vendor deck screams "vector DB," here's why pgvector became genuinely good, where its ceiling sits, and the mistakes that make people wrongly conclude it doesn't scale.

What pgvector actually is (and why it stopped being a toy)

pgvector is a Postgres extension that adds a vector data type and approximate-nearest-neighbor indexes. You install it once, add a column to a table, and query it with normal SQL. No new service to run, no new SDK to learn, no separate cluster to keep alive at 3 a.m.

The "toy" critique once had teeth — early versions were single-threaded on index builds and weak on filtered search. Those complaints are out of date. Recent pgvector releases brought parallel HNSW index builds, iterative index scans that make filtered queries work well, and halfvec, a half-precision storage format that cuts vector storage roughly in half with almost no measurable recall loss on typical embedding models.

Every managed Postgres of any consequence ships pgvector today: AWS RDS and Aurora, Google Cloud SQL and AlloyDB, Azure Database for PostgreSQL, Supabase, Neon, Crunchy Bridge. If your app already uses Postgres somewhere, you almost certainly have access with one CREATE EXTENSION.

What "enough" actually looks like

The question isn't whether pgvector is fast — it's whether it's fast enough for what you're building. For most products, "enough" is a generous envelope.

With an HNSW index at reasonable settings, pgvector routinely serves sub-20ms similarity queries on around a million vectors while holding 95%+ recall. That is well inside the latency budget of a chatbot, an internal search box, or a support-ticket triage tool. If retrieval disappears into the noise of the model call that follows it, you have the performance you need.

Scale up further and pgvector still holds up for the workloads most teams actually have — tens of millions of vectors, moderate QPS, mixed read/write. The failure mode people expect — "Postgres will collapse the first time we go past a million rows" — mostly doesn't happen. What happens instead is that schema mistakes and missing indexes make queries slow, and the team blames pgvector rather than its configuration.

The killer feature: filtered queries

Almost no real RAG query is "give me the top-K nearest chunks globally." What you want is closer to:

Give me the top-K nearest chunks where tenant_id = 42, where document.status = 'published', where updated_at > last week, and where the user is allowed to see them.

This pattern — ANN plus SQL predicates — is where dedicated vector stores historically struggled. Some faked it by fetching an oversized candidate pool from the index and filtering after the fact, which quietly wrecked recall. pgvector's iterative scan approach handles it as first-class SQL: your filters and your similarity search live in the same query planner, so you write it the way you'd write any other Postgres query.

Your RAG query is a WHERE clause with a vector on the end. Postgres has been the world's best WHERE-clause engine for thirty years. Let it do its job.

The operational win nobody puts on a slide

Even if a dedicated vector store were 2× faster on a benchmark you'll never actually run, staying in Postgres wins on operational reality. One system to back up and monitor. One place for permissions and row-level security. One SQL dialect your team already knows. Transactions that keep relational and vector data consistent instead of two eventually-consistent stores. It's the same math as SQL vs NoSQL: the boring choice is usually the right one.

Your documents table gets an embedding vector(1024) column, and the embedding travels with the document through every join, every audit, every soft-delete. That's the difference between one clean system and two systems that drift.

pgvector vs a dedicated vector database, honestly

There are real trade-offs. Here's the shape of them, without the gloss.

Notice what the right column optimizes for: extreme scale, extreme latency, extreme QPS. Those are real problems — but problems you have later, if you're lucky, and by then you'll know which one is biting you.

When you actually should leave pgvector

Dedicated vector databases aren't bad; they exist because real workloads eventually outgrow the general-purpose tool. The mistake is picking one on day one for a problem you don't have yet. Legitimate reasons to migrate:

• Corpus above roughly 10–50 million vectors, especially with heavy write throughput or frequent re-embedding.
• Sub-10ms P99 latency at very high QPS — typically a public-facing search product with tight SLAs, not an internal tool.
• Multi-region writes where your Postgres topology can't reasonably support the pattern.
• Specialized ANN needs — exotic distance metrics, learned indexes, or GPU-accelerated recall that pgvector doesn't expose.

If any of those describe you, the migration is worth the operational cost. If none do, migrating early means paying that cost for benefits you can't measure.

The pattern we recommend

Start in pgvector. Keep a thin abstraction over your embedding writes and similarity queries — a couple of repository functions, not a framework — so the surface you'd have to swap later is small. Instrument P95 and P99 retrieval latency, recall against a small golden set, and index build times. Migrate the day a metric tells you to, not the day a vendor emails you.

The same posture applies to the whole RAG stack: buy simplicity now, buy specialization later, and never in the other order.

Where people go wrong (and when to call a pro)

Most "pgvector doesn't scale" stories are configuration stories in disguise. The recurring patterns:

The value of bringing in someone who's shipped a few of these isn't the SQL — it's the schema and the evaluation harness. What to embed, at what granularity, with which model, and how to measure whether retrieval is actually helping the LLM. That's what decides whether your RAG feature is good, not which store the vectors sit in. If you'd rather not learn that the hard way, that's the kind of thing we do.

Frequently asked questions