Vector databases became suddenly prominent because of AI retrieval, and the explanations tend to be either hand-wavy or impenetrable. The underlying idea is straightforward.
What an embedding is
An embedding model converts text into a list of numbers — typically several hundred to a few thousand of them. That list is a position in a high-dimensional space.
The useful property: text with similar meaning ends up in similar positions. "Can I return this item" and "what is your refund policy" land close together, despite sharing almost no words. "The weather in March" lands somewhere else entirely.
A vector database stores these positions and answers one question efficiently: given this position, what are the nearest others?
That is it. The sophistication is in doing it fast across millions of entries.
Why normal search is not enough
Traditional keyword search matches words. If the document says "refund" and the user types "money back", keyword search finds nothing.
Semantic search matches meaning, so it finds the passage anyway. For AI retrieval — where you are looking for the chunk that answers a question nobody phrased in your terms — this is essential.
Why keyword search is still needed
The reverse is also true, and this is the part people miss when they first adopt vector search.
Semantic search is bad at exact terms. Product codes, clause numbers, version strings, names. Someone searching for "SKU-4471" wants the chunk containing that exact string, and semantic similarity is the wrong tool for it.
This is why production systems use hybrid search: run both, merge the results, re-rank. It consistently outperforms either alone on real business content, and the difference is large enough to matter.
Metadata filtering
Equally important and less discussed. Each stored vector carries metadata — document type, date, department, access level, source.
You filter on that before or during the similarity search. This is how you:
- Enforce permissions — a user never receives chunks from documents they cannot read. For internal knowledge systems this is the most important feature in the list.
- Exclude superseded content — so a 2023 policy is not quoted as current
- Scope a search to one product line or one client
A vector database without good metadata filtering is unsuitable for anything with access control, which is most business use.
The options
pgvector
An extension that adds vector support to PostgreSQL. You keep one database.
Why it is usually right: your relational data and your vectors live together, so filtering by anything in your schema is a normal SQL join. One system to back up, monitor and secure. No additional cost. Transactional consistency between your records and your embeddings.
When it is not: very large scale — tens of millions of vectors with demanding latency requirements — where a purpose-built engine pulls ahead.
For the large majority of business RAG systems, pgvector is sufficient and the operational simplicity is worth a great deal.
Pinecone
Fully managed, purpose-built. No infrastructure to run, scales without thought, good filtering.
Trade-offs: a separate system to keep in sync with your primary database, ongoing cost, and your embeddings live with a third party.
Qdrant
Open source, self-hostable or managed. Strong filtering, good performance, straightforward to operate.
Good when: you want a dedicated engine without a managed-service dependency.
Weaviate
Open source with built-in hybrid search and modules that handle embedding generation.
Good when: you want more of the retrieval pipeline handled for you.
Elasticsearch / OpenSearch
Mature keyword search engines that have added vector support.
Good when: you already run one. Hybrid search is native, and adding vectors to an existing deployment beats introducing a new system.
Chroma
Lightweight, developer-friendly, excellent for prototyping and local development.
Good when: building and testing. Most teams move to something else for production.
Choosing
A short decision path that covers most cases:
- Already using PostgreSQL? Start with pgvector. Revisit only if you hit a limit you can demonstrate.
- Already running Elasticsearch? Use its vector support.
- Need managed and do not want to run anything? Pinecone.
- Want dedicated and self-hosted? Qdrant.
- Prototyping? Chroma, then migrate.
The most common mistake is starting with a dedicated vector database for a dataset of a few thousand documents, which adds a second system to operate for no measurable benefit.
Things that matter more than the database
Worth stating plainly, because the database choice attracts disproportionate attention:
Chunking strategy affects retrieval quality more than the database does. How you split documents determines whether the right passage can be found at all.
Which embedding model you use matters more than where you store the output. Models differ meaningfully in quality and in cost, and in how well they handle domain-specific language.
Re-ranking retrieved candidates before passing them to the model produces a larger improvement than almost any infrastructure change.
Content hygiene — deciding what not to index — determines whether answers are consistent.
Get those right with pgvector and you will outperform a poorly tuned system on expensive infrastructure.
Operational notes
Re-indexing. When content changes, embeddings must be regenerated. Build this as a scheduled job or a trigger on content updates, not a manual task someone remembers.
Model changes. Switching embedding models means re-embedding everything, because vectors from different models are not comparable. Budget for it when you plan a model upgrade.
Dimensionality. Larger embeddings are more expressive and cost more to store and search. Most models offer a few sizes; the smaller ones are frequently good enough.
If you are planning a retrieval system and want a view on the architecture before committing, get in touch.
