Let me start with a confession that will probably annoy half of AI LinkedIn: I run two retrieval systems in production, every day, for months now, and neither of them has a vector DB. No chunking. No embeddings. No Pinecone. Retrieval is grep plus reading structured files plus a long-context model doing the fine filtering. And it works better than classic RAG would for these cases.
The thesis that became dogma around 2023 was simple: the reader (the LLM) is small and expensive, so the retriever needs to be smart. Embed everything, index it, search by similarity, send only the top-K. It was the right architecture for that moment. The problem is that a lot of people keep paying the price of that architecture without noticing the bottleneck has already flipped.
The bottleneck flipped (and almost nobody updated their mental model)
In 2026 the reader is the smartest component at the table, and the window fits an entire book. When the reader became brilliant and cheap with prompt caching, the retriever could go back to being dumb. And the dumbest, most transparent retriever there is happens to be grep.
This is not theory. Today’s most advanced agent tools went exactly in that direction: instead of a vector index, file-based text memory, an index file of pointers, topic files holding the facts, and direct text search to consolidate. The memory of a state-of-the-art agent is, quite often, a subagent grepping through text files. That should reorganize your intuition about what “state of the art” means.
What I measured in my own setup
Here come the numbers that matter, because I measured them, I did not guess. My second brain (an Obsidian vault) has 3,323 curated notes across literature and atomic notes. One of my knowledge agents runs over 1,192 files of structured base. In both, retrieval is grep plus reading the whole file plus long context. Zero embeddings. Zero vector DB. In daily production.
Why does this beat classic RAG in these cases? Three practical reasons.
First, transparent failure. When grep finds nothing, it tells me “the word is not here”. When the vector DB gets it wrong, it confidently returns the wrong neighbor, and you find out late. Debugging a false neighbor in production costs more than any server.
Second, no index going stale. Edited a note just now? It is already in the search, because the search reads the disk. There is no reindexing, no drift between what is in the database and what is in the file.
Third, the cost math is a trap. “RAG saves tokens” ignores that engineers are expensive and production bugs are more expensive. For a small or medium base, setting up and maintaining a Pinecone eats more than you save. In the Brazilian context this weighs twice as much: lean teams, tight cloud budgets, and every extra piece of infra is one more piece for someone to be on call for.
When the vector DB STILL wins (being honest)
Now the part the “RAG is dead” heralds leave out. The vector DB did not die. It just stopped being the mandatory default.
LaRA, the ICML 2025 benchmark, was direct: there is no silver bullet, the choice depends on the model, the corpus size and the task type. And the quantitative verdict is clear. RAG dominates when the corpus goes beyond a few million tokens, when fuzzy semantic search truly matters (the user describes by synonym, not by the exact word), when you need freshness at scale and source attribution. Long context in those scenarios gets 8 to 82 times more expensive than retrieving only what is relevant (Li et al., 2024, arXiv:2407.16833).
Add to that the old “Lost in the Middle” (Liu, 2023): model accuracy draws a U, high at the beginning and end of the context, dropping more than 30% in the middle. The 2026 models improved a lot on this, but the yardstick still holds: stuffing 1M tokens is not free in quality. For a giant corpus and real semantic search, retrieving well still beats dumping everything.
The test question I use: does the right answer fit in a few files I can find by keyword, or is it spread across thousands of documents I can only find by meaning? First case, grep plus long context. Second case, it is time for a vector DB (or GraphRAG, if the question is global over the entire corpus).
The verdict
Start simple. Raw documents on disk, a fast lexical filter, load the whole file, let the model do the filtering. Add embeddings only when you miss them, with real data showing where lexical search failed. Earlier in my career I tried the complex version first and regretted it. Today my yardstick is the opposite: the sophisticated stack has to justify its own existence, not the other way around.
The vector DB is an excellent tool for the right problem. The mistake is treating “RAG” and “vector DB” as synonyms and installing Pinecone reflexively before trying grep.