Mintlify just deleted their vector search RAG pipeline and replaced it with cat, ls, and grep. Their agent performance went up. Their compute costs dropped to near zero. And the Hacker News thread hit 217 upvotes because everyone's been thinking the same thing: RAG is broken.
The Problem Nobody Wanted to Admit
Here's what happened across the industry over the past two years: someone said "RAG" and everyone heard "vector embeddings." Teams spent months building chunking pipelines, tuning embedding models, tuning similarity thresholds, reranking results — all to answer questions about documentation that already had structure.
Mintlify calculated the cost of this approach: $70,000 per year just to let an AI read their static docs. For 850,000 conversations a month, that's embedding generation, vector store queries, reranking API calls — all to answer "how do I install this?" from docs that have a table of contents.
The dirty secret? For structured knowledge — docs, codebases, configs, wikis — vector search is the wrong tool. You're converting human-readable text into numbers, searching the numbers, then converting back. It's like translating a book into French to search for an English word.
A new research paper on Neuro-RIT confirms what practitioners suspected: even improved RAG pipelines struggle with noisy context. The precision drops aren't marginal — they're structural.
The Solution: Virtual Filesystems for Agents
Mintlify built ChromaFs — a virtual filesystem that lets their AI agent navigate documentation the same way a developer does. ls to list directories. cat to read files. grep to search. No embeddings. No vector store. No reranking.
The key insight is simple: directory hierarchies are already human-curated knowledge graphs. When someone organizes docs into /guides/, /api/, /changelog/, they've already done the retrieval work. You don't need cosine similarity to find the installation guide — it's in /guides/installation.md.
Here's how the approach works:
- Agents use filesystem tools —
ls,cat,grep,find— to navigate structured content - Directory structure provides context — the path itself is metadata
- File boundaries replace chunking — no arbitrary token splits, no overlap tuning
- Zero marginal compute — reading a file costs nothing compared to embedding a query
The agent doesn't need to "understand" the semantic relationship between "setup" and "installation." It just needs to look in the right folder.
Benchmarks: Where Filesystem Wins (and Where It Doesn't)
Let's be honest about where each approach works:
Filesystem-based retrieval wins when:
- Content has clear structure (docs, code, configs)
- Answers are contained in identifiable files or sections
- You need deterministic, reproducible results
- Cost matters — marginal compute is near zero
Vector RAG still wins when:
- Content is unstructured (emails, chat logs, transcripts)
- You need cross-document synthesis ("compare what A said about X with what B said about Y")
- Queries are fuzzy or conceptual ("explain our pricing philosophy")
- Content spans multiple languages with different embedding spaces
The real numbers:
- Mintlify: eliminated $70K/year in embedding compute costs
- Agent response accuracy improved for structured doc queries
- Response latency dropped — no vector store round-trip
- New research shows even optimized RAG (Neuro-RIT precision-driven alignment) still loses to structured retrieval for structured content
The honest caveat: This isn't "RAG is dead for everything." It's "RAG-as-vector-search is dead for structured knowledge." For unstructured, messy, cross-domain retrieval — hybrid approaches still matter.
The Business Impact: Stop Paying for What You Don't Need
Let's do the math most teams never do.
If you're running a documentation chatbot, customer support agent, or internal knowledge assistant, your costs break down roughly like this:
- Embedding generation: $0.02-0.10 per 1K queries (depending on model)
- Vector store: $50-500/month depending on scale
- Reranking API: $0.001-0.01 per query
- Engineering time: 2-4 weeks to build and tune the pipeline
For Mintlify at 850K monthly conversations, that's $70K/year. For a smaller SaaS doing 50K monthly queries, it's still $5-10K/year in infrastructure alone — plus the engineering debt of maintaining embedding pipelines.
The filesystem approach flips the economics. Your cost per query is the cost of reading a file from disk. At any scale, that rounds to zero.
The framework is simple: Before building a RAG pipeline, ask one question — Is my content already structured? If yes, start with a virtual filesystem. You can always add vector search later for the fuzzy stuff. But you can't un-build an over-engineered pipeline without rewriting half your codebase.
The Takeaway
The industry over-invested in vector search because it felt sophisticated. Embeddings are sexy. Cosine similarity is elegant. Building a chunking pipeline with overlap windows and reranking sounds impressive in architecture reviews.
But cat /docs/api/authentication.md answers "how do I authenticate?" faster, cheaper, and more accurately than any embedding pipeline ever will. The best retrieval system is the one that respects how humans already organize information.
RAG isn't dead. But the reflex to reach for vector search first? That should be.