Why RAG fails without an acquisition strategy

"We'll just embed everything" is the sentence I hear right before a RAG product plateaus. It sounds like a plan. It's actually the absence of one. Embedding is the easy 20% that everyone demos. The data that goes into the embeddings is the hard 80% that decides whether the product is any good, and it's the part most teams treat as already solved.

I've built the acquisition side of this at scale, the extraction and ingestion pipelines feeding LLMs and smaller domain-specific models, end to end across the whole chain: crawl, chunk, embed, retrieve, generate, with cost controls at each step. What follows is where AI products actually break, in order of how often I see it.

Garbage in, confident garbage out

A retrieval system is only as good as the corpus behind it. Feed it stale, partial, or dirty data and it will retrieve stale, partial, dirty data and present it with total confidence, which is worse than retrieving nothing, because the user believes it. The model can't tell that the document it's citing is eighteen months out of date. It just answers.

So the quality of your AI product is mostly the quality of your acquisition, and acquisition is the part nobody on the ML team owns. The model gets the attention. The data gets a cron job someone wrote once and a quiet assumption that it's fine. It usually isn't.

Source selection is a product decision, not a scraping task

Which sources you pull from determines what your product can credibly answer. That's a product and strategy call, and it keeps getting delegated to whoever's writing the crawler, as if it were a technical detail. It isn't. Choosing to cover three authoritative sources deeply beats covering thirty shallow ones, and that choice shapes everything downstream, including what you're allowed to promise a customer. Decide it deliberately, at the top, before anyone writes a fetcher.

The extraction tier is where quality is won or lost

Raw HTML is not knowledge. Between the page and the embedding there's an extraction step, and that step is where you either get clean structured text or you get navigation menus, cookie banners, and boilerplate embedded right alongside the content you wanted.

This matters more than people expect, because junk in the chunk poisons retrieval. If your embeddings are full of footer text and "accept cookies" prompts, your similarity search surfaces noise, and no amount of prompt engineering downstream fixes a corpus that was dirty going in. Markdown-native, content-only extraction isn't a nicety. It's the difference between retrieval that works and retrieval that's subtly, expensively wrong in ways you won't catch in a demo and will catch in production.

But clean text is only the floor. Extraction isn't one technique, it's a decision you make per source. A clean, consistent source with stable markup wants a targeted DOM or rule-based parse: cheapest and most precise. An article-style page wants a readability pass that strips the chrome and emits markdown. The trouble is the sources that are messy, inconsistent, or dense with the entities you actually care about, because that's exactly where a rule-based parse quietly drops half of what matters and a readability pass flattens the structure you needed to keep.

That's where model-based extraction earns its cost. Named-entity recognition and layout-aware document models don't just separate content from boilerplate, they recognize what the content is: the people, organizations, dates, prices, and clauses, plus the table and section boundaries that give a chunk its shape. Run that and your chunks carry structured metadata, not just a wall of text. The payoff lands on retrieval, because you can filter and route by entity or field instead of asking vector similarity to guess. It's the difference between "find documents that mention Acme" and "find the renewal date in Acme's contract."

The strategy mistake is going to one extreme. Teams either regex everything, which breaks on the first layout change, or push every page through an LLM extractor, which is accurate and ruinous at volume. The right move is the same tiered discipline I use for anti-bot: cheap deterministic extraction first, escalate to model-based recognition only on the sources valuable or messy enough to deserve it. Extraction method is a cost-versus-fidelity decision, and like the rest of acquisition, it should be made on purpose.

Structure-first versus structure-later, again

I made this point in the log on why scraping projects die, and it's even more lethal for RAG.

The tempting path is to embed raw text now and figure out structure later. The problem is that "later" means re-running extraction and re-embedding across your entire corpus every time you realize you needed a field, a date, a source attribution, a chunk boundary that respects document structure. If you embedded blobs, you re-embed everything. If you extracted structure first, you adjust and move on. At small scale this is annoying. At large scale, re-embedding the whole corpus is a budget event you have to schedule and defend, and you'll do almost anything to avoid it, including shipping a worse product.

Decide your chunking and your structure before you embed at volume. Re-embedding is the most expensive way to learn this lesson and the most common.

Refresh cadence is the question nobody answers

Here's the one that separates a demo from a product. Your corpus was current the day you built it. What about next month? Sources change. Documents get superseded. Facts go stale. If your acquisition has no refresh strategy, your product degrades quietly from the day you launch, and the degradation is invisible until a user catches the model citing something that's no longer true.

Different data has different half-lives. Pricing data is stale in hours. A legal statute might be good for years until the day it very much isn't. Your refresh cadence has to match the half-life of each source, and that's an acquisition design decision that has to exist before launch, not a patch you apply after the first embarrassing answer.

Continuous coverage, the part nobody talks about

One-time scrapes feel like acquisition and aren't. A real ingestion pipeline keeps acquiring: new documents as they appear, changes as they happen, with monitoring so you know when a source goes dark instead of finding out from a user. This is operations, not a project, and it's the gap between "we built a RAG demo" and "we run a RAG product." Continuous coverage is unglamorous and it's most of the actual work, which is exactly why it gets skipped.

The uncomfortable summary

If you're building an AI product, your real moat probably isn't the model. You're likely using the same foundation models as everyone else. The moat is the data, and the data is an acquisition problem: which sources, extracted how cleanly, structured how deliberately, refreshed how often, monitored by whom. Teams pour their attention into the model layer because it's the interesting part, and let the data layer run on assumptions. Then they wonder why the product feels thin next to a competitor that took acquisition seriously.

"We'll just embed everything" isn't a strategy. The strategy is the five decisions above, made on purpose, before you scale. Get them right and the model layer mostly takes care of itself. Get them wrong and no model saves you.