--- title: "RAG in practice: when to use it (and when not to)" description: "RAG (Retrieval-Augmented Generation) explained without the hype: what it is, how it works, when it actually solves your problem and when it's over-engineering. The guide from someone who runs it in production." slug: rag-na-pratica-quando-usar lang: en date: 2026-07-22 updated: 2026-07-22 author: Lucas Silva category: deep-dive tags: [RAG, LLM, Retrieval, Vector Database, AI Engineering, Context] reading_time: 9 featured: false faq: - q: "What is RAG (Retrieval-Augmented Generation)?" a: "RAG is a technique where, before answering, the system searches an external knowledge base (documents, FAQs, company data) for relevant information and injects that content into the language model's context. The LLM then answers based on real, up-to-date data instead of just what it 'knows' from training — reducing hallucination and letting you use private knowledge." - q: "What's the difference between RAG and fine-tuning?" a: "Fine-tuning changes the model's weights, teaching it a behavior or style; it's expensive and static. RAG doesn't train anything: it retrieves information on the fly and injects it into the context, so the knowledge base can change at any time without retraining. To answer based on documents that change, RAG is almost always the right choice; fine-tuning is better suited to the style/format of the response." - q: "When should you NOT use RAG?" a: "When the information already fits in the model's context, when the answer doesn't depend on specific external knowledge, or when a simple function call to an API/database does the job better. RAG adds infrastructure (embeddings, a vector database, an ingestion pipeline) — if the problem doesn't require semantic search across many documents, it's over-engineering." --- RAG has become the default answer to everything. "How do I make the AI know about my data?" — "RAG!". "How do I reduce hallucination?" — "RAG!". And most of the time that's right — but RAG is also one of the most misused things around right now, thrown at problems that don't call for it. This post is the honest guide: what RAG really is, when it solves your problem, and when you're building expensive infrastructure for nothing. ## What RAG is, without the hype A language model knows what was in its training data and nothing beyond that. It doesn't know your internal documents, your catalog, your company's policy. And its context is limited — you can't just paste 10,000 pages into the conversation. **RAG (Retrieval-Augmented Generation) solves this by fetching, at question time, the relevant pieces of information and injecting them into the model's context.** The flow: 1. You break your knowledge base into pieces (chunks) and generate **embeddings** — numerical representations of the meaning of each piece. 2. You store those vectors in a **vector database**. 3. When a question comes in, you turn it into a vector and search for the most **semantically similar** pieces. 4. You inject those pieces into the prompt and the model answers based on them. The result: the LLM answers about *your* data, up to date, citing the source — without making things up. It's "giving the model the right cheat sheet before the exam." ## When RAG actually solves your problem RAG shines when you have **a lot of external knowledge, that changes, and that the model needs to consult**: - **Knowledge base / support:** hundreds of articles, manuals, FAQs. The agent finds the right passage and answers accurately. - **Documents that change:** policies, catalogs, prices. Since RAG doesn't train anything, you update the document and the answer changes instantly. - **Private knowledge:** your company's data that was never in any training set. - **The need to cite the source:** RAG knows where it pulled the answer from, which gives you traceability. If your problem is "the AI needs to answer based on a lot of my documents that change," RAG is the tool. ## When NOT to use RAG (the part nobody talks about) Here's the counterpoint that saves months of unnecessary work: **If the information fits in the context, you don't need RAG.** Modern models have huge context windows. If your knowledge is a 20-page document, put it in the prompt and be done. RAG for that is using a cannon to kill a fly. **If the answer comes from a system, use function calling, not RAG.** "What's the status of my order?" is not a semantic-search question — it's a database query. The right tool is [function calling](https://www.lucassilva.io/blog/function-calling-llm-que-age): the agent calls `get_order(id)` and gets the exact data. RAG would search for "documents similar to the question," which is wrong for structured data. **If you don't have many documents, RAG is over-engineering.** Embeddings, a vector database, an ingestion pipeline, reindexing — that's real infrastructure. It only pays off when the volume justifies it. The filter question: *does my answer depend on searching for meaning across many texts, or on looking up a specific piece of data?* If it's searching texts, RAG. If it's looking up data, function calling. ## The details that decide whether your RAG works A badly built RAG is worse than no RAG — it brings back the wrong piece with confidence. What separates a RAG that works: - **Smart chunking.** How you split the documents matters more than it seems. Chunks that are too large dilute relevance; too small lose context. Respect the document's structure (sections, paragraphs). - **Embedding quality.** The embedding model defines what "similar" means. A good embedding model is half the battle. - **Retrieve enough, not the maximum.** Injecting 50 chunks clogs the context and confuses the model. Retrieve the few most relevant ones. - **Reranking.** A second pass that reorders the results by real relevance greatly improves precision. - **Evaluate.** If you don't measure whether the RAG is bringing back the right passages, you don't know whether it works. Measure it. ## The product view RAG is not a goal — it's a means, like every AI technology. The mistake I see most is teams building an elaborate RAG pipeline before asking whether the problem calls for it. Often, what solves it is [function calling into a system](https://www.lucassilva.io/blog/function-calling-llm-que-age), or simply putting the document in the context. RAG comes in when the knowledge is large, textual, and changing — and then it's powerful. As always: the right question isn't "which trendy AI technique am I going to use?". It's "what's the problem, and what's the simplest tool that solves it?". Sometimes it's RAG. Often it isn't. Knowing the difference is what separates the people who ship from the people who just follow hype. --- *I build AI systems that solve the right problem with the right tool — RAG, agents, integration — in production. If you're not sure whether you need RAG, [let's talk](https://www.lucassilva.io).*