What is RAG (retrieval-augmented generation)? A guide for business readers
Easy Insight Team ·
Retrieval-augmented generation (RAG) is a way of making an AI assistant answer from your own content. When someone asks a question, the system first searches your documents for the most relevant passages, then hands those passages to a language model with the question, so the answer is based on your material rather than the model's general training.
It is the pattern behind most "chat with your documents" tools, and the main reason a business assistant can quote your returns policy instead of guessing it.
How does RAG work?
The UK Government's AI Insights guide to RAG systems (updated August 2026) breaks the name into its three parts:
- Retrieval. Your documents are split into small pieces ("chunks"), each turned into a numerical representation of its meaning called an embedding, and stored in a searchable index. A question is matched against that index by meaning, not just keywords.
- Augmentation. The best-matching chunks are added to the prompt alongside the question.
- Generation. A large language model writes the answer from that supplied context.
The guide stresses that RAG does not change the model itself. It supplements it with outside knowledge at the moment of asking. Microsoft Learn makes the same division of labour clear: the search layer finds content, and only the model generates the answer.
Why use RAG instead of training a model on your data?
Three practical reasons, all drawn from the GOV.UK guide:
- It stays current. Update a document and the next answer uses it. No retraining.
- It is cheaper. Retraining or fine-tuning a model is expensive and slow; indexing documents is not.
- You can take things out. Removing a document from the index removes it from the answers. The guide notes that making a trained model "unlearn" data is expensive.
That last point matters under UK GDPR. If a customer's details should no longer be used, deleting them from a source system is far simpler than reworking a model. Our guide to GDPR and AI for UK SMEs covers the wider obligations.
What does a worked example look like?
An illustrative case: a 30-person UK wholesaler wants staff to stop emailing the operations manager with the same delivery and returns questions.
- Sources. The delivery policy, returns policy and carrier price list, all in one SharePoint folder that has a named owner.
- Question. A sales rep asks, "Can a customer in Jersey return opened stock?"
- Retrieval. The system finds the returns policy paragraph on opened goods and the delivery policy section on the Channel Islands.
- Answer. The model replies from those two passages, cites both, and is instructed to say "not found, ask operations" if the documents do not cover it.
The quality of step 3 decides everything. If the returns policy exists in three versions, the assistant may quote the wrong one. In our view, tidying the sources is usually the largest job in a RAG project, ahead of any choice of model or prompt engineering.
Where does RAG go wrong?
- Poor retrieval. The GOV.UK guide names retrieval quality as the first challenge: badly split or out-of-date documents produce confident, wrong answers.
- Permissions. Microsoft's example is finance data that only the finance team should see, even when an executive asks the chatbot. A RAG system must respect who is allowed to read what.
- Leftover hallucination. Grounding reduces invented answers; it does not end them. Outputs that matter still need checking.
On the permissions point, Microsoft's Copilot Studio knowledge sources documentation (as of November 2026) says SharePoint and Dataverse sources use the asking user's own Microsoft Entra ID sign-in, so the agent only surfaces content that person can already access. For many SMEs that low-code route is the right first step. A bespoke build, with costs set out in our AI chatbot development cost guide, earns its place when you need other sources, tighter control or integrations.
We design and build grounded assistants in our AI development work, part of our wider AI practice.
Frequently asked questions
What does RAG stand for?
Retrieval-augmented generation. The system retrieves relevant passages from your own content, adds them to the question it sends to a language model, and the model generates an answer from them.
Is RAG the same as training an AI on my data?
No. RAG leaves the model unchanged and looks your documents up each time a question is asked. Training or fine-tuning changes the model itself, costs more, and makes it much harder to remove information later.
Does RAG stop AI hallucinations?
It reduces them but does not remove them. If retrieval finds the wrong passage, or the right document is out of date, the model can still give a confident wrong answer. Citations and a "not found" rule make errors easier to catch.
Can a small business use RAG without building anything?
Often, yes. Low-code tools such as Microsoft Copilot Studio can point an assistant at SharePoint, uploaded files or your website and handle the retrieval for you. A bespoke build makes sense when you need tighter control, unusual sources or integrations.
Easy Insight is a UK consultancy for AI, web, apps and data — senior specialists only, no juniors.
Next step
Wondering where AI would actually pay in your business?
An AI strategy and readiness review (from £2,500) tells you where AI pays, where it doesn't, and what to leave alone. The price is fixed in writing before we start, and the advice comes from a team that runs AI in its own products.

