# What is RAG? Retrieval-augmented generation in plain words

> RAG lets an AI answer from your own documents: it finds the passages that fit a question and answers from them. How it works, and where it goes wrong.

Canonical: https://heykiko.io/blog/what-is-rag
Last updated: 2026-11-05

AI agent guides · by The HeyKiko team · published 5 November 2026 · 7 min read

RAG, short for retrieval-augmented generation, is how an AI answers from your own documents instead of from memory. Before it replies, it searches your pages and files for the passages that fit the question, then writes its answer from those passages, and can say where each fact came from. It's the technique behind almost every chatbot that's “trained on your website”.

## An open-book exam

A language model on its own answers like a student in a closed-book exam: from what it remembers, which may be out of date, and with a guess where memory runs out. RAG makes it an open-book exam. For every question:

1. **Understand the question**: Work out what's really being asked, including follow-ups like “and for dining sets?”.
2. **Look it up**: Search your documents for the passages most likely to hold the answer.
3. **Pick the best**: Keep the few passages that fit best and drop the rest.
4. **Answer from them**: Write the reply using those passages, and name the sources.

The idea comes from a 2020 research paper that paired a language model with a searchable index of text, so its answers could draw on facts it was never trained on. [Lewis et al.: retrieval-augmented generation](https://arxiv.org/abs/2005.11401)

## Why not just train the AI on my data?

You could fine-tune a model on your documents, but for a business's facts RAG is usually the better fit. AWS sums up the advantages: it's cheaper than retraining, answers can be kept current, they can cite sources, and you control which sources are used and who may see them. [AWS: what is RAG?](https://aws.amazon.com/what-is/retrieval-augmented-generation/)

Retrieval-augmented generation compared with fine-tuning a model on your documents

|  | RAG | Fine-tuning |
| --- | --- | --- |
| A price changes | Update the page; the next answer uses it | Retrain the model |
| Where an answer came from | It can name the page or file | It can't say |
| Fixing a wrong answer | Fix the passage it used | Add examples and retrain |
| Different readers | Search only what each person may see | Everyone gets the same model |
| Cost to start | Low | Higher |

## The parts, in plain words

- Chunking: long pages are cut into short passages, so an answer can use just the part that matters.
- Embeddings: each passage is turned into a list of numbers that captures its meaning, so “refund” and “money back” end up close together.
- Vector search: the question is turned into numbers the same way, and the passages with the closest numbers are found. A vector database is where those numbers are kept.
- Ranking: a second look at the passages found, to put the ones that really answer the question first.
- The prompt: the chosen passages are handed to the language model with instructions such as “answer only from these”.
- Citations: the reply keeps track of which passages it used, so it can show its sources.

## Where RAG goes wrong

Common ways a RAG chatbot gives a wrong answer, and the fix for each

| What happens | Why | The fix |
| --- | --- | --- |
| It invents an answer | Nothing in your documents covers the question | Add the answer, and let it say it doesn't know |
| It quotes an old price | An out-of-date page was found | Remove or update the old page |
| It misses part of the answer | The facts are on two pages and only one was found | Put related facts together, or let it combine sources |
| It applies the rule, not the exception | The exception is far from the rule | Write the exception next to the rule |
| It answers the wrong question | A short follow-up was read on its own | Use a chatbot that rewrites follow-ups with the conversation |

Our guide to [stopping a chatbot from making things up](https://heykiko.io/blog/how-to-stop-a-chatbot-making-things-up) covers the fixes in more depth.

## How HeyKiko does it

- It plans each question first, rewriting a follow-up with the conversation and noticing when the customer is shopping or wants to book.
- It searches several ways at once, ranks what it finds, and hands the model about a dozen short passages.
- It searches only the knowledge the person's role may see.
- Live facts, such as stock or an order's status, come from your database at the moment they're asked, not from the index.
- If the search is ever unavailable, the agent says it can't check facts right now rather than answering from memory; bookings and cases still work.
- Your content is sent to the AI provider only to index it and to answer; it isn't used to train models.

In the [Playground](https://heykiko.io/features/test-your-agent) you can open “Why this answer” to see how confident it was and the passages it used. [Training on your website and files](https://heykiko.io/features/train-on-your-website) shows what it can read, and you can try it free for 14 days.

## Sources

Checked on 5 November 2026. We make HeyKiko, which this guide uses as its example.

- [Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)](https://arxiv.org/abs/2005.11401)
- [AWS: what is retrieval-augmented generation?](https://aws.amazon.com/what-is/retrieval-augmented-generation/)

## Questions people ask

### What is RAG in simple terms?

Retrieval-augmented generation: before an AI answers, it searches your documents for the passages that fit the question and writes its answer from them, like an open-book exam instead of answering from memory.

### What is the difference between RAG and fine-tuning?

RAG looks facts up in your documents for each question, so a change to a page shows up in the next answer and sources can be named. Fine-tuning changes the model itself, so updating a fact means training it again.

### Does RAG stop AI hallucinations?

It reduces them, because answers come from your documents, but it doesn't remove them. If nothing in your documents covers a question, the model can still guess unless it's told to say it doesn't know.

### What is a vector database?

A store for passages turned into lists of numbers that capture their meaning, so the passages closest in meaning to a question can be found quickly.

### What is chunking?

Cutting long documents into short passages before they're stored, so a search can return just the part that answers the question.

### Is my data used to train the AI with RAG?

Not by RAG itself: your documents are searched, not used to change the model. In HeyKiko, your content is sent to the AI provider only to index it and to answer, and isn't used to train models.

### Do I need to understand RAG to use a chatbot?

No. What matters is that your documents are current and complete, and that you test the chatbot with real questions.

## Related

- [How to train a chatbot on your website](https://heykiko.io/blog/how-to-train-a-chatbot-on-your-website)
- [How to stop a chatbot from making things up](https://heykiko.io/blog/how-to-stop-a-chatbot-making-things-up)
- [AI agent vs chatbot](https://heykiko.io/blog/ai-agent-vs-chatbot)
- [Train on your website and files](https://heykiko.io/features/train-on-your-website)
- [All guides](https://heykiko.io/blog)