What RAG is, and why a website chatbot should use it

A large language model on its own answers from whatever it learned during training. It doesn’t know your prices, your opening hours or last week’s policy change, and when it doesn’t know, it can still produce a confident, fluent answer that is wrong. For a chatbot on your website, that’s the worst kind of mistake.

Retrieval-augmented generation (RAG) is the usual way around this. Instead of asking the model to remember your business, you give it the relevant parts of your own content at the moment a visitor asks a question.

How it works

1. Read and split your content. Your web pages, documents and written answers are split into short passages, each kept with the page it came from.
2. Index the passages. Each passage is turned into an embedding, a list of numbers that captures its meaning, so passages about the same thing end up close together even when they use different words.
3. Retrieve. When a visitor asks something, the question is turned into an embedding too, and the closest passages are found. Combining this with ordinary keyword matching helps with names, codes and prices.
4. Generate. The model gets the question and those passages, with instructions to answer only from them and to say so when they don’t contain the answer.

Why it matters for a website chatbot

• Answers can cite sources. Because every passage remembers its page, the answer can link to it. Visitors can check, and you can see why it said what it said.
• Updates are quick. Change a page, read it again, and the next answer uses the new text. No retraining.
• “I don’t know” becomes possible. If nothing relevant is retrieved, the honest reply is that the content doesn’t cover it, and the question can go to a person.

Where it still goes wrong

RAG reduces made-up answers; it doesn’t eliminate them. The common failures are worth knowing:

• The answer isn’t written down. If your site never says whether you open on Sundays, no retrieval will find it. The questions a chatbot couldn’t answer are the best list of what to add.
• The wrong passage wins. An old pricing page can outrank the current one. Removing or excluding outdated pages matters more than any setting.
• Pieces get lost in splitting. A table or a list split across passages can lose its meaning. Splitting along headings helps.
• The model over-reaches. Even with good passages, a model can blend in general knowledge. Clear instructions, test questions run before every change, and reading real conversations are how you catch it.

RAG or fine-tuning?

Fine-tuning changes how a model writes; it’s a poor way to teach it facts that change. For answering questions about your business, retrieval is simpler, cheaper to keep current, and the only one of the two that can show where an answer came from.

AI Chatbot Engine is built this way: it reads your website, files and written answers, answers with links to its sources, and hands questions it can’t answer to you. You can try it on your own site.