RAG lets a language model answer using documents you provide instead of relying on what it learned in training. It is a useful pattern, not a universal one.
How it works in brief
Your documents are split into passages and indexed. When someone asks a question, the system finds the most relevant passages and gives them to the model, which writes an answer based on them.
Because the passages are known, the answer can cite its sources, so a person can verify it.
Where it fits
RAG tends to work well when these are true.
- The answers live in a defined set of documents
- The documents change, so retraining would not keep up
- Users need to see where an answer came from
- The text is reasonably well organised
Where it struggles
If the answer requires combining many scattered facts, doing calculations, or the source documents are contradictory or out of date, results will suffer. Quality also depends on how the documents are prepared and searched, which is most of the engineering work.
This topic is part of our AI Development & Automation work.