Retrieval augmented generation in Data 360
Key Takeaway
RAG grounds a prompt at run time with relevant content from a Data 360 search index. A retriever is the bridge between the search index and the prompt template: you create and activate the retriever in AI Models, then insert and configure it in Prompt Builder.
Retrieval Augmented Generation (RAG) in Data 360 is a framework for grounding large language model (LLM) prompts with your own accurate, current information. RAG exists because many LLMs were trained on older, publicly available internet content. RAG is the alternative to retraining or fine-tuning a model: it lets you bring your proprietary data to the LLM without retraining or fine-tuning it.
How RAG works
When a prompt is submitted, RAG works in three steps:
- Retrieve: RAG retrieves relevant information from structured and unstructured content indexed in Data 360.
- Augment: RAG combines that information with the original prompt.
- Generate: the LLM generates a response.
Think of RAG as an open-book exam. The LLM is the student, and the facts you hand it at run time are the open book. The LLM answers from those facts instead of relying only on what it learned in training.
RAG adds information that is accurate, up to date, and not part of the LLM's trained knowledge, such as your company's proprietary data, so the responses fit your context and use case.