RAG · Module 12 of 14 · 8 concepts · about 11 min

Advanced RAG Patterns

What to do when one search followed by one answer is not enough: questions needing several lookups, self-checking pipelines, knowledge graphs, images and tables, databases, and very long prompts.

At work: The designs behind research assistants, analytics copilots and document-heavy products, where questions span several sources or types of data.

Multi-hop questions and iterative retrieval

What it is

A multi-hop question needs facts from more than one place, where the second lookup depends on the first.

"What is the refund for the membership plan that includes free delivery?"
First find which plan includes free delivery (Kart Plus), then find that plan's refund rule.
Iterative retrieval does this in a loop: retrieve, read, decide what is still missing, retrieve again.

Why it matters

A single search for the original question finds chunks about free delivery and chunks about refunds in general, but not the Kart Plus refund rule, because the question never names Kart Plus.

The name only becomes known after the first hop.
Plain RAG cannot answer such questions however good the retriever is.

In the real world: A due-diligence assistant answers 'who owns the supplier named in clause 12?' by first reading the clause for the supplier's name and then searching the company registry documents.

Question: What is the refund for the membership plan that
          includes free delivery?

Hop 1  search: membership plan with free delivery
       found:  Kart Plus gives free delivery on every order.

Hop 2  search: Kart Plus refund policy        <- uses hop 1's answer
       found:  Kart Plus can be cancelled within 7 days for a
               full refund.

Answer from both passages.

Looping without a limit. Each hop adds a model call and a search. Cap the number of hops (two or three is typical) and stop as soon as the model judges it has enough.

Agentic RAG: retrieval as a tool

What it is

In agentic RAG, the language model is given search as a tool it can choose to call, instead of search running automatically before every answer. The model decides whether to search, what to search for, which source to use, and whether to search again after reading the results.

Why it matters

It handles the questions fixed pipelines handle badly: multi-hop questions, questions that need two sources, and messages that need no search at all.

The price is predictability.
Cost and response time vary from question to question, behaviour is harder to test, and a model can loop or stop too early.
A fixed pipeline is the right default; an agent is worth it when questions are varied and complex.

In the real world: A coding assistant decides for itself to search the codebase, read a file, then search again for where a function is called, before answering a question about a bug.

Visual example

Comparison

An agent buys flexibility with predictability.

Fixed pipeline versus agentic RAG

	fixed pipeline	agentic
Who decides to search	Always searches, once	The model, each step
Multi-hop questions	Poor	Good
Cost and time per question	Predictable	Variable, often higher
Testing and debugging	Simpler	Harder: many possible paths
Best for	High-volume, simple questions	Complex, varied questions

tools = [search_help_centre, search_product_catalogue, lookup_order]

messages = [{"role": "user", "content": question}]
for _ in range(MAX_STEPS):                    # always bound the loop
    reply = llm(messages, tools=tools)
    if not reply.tool_calls:
        break                                 # the model chose to answer
    for call in reply.tool_calls:
        result = run_tool(call, user=current_user)   # permissions enforced here
        messages.append(tool_result(call, result))

Checking permissions in the prompt instead of in the tool. An agent chooses its own queries, so the search tool itself must enforce what this user may see. Never rely on the model to respect an instruction about access.

Corrective and self-checking RAG

What it is

The pipeline inspects its own work and reacts.

Corrective RAG grades the retrieved chunks before answering: if they look relevant, proceed; if not, rewrite the query and search again, or fall back to another source such as web search.
Self-RAG has the model decide whether retrieval is needed, then judge whether each passage is relevant and whether its own answer is supported.

Why it matters

A plain pipeline uses whatever was retrieved, good or bad.

Adding a grading step means bad retrieval is caught and retried instead of being turned into a confident wrong answer.
It trades extra model calls for reliability, which suits settings where a wrong answer is costly.

In the real world: A medical-information assistant grades its retrieved passages, and when none is judged relevant it tells the user it cannot answer, instead of answering from the nearest unrelated passage.

Adding a grader without measuring it. The grader is another model that can be wrong: too strict and it triggers retries on good results, doubling the cost; too lenient and it adds delay for nothing.

GraphRAG and knowledge graphs

What it is

A knowledge graph stores facts as entities (people, companies, products) connected by relationships ("supplies", "owns", "reports to").

GraphRAG builds such a graph from the documents, using a language model to extract entities and relationships, and retrieves by following connections.
Some versions also group the graph into communities and write a summary of each, to answer broad questions.

Why it matters

Vector search finds passages similar to the question.

It struggles with questions about connections across many documents ("which suppliers are linked to both of these companies?") and with whole-collection questions ("what are the main themes in these reports?"), where no single passage holds the answer.
A graph makes those relationships directly searchable.

In the real world: A compliance team traces ownership chains across thousands of filings with a graph, a question that similarity search over individual passages could not answer.

Building a graph by default. Extraction means a model call over every chunk, which is expensive, and extraction errors (two spellings of one company treated as two entities) weaken the graph. Use it when the questions are truly about relationships; otherwise hybrid search with reranking is cheaper and usually enough.

Multimodal RAG

What it is

Multimodal RAG retrieves and reasons over more than plain text: images, charts, diagrams, scanned pages, slides. Three approaches:

Describe: a vision model writes a text description of each image, which is indexed as text.
Embed together: a multimodal embedding model places images and text in the same vector space, so a text question can find an image.
Embed the page: treat each page as an image and embed it whole, skipping text extraction.

Why it matters

Much real knowledge is visual: a wiring diagram, a chart in an annual report, a screenshot in a manual.

Text-only pipelines drop it at parsing.
At answer time, the retrieved image itself can be passed to a model that reads images, so it answers from the chart, not from a description of it.

In the real world: A field-service app lets technicians ask about a machine and returns the right page of the manual with its diagram, which a text-only index had never been able to find.

Indexing only image descriptions and discarding the images. A description loses detail, such as the exact value of a bar in a chart. Keep the original image and give it to the model when answering.

Practise it: Design a Document Intelligence Platform

RAG over structured data: text-to-SQL

What it is

Questions about rows and numbers ("how many orders were refunded last week?") are answered by a database query, not a text search.

In text-to-SQL, the model is shown the table definitions and writes a SQL query; the system runs it and the model explains the result.
Retrieval still helps here: it finds the relevant tables, column descriptions and example queries to put in the prompt.

Why it matters

Counting, totalling, averaging and filtering need every matching row.

Vector search returns a handful of similar rows and the model guesses a total from them. A query computes the exact answer.
Mature assistants route between the two: documents for 'what is the policy?', the database for 'how many?'.

In the real world: A retail analytics assistant answers 'which store had the highest returns in March?' with a generated query over the sales tables, and uses document search only for the returns policy.

Running model-written SQL with full database permissions. Use a read-only account limited to the permitted tables, apply row-level access rules for the user, set time and row limits, and validate the query before it runs.

Summary indexes and hierarchical retrieval

What it is

Alongside the ordinary chunks, the system indexes summaries: one per section, one per document, sometimes one per group of related documents.

Search can then match at whichever level fits.
A well-known version, RAPTOR, clusters chunks, summarises each cluster, then clusters and summarises the summaries, forming a tree.

Why it matters

Chunk-level search serves detail questions.

It serves broad questions badly: 'what is this 80-page contract about?' is answered by no single chunk.
A summary is a chunk that exists to answer exactly that.
It also allows a two-step search: find the right document by its summary, then search for chunks inside it.

In the real world: A consulting firm's assistant first picks the right three reports by their summaries, then searches inside only those, which removes look-alike passages from unrelated projects.

Returning summaries for questions that need exact wording. A summary drops figures, conditions and exceptions. Use summaries to locate, and the original chunks to answer and cite.

Long-context models and cache-augmented generation

What it is

With very large context windows, a small collection can be placed in the prompt whole, with no retrieval. Cache-augmented generation makes this affordable: the model provider stores the processed form of the long, unchanging part of the prompt (prompt caching), so later questions reuse it at a fraction of the cost and time.

Why it matters

It removes retrieval failures entirely, since nothing is left out, and it is far simpler to build.

It works while the collection fits comfortably in the window and changes rarely.
It stops working as the collection grows, changes often, or needs per-user access control, and accuracy on a fact buried in a huge prompt is lower than on a fact in a short, focused one.

In the real world: A startup's assistant over a single product manual loads the whole manual into a cached prompt; when the company grows to forty manuals and regional versions, it moves to retrieval.

Visual example

Decision

The answer is a threshold, not a belief.

Put everything in the prompt, or retrieve?

condition	whole collection in a cached prompt	retrieval (RAG)
Collection size	Fits well inside the context window	Any size
How often it changes	Rarely	Often
Per-user permissions	Everyone may see everything	Filtered per user
Cost per question	High without caching	Lower: only a few chunks
Failure to watch for	Facts overlooked in a long prompt	The right chunk not retrieved

Arguing either extreme in an interview. 'Long context replaces RAG' ignores cost, scale and permissions. 'Always use RAG' ignores that a 50-page handbook does not need a vector database. State the conditions under which each wins.