Agentic Architectures

Catalogue / Retrieval

Self-RAG

Judge every step, and answer only from passages that hold up

Shape
gate
Model calls
1 to decide + 1 per passage graded + 1 per draft + 1 per support check (often 8 to 15)
Latency
high
Complexity
medium

Also known as Self-reflective retrieval

1

How do I know each sentence of the answer is actually backed by a source?

Citing a document is not the same as being supported by it. Self-RAG makes three explicit judgements: does this question need a lookup, is this passage relevant, and does the passage really support what was written. Only text that passes all three reaches the reader, with the passage that backs it.

You may need it if

  • Answers cite a document, but the document does not say what the answer claims.
  • Your answers carry risk: contract terms, compliance rules, medical or financial facts.
  • Reviewers spend their time checking citations by hand.
2

When are three checks per answer too many?

Use it when

  • A wrong claim with a citation is worse than no answer, because readers trust citations.
  • Your documents are the authority and the answer must not go beyond them.
  • Some questions need no lookup at all and you do not want to search for those.

Avoid it when

What it costs

Model calls
1 to decide + 1 per passage graded + 1 per draft + 1 per support check (often 8 to 15)
Latency
high
Complexity
medium
3

Where do the judgements happen as the answer is built?

A first judgement decides whether to retrieve. Retrieved passages are graded one by one and only relevant ones are kept. The writer drafts from them, citing each passage. A support check compares every claim with its passage; unsupported drafts are rewritten once, then the answer leaves.

depends ondocumentsno lookupneededunsupportedclaims, onerewrite leftall supported,or rewrite usedQuestion: The user's question in plain language.QuestionRetrieve?: Decides whether the question depends on your documents or can be answered without them.Retrieve?Retriever: Searches the index and returns the top passages with their sources.RetrieverDocument index: Your reference documents, split into passages and indexed ahead of time.DocumentindexRelevant?: Grades each passage on its own and keeps only those that bear on the question.Relevant?Writer: Drafts the answer from the endorsed passages, tagging each claim with the passage behind it.WriterSupported?: Checks every claim against its passage: fully supported, partly, or not at all.Supported?Answer: The answer made only of supported claims, each with its citation.Answer
  1. Step 1 of 5

    Does this need a lookup?

    The first judgement is whether to search at all. "Summarise this paragraph" does not. "What is our refund window?" does. Skipping needless searches saves cost and keeps unrelated passages out.

  2. Step 2 of 5

    Retrieve, then judge each passage

    The retriever returns a handful of passages. Each one is graded alone against the question. Only endorsed passages move on; the rest are dropped before any writing.

  3. Step 3 of 5

    Draft with a passage behind every claim

    The writer drafts from the endorsed passages and tags each claim with the passage it relies on. A claim without a passage must be removed or marked as general knowledge.

  4. Step 4 of 5

    Check support, claim by claim

    A separate check reads each claim next to its passage. Does the passage say this, in substance? Partly supported claims count as failures.

  5. Step 5 of 5

    Rewrite once, then release

    Failed claims go back to the writer once, to be corrected from the passage or dropped. The final answer keeps only supported claims, and says what it could not confirm.

What each part does
Question
The user's question in plain language.
Retrieve?
Decides whether the question depends on your documents or can be answered without them.
Retriever
Searches the index and returns the top passages with their sources.
Document index
Your reference documents, split into passages and indexed ahead of time.
Relevant?
Grades each passage on its own and keeps only those that bear on the question.
Writer
Drafts the answer from the endorsed passages, tagging each claim with the passage behind it.
Supported?
Checks every claim against its passage: fully supported, partly, or not at all.
Answer
The answer made only of supported claims, each with its citation.
  • Retrieve? → Retriever: depends on documents
  • Retrieve? → Writer: no lookup needed
  • Retriever → Relevant?: passages
  • Relevant? → Writer: endorsed passages
  • Writer → Supported?: draft with citations
  • Supported? → Writer: unsupported claims, one rewrite left
  • Supported? → Answer: all supported, or rewrite used

Where it connects

Entry
question
Question

A question in plain language. Optionally, a note on how strict the support check should be.

Exit
answer
AnswerEvidence

When: Every remaining claim is supported, or the single rewrite has been used.

The answer, each claim with its supporting passage and source, and a list of claims that were dropped as unsupported.

Memory kept between runs
index
Evidence

Passages from your reference documents with source and date. Read during the run, never written by it.

4

What do I ask my assistant to set up?

The agents to launch

Writer

First say whether the question needs your documents, in one word. When writing, use only the endorsed passages. Write short claims, each followed by the passage number it relies on. When given failed claims, correct them from their passage or delete them. Never cite a passage for something it does not say.

Sees: The question, the endorsed passages and, on a rewrite, the list of failed claims. Not the rejected passages.

Judge, One call per passage, then one call per claim, run in parallel

You make one judgement per call. For relevance, given a question and one passage, answer relevant or irrelevant with the deciding sentence. For support, given one claim and one passage, answer supported, partly or unsupported, quoting the passage.

Sees: One question and one passage, or one claim and one passage. Never the full draft, the other passages or the writer's reasoning.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Self-RAG" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Self-RAG
Judge every step, and answer only from passages that hold up.
Input: the task above. It must provide: A question in plain language. Optionally, a note on how strict the support check should be. If something essential is missing, ask for it before starting.

Agents to launch:
- Writer: First say whether the question needs your documents, in one word. When writing, use only the endorsed passages. Write short claims, each followed by the passage number it relies on. When given failed claims, correct them from their passage or delete them. Never cite a passage for something it does not say.
  Context for this agent: The question, the endorsed passages and, on a rewrite, the list of failed claims. Not the rejected passages.
- Judge (One call per passage, then one call per claim, run in parallel): You make one judgement per call. For relevance, given a question and one passage, answer relevant or irrelevant with the deciding sentence. For support, given one claim and one passage, answer supported, partly or unsupported, quoting the passage.
  Context for this agent: One question and one passage, or one claim and one passage. Never the full draft, the other passages or the writer's reasoning.

Procedure:
1. Confirm the documents are indexed as passages with their source. Build the index once, outside the run.
2. Ask the writer whether the question needs a lookup. If not, have it answer directly and mark the answer as not based on your documents.
3. Otherwise search the index and keep the top five passages. Launch one judge call per passage for relevance.
4. If no passage is endorsed, stop and report that the documents do not cover the question.
5. Launch the writer with the endorsed passages. Split its draft into claims and launch one judge call per claim for support.
6. Send any claim judged partly or unsupported back to the writer once. Drop any claim that still fails.

Stop rule: Stop after the support check that follows at most one rewrite. Claims still unsupported are removed, never released.

Exit:
- `answer`, when: Every remaining claim is supported, or the single rewrite has been used. Deliver: The answer, each claim with its supporting passage and source, and a list of claims that were dropped as unsupported.

# Final deliverable
The answer built from supported claims, each with its passage and source, then the list of claims dropped as unsupported and the passages rejected as irrelevant.

The same thing in code

LangGraph, python, package langgraph
import asyncio
import operator
from typing import Annotated, Literal, TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel

llm = init_chat_model("anthropic:claude-sonnet-5-5")
WRITER = ("Use only the endorsed passages. Write short claims, one fact each, each tagged with the number of the "
          "passage it relies on. When given failed claims, correct them from their passage or delete them. "
          "Never cite a passage for something it does not say.")
JUDGE = ("You make one judgement per call. For relevance, given a question and one passage, answer relevant or "
         "irrelevant with the deciding sentence. For support, given one claim and one passage, answer supported, "
         "partly or unsupported, quoting the passage.")

class Need(BaseModel):
    needs_documents: bool

class Relevance(BaseModel):
    relevant: bool
    sentence: str

class Claim(BaseModel):
    text: str
    passage: int

class Draft(BaseModel):
    claims: list[Claim]

class Support(BaseModel):
    verdict: Literal["supported", "partly", "unsupported"]
    quote: str

class Input(TypedDict):
    task: str

class State(Input):
    needs: bool
    direct: str
    endorsed: list[str]
    rejected: list[str]
    pending: list[Claim]
    kept: Annotated[list[Claim], operator.add]
    failed: list[Claim]
    rewrites: int

def search_index(query: str, k: int = 5) -> list[str]:
    """Stub: search your passage index; return each passage with its source."""
    return [f"(guide.pdf) passage about {query}"][:k]

async def ask[T: BaseModel](schema: type[T], system: str, user: str) -> T:
    return schema.model_validate(await llm.with_structured_output(schema).ainvoke([("system", system), ("user", user)]))

async def need(state: State) -> dict:
    r = await ask(Need, WRITER, f"Does this question need the documents? Question: {state['task']}")
    return {"needs": r.needs_documents}

async def direct(state: State) -> dict:
    return {"direct": (await llm.ainvoke(state["task"])).text}

async def relevance(state: State) -> dict:
    # one judge call per passage: one question and one passage, nothing else
    passages = search_index(state["task"])
    grades = await asyncio.gather(*(ask(Relevance, JUDGE, f"Question: {state['task']}\nPassage: {p}") for p in passages))
    return {"endorsed": [p for p, g in zip(passages, grades) if g.relevant],
            "rejected": [p for p, g in zip(passages, grades) if not g.relevant], "rewrites": 0}

async def write(state: State) -> dict:
    # the writer sees the endorsed passages and, on a rewrite, the failed claims; never the rejected ones
    numbered = "\n".join(f"[{i}] {p}" for i, p in enumerate(state["endorsed"]))
    failed = "\n".join(f"- {c.text} [{c.passage}]" for c in state.get("failed", []))
    ask_for = f"Correct or delete these failed claims:\n{failed}" if failed else "Draft the answer."
    draft = await ask(Draft, WRITER, f"Question: {state['task']}\nEndorsed passages:\n{numbered}\n{ask_for}")
    return {"pending": draft.claims, "rewrites": state["rewrites"] + (1 if failed else 0)}

async def support(state: State) -> dict:
    # one judge call per claim, with only that claim and its passage
    ok = [c for c in state["pending"] if 0 <= c.passage < len(state["endorsed"])]
    checks = await asyncio.gather(*(ask(Support, JUDGE, f"Claim: {c.text}\nPassage: {state['endorsed'][c.passage]}") for c in ok))
    return {"kept": [c for c, s in zip(ok, checks) if s.verdict == "supported"],
            "failed": [c for c, s in zip(ok, checks) if s.verdict != "supported"]}

builder = StateGraph(State, input_schema=Input)
for name, fn in [("need", need), ("direct", direct), ("relevance", relevance), ("write", write), ("support", support)]:
    builder.add_node(name, fn)
builder.add_edge(START, "need")
builder.add_conditional_edges("need", lambda s: "relevance" if s["needs"] else "direct", ["relevance", "direct"])
builder.add_conditional_edges("relevance", lambda s: "write" if s["endorsed"] else END, ["write", END])
builder.add_edge("write", "support")
# the gate: one rewrite at most; claims that still fail are dropped
builder.add_conditional_edges("support", lambda s: "write" if s["failed"] and s["rewrites"] == 0 else END, ["write", END])
builder.add_edge("direct", END)
graph = builder.compile()

async def run(task: str):
    s = await graph.ainvoke({"task": task})
    if not s["needs"]:
        return {"answer": s["direct"], "based_on_documents": False}
    if not s["endorsed"]:
        return {"answer": "The documents do not cover this question.", "rejected": s["rejected"]}
    return {"claims": [(c.text, s["endorsed"][c.passage]) for c in s["kept"]],
            "dropped": [c.text for c in s.get("failed", [])], "rejected": s["rejected"]}

Replace search_index with your passage index, and check that the writer keeps one fact per claim, or partly supported claims slip through.

Claude Agent SDK, python, package claude-agent-sdk
from typing import Any

from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query

AGENTS = {
    "writer": AgentDefinition(
        description="Decides whether a lookup is needed, then writes short cited claims from endorsed passages.",
        prompt=("First say whether the question needs the documents. When writing, use only the endorsed passages: "
                "short claims, one fact each, each tagged with the passage number it relies on. When given failed "
                "claims, correct them from their passage or delete them. Never cite a passage for something it does not say."),
        tools=[]),
    "judge": AgentDefinition(
        description="Makes one relevance or support judgement on one passage.",
        prompt=("You make one judgement per call. For relevance, given a question and one passage, answer relevant or "
                "irrelevant with the deciding sentence. For support, given one claim and one passage, answer "
                "supported, partly or unsupported, quoting the passage."),
        tools=[]),
}

def obj(**props: Any) -> dict[str, Any]:
    return {"type": "object", "required": list(props), "properties": props}
NEED = obj(needs_documents={"type": "boolean"})
VERDICTS = obj(verdicts={"type": "array", "items": {"type": "string"}})
CLAIMS = obj(claims={"type": "array", "items": obj(text={"type": "string"}, passage={"type": "integer"})})

def search_index(query: str, k: int = 5) -> list[str]:
    """Stub: search your passage index; return each passage with its source."""
    return [f"(guide.pdf) passage about {query}"][:k]

async def ask(prompt: str, schema: dict[str, Any] | None = None) -> Any:
    fmt = {"type": "json_schema", "schema": schema} if schema else None
    options = ClaudeAgentOptions(agents=AGENTS, tools=["Agent"], allowed_tools=["Agent"], output_format=fmt)
    async for message in query(prompt=prompt, options=options):
        if isinstance(message, ResultMessage):
            return message.structured_output if schema else message.result
    raise RuntimeError("no result")

async def judge_each(items: list[str], kind: str) -> list[str]:
    # one judge sub-agent per item, in parallel; each sees one pair, never the full draft
    listing = "\n".join(f"{i}. {x}" for i, x in enumerate(items))
    out = await ask(f"Launch one judge sub-agent per item below, in parallel, for a {kind} judgement. Give each only "
                    f"its own item.\n{listing}\nReturn each verdict word in item order.", VERDICTS)
    return out["verdicts"]

async def check(claims: list[dict[str, Any]], endorsed: list[str]) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
    claims = [c for c in claims if 0 <= c["passage"] < len(endorsed)]
    verdicts = await judge_each([f"Claim: {c['text']}\nPassage: {endorsed[c['passage']]}" for c in claims], "support")
    return ([c for c, v in zip(claims, verdicts) if v == "supported"],
            [c for c, v in zip(claims, verdicts) if v != "supported"])

async def run(task: str):
    need = await ask(f"Ask the writer sub-agent whether this question needs the documents: {task}", NEED)
    if not need["needs_documents"]:
        answer = await ask(f"Have the writer sub-agent answer directly: {task}\nReturn its answer verbatim.")
        return {"answer": answer, "based_on_documents": False}
    passages = search_index(task)
    grades = await judge_each([f"Question: {task}\nPassage: {p}" for p in passages], "relevance")
    endorsed = [p for p, g in zip(passages, grades) if g == "relevant"]
    rejected = [p for p, g in zip(passages, grades) if g != "relevant"]
    if not endorsed:
        return {"answer": "The documents do not cover this question.", "rejected": rejected}
    numbered = "\n".join(f"[{i}] {p}" for i, p in enumerate(endorsed))
    # the writer sees the endorsed passages only, never the rejected ones
    draft = await ask(f"Use the writer sub-agent. Give it only:\nQuestion: {task}\nEndorsed passages:\n{numbered}\n"
                      "Return its claims.", CLAIMS)
    kept, failed = await check(draft["claims"], endorsed)
    if failed:  # the gate: one rewrite, then anything still failing is dropped
        listing = "\n".join(f"- {c['text']} [{c['passage']}]" for c in failed)
        fixed = await ask(f"Use the writer sub-agent. Give it only:\nEndorsed passages:\n{numbered}\n"
                          f"Correct or delete these failed claims:\n{listing}\nReturn its claims.", CLAIMS)
        more, failed = await check(fixed["claims"], endorsed)
        kept += more
    return {"claims": [(c["text"], endorsed[c["passage"]]) for c in kept],
            "dropped": [c["text"] for c in failed], "rejected": rejected}

Replace search_index with your passage index, and check that the writer keeps one fact per claim, or partly supported claims slip through.

OpenAI Agents SDK, python, package openai-agents
import asyncio
from typing import Literal

from agents import Agent, Runner
from pydantic import BaseModel

class Need(BaseModel):
    needs_documents: bool

class Claim(BaseModel):
    text: str
    passage: int

class Draft(BaseModel):
    claims: list[Claim]

class Relevance(BaseModel):
    relevant: bool
    sentence: str

class Support(BaseModel):
    verdict: Literal["supported", "partly", "unsupported"]
    quote: str

writer = Agent(
    name="Writer",
    instructions=("First say whether the question needs the documents. When writing, use only the endorsed passages: "
                  "short claims, one fact each, each tagged with the passage number it relies on. When given failed "
                  "claims, correct them from their passage or delete them. Never cite a passage for something it does not say."))
judge = Agent(
    name="Judge",
    instructions=("You make one judgement per call. For relevance, given a question and one passage, answer relevant or "
                  "irrelevant with the deciding sentence. For support, given one claim and one passage, answer "
                  "supported, partly or unsupported, quoting the passage."))
# same briefs, one structured output per kind of judgement
need_writer, draft_writer = writer.clone(output_type=Need), writer.clone(output_type=Draft)
relevance_judge, support_judge = judge.clone(output_type=Relevance), judge.clone(output_type=Support)

def search_index(query: str, k: int = 5) -> list[str]:
    """Stub: search your passage index; return each passage with its source."""
    return [f"(guide.pdf) passage about {query}"][:k]

async def check(claims: list[Claim], endorsed: list[str]) -> tuple[list[Claim], list[Claim]]:
    # one judge run per claim, with only that claim and its passage
    claims = [c for c in claims if 0 <= c.passage < len(endorsed)]
    runs = await asyncio.gather(*(Runner.run(support_judge, f"Claim: {c.text}\nPassage: {endorsed[c.passage]}") for c in claims))
    ok = [r.final_output_as(Support).verdict == "supported" for r in runs]
    return [c for c, k in zip(claims, ok) if k], [c for c, k in zip(claims, ok) if not k]

async def run(task: str):
    need = await Runner.run(need_writer, f"Does this question need the documents? {task}")
    if not need.final_output_as(Need).needs_documents:
        direct = await Runner.run(writer, task)
        return {"answer": str(direct.final_output), "based_on_documents": False}
    passages = search_index(task)
    # one judge run per passage: one question and one passage, nothing else
    runs = await asyncio.gather(*(Runner.run(relevance_judge, f"Question: {task}\nPassage: {p}") for p in passages))
    grades = [r.final_output_as(Relevance).relevant for r in runs]
    endorsed = [p for p, g in zip(passages, grades) if g]
    rejected = [p for p, g in zip(passages, grades) if not g]
    if not endorsed:
        return {"answer": "The documents do not cover this question.", "rejected": rejected}
    numbered = "\n".join(f"[{i}] {p}" for i, p in enumerate(endorsed))
    draft = await Runner.run(draft_writer, f"Question: {task}\nEndorsed passages:\n{numbered}")
    kept, failed = await check(draft.final_output_as(Draft).claims, endorsed)
    if failed:  # the gate: one rewrite, then anything still failing is dropped
        listing = "\n".join(f"- {c.text} [{c.passage}]" for c in failed)
        fixed = await Runner.run(draft_writer, f"Endorsed passages:\n{numbered}\nCorrect or delete these failed claims:\n{listing}")
        more, failed = await check(fixed.final_output_as(Draft).claims, endorsed)
        kept += more
    return {"claims": [(c.text, endorsed[c.passage]) for c in kept],
            "dropped": [c.text for c in failed], "rejected": rejected}

Replace search_index with your passage index, and check that the writer keeps one fact per claim, or partly supported claims slip through.

Vercel AI SDK, typescript, package ai
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, Output } from 'ai';
import { z } from 'zod';

const model = anthropic('claude-sonnet-5-5');

const WRITER =
  'First say whether the question needs the documents. When writing, use only the endorsed passages: short ' +
  'claims, one fact each, each tagged with the passage number it relies on. When given failed claims, correct ' +
  'them from their passage or delete them. Never cite a passage for something it does not say.';
const JUDGE =
  'You make one judgement per call. For relevance, given a question and one passage, answer relevant or ' +
  'irrelevant with the deciding sentence. For support, given one claim and one passage, answer supported, ' +
  'partly or unsupported, quoting the passage.';
const draftSchema = z.object({ claims: z.array(z.object({ text: z.string(), passage: z.number().int() })) });
type Claim = z.infer<typeof draftSchema>['claims'][number];

// Stub: search your passage index; return each passage with its source.
async function searchIndex(query: string, k = 5): Promise<string[]> {
  return [`(guide.pdf) passage about ${query}`].slice(0, k);
}

async function ask<T>(instructions: string, prompt: string, schema: z.ZodType<T>): Promise<T> {
  return (await generateText({ model, instructions, prompt, output: Output.object({ schema }) })).output;
}

async function check(claims: Claim[], endorsed: string[]): Promise<[Claim[], Claim[]]> {
  // one judge call per claim, with only that claim and its passage
  const valid = claims.filter((c) => c.passage >= 0 && c.passage < endorsed.length);
  const verdicts = await Promise.all(valid.map((c) => ask(JUDGE, `Claim: ${c.text}\nPassage: ${endorsed[c.passage]}`,
    z.object({ verdict: z.enum(['supported', 'partly', 'unsupported']), quote: z.string() }))));
  return [valid.filter((_, i) => verdicts[i].verdict === 'supported'), valid.filter((_, i) => verdicts[i].verdict !== 'supported')];
}

export async function run(task: string) {
  const need = await ask(WRITER, `Does this question need the documents? ${task}`, z.object({ needsDocuments: z.boolean() }));
  if (!need.needsDocuments) {
    const { text } = await generateText({ model, instructions: WRITER, prompt: task });
    return { answer: text, basedOnDocuments: false };
  }
  const passages = await searchIndex(task);
  // one judge call per passage: one question and one passage, nothing else
  const grades = await Promise.all(passages.map((p) => ask(JUDGE, `Question: ${task}\nPassage: ${p}`,
    z.object({ relevant: z.boolean(), sentence: z.string() }))));
  const endorsed = passages.filter((_, i) => grades[i].relevant);
  const rejected = passages.filter((_, i) => !grades[i].relevant);
  if (!endorsed.length) return { answer: 'The documents do not cover this question.', rejected };
  const numbered = endorsed.map((p, i) => `[${i}] ${p}`).join('\n');
  const draft = await ask(WRITER, `Question: ${task}\nEndorsed passages:\n${numbered}`, draftSchema);
  let [kept, failed] = await check(draft.claims, endorsed);
  if (failed.length) { // the gate: one rewrite, then anything still failing is dropped
    const listing = failed.map((c) => `- ${c.text} [${c.passage}]`).join('\n');
    const fixed = await ask(WRITER, `Endorsed passages:\n${numbered}\nCorrect or delete these failed claims:\n${listing}`, draftSchema);
    const [more, still] = await check(fixed.claims, endorsed);
    kept = kept.concat(more);
    failed = still;
  }
  return {
    claims: kept.map((c) => ({ claim: c.text, passage: endorsed[c.passage] })),
    dropped: failed.map((c) => c.text),
    rejected,
  };
}

Replace searchIndex with your passage index, and check that the writer keeps one fact per claim, or partly supported claims slip through.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • Long claims mix supported and unsupported parts and pass as "partly". Ask the writer for one fact per claim.
  • A judge that sees the whole draft grades the story, not the claim. Give it one claim and one passage at a time.
  • Dropping failed claims can leave the main question unanswered. Say so in the answer instead of releasing a hollow one.

Chain it

Its result can feed

Source