Agentic Architectures

Catalogue / Guardrails & Routing

Metacognitive Triage

Judge your own competence and the stakes before you answer

Shape
router
Model calls
1 assessment + 1 answer or handoff note = 2
Latency
low
Complexity
low

Also known as Know-your-limits routing, Confidence triage

1

How do I keep an agent from answering what it should not?

Before answering, the agent rates two things: how well this request falls within what it knows and can check, and how much a wrong answer would cost. A fixed rule then sends it to a plain answer, an answer with explicit caveats, or a person.

You may need it if

  • The agent answers billing, legal or safety questions with the same confidence as FAQ questions.
  • Customers sometimes act on answers that a staff member would never have given.
  • Your support team wants hard cases handed over with context, not discovered after the fact.
2

When is self-assessment the wrong safeguard?

Use it when

  • Requests vary widely in risk, and some must reach a person.
  • You can describe the agent's scope in a few lines, and what lies outside it.
  • A handover path exists: a queue, a ticket or a person on call.

Avoid it when

  • The risk is an action on a system, not the content of an answer.
    Use Dry Run instead
  • The answer's facts can be checked one by one against sources, which beats a self-rating.
    Use Chain of Verification instead
  • Requests differ in kind and need different machinery, but all of them can be answered.
    Use Meta-Controller instead
  • Nobody is available to take escalations. The escalation branch then leads nowhere.

What it costs

Model calls
1 assessment + 1 answer or handoff note = 2
Latency
low
Complexity
low
3

How does a request get routed by confidence and stakes?

One assessment, then a fork. An assessor rates competence and stakes. A fixed table picks one of three routes. Plain and caveated answers leave through one exit. Escalated requests leave through another, with a handoff note for a person.

competent, lowstakespartly competent ormedium stakesout of scope orhigh stakescannot answer evenwith caveatsRequest: The user's request, with any account or context information available.RequestSelf-assessment: Rates its competence and the stakes from 1 to 3, with one reason for each rating.Self-assessmentWhich route?: A fixed table maps the two ratings to answer, answer with caveats, or escalate.Which route?Plain answer: Answers directly when competence is high and stakes are low.Plain answerCaveated answer: Answers, but states what it is unsure of and what the user should verify.CaveatedanswerHandoff note: Writes a short case summary for a person: the request, the ratings and what is missing.Handoff noteAnswered: The answer, labelled plain or caveated.AnsweredEscalated: The handoff to a person. The user has been told a person will reply.Escalated
  1. Step 1 of 4

    Rate before answering

    The assessor does not answer. It rates its competence, meaning whether the request is in scope and the needed facts are available, and the stakes, meaning money, legal exposure, health or an irreversible decision.

  2. Step 2 of 4

    A table, not a feeling

    The two ratings go into a fixed table that you wrote in advance. High stakes with anything less than full competence always escalates, whatever the model would prefer.

  3. Step 3 of 4

    Answer plainly or with caveats

    Easy cases get a direct answer. Middle cases get an answer that names what is uncertain and what the user should check. If the caveated answerer finds it cannot answer honestly, it escalates.

  4. Step 4 of 4

    Hand it to a person

    Escalated requests get a handoff note, so the person who picks it up does not start from zero. They leave through a separate exit that your ticketing or on-call process can listen to.

What each part does
Request
The user's request, with any account or context information available.
Self-assessment
Rates its competence and the stakes from 1 to 3, with one reason for each rating.
Which route?
A fixed table maps the two ratings to answer, answer with caveats, or escalate.
Plain answer
Answers directly when competence is high and stakes are low.
Caveated answer
Answers, but states what it is unsure of and what the user should verify.
Handoff note
Writes a short case summary for a person: the request, the ratings and what is missing.
Answered
The answer, labelled plain or caveated.
Escalated
The handoff to a person. The user has been told a person will reply.
  • Self-assessment → Which route?: two ratings and reasons
  • Which route? → Plain answer: competent, low stakes
  • Which route? → Caveated answer: partly competent or medium stakes
  • Which route? → Handoff note: out of scope or high stakes
  • Caveated answer → Handoff note: cannot answer even with caveats

Where it connects

Entry
request
TaskQuestion

A user request in plain language, with any context that affects the stakes, such as account type or amounts involved.

Exit
answered
Answer

When: The route was plain or caveated, and an answer was given.

The answer, labelled plain or caveated, with the caveats listed and the two ratings that led there.

Exit
escalated
Verdict

When: The request was out of scope or high stakes, or the caveated answerer could not answer.

The decision to escalate, with the ratings, the reasons, the original request and a handoff note for the person who takes it.

4

What do I ask my assistant to set up?

The agents to launch

Assessor

Do not answer the request. Rate your competence from 1 to 3: is it in the stated scope, and are the facts needed available to you? Rate the stakes from 1 to 3: money, legal, health or irreversible consequences. Give one reason for each.

Sees: The request, its context, the scope description and the stakes list. Not any draft answer.

Answerer

Answer the request directly and briefly. Stay within the stated scope.

Sees: The request and its context. Not the ratings, so it does not hedge without reason.

Cautious answerer

Answer the request, then list what you are unsure of and what the user should verify, and with whom. If you cannot give an honest answer even with caveats, reply "escalate" with one line of reasons.

Sees: The request, its context and the assessor's ratings and reasons.

Handoff writer

Write a handoff note of at most five lines for a person: what the user wants, why it was escalated, what is known and what is missing. Also write one polite line telling the user a person will follow up.

Sees: The request, its context, the ratings and reasons, and any "escalate" reply from the cautious answerer.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Metacognitive Triage" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Metacognitive Triage
Judge your own competence and the stakes before you answer.
Input: the task above. It must provide: A user request in plain language, with any context that affects the stakes, such as account type or amounts involved. If something essential is missing, ask for it before starting.

Agents to launch:
- Assessor: Do not answer the request. Rate your competence from 1 to 3: is it in the stated scope, and are the facts needed available to you? Rate the stakes from 1 to 3: money, legal, health or irreversible consequences. Give one reason for each.
  Context for this agent: The request, its context, the scope description and the stakes list. Not any draft answer.
- Answerer: Answer the request directly and briefly. Stay within the stated scope.
  Context for this agent: The request and its context. Not the ratings, so it does not hedge without reason.
- Cautious answerer: Answer the request, then list what you are unsure of and what the user should verify, and with whom. If you cannot give an honest answer even with caveats, reply "escalate" with one line of reasons.
  Context for this agent: The request, its context and the assessor's ratings and reasons.
- Handoff writer: Write a handoff note of at most five lines for a person: what the user wants, why it was escalated, what is known and what is missing. Also write one polite line telling the user a person will follow up.
  Context for this agent: The request, its context, the ratings and reasons, and any "escalate" reply from the cautious answerer.

Procedure:
1. Write the scope in a few lines, the stakes list, and the routing table. Example: stakes 3 with competence below 3 escalates; competence 1 escalates; competence 3 with stakes 1 answers plainly; everything else answers with caveats.
2. Launch the assessor on the request, as a separate agent. Record both ratings and reasons.
3. Apply the routing table yourself. Do not ask a model to choose the route.
4. Launch the answerer, the cautious answerer or the handoff writer according to the route.
5. If the cautious answerer replies "escalate", launch the handoff writer with its reasons.

Stop rule: Stop after one route. There is no loop. A caveated answer may fall through to escalation once, and an escalation always ends the run.

Exits:
- `answered`, when: The route was plain or caveated, and an answer was given. Deliver: The answer, labelled plain or caveated, with the caveats listed and the two ratings that led there.
- `escalated`, when: The request was out of scope or high stakes, or the caveated answerer could not answer. Deliver: The decision to escalate, with the ratings, the reasons, the original request and a handoff note for the person who takes it.

# Final deliverable
Either the answer labelled plain or caveated, with caveats listed, or the handoff note and the user message, labelled escalated. Both include the two ratings.

The same thing in code

LangGraph, python, package langgraph
from typing import Literal, TypedDict, cast

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel, Field

model = init_chat_model("anthropic:claude-sonnet-5-5")
SCOPE = "Billing questions for our subscription plans, using the plan and invoice data given."
STAKES = "money above one invoice, legal, health, or anything irreversible"


class Assessment(BaseModel):
    competence: int = Field(ge=1, le=3)
    competence_reason: str
    stakes: int = Field(ge=1, le=3)
    stakes_reason: str


class Input(TypedDict):
    request: str


class State(TypedDict):
    request: str
    ratings: Assessment
    route: str
    answer: str
    escalate_reason: str


def table(a: Assessment) -> Literal["plain", "caveated", "handoff"]:
    # the routing table is plain code, never a model's choice
    if a.competence == 1 or (a.stakes == 3 and a.competence < 3):
        return "handoff"
    return "plain" if a.competence == 3 and a.stakes == 1 else "caveated"


async def assessor(state: State) -> dict:
    a = cast(Assessment, await model.with_structured_output(Assessment).ainvoke(
        f"Do not answer. Scope: {SCOPE}\nStakes list: {STAKES}\nRate your competence 1-3 "
        "(in scope, facts available) and the stakes 1-3, one reason each.\n\n" + state["request"]))
    return {"ratings": a, "route": table(a)}


async def plain(state: State) -> dict:
    # the answerer never sees the ratings, so it does not hedge without reason
    msg = await model.ainvoke(f"Answer directly and briefly, within scope: {SCOPE}\n\n{state['request']}")
    return {"answer": msg.text, "route": "plain"}


async def caveated(state: State) -> dict:
    msg = await model.ainvoke(
        "Answer, then list what you are unsure of and what the user should verify, and with "
        "whom. If no honest answer is possible even with caveats, reply 'escalate' and one "
        f"line of reasons.\nRatings: {state['ratings'].model_dump()}\n\n{state['request']}")
    if msg.text.strip().lower().startswith("escalate"):
        return {"route": "handoff", "escalate_reason": msg.text}
    return {"answer": msg.text, "route": "caveated"}


async def handoff(state: State) -> dict:
    msg = await model.ainvoke(
        "Write a handoff note of at most five lines for a person: what the user wants, why it "
        "was escalated, what is known and what is missing. Then one polite line telling the "
        f"user a person will follow up.\nRatings: {state['ratings'].model_dump()}\n"
        f"Escalate reply: {state.get('escalate_reason', '')}\n\n{state['request']}")
    return {"answer": msg.text, "route": "escalated"}


graph = StateGraph(State, input_schema=Input)
graph.add_node("assessor", assessor)
graph.add_node("plain", plain)
graph.add_node("caveated", caveated)
graph.add_node("handoff", handoff)
graph.add_edge(START, "assessor")
graph.add_conditional_edges("assessor", lambda state: state["route"], ["plain", "caveated", "handoff"])
# one fall-through allowed: caveated may escalate once, then the run ends
graph.add_conditional_edges("caveated", lambda state: state["route"], {"handoff": "handoff", "caveated": END})
graph.add_edge("plain", END)
graph.add_edge("handoff", END)
app = graph.compile()


async def run(task: str):
    return await app.ainvoke({"request": task})

Replace SCOPE and STAKES with your own written scope and stakes list, and send the escalated handoff to a real queue a person watches.

Claude Agent SDK, python, package claude-agent-sdk
from typing import Any

from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query

SCOPE = "Billing questions for our subscription plans, using the plan and invoice data given."
STAKES = "money above one invoice, legal, health, or anything irreversible"

AGENTS = {
    "assessor": AgentDefinition(
        description="Rates competence and stakes for a request without answering it.",
        prompt=f"Do not answer the request. Scope: {SCOPE}. Stakes list: {STAKES}. Rate your "
               "competence 1-3 and the stakes 1-3, with one reason for each."),
    "answerer": AgentDefinition(
        description="Answers an in-scope, low-stakes request directly.",
        prompt=f"Answer the request directly and briefly. Stay within this scope: {SCOPE}"),
    "cautious-answerer": AgentDefinition(
        description="Answers with explicit caveats, or replies 'escalate'.",
        prompt="Answer, then list what you are unsure of and what the user should verify, and "
               "with whom. If you cannot answer honestly even with caveats, reply 'escalate' "
               "with one line of reasons."),
    "handoff-writer": AgentDefinition(
        description="Writes a short handoff note for a person.",
        prompt="Write a handoff note of at most five lines: what the user wants, why it was "
               "escalated, what is known and what is missing. Add one polite line telling the "
               "user a person will follow up."),
}

RATINGS = {"type": "object", "required": ["competence", "stakes", "competence_reason", "stakes_reason"],
           "properties": {"competence": {"type": "integer", "minimum": 1, "maximum": 3},
                          "stakes": {"type": "integer", "minimum": 1, "maximum": 3},
                          "competence_reason": {"type": "string"}, "stakes_reason": {"type": "string"}}}


async def delegate(agent: str, payload: str, schema: dict[str, Any] | None = None) -> ResultMessage:
    options = ClaudeAgentOptions(
        agents=AGENTS, allowed_tools=["Agent"],
        output_format={"type": "json_schema", "schema": schema} if schema else None)
    prompt = f"Launch the {agent} agent with exactly this input and return its reply:\n{payload}"
    async for msg in query(prompt=prompt, options=options):
        if isinstance(msg, ResultMessage):
            return msg
    raise RuntimeError("no result")


def table(competence: int, stakes: int) -> str:
    # the routing table runs in Python, never in a model
    if competence == 1 or (stakes == 3 and competence < 3):
        return "escalate"
    return "plain" if competence == 3 and stakes == 1 else "caveated"


async def run(task: str):
    r = (await delegate("assessor", task, RATINGS)).structured_output
    route = table(r["competence"], r["stakes"])
    if route == "plain":
        # the answerer gets the request only, not the ratings
        answer = await delegate("answerer", task)
        return {"route": "plain", "ratings": r, "answer": answer.result}
    reason = ""
    if route == "caveated":
        answer = (await delegate("cautious-answerer", f"Ratings: {r}\nRequest: {task}")).result or ""
        if not answer.strip().lower().startswith("escalate"):
            return {"route": "caveated", "ratings": r, "answer": answer}
        reason = answer  # the single allowed fall-through to escalation
    note = await delegate("handoff-writer", f"Ratings: {r}\nEscalate reply: {reason}\nRequest: {task}")
    return {"route": "escalated", "ratings": r, "handoff": note.result}

Replace SCOPE and STAKES with your own written scope and stakes list, and send the escalated handoff to a real queue a person watches.

OpenAI Agents SDK, python, package openai-agents
from agents import Agent, Runner
from pydantic import BaseModel, Field

SCOPE = "Billing questions for our subscription plans, using the plan and invoice data given."
STAKES = "money above one invoice, legal, health, or anything irreversible"


class Assessment(BaseModel):
    competence: int = Field(ge=1, le=3)
    competence_reason: str
    stakes: int = Field(ge=1, le=3)
    stakes_reason: str


assessor = Agent(
    name="Assessor", output_type=Assessment,
    instructions=f"Do not answer the request. Scope: {SCOPE}. Stakes list: {STAKES}. Rate your "
                 "competence 1-3 (in scope, facts available) and the stakes 1-3, one reason each.")
answerer = Agent(
    name="Answerer",
    instructions=f"Answer the request directly and briefly. Stay within this scope: {SCOPE}")
cautious = Agent(
    name="Cautious answerer",
    instructions="Answer, then list what you are unsure of and what the user should verify, and "
                 "with whom. If you cannot answer honestly even with caveats, reply 'escalate' "
                 "with one line of reasons.")
handoff_writer = Agent(
    name="Handoff writer",
    instructions="Write a handoff note of at most five lines for a person: what the user wants, "
                 "why it was escalated, what is known and what is missing. Then one polite line "
                 "telling the user a person will follow up.")


def table(a: Assessment) -> str:
    # the fixed routing table, applied in code
    if a.competence == 1 or (a.stakes == 3 and a.competence < 3):
        return "escalate"
    return "plain" if a.competence == 3 and a.stakes == 1 else "caveated"


async def run(task: str):
    a = (await Runner.run(assessor, task)).final_output_as(Assessment)
    route = table(a)
    if route == "plain":
        # the answerer sees the request only, not the ratings
        out = await Runner.run(answerer, task)
        return {"route": "plain", "ratings": a, "answer": out.final_output}
    reason = ""
    if route == "caveated":
        out = await Runner.run(cautious, f"Ratings: {a.model_dump()}\nRequest: {task}")
        text = str(out.final_output)
        if not text.strip().lower().startswith("escalate"):
            return {"route": "caveated", "ratings": a, "answer": text}
        reason = text  # one fall-through to escalation, then the run ends
    note = await Runner.run(
        handoff_writer, f"Ratings: {a.model_dump()}\nEscalate reply: {reason}\nRequest: {task}")
    return {"route": "escalated", "ratings": a, "handoff": note.final_output}

Replace SCOPE and STAKES with your own written scope and stakes list, and send the escalated handoff to a real queue a person watches.

Vercel AI SDK, typescript, package ai
import { generateText, Output } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
import { z } from 'zod';

const model = anthropic('claude-sonnet-5-5');
const SCOPE = 'Billing questions for our subscription plans, using the plan and invoice data given.';
const STAKES = 'money above one invoice, legal, health, or anything irreversible';

const Assessment = z.object({
  competence: z.number().int().min(1).max(3),
  competenceReason: z.string(),
  stakes: z.number().int().min(1).max(3),
  stakesReason: z.string(),
});
type Assessment = z.infer<typeof Assessment>;

// the fixed routing table, applied in code rather than by a model
function table(a: Assessment): 'plain' | 'caveated' | 'escalate' {
  if (a.competence === 1 || (a.stakes === 3 && a.competence < 3)) return 'escalate';
  return a.competence === 3 && a.stakes === 1 ? 'plain' : 'caveated';
}

export async function run(task: string) {
  const { output: ratings } = await generateText({
    model,
    instructions:
      `Do not answer the request. Scope: ${SCOPE}. Stakes list: ${STAKES}. Rate your ` +
      'competence 1-3 (in scope, facts available) and the stakes 1-3, one reason each.',
    prompt: task,
    output: Output.object({ schema: Assessment }),
  });
  const route = table(ratings);

  if (route === 'plain') {
    // the answerer sees the request only, never the ratings
    const { text } = await generateText({
      model,
      instructions: `Answer the request directly and briefly. Stay within this scope: ${SCOPE}`,
      prompt: task,
    });
    return { route, ratings, answer: text };
  }

  let reason = '';
  if (route === 'caveated') {
    const { text } = await generateText({
      model,
      instructions:
        'Answer, then list what you are unsure of and what the user should verify, and with ' +
        'whom. If you cannot answer honestly even with caveats, reply "escalate" with one line of reasons.',
      prompt: `Ratings: ${JSON.stringify(ratings)}\nRequest: ${task}`,
    });
    if (!text.trim().toLowerCase().startsWith('escalate')) return { route, ratings, answer: text };
    reason = text; // the single fall-through to escalation
  }

  const { text: handoff } = await generateText({
    model,
    instructions:
      'Write a handoff note of at most five lines for a person: what the user wants, why it was ' +
      'escalated, what is known and what is missing. Then one polite line telling the user a person will follow up.',
    prompt: `Ratings: ${JSON.stringify(ratings)}\nEscalate reply: ${reason}\nRequest: ${task}`,
  });
  return { route: 'escalated' as const, ratings, handoff };
}

Replace SCOPE and STAKES with your own written scope and stakes list, and send the escalated handoff to a real queue a person watches.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • Models overrate their own competence. Anchor the scope in a written list of topics and facts available, not in the model's sense of ease.
  • Caveats on every answer train users to ignore them. Reserve the caveated route for real uncertainty.
  • An escalation that reaches no one is worse than an answer. Test the handoff path end to end before launch.

Chain it

Its result can feed