Agentic Architectures

Catalogue / Memory

Tiered Memory

A small working memory, a large archive, and paging between them

Shape
memory-loop
Model calls
1 per turn + 1 per memory operation (usually 1 to 4)
Latency
medium
Complexity
high

Also known as MemGPT, Virtual context

1

What happens when the conversation no longer fits in the model's context?

Memory is split in two tiers, as in an operating system. A small core stays in the context at all times. Everything else lives in an archive outside it. The agent itself decides what to fetch from the archive and what to move out of the core.

You may need it if

  • Conversations last for weeks and early details are forgotten or cut off.
  • The assistant must remember a user's preferences and decisions between sessions.
  • Sending the whole history every time has become slow and expensive.
2

When is managed memory more trouble than it is worth?

Use it when

  • One assistant serves the same user or case over many sessions.
  • Most past content is irrelevant to any given turn, but some of it is essential.
  • You can store data per user between sessions.

Avoid it when

What it costs

Model calls
1 per turn + 1 per memory operation (usually 1 to 4)
Latency
medium
Complexity
high
3

How does a fact travel between the conversation and long-term memory?

Each message meets the responder, which always has the core memory in view. If a needed fact is missing, a memory keeper fetches it from the archive. Before replying, new facts are saved and the core is trimmed. Both tiers persist after the run.

fact missing orworth savingmemory resultready to answerMessage: The user's new message or request in an ongoing relationship.MessageCore memory: A small set of facts always placed in the context, such as the user profile and current goals.Core memoryResponder: Answers the user and decides when memory must be read or updated.ResponderMemory keeper: Searches the archive, stores new facts and moves old core content out.Memory keeperArchive: Unlimited long-term storage outside the context, searched on demand.ArchiveReply: The answer to the user, produced after memory has been updated.Reply
  1. Step 1 of 4

    The core is always in view

    The responder reads the message with the core memory already in its context. For most turns this is all it needs.

  2. Step 2 of 4

    Paging a fact in

    When the message refers to something not in the core, the responder asks the memory keeper, which searches the archive and returns only the relevant entries.

  3. Step 3 of 4

    Saving what is new

    New durable facts go to the archive. Facts needed on every turn, such as a changed preference, are written into the core, replacing what they supersede.

  4. Step 4 of 4

    Reply, with memory already updated

    The reply leaves only after the memory writes. The next session starts from the updated core and archive, not from the transcript.

What each part does
Message
The user's new message or request in an ongoing relationship.
Core memory
A small set of facts always placed in the context, such as the user profile and current goals.
Responder
Answers the user and decides when memory must be read or updated.
Memory keeper
Searches the archive, stores new facts and moves old core content out.
Archive
Unlimited long-term storage outside the context, searched on demand.
Reply
The answer to the user, produced after memory has been updated.
  • Responder → Memory keeper: fact missing or worth saving
  • Memory keeper → Responder: memory result
  • Responder → Core memory: update profile
  • Responder → Reply: ready to answer

Where it connects

Entry
message
TaskQuestion

One message from a known user or case, with an identifier that selects the right memory.

Exit
reply
Answer

When: The responder has what it needs and the memory writes for this turn are done.

The reply to the user, with a list of the memory entries read and written during the turn.

Memory kept between runs
core
Knowledge

A size-limited block of facts loaded into every turn. Entries are replaced, not accumulated.

Memory kept between runs
archive
KnowledgeEvidence

Dated entries stored outside the context and retrieved by search. Never loaded in full.

4

What do I ask my assistant to set up?

The agents to launch

Responder

Answer the user's message using the core memory. If you need a fact that is not in it, ask the memory keeper. Before replying, list any new durable fact to save and any core entry to replace. Never guess a remembered fact.

Sees: The message, the core memory and the entries returned by the memory keeper this turn. Not the full archive.

Memory keeper

Maintain the archive. On a search request, return at most five relevant entries with their dates. On a save request, store the fact as one dated entry and report any earlier entry it contradicts. Do not answer the user.

Sees: The search or save request and the archive. Not the conversation.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Tiered Memory" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Tiered Memory
A small working memory, a large archive, and paging between them.
Input: the task above. It must provide: One message from a known user or case, with an identifier that selects the right memory. If something essential is missing, ask for it before starting.

Agents to launch:
- Responder: Answer the user's message using the core memory. If you need a fact that is not in it, ask the memory keeper. Before replying, list any new durable fact to save and any core entry to replace. Never guess a remembered fact.
  Context for this agent: The message, the core memory and the entries returned by the memory keeper this turn. Not the full archive.
- Memory keeper: Maintain the archive. On a search request, return at most five relevant entries with their dates. On a save request, store the fact as one dated entry and report any earlier entry it contradicts. Do not answer the user.
  Context for this agent: The search or save request and the archive. Not the conversation.

Procedure:
1. Create two stores for this user. A core memory file limited to about 300 words, and an archive of dated entries that can be searched.
2. Launch the responder with the message and the core memory. Launch the memory keeper as a separate agent with access to the archive.
3. Whenever the responder asks for a fact, pass the request to the memory keeper and return its entries. Allow at most three requests per turn.
4. Before the reply is released, have the memory keeper save each new durable fact, and apply the responder's replacements to the core memory. If the core exceeds its limit, move its least-used entry to the archive.
5. Release the reply to the user once both stores are updated.

Stop rule: Each turn ends when the reply is released after the memory writes, or after three memory requests. The stores are kept for the next turn.

Exit:
- `reply`, when: The responder has what it needs and the memory writes for this turn are done. Deliver: The reply to the user, with a list of the memory entries read and written during the turn.

# Final deliverable
The reply, then the list of archive entries read, archive entries written and core entries changed during the turn.

The same thing in code

LangGraph, python, package langgraph
from datetime import date
from typing import TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel

model = init_chat_model("anthropic:claude-sonnet-5-5")

CORE: list[str] = []     # core memory: in every turn's context, about 300 words
ARCHIVE: list[str] = []  # archive: dated entries, never loaded in full
CORE_WORD_LIMIT = 300
MAX_REQUESTS = 3

def search_archive(query: str) -> list[str]:
    # plug in a vector or full-text search over the archive
    words = query.lower().split()
    return [e for e in ARCHIVE if any(w in e.lower() for w in words)][:20]

class CoreEdit(BaseModel):
    replaces: str  # the core entry it supersedes, or "" to add
    new: str

class Turn(BaseModel):
    fetch: str | None  # a fact to page in from the archive, if missing
    reply: str
    save: list[str]
    core_edits: list[CoreEdit]

class Entries(BaseModel):
    entries: list[str]

class State(TypedDict):
    message: str
    fetched: list[str]
    requests: int
    turn: Turn | None

RESPONDER = ("Answer the user using the core memory. If a fact you need is missing, set fetch. "
             "List new durable facts in save and superseded core entries in core_edits. "
             "Never guess a remembered fact.")
KEEPER = "Return at most five archive entries relevant to the request, with their dates. Do not answer the user."

async def responder(state: State) -> dict:
    # sees the message, the core and this turn's fetched entries, never the archive
    prompt = (f"{RESPONDER}\n\nCore memory:\n" + "\n".join(CORE) +
              "\n\nFetched this turn:\n" + "\n".join(state["fetched"]) + f"\n\nMessage: {state['message']}")
    turn = Turn.model_validate(await model.with_structured_output(Turn).ainvoke(prompt))
    return {"turn": turn}

async def keeper(state: State) -> dict:
    # sees the request and the archive candidates, not the conversation
    assert state["turn"] and state["turn"].fetch
    request = state["turn"].fetch
    candidates = "\n".join(search_archive(request))
    picked = Entries.model_validate(await model.with_structured_output(Entries).ainvoke(
        f"{KEEPER}\n\nRequest: {request}\n\nArchive candidates:\n{candidates}"))
    return {"fetched": state["fetched"] + picked.entries[:5], "requests": state["requests"] + 1}

async def commit(state: State) -> dict:
    # saving is a required step: the reply is released only after both stores are updated
    assert state["turn"]
    for fact in state["turn"].save:
        ARCHIVE.append(f"{date.today().isoformat()}: {fact}")
    for edit in state["turn"].core_edits:
        if edit.replaces in CORE:
            CORE.remove(edit.replaces)
        CORE.append(edit.new)
    while sum(len(e.split()) for e in CORE) > CORE_WORD_LIMIT:
        ARCHIVE.append(f"{date.today().isoformat()}: {CORE.pop(0)}")  # oldest core entry pages out
    return {}

def route(state: State) -> str:
    turn = state["turn"]
    if turn and turn.fetch and state["requests"] < MAX_REQUESTS:  # at most three memory requests per turn
        return "keeper"
    return "commit"

graph = StateGraph(State)
graph.add_node("responder", responder)
graph.add_node("keeper", keeper)
graph.add_node("commit", commit)
graph.add_edge(START, "responder")
graph.add_conditional_edges("responder", route, ["keeper", "commit"])
graph.add_edge("keeper", "responder")
graph.add_edge("commit", END)
app = graph.compile()

async def run(task: str):
    state = await app.ainvoke({"message": task, "fetched": [], "requests": 0, "turn": None})
    turn = state["turn"]
    return {"reply": turn.reply, "read": state["fetched"], "written": turn.save, "core": list(CORE)}

Key CORE and ARCHIVE by user id and persist them in a real store; module-level lists would leak one user's memory into another's.

Claude Agent SDK, python, package claude-agent-sdk
from datetime import date
from typing import Any

from claude_agent_sdk import (AgentDefinition, ClaudeAgentOptions, ResultMessage,
                              create_sdk_mcp_server, query, tool)
from pydantic import BaseModel

CORE: list[str] = []     # core memory: placed in the responder's prompt every turn
ARCHIVE: list[str] = []  # archive: reachable only through the keeper's tools
CORE_WORD_LIMIT = 300
MAX_REQUESTS = 3
turn_log: dict[str, list[str]] = {"read": [], "written": []}

def text(s: str, error: bool = False) -> dict[str, Any]:
    return {"content": [{"type": "text", "text": s}], "is_error": error}

@tool("search_archive", "Search the archive for dated entries.", {"query": str})
async def search_archive(args: dict[str, Any]) -> dict[str, Any]:
    if len(turn_log["read"]) >= MAX_REQUESTS:  # hard limit: three memory requests per turn
        return text("Memory request limit reached for this turn.", error=True)
    turn_log["read"].append(args["query"])
    # plug in a vector or full-text search over the archive
    words = args["query"].lower().split()
    return text("\n".join([e for e in ARCHIVE if any(w in e.lower() for w in words)][:5]) or "Nothing found.")

@tool("save_entry", "Store one durable fact as a dated archive entry.", {"fact": str})
async def save_entry(args: dict[str, Any]) -> dict[str, Any]:
    ARCHIVE.append(f"{date.today().isoformat()}: {args['fact']}")
    turn_log["written"].append(args["fact"])
    return text("Saved.")

archive_server = create_sdk_mcp_server("archive", tools=[search_archive, save_entry])
KEEPER_TOOLS = ["mcp__archive__search_archive", "mcp__archive__save_entry"]

class TurnResult(BaseModel):
    reply: str
    save: list[str]
    core_edits: dict[str, str]  # superseded core entry (or "") -> new entry

async def run(task: str):
    turn_log["read"], turn_log["written"] = [], []
    agents = {
        "responder": AgentDefinition(
            description="Answers the user from the core memory and the entries it is given.",
            prompt=("Answer the message using the core memory below. If a fact is missing, say which. "
                    "List new durable facts to save and core entries to replace. Never guess a remembered fact.\n\n"
                    "Core memory:\n" + "\n".join(CORE)),
            tools=[]),  # no archive access: sees the core and what the keeper returns
        "keeper": AgentDefinition(
            description="Searches and maintains the archive. Never answers the user.",
            prompt=("On a search request return at most five relevant entries with dates. On a save request "
                    "store the fact as one dated entry and report any earlier entry it contradicts."),
            tools=KEEPER_TOOLS),  # sees the request and the archive, not the conversation
    }
    options = ClaudeAgentOptions(
        agents=agents, mcp_servers={"archive": archive_server},
        allowed_tools=["Agent", *KEEPER_TOOLS], permission_mode="dontAsk",
        output_format={"type": "json_schema", "schema": TurnResult.model_json_schema()})
    prompt = (f"User message: {task}\n\n1. Give the message to the responder agent. "
              "2. If it names a missing fact, ask the keeper agent to search, then give the entries back to "
              "the responder. 3. Ask the keeper to save every fact the responder lists. "
              "4. Return the reply, the facts to save and the core edits.")
    result: TurnResult | None = None
    async for msg in query(prompt=prompt, options=options):
        if isinstance(msg, ResultMessage) and msg.structured_output:
            result = TurnResult.model_validate(msg.structured_output)
    if result is None:
        raise RuntimeError("the turn produced no result")
    for fact in result.save:  # saving is required before release, even if the keeper skipped it
        if fact not in turn_log["written"]:
            await save_entry.handler({"fact": fact})
    for old, new in result.core_edits.items():
        if old in CORE:
            CORE.remove(old)
        CORE.append(new)
    while sum(len(e.split()) for e in CORE) > CORE_WORD_LIMIT:
        ARCHIVE.append(f"{date.today().isoformat()}: {CORE.pop(0)}")  # oldest core entry pages out
    return {"reply": result.reply, **turn_log, "core": list(CORE)}

Key CORE and ARCHIVE by user id and back them with a persistent store; the in-process lists here are shared by every caller.

OpenAI Agents SDK, python, package openai-agents
from datetime import date

from agents import Agent, Runner, function_tool
from pydantic import BaseModel

CORE: list[str] = []     # core memory: in the responder's instructions every turn
ARCHIVE: list[str] = []  # archive: only the keeper's tools touch it
CORE_WORD_LIMIT = 300
MAX_REQUESTS = 3
turn_log: dict[str, list[str]] = {"read": [], "written": []}

@function_tool
def search_archive(query: str) -> list[str]:
    """Search the archive for dated entries."""
    # plug in a vector or full-text search over the archive
    words = query.lower().split()
    return [e for e in ARCHIVE if any(w in e.lower() for w in words)][:5]

@function_tool
def save_entry(fact: str) -> str:
    """Store one durable fact as a dated archive entry."""
    ARCHIVE.append(f"{date.today().isoformat()}: {fact}")
    turn_log["written"].append(fact)
    return "saved"

keeper = Agent(
    name="Memory keeper",
    instructions=("On a search request return at most five relevant entries with dates. On a save request "
                  "store the fact as one dated entry and report any earlier entry it contradicts. "
                  "Do not answer the user."),
    tools=[search_archive, save_entry])

@function_tool
async def ask_keeper(request: str) -> str:
    """Ask the memory keeper for a fact that is not in the core memory."""
    if len(turn_log["read"]) >= MAX_REQUESTS:  # hard limit: three memory requests per turn
        return "Memory request limit reached; answer with what you have."
    turn_log["read"].append(request)
    result = await Runner.run(keeper, request)  # the keeper sees the request, not the conversation
    return str(result.final_output)

class CoreEdit(BaseModel):
    replaces: str  # the superseded core entry, or "" to add
    new: str

class Turn(BaseModel):
    reply: str
    save: list[str]
    core_edits: list[CoreEdit]

async def run(task: str):
    turn_log["read"], turn_log["written"] = [], []
    responder = Agent(
        name="Responder",
        instructions=("Answer the user using the core memory. If you need a fact not in it, call ask_keeper. "
                      "List new durable facts in save and superseded core entries in core_edits. "
                      "Never guess a remembered fact.\n\nCore memory:\n" + "\n".join(CORE)),
        tools=[ask_keeper],
        output_type=Turn)
    turn = (await Runner.run(responder, task)).final_output_as(Turn)
    for fact in turn.save:  # the save step is required before the reply is released
        await Runner.run(keeper, f"Save this fact: {fact}")
    for edit in turn.core_edits:
        if edit.replaces in CORE:
            CORE.remove(edit.replaces)
        CORE.append(edit.new)
    while sum(len(e.split()) for e in CORE) > CORE_WORD_LIMIT:
        ARCHIVE.append(f"{date.today().isoformat()}: {CORE.pop(0)}")  # oldest core entry pages out
    return {"reply": turn.reply, **turn_log, "core": list(CORE)}

Key CORE and ARCHIVE by user id and persist them; module-level lists would mix users' memories.

Vercel AI SDK, typescript, package ai
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, isStepCount, Output, tool } from 'ai';
import { z } from 'zod';

const model = anthropic('claude-sonnet-5-5');
const CORE: string[] = [];    // core memory: in the responder's instructions every turn
const ARCHIVE: string[] = []; // archive: only the keeper's tools touch it
const CORE_WORD_LIMIT = 300;
const MAX_REQUESTS = 3;
const today = () => new Date().toISOString().slice(0, 10);

const keeperTools = {
  searchArchive: tool({
    description: 'Search the archive for dated entries.',
    inputSchema: z.object({ query: z.string() }),
    // plug in a vector or full-text search over the archive
    execute: async ({ query }) => {
      const words = query.toLowerCase().split(/\s+/);
      return ARCHIVE.filter((e) => words.some((w) => e.toLowerCase().includes(w))).slice(0, 5);
    },
  }),
  saveEntry: tool({
    description: 'Store one durable fact as a dated archive entry.',
    inputSchema: z.object({ fact: z.string() }),
    execute: async ({ fact }) => { ARCHIVE.push(`${today()}: ${fact}`); return 'saved'; },
  }),
};

// the keeper sees one request and the archive, never the conversation
async function keeper(request: string): Promise<string> {
  const { text } = await generateText({
    model,
    instructions: 'On a search request return at most five relevant entries with dates. On a save request store '
      + 'the fact as one dated entry and report any earlier entry it contradicts. Do not answer the user.',
    prompt: request,
    tools: keeperTools,
    stopWhen: isStepCount(4),
  });
  return text;
}

const Turn = z.object({
  reply: z.string(),
  save: z.array(z.string()),
  coreEdits: z.array(z.object({ replaces: z.string(), next: z.string() })),
});

export async function run(task: string) {
  const read: string[] = [];
  const { output: turn } = await generateText({
    model,
    instructions: 'Answer the user using the core memory. If you need a fact not in it, call askKeeper. '
      + 'List new durable facts in save and superseded core entries in coreEdits. Never guess a remembered fact.'
      + `\n\nCore memory:\n${CORE.join('\n')}`,
    prompt: task,
    tools: {
      askKeeper: tool({
        description: 'Ask the memory keeper for a fact that is not in the core memory.',
        inputSchema: z.object({ request: z.string() }),
        execute: async ({ request }) => {
          if (read.length >= MAX_REQUESTS) return 'Memory request limit reached; answer with what you have.';
          read.push(request); // hard limit: three memory requests per turn
          return keeper(request);
        },
      }),
    },
    output: Output.object({ schema: Turn }),
    stopWhen: isStepCount(MAX_REQUESTS + 2),
  });
  for (const fact of turn.save) await keeper(`Save this fact: ${fact}`); // required before release
  for (const edit of turn.coreEdits) {
    const i = CORE.indexOf(edit.replaces);
    if (i >= 0) CORE.splice(i, 1);
    CORE.push(edit.next);
  }
  const words = () => CORE.reduce((n, e) => n + e.split(/\s+/).length, 0);
  while (words() > CORE_WORD_LIMIT) ARCHIVE.push(`${today()}: ${CORE.shift()}`); // oldest entry pages out
  return { reply: turn.reply, read, written: turn.save, core: [...CORE] };
}

Key CORE and ARCHIVE by user id and persist them in a database; module-level arrays would mix users' memories.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • Agents skip the save step when the reply feels complete. Make saving a required step before release, not an option.
  • Appending to the core without replacing fills it with stale, contradictory facts.
  • Memory is per user. A shared store leaks one user's facts into another's conversation.

Chain it

Its result can feed

Source