1What happens when the conversation no longer fits in the model's context?
Memory is split in two tiers, as in an operating system. A small core stays in the context at all times. Everything else lives in an archive outside it. The agent itself decides what to fetch from the archive and what to move out of the core.
You may need it if
- Conversations last for weeks and early details are forgotten or cut off.
- The assistant must remember a user's preferences and decisions between sessions.
- Sending the whole history every time has become slow and expensive.
2When is managed memory more trouble than it is worth?
Use it when
- One assistant serves the same user or case over many sessions.
- Most past content is irrelevant to any given turn, but some of it is essential.
- You can store data per user between sessions.
Avoid it when
- Each request is independent and nothing needs to be remembered afterwards.
Use Tool Use instead - The knowledge is a body of documents to consult, not a history to maintain.
Use Agentic RAG instead - What must be remembered is how facts relate to each other, such as who reports to whom.
Use Graph Memory instead
What it costs
- Model calls
- 1 per turn + 1 per memory operation (usually 1 to 4)
- Latency
- medium
- Complexity
- high
3How does a fact travel between the conversation and long-term memory?
Each message meets the responder, which always has the core memory in view. If a needed fact is missing, a memory keeper fetches it from the archive. Before replying, new facts are saved and the core is trimmed. Both tiers persist after the run.
- Step 1 of 4
The core is always in view
The responder reads the message with the core memory already in its context. For most turns this is all it needs.
- Step 2 of 4
Paging a fact in
When the message refers to something not in the core, the responder asks the memory keeper, which searches the archive and returns only the relevant entries.
- Step 3 of 4
Saving what is new
New durable facts go to the archive. Facts needed on every turn, such as a changed preference, are written into the core, replacing what they supersede.
- Step 4 of 4
Reply, with memory already updated
The reply leaves only after the memory writes. The next session starts from the updated core and archive, not from the transcript.
What each part does
- Message
- The user's new message or request in an ongoing relationship.
- Core memory
- A small set of facts always placed in the context, such as the user profile and current goals.
- Responder
- Answers the user and decides when memory must be read or updated.
- Memory keeper
- Searches the archive, stores new facts and moves old core content out.
- Archive
- Unlimited long-term storage outside the context, searched on demand.
- Reply
- The answer to the user, produced after memory has been updated.
- Responder → Memory keeper: fact missing or worth saving
- Memory keeper → Responder: memory result
- Responder → Core memory: update profile
- Responder → Reply: ready to answer
Where it connects
One message from a known user or case, with an identifier that selects the right memory.
When: The responder has what it needs and the memory writes for this turn are done.
The reply to the user, with a list of the memory entries read and written during the turn.
A size-limited block of facts loaded into every turn. Entries are replaced, not accumulated.
Dated entries stored outside the context and retrieved by search. Never loaded in full.
4What do I ask my assistant to set up?
The agents to launch
Responder
Answer the user's message using the core memory. If you need a fact that is not in it, ask the memory keeper. Before replying, list any new durable fact to save and any core entry to replace. Never guess a remembered fact.
Sees: The message, the core memory and the entries returned by the memory keeper this turn. Not the full archive.
Memory keeper
Maintain the archive. On a search request, return at most five relevant entries with their dates. On a save request, store the fact as one dated entry and report any earlier entry it contradicts. Do not answer the user.
Sees: The search or save request and the archive. Not the conversation.
A prompt you can paste
Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.
You are the orchestrator of a multi-agent workflow that follows the "Tiered Memory" architecture. Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it. # Task [Describe your task here] # Tiered Memory A small working memory, a large archive, and paging between them. Input: the task above. It must provide: One message from a known user or case, with an identifier that selects the right memory. If something essential is missing, ask for it before starting. Agents to launch: - Responder: Answer the user's message using the core memory. If you need a fact that is not in it, ask the memory keeper. Before replying, list any new durable fact to save and any core entry to replace. Never guess a remembered fact. Context for this agent: The message, the core memory and the entries returned by the memory keeper this turn. Not the full archive. - Memory keeper: Maintain the archive. On a search request, return at most five relevant entries with their dates. On a save request, store the fact as one dated entry and report any earlier entry it contradicts. Do not answer the user. Context for this agent: The search or save request and the archive. Not the conversation. Procedure: 1. Create two stores for this user. A core memory file limited to about 300 words, and an archive of dated entries that can be searched. 2. Launch the responder with the message and the core memory. Launch the memory keeper as a separate agent with access to the archive. 3. Whenever the responder asks for a fact, pass the request to the memory keeper and return its entries. Allow at most three requests per turn. 4. Before the reply is released, have the memory keeper save each new durable fact, and apply the responder's replacements to the core memory. If the core exceeds its limit, move its least-used entry to the archive. 5. Release the reply to the user once both stores are updated. Stop rule: Each turn ends when the reply is released after the memory writes, or after three memory requests. The stores are kept for the next turn. Exit: - `reply`, when: The responder has what it needs and the memory writes for this turn are done. Deliver: The reply to the user, with a list of the memory entries read and written during the turn. # Final deliverable The reply, then the list of archive entries read, archive entries written and core entries changed during the turn.
The same thing in code
from datetime import date
from typing import TypedDict
from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel
model = init_chat_model("anthropic:claude-sonnet-5-5")
CORE: list[str] = [] # core memory: in every turn's context, about 300 words
ARCHIVE: list[str] = [] # archive: dated entries, never loaded in full
CORE_WORD_LIMIT = 300
MAX_REQUESTS = 3
def search_archive(query: str) -> list[str]:
# plug in a vector or full-text search over the archive
words = query.lower().split()
return [e for e in ARCHIVE if any(w in e.lower() for w in words)][:20]
class CoreEdit(BaseModel):
replaces: str # the core entry it supersedes, or "" to add
new: str
class Turn(BaseModel):
fetch: str | None # a fact to page in from the archive, if missing
reply: str
save: list[str]
core_edits: list[CoreEdit]
class Entries(BaseModel):
entries: list[str]
class State(TypedDict):
message: str
fetched: list[str]
requests: int
turn: Turn | None
RESPONDER = ("Answer the user using the core memory. If a fact you need is missing, set fetch. "
"List new durable facts in save and superseded core entries in core_edits. "
"Never guess a remembered fact.")
KEEPER = "Return at most five archive entries relevant to the request, with their dates. Do not answer the user."
async def responder(state: State) -> dict:
# sees the message, the core and this turn's fetched entries, never the archive
prompt = (f"{RESPONDER}\n\nCore memory:\n" + "\n".join(CORE) +
"\n\nFetched this turn:\n" + "\n".join(state["fetched"]) + f"\n\nMessage: {state['message']}")
turn = Turn.model_validate(await model.with_structured_output(Turn).ainvoke(prompt))
return {"turn": turn}
async def keeper(state: State) -> dict:
# sees the request and the archive candidates, not the conversation
assert state["turn"] and state["turn"].fetch
request = state["turn"].fetch
candidates = "\n".join(search_archive(request))
picked = Entries.model_validate(await model.with_structured_output(Entries).ainvoke(
f"{KEEPER}\n\nRequest: {request}\n\nArchive candidates:\n{candidates}"))
return {"fetched": state["fetched"] + picked.entries[:5], "requests": state["requests"] + 1}
async def commit(state: State) -> dict:
# saving is a required step: the reply is released only after both stores are updated
assert state["turn"]
for fact in state["turn"].save:
ARCHIVE.append(f"{date.today().isoformat()}: {fact}")
for edit in state["turn"].core_edits:
if edit.replaces in CORE:
CORE.remove(edit.replaces)
CORE.append(edit.new)
while sum(len(e.split()) for e in CORE) > CORE_WORD_LIMIT:
ARCHIVE.append(f"{date.today().isoformat()}: {CORE.pop(0)}") # oldest core entry pages out
return {}
def route(state: State) -> str:
turn = state["turn"]
if turn and turn.fetch and state["requests"] < MAX_REQUESTS: # at most three memory requests per turn
return "keeper"
return "commit"
graph = StateGraph(State)
graph.add_node("responder", responder)
graph.add_node("keeper", keeper)
graph.add_node("commit", commit)
graph.add_edge(START, "responder")
graph.add_conditional_edges("responder", route, ["keeper", "commit"])
graph.add_edge("keeper", "responder")
graph.add_edge("commit", END)
app = graph.compile()
async def run(task: str):
state = await app.ainvoke({"message": task, "fetched": [], "requests": 0, "turn": None})
turn = state["turn"]
return {"reply": turn.reply, "read": state["fetched"], "written": turn.save, "core": list(CORE)}
Key CORE and ARCHIVE by user id and persist them in a real store; module-level lists would leak one user's memory into another's.
from datetime import date
from typing import Any
from claude_agent_sdk import (AgentDefinition, ClaudeAgentOptions, ResultMessage,
create_sdk_mcp_server, query, tool)
from pydantic import BaseModel
CORE: list[str] = [] # core memory: placed in the responder's prompt every turn
ARCHIVE: list[str] = [] # archive: reachable only through the keeper's tools
CORE_WORD_LIMIT = 300
MAX_REQUESTS = 3
turn_log: dict[str, list[str]] = {"read": [], "written": []}
def text(s: str, error: bool = False) -> dict[str, Any]:
return {"content": [{"type": "text", "text": s}], "is_error": error}
@tool("search_archive", "Search the archive for dated entries.", {"query": str})
async def search_archive(args: dict[str, Any]) -> dict[str, Any]:
if len(turn_log["read"]) >= MAX_REQUESTS: # hard limit: three memory requests per turn
return text("Memory request limit reached for this turn.", error=True)
turn_log["read"].append(args["query"])
# plug in a vector or full-text search over the archive
words = args["query"].lower().split()
return text("\n".join([e for e in ARCHIVE if any(w in e.lower() for w in words)][:5]) or "Nothing found.")
@tool("save_entry", "Store one durable fact as a dated archive entry.", {"fact": str})
async def save_entry(args: dict[str, Any]) -> dict[str, Any]:
ARCHIVE.append(f"{date.today().isoformat()}: {args['fact']}")
turn_log["written"].append(args["fact"])
return text("Saved.")
archive_server = create_sdk_mcp_server("archive", tools=[search_archive, save_entry])
KEEPER_TOOLS = ["mcp__archive__search_archive", "mcp__archive__save_entry"]
class TurnResult(BaseModel):
reply: str
save: list[str]
core_edits: dict[str, str] # superseded core entry (or "") -> new entry
async def run(task: str):
turn_log["read"], turn_log["written"] = [], []
agents = {
"responder": AgentDefinition(
description="Answers the user from the core memory and the entries it is given.",
prompt=("Answer the message using the core memory below. If a fact is missing, say which. "
"List new durable facts to save and core entries to replace. Never guess a remembered fact.\n\n"
"Core memory:\n" + "\n".join(CORE)),
tools=[]), # no archive access: sees the core and what the keeper returns
"keeper": AgentDefinition(
description="Searches and maintains the archive. Never answers the user.",
prompt=("On a search request return at most five relevant entries with dates. On a save request "
"store the fact as one dated entry and report any earlier entry it contradicts."),
tools=KEEPER_TOOLS), # sees the request and the archive, not the conversation
}
options = ClaudeAgentOptions(
agents=agents, mcp_servers={"archive": archive_server},
allowed_tools=["Agent", *KEEPER_TOOLS], permission_mode="dontAsk",
output_format={"type": "json_schema", "schema": TurnResult.model_json_schema()})
prompt = (f"User message: {task}\n\n1. Give the message to the responder agent. "
"2. If it names a missing fact, ask the keeper agent to search, then give the entries back to "
"the responder. 3. Ask the keeper to save every fact the responder lists. "
"4. Return the reply, the facts to save and the core edits.")
result: TurnResult | None = None
async for msg in query(prompt=prompt, options=options):
if isinstance(msg, ResultMessage) and msg.structured_output:
result = TurnResult.model_validate(msg.structured_output)
if result is None:
raise RuntimeError("the turn produced no result")
for fact in result.save: # saving is required before release, even if the keeper skipped it
if fact not in turn_log["written"]:
await save_entry.handler({"fact": fact})
for old, new in result.core_edits.items():
if old in CORE:
CORE.remove(old)
CORE.append(new)
while sum(len(e.split()) for e in CORE) > CORE_WORD_LIMIT:
ARCHIVE.append(f"{date.today().isoformat()}: {CORE.pop(0)}") # oldest core entry pages out
return {"reply": result.reply, **turn_log, "core": list(CORE)}
Key CORE and ARCHIVE by user id and back them with a persistent store; the in-process lists here are shared by every caller.
from datetime import date
from agents import Agent, Runner, function_tool
from pydantic import BaseModel
CORE: list[str] = [] # core memory: in the responder's instructions every turn
ARCHIVE: list[str] = [] # archive: only the keeper's tools touch it
CORE_WORD_LIMIT = 300
MAX_REQUESTS = 3
turn_log: dict[str, list[str]] = {"read": [], "written": []}
@function_tool
def search_archive(query: str) -> list[str]:
"""Search the archive for dated entries."""
# plug in a vector or full-text search over the archive
words = query.lower().split()
return [e for e in ARCHIVE if any(w in e.lower() for w in words)][:5]
@function_tool
def save_entry(fact: str) -> str:
"""Store one durable fact as a dated archive entry."""
ARCHIVE.append(f"{date.today().isoformat()}: {fact}")
turn_log["written"].append(fact)
return "saved"
keeper = Agent(
name="Memory keeper",
instructions=("On a search request return at most five relevant entries with dates. On a save request "
"store the fact as one dated entry and report any earlier entry it contradicts. "
"Do not answer the user."),
tools=[search_archive, save_entry])
@function_tool
async def ask_keeper(request: str) -> str:
"""Ask the memory keeper for a fact that is not in the core memory."""
if len(turn_log["read"]) >= MAX_REQUESTS: # hard limit: three memory requests per turn
return "Memory request limit reached; answer with what you have."
turn_log["read"].append(request)
result = await Runner.run(keeper, request) # the keeper sees the request, not the conversation
return str(result.final_output)
class CoreEdit(BaseModel):
replaces: str # the superseded core entry, or "" to add
new: str
class Turn(BaseModel):
reply: str
save: list[str]
core_edits: list[CoreEdit]
async def run(task: str):
turn_log["read"], turn_log["written"] = [], []
responder = Agent(
name="Responder",
instructions=("Answer the user using the core memory. If you need a fact not in it, call ask_keeper. "
"List new durable facts in save and superseded core entries in core_edits. "
"Never guess a remembered fact.\n\nCore memory:\n" + "\n".join(CORE)),
tools=[ask_keeper],
output_type=Turn)
turn = (await Runner.run(responder, task)).final_output_as(Turn)
for fact in turn.save: # the save step is required before the reply is released
await Runner.run(keeper, f"Save this fact: {fact}")
for edit in turn.core_edits:
if edit.replaces in CORE:
CORE.remove(edit.replaces)
CORE.append(edit.new)
while sum(len(e.split()) for e in CORE) > CORE_WORD_LIMIT:
ARCHIVE.append(f"{date.today().isoformat()}: {CORE.pop(0)}") # oldest core entry pages out
return {"reply": turn.reply, **turn_log, "core": list(CORE)}
Key CORE and ARCHIVE by user id and persist them; module-level lists would mix users' memories.
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, isStepCount, Output, tool } from 'ai';
import { z } from 'zod';
const model = anthropic('claude-sonnet-5-5');
const CORE: string[] = []; // core memory: in the responder's instructions every turn
const ARCHIVE: string[] = []; // archive: only the keeper's tools touch it
const CORE_WORD_LIMIT = 300;
const MAX_REQUESTS = 3;
const today = () => new Date().toISOString().slice(0, 10);
const keeperTools = {
searchArchive: tool({
description: 'Search the archive for dated entries.',
inputSchema: z.object({ query: z.string() }),
// plug in a vector or full-text search over the archive
execute: async ({ query }) => {
const words = query.toLowerCase().split(/\s+/);
return ARCHIVE.filter((e) => words.some((w) => e.toLowerCase().includes(w))).slice(0, 5);
},
}),
saveEntry: tool({
description: 'Store one durable fact as a dated archive entry.',
inputSchema: z.object({ fact: z.string() }),
execute: async ({ fact }) => { ARCHIVE.push(`${today()}: ${fact}`); return 'saved'; },
}),
};
// the keeper sees one request and the archive, never the conversation
async function keeper(request: string): Promise<string> {
const { text } = await generateText({
model,
instructions: 'On a search request return at most five relevant entries with dates. On a save request store '
+ 'the fact as one dated entry and report any earlier entry it contradicts. Do not answer the user.',
prompt: request,
tools: keeperTools,
stopWhen: isStepCount(4),
});
return text;
}
const Turn = z.object({
reply: z.string(),
save: z.array(z.string()),
coreEdits: z.array(z.object({ replaces: z.string(), next: z.string() })),
});
export async function run(task: string) {
const read: string[] = [];
const { output: turn } = await generateText({
model,
instructions: 'Answer the user using the core memory. If you need a fact not in it, call askKeeper. '
+ 'List new durable facts in save and superseded core entries in coreEdits. Never guess a remembered fact.'
+ `\n\nCore memory:\n${CORE.join('\n')}`,
prompt: task,
tools: {
askKeeper: tool({
description: 'Ask the memory keeper for a fact that is not in the core memory.',
inputSchema: z.object({ request: z.string() }),
execute: async ({ request }) => {
if (read.length >= MAX_REQUESTS) return 'Memory request limit reached; answer with what you have.';
read.push(request); // hard limit: three memory requests per turn
return keeper(request);
},
}),
},
output: Output.object({ schema: Turn }),
stopWhen: isStepCount(MAX_REQUESTS + 2),
});
for (const fact of turn.save) await keeper(`Save this fact: ${fact}`); // required before release
for (const edit of turn.coreEdits) {
const i = CORE.indexOf(edit.replaces);
if (i >= 0) CORE.splice(i, 1);
CORE.push(edit.next);
}
const words = () => CORE.reduce((n, e) => n + e.split(/\s+/).length, 0);
while (words() > CORE_WORD_LIMIT) ARCHIVE.push(`${today()}: ${CORE.shift()}`); // oldest entry pages out
return { reply: turn.reply, read, written: turn.save, core: [...CORE] };
}
Key CORE and ARCHIVE by user id and persist them in a database; module-level arrays would mix users' memories.
Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.
What goes wrong
- Agents skip the save step when the reply feels complete. Make saving a required step before release, not an option.
- Appending to the core without replacing fills it with stale, contradictory facts.
- Memory is per user. A shared store leaks one user's facts into another's conversation.
Chain it
Its result can feed
- Reflection direct
Treat the reply as an existing answer to improve against criteria for the original message. Keep the facts recalled from memory as given, and keep the memory read/write list as context only.
- Chain of Verification conditional
Condition: Use the link when the reply makes factual claims about the world, not only recalls user- or case-specific facts held in memory.
- Debate conditional
Condition: Use the link when the user's message asks a question or decision whose answer is open to argument, not a conversational turn or a plain recall of stored facts.
- Constitutional Critique direct
Treat the reply as a draft to check against every rule of the constitution. Fix only the failed rules and keep its memory-based content intact.
- Textual Backpropagation direct
Treat the memory agent's reply as an existing answer to the original task, and evaluate and improve it through the configured pipeline. Its list of memory reads and writes is context only.