1What if the memory work was done while nobody is waiting?
Two runs share one memory. During a conversation, a primary agent answers from it and does not reorganise it. Between conversations, with no message at all, a second agent rereads what has accumulated and rewrites the memory: it merges, corrects and works out in advance what later questions are likely to need.
You may need it if
- Replies are slow because the agent sorts out its memory or rereads raw history while the user waits.
- The memory fills with duplicates and facts that later conversations have contradicted.
- Many questions about the same context each redo the same reasoning from the raw material.
- The system sits idle between sessions, and that time is not used.
2When does thinking ahead of the question not pay off?
Use it when
- The same context, such as one user's history, a case file or a codebase, is questioned again and again.
- Later questions can largely be guessed from what is stored.
- There are idle periods between conversations in which background runs are acceptable.
- Replies must stay fast, so the request path cannot carry the memory upkeep.
Avoid it when
- A fact must be usable right away, in the same session or the next minute. The rewritten memory only exists after the next pass.
Use Tiered Memory instead - Questions cannot be guessed from the stored context. The conclusions prepared in advance go unused, so the work belongs at question time.
Use Agentic RAG instead - Each context is questioned once and then dropped. The pass costs more calls than the single question it would prepare.
- What should be kept is the method for a recurring kind of task, not facts about a user or a case.
Use Workflow Memory instead
What it costs
- Model calls
- 1 per turn on the request path; up to 10 per idle pass, off it
- Latency
- low
- Complexity
- high
3How does a run with no user message get started, and what does it do?
Two separate runs around one memory. A message goes to the primary agent, which reads the memory and replies. When the system has been idle, a trigger starts the consolidation agent on the raw context gathered since the last pass. It rewrites the memory. The next conversation starts from the new version.
- Step 1 of 6
A turn reads the memory and replies
A customer asks a support assistant where their refund stands. The primary agent has the memory of this account in view and answers in one call. It does not tidy anything. The exchange is added to the raw context kept for later.
- Step 2 of 6
Nobody writes, and a run starts
Thirty minutes pass without a message. A timer fires. It is not a user turn and carries no question. It hands over the three conversations of the day, exactly as they happened.
- Step 3 of 6
The consolidation agent rereads
It reads the transcripts next to the current memory. The memory says twice that the customer prefers email. It still holds the address that was corrected on Tuesday. Two open tickets turn out to concern the same invoice.
- Step 4 of 6
It rewrites, and thinks ahead
It merges the duplicate, replaces the address and adds a conclusion marked as inferred: both tickets dispute one invoice, 340 euros in total. It may rewrite up to ten times. Then the new version replaces the old one, which is kept.
- Step 5 of 6
The run reports and ends
No reply goes to anyone. The run leaves a short report: one duplicate merged, one address corrected, one conclusion added. The transcripts it processed are archived, not deleted.
- Step 6 of 6
The next conversation starts ahead
The next morning the customer asks how much is in dispute. The primary agent finds the total in the memory and answers at once. The combining was done during the night, when nobody was waiting.
What each part does
- Message
- The user's message in an ongoing relationship: a task or a question.
- Primary agent
- Answers the user from the memory as it stands. Does not reorganise it.
- Reply
- The answer to the user. The exchange is added to the raw context for the next pass.
- Memory
- The rewritten context: facts, corrections and conclusions prepared in advance. Read on every turn.
- Idle trigger
- Fired by a timer or by inactivity. Carries the raw context gathered since the last pass and no question.
- Consolidation agent
- Rereads the raw context and the current memory, then rewrites the memory in a few passes.
- Memory rewritten
- The report of one consolidation run: what was merged, corrected and added.
- Memory → Primary agent: memory as it stands
- Idle trigger → Consolidation agent: raw context, no question
- Memory → Consolidation agent: current version
- Consolidation agent → Memory: rewritten version
Where it connects
One message from a known user or case, with an identifier that selects the right memory.
The raw context gathered since the last pass, such as transcripts or new documents, with no question. Fired by a timer or by inactivity, never by a chain.
When: The primary agent has answered the message from the memory as it stands.
The reply to the user and the version of the memory it was written from.
When: A consolidation run has finished, by its own decision or after its last allowed rewrite.
The entries merged, corrected and added, the number of rewrites used, and the identifiers of the new and the previous memory versions.
One size-limited memory per user or case, with conclusions marked as inferred. Rewritten only by consolidation runs, read by every turn. Earlier versions are kept.
4What do I ask my assistant to set up?
The agents to launch
Primary agent
Answer the user's message using the memory you are given. Treat entries marked as inferred as likely, not certain. If the memory does not hold a fact you need, say so. Never guess a remembered fact. Do not reorganise the memory.
Sees: The message, the current conversation and the current version of the memory. Not the raw transcripts of earlier sessions.
Consolidation agent
Nobody is waiting for an answer. You receive the raw context gathered since the last pass and the current memory. Rewrite the memory: merge duplicates, replace what the raw context corrects, and add the conclusions that later questions are likely to need, each marked as inferred. Stop when another rewrite would change nothing.
Sees: The raw context since the last pass and the current memory. No user question, and nobody to ask.
A prompt you can paste
Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.
You are the orchestrator of a multi-agent workflow that follows the "Sleep-time Consolidation" architecture. Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it. # Task [Describe your task here] # Sleep-time Consolidation While nobody is waiting, a second agent rewrites the memory. Input: the task above. It must provide: One message from a known user or case, with an identifier that selects the right memory. If something essential is missing, ask for it before starting. Agents to launch: - Primary agent: Answer the user's message using the memory you are given. Treat entries marked as inferred as likely, not certain. If the memory does not hold a fact you need, say so. Never guess a remembered fact. Do not reorganise the memory. Context for this agent: The message, the current conversation and the current version of the memory. Not the raw transcripts of earlier sessions. - Consolidation agent: Nobody is waiting for an answer. You receive the raw context gathered since the last pass and the current memory. Rewrite the memory: merge duplicates, replace what the raw context corrects, and add the conclusions that later questions are likely to need, each marked as inferred. Stop when another rewrite would change nothing. Context for this agent: The raw context since the last pass and the current memory. No user question, and nobody to ask. Procedure: 1. Create one memory per user or case, limited to about 2,000 words and stored with its earlier versions. Create a log for the raw context that waits for the next pass. 2. On each message, launch the primary agent with the message and the current memory. It may read the memory and may not rewrite it. Append the exchange to the log. 3. Set the trigger in plain code: 30 minutes without a message, or 20 exchanges waiting in the log, whichever comes first. A message never starts a consolidation run. 4. When the trigger fires, launch the consolidation agent with the waiting raw context and the current memory. Give it no question. 5. Let it rewrite the memory at most 10 times. Each rewrite is a full new version that it can then reread and improve. 6. Replace the memory in one step when the run ends, and keep the previous version. Archive the raw context that was processed. 7. If a message arrives during a consolidation run, answer it from the last complete version. Stop rule: A turn ends when the reply is sent. A consolidation run ends when the agent finds nothing left to change, or after 10 rewrites. The next run waits for 30 idle minutes or 20 logged exchanges. Exit: - `answer`, when: The primary agent has answered the message from the memory as it stands. Deliver: The reply to the user and the version of the memory it was written from. # Final deliverable For a turn: the reply and the memory version it used. For a consolidation run: what was merged, corrected and added, the number of rewrites and the new version's identifier.
The same thing in code
from typing import TypedDict
from langchain.chat_models import init_chat_model
from langchain_core.messages import HumanMessage, SystemMessage
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel
MAX_REWRITES, MEMORY_WORDS = 10, 2000
IDLE_MINUTES, MAX_PENDING = 30, 20 # the trigger: inactivity, or enough raw context waiting
MEMORY: list[str] = [""] # versions of the memory; the last one is current, earlier ones allow a rollback
PENDING: list[str] = [] # raw exchanges gathered since the last pass
PRIMARY = ("Answer the user's message using the memory. Treat entries marked as inferred as likely, not "
"certain. If the memory lacks a fact you need, say so. Never guess a remembered fact.")
CONSOLIDATOR = ("Nobody is waiting for an answer. You receive the raw context gathered since the last pass and "
"the current memory. Rewrite the memory in full: merge duplicates, replace what the raw "
"context corrects, and add the conclusions later questions are likely to need, each marked "
f"as inferred. Stay under {MEMORY_WORDS} words. Set finished when another rewrite would "
"change nothing.")
model = init_chat_model("anthropic:claude-sonnet-5-5")
class Rewrite(BaseModel):
memory: str # the full new version
changes: list[str] # what was merged, corrected or added in this rewrite
finished: bool
class Pass(TypedDict):
raw: str
draft: str
changes: list[str]
rewrites: int
finished: bool
async def rewrite(state: Pass) -> dict:
# no question in this prompt: only the raw context and the draft to improve
prompt = f"Raw context since the last pass:\n{state['raw']}\n\nCurrent memory:\n{state['draft']}"
step = Rewrite.model_validate(
await model.with_structured_output(Rewrite).ainvoke([SystemMessage(CONSOLIDATOR), HumanMessage(prompt)]))
return {"draft": step.memory, "changes": state["changes"] + step.changes,
"rewrites": state["rewrites"] + 1, "finished": step.finished}
def again(state: Pass) -> str:
return END if state["finished"] or state["rewrites"] >= MAX_REWRITES else "rewrite"
builder = StateGraph(Pass)
builder.add_node("rewrite", rewrite)
builder.add_edge(START, "rewrite")
builder.add_conditional_edges("rewrite", again, ["rewrite", END])
consolidation = builder.compile()
def due(idle_minutes: float) -> bool:
# plain code decides when a pass runs; a message never does
return bool(PENDING) and (idle_minutes >= IDLE_MINUTES or len(PENDING) >= MAX_PENDING)
async def consolidate() -> dict:
# entry 2: called by a timer when due() is true. No user message, no reply to anyone.
taken = len(PENDING)
start: Pass = {"raw": "\n\n".join(PENDING), "draft": MEMORY[-1], "changes": [], "rewrites": 0, "finished": False}
done = await consolidation.ainvoke(start)
MEMORY.append(" ".join(done["draft"].split()[:MEMORY_WORDS])) # one swap; the previous version stays
del PENDING[:taken] # archive these in a real system; exchanges that arrived meanwhile wait
return {"version": len(MEMORY) - 1, "previous": len(MEMORY) - 2, "rewrites": done["rewrites"], "changes": done["changes"]}
async def run(message: str) -> dict:
# entry 1: a user turn. It reads the memory as it stands and never rewrites it.
version = len(MEMORY) - 1
reply = await model.ainvoke([SystemMessage(f"{PRIMARY}\n\nMemory:\n{MEMORY[version]}"), HumanMessage(message)])
PENDING.append(f"User: {message}\nAssistant: {reply.text}")
return {"reply": reply.text, "memory_version": version}
Two entry points: run for a message, consolidate for the timer. Key MEMORY and PENDING by user id, persist them, and call consolidate from your scheduler when due() is true.
from typing import Any
from claude_agent_sdk import ClaudeAgentOptions, ResultMessage, create_sdk_mcp_server, query, tool
MAX_REWRITES, MEMORY_WORDS = 10, 2000
IDLE_MINUTES, MAX_PENDING = 30, 20 # the trigger: inactivity, or enough raw context waiting
MEMORY: list[str] = [""] # versions of the memory; the last one is current, earlier ones allow a rollback
PENDING: list[str] = [] # raw exchanges gathered since the last pass
DRAFTS: list[str] = [] # the rewrites of the consolidation run in progress
PRIMARY = ("Answer the user's message using the memory. Treat entries marked as inferred as likely, not "
"certain. If the memory lacks a fact you need, say so. Never guess a remembered fact.")
CONSOLIDATOR = ("Nobody is waiting for an answer. You receive the raw context gathered since the last pass and "
"the current memory. Rewrite the memory with rethink_memory, always in full: merge duplicates, "
"replace what the raw context corrects, and add the conclusions later questions are likely to "
f"need, each marked as inferred. Stay under {MEMORY_WORDS} words. You may rewrite several "
"times. When another rewrite would change nothing, stop and list what you changed.")
def text(s: str, error: bool = False) -> dict[str, Any]:
return {"content": [{"type": "text", "text": s}], "is_error": error}
@tool("rethink_memory", "Replace the memory with a full new version.", {"new_memory": str})
async def rethink_memory(args: dict[str, Any]) -> dict[str, Any]:
if len(DRAFTS) >= MAX_REWRITES: # hard limit: ten rewrites per run
return text("Rewrite limit reached. Stop and list what you changed.", error=True)
DRAFTS.append(" ".join(args["new_memory"].split()[:MEMORY_WORDS]))
return text("Stored as the current draft. Reread it, then rewrite again or stop.")
memory_server = create_sdk_mcp_server("memory", tools=[rethink_memory])
async def result_of(prompt: str, options: ClaudeAgentOptions) -> ResultMessage:
async for msg in query(prompt=prompt, options=options):
if isinstance(msg, ResultMessage):
return msg
raise RuntimeError("no result")
def due(idle_minutes: float) -> bool:
# plain code decides when a pass runs; a message never does
return bool(PENDING) and (idle_minutes >= IDLE_MINUTES or len(PENDING) >= MAX_PENDING)
async def consolidate() -> dict:
# entry 2: called by a timer when due() is true. No user message, no reply to anyone.
taken = len(PENDING)
DRAFTS.clear()
options = ClaudeAgentOptions(
system_prompt=CONSOLIDATOR,
mcp_servers={"memory": memory_server},
allowed_tools=["mcp__memory__rethink_memory"], # only this run can write the memory
permission_mode="dontAsk",
)
raw = "\n\n".join(PENDING)
report = await result_of(f"Raw context since the last pass:\n{raw}\n\nCurrent memory:\n{MEMORY[-1]}", options)
if DRAFTS:
MEMORY.append(DRAFTS[-1]) # one swap; the previous version stays
del PENDING[:taken] # archive these in a real system; exchanges that arrived meanwhile wait
return {"version": len(MEMORY) - 1, "rewrites": len(DRAFTS), "changes": report.result}
async def run(message: str) -> dict:
# entry 1: a user turn. It reads the memory as it stands and has no tool to rewrite it.
version = len(MEMORY) - 1
options = ClaudeAgentOptions(system_prompt=f"{PRIMARY}\n\nMemory:\n{MEMORY[version]}")
reply = (await result_of(message, options)).result or ""
PENDING.append(f"User: {message}\nAssistant: {reply}")
return {"reply": reply, "memory_version": version}
Two entry points: run for a message, consolidate for the timer. Key the three lists by user id, persist them, and call consolidate from your scheduler when due() is true.
from agents import Agent, Runner, function_tool
MAX_REWRITES, MEMORY_WORDS = 10, 2000
IDLE_MINUTES, MAX_PENDING = 30, 20 # the trigger: inactivity, or enough raw context waiting
MEMORY: list[str] = [""] # versions of the memory; the last one is current, earlier ones allow a rollback
PENDING: list[str] = [] # raw exchanges gathered since the last pass
DRAFTS: list[str] = [] # the rewrites of the consolidation run in progress
PRIMARY = ("Answer the user's message using the memory. Treat entries marked as inferred as likely, not "
"certain. If the memory lacks a fact you need, say so. Never guess a remembered fact.")
@function_tool
def rethink_memory(new_memory: str) -> str:
"""Replace the memory with a full new version."""
if len(DRAFTS) >= MAX_REWRITES: # hard limit: ten rewrites per run
return "Rewrite limit reached. Stop and list what you changed."
DRAFTS.append(" ".join(new_memory.split()[:MEMORY_WORDS]))
return "Stored as the current draft. Reread it, then rewrite again or stop."
consolidator = Agent( # only this agent has the tool that writes the memory
name="Consolidation agent",
instructions="Nobody is waiting for an answer. You receive the raw context gathered since the last pass and "
"the current memory. Rewrite the memory with rethink_memory, always in full: merge duplicates, replace "
"what the raw context corrects, and add the conclusions later questions are likely to need, each marked "
f"as inferred. Stay under {MEMORY_WORDS} words. You may rewrite several times. When another rewrite "
"would change nothing, stop and list what you changed.",
tools=[rethink_memory],
)
def due(idle_minutes: float) -> bool:
# plain code decides when a pass runs; a message never does
return bool(PENDING) and (idle_minutes >= IDLE_MINUTES or len(PENDING) >= MAX_PENDING)
async def consolidate() -> dict:
# entry 2: called by a timer when due() is true. No user message, no reply to anyone.
taken = len(PENDING)
DRAFTS.clear()
raw = "\n\n".join(PENDING)
result = await Runner.run(
consolidator,
f"Raw context since the last pass:\n{raw}\n\nCurrent memory:\n{MEMORY[-1]}",
max_turns=MAX_REWRITES + 6,
)
if DRAFTS:
MEMORY.append(DRAFTS[-1]) # one swap; the previous version stays
del PENDING[:taken] # archive these in a real system; exchanges that arrived meanwhile wait
return {"version": len(MEMORY) - 1, "rewrites": len(DRAFTS), "changes": str(result.final_output)}
async def run(message: str) -> dict:
# entry 1: a user turn. It reads the memory as it stands and has no tool to rewrite it.
version = len(MEMORY) - 1
primary = Agent(name="Primary agent", instructions=f"{PRIMARY}\n\nMemory:\n{MEMORY[version]}")
reply = str((await Runner.run(primary, message)).final_output)
PENDING.append(f"User: {message}\nAssistant: {reply}")
return {"reply": reply, "memory_version": version}
Two entry points: run for a message, consolidate for the timer. Key the three lists by user id, persist them, and call consolidate from your scheduler when due() is true.
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, isStepCount, tool } from 'ai';
import { z } from 'zod';
const model = anthropic('claude-sonnet-5-5');
const [MAX_REWRITES, MEMORY_WORDS] = [10, 2000];
const [IDLE_MINUTES, MAX_PENDING] = [30, 20]; // the trigger: inactivity, or enough raw context waiting
const MEMORY: string[] = ['']; // versions of the memory; the last one is current, earlier ones allow a rollback
const PENDING: string[] = []; // raw exchanges gathered since the last pass
const PRIMARY =
'Answer the user\'s message using the memory. Treat entries marked as inferred as likely, not ' +
'certain. If the memory lacks a fact you need, say so. Never guess a remembered fact.';
const CONSOLIDATOR =
'Nobody is waiting for an answer. You receive the raw context gathered since the last pass and the ' +
'current memory. Rewrite the memory with rethinkMemory, always in full: merge duplicates, replace what ' +
'the raw context corrects, and add the conclusions later questions are likely to need, each marked as ' +
`inferred. Stay under ${MEMORY_WORDS} words. You may rewrite several times. When another rewrite ` +
'would change nothing, stop and list what you changed.';
// plain code decides when a pass runs; a message never does
export const due = (idleMinutes: number) =>
PENDING.length > 0 && (idleMinutes >= IDLE_MINUTES || PENDING.length >= MAX_PENDING);
// entry 2: called by a timer when due() is true. No user message, no reply to anyone.
export async function consolidate() {
const taken = PENDING.length;
const drafts: string[] = [];
const { text: changes } = await generateText({
model,
instructions: CONSOLIDATOR,
prompt: `Raw context since the last pass:\n${PENDING.join('\n\n')}\n\nCurrent memory:\n${MEMORY[MEMORY.length - 1]}`,
tools: {
// only this run has the tool that writes the memory
rethinkMemory: tool({
description: 'Replace the memory with a full new version.',
inputSchema: z.object({ newMemory: z.string() }),
execute: async ({ newMemory }) => {
if (drafts.length >= MAX_REWRITES) return 'Rewrite limit reached. Stop and list what you changed.';
drafts.push(newMemory.split(/\s+/).slice(0, MEMORY_WORDS).join(' '));
return 'Stored as the current draft. Reread it, then rewrite again or stop.';
},
}),
},
stopWhen: isStepCount(MAX_REWRITES + 2), // hard limit: ten rewrites per run
});
if (drafts.length > 0) MEMORY.push(drafts[drafts.length - 1]); // one swap; the previous version stays
PENDING.splice(0, taken); // archive these in a real system; exchanges that arrived meanwhile wait
return { version: MEMORY.length - 1, rewrites: drafts.length, changes };
}
// entry 1: a user turn. It reads the memory as it stands and has no tool to rewrite it.
export async function run(message: string) {
const version = MEMORY.length - 1;
const { text: reply } = await generateText({
model,
instructions: `${PRIMARY}\n\nMemory:\n${MEMORY[version]}`,
prompt: message,
});
PENDING.push(`User: ${message}\nAssistant: ${reply}`);
return { reply, memoryVersion: version };
}
Two entry points: run for a message, consolidate for the timer. Key MEMORY and PENDING by user id, persist them, and call consolidate from your scheduler when due() is true.
Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.
What goes wrong
- A rewrite can drop a fact that a later question needs. Keep the raw context and the previous version so that a pass can be undone and rerun.
- Conclusions written next to facts soon look like facts. Mark what was inferred, and tell the primary agent how to treat it.
- A memory that grows at every pass becomes the long context you wanted to avoid. Enforce a size limit at each rewrite.
- A reply that reads the memory while it is being rewritten sees half of each version. Swap versions in one step.
- Nothing new reaches the memory before the next pass. If a correction must hold at once, also write it during the turn.
Chain it
Its result can feed
- Reflection direct
Treat the primary agent's reply as an existing answer to improve against criteria for the user's message, keeping the memory version it came from as context.
- Chain of Verification direct
Treat the reply as an existing answer whose factual claims, including any drawn from memory conclusions marked as inferred, are to be checked one by one.
- Debate conditional
Condition: Use this link only when the user's message asks for a judgment, recommendation or decision, not a recall of stored facts or a conversational reply.
- Constitutional Critique direct
Treat the reply as a finished text to check rule by rule against the stored constitution and revise where rules fail.
- Textual Backpropagation direct
Give the reply to the pipeline's first step as an existing result to improve for the user's message, under the configured evaluation criteria.