The Memory Boundary: Three Tiers That Keep Production Agents Useful Without Poisoning Context

Why most agent memory designs fail in production

Agents that “remember everything” quickly become expensive, unreliable, or unsafe. Raw transcripts bloat the context window, dilute attention, and raise costs. Extracted facts written without provenance become permanent and hard to correct. Persistent memory also becomes an attack channel: a malicious instruction can hide in retrieved content, get written by the agent itself, and fire in a later session.

The strongest evidence from recent work is simple. Keep the raw history where it is cheap. Spend effort on retrieval. Version facts that change. Bind every durable record to its origin at write time. Retrieval method moves accuracy far more than elaborate write strategies. Full context or well-tuned BM25 over history often matches complex memory systems until the conversation grows large.

This post gives a concrete three-tier boundary you can implement without adopting a full cognitive architecture.

The three tiers

Treat memory as three distinct stores with different lifetimes, latency budgets, and failure modes.

• Working memory (in-context): The current turn or short session window already inside the model. Zero retrieval latency. Volatile. Holds the live task, recent tool outputs, and the active plan.
• Session state: Redis or a checkpointer (PostgresSaver, RedisSaver, or equivalent). Lives for the thread lifetime (minutes to hours, or a defined TTL). Holds conversation history, completed steps, and intermediate tool results. Used for crash recovery and multi-instance handoff.
• Durable facts: Structured tables (Postgres) for exact facts and preferences, plus optional vector or graph store for semantic/episodic recall. Cross-session. Written only after an explicit policy check.

Do not collapse them. Putting exact facts into a vector store produces “similar” answers when you need exact ones. Leaving session state only in process memory means a restart loses progress. Dumping every message into long-term storage creates unbounded growth and poisoned recall.

What may cross the boundary

Define a write policy before any extraction runs.

Store when the item is:

• Explicitly stated by the user or confirmed by an authoritative system
• Durable and likely to change a future decision
• Scoped to a tenant, user, or project
• Free of secrets or one-time credentials

Do not store:

• Intermediate reasoning or speculative conclusions
• Raw tool output that can be re-fetched
• Temporary task details that die with the session
• Anything the user would not want retained on request

Prefer structured fields over free-form prose for workflow-critical state. A row with preferred_language, valid_from, and source is easier to update and audit than a paragraph.

Hot-path rule

During the active turn the agent may read from durable memory and write only to session state. Extraction and consolidation happen asynchronously after the turn or at session boundary. A user waiting on a memory-extraction LLM call is waiting for work that benefits a future conversation, not the current one. Batch consolidation; extraction calls can cost 10–100× more than the storage write itself.

Practical write contract

Every durable record should carry at least:

• tenant_id / user_id — structural prefix, not a post-filter
• content — the fact or preference
• source — user statement, tool result, system of record, or inferred
• provenance — session_id, message_id or tool_call_id, timestamp
• valid_from / valid_to — for facts that change
• confidence or confirmation_status — explicit vs inferred

On conflict, supersede rather than accumulate. Keep the old version for audit; remove it from the active retrieval index.

Minimal schema (Postgres)

CREATE TABLE agent_facts (
id            uuid PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id     text NOT NULL,
user_id       text NOT NULL,
key           text NOT NULL,          -- e.g. 'preferred_hotel_style'
value         jsonb NOT NULL,
source        text NOT NULL,          -- 'user' | 'tool' | 'inferred' | 'system'
session_id    text,
created_at    timestamptz DEFAULT now(),
valid_from    timestamptz DEFAULT now(),
valid_to      timestamptz,
superseded_by uuid,
UNIQUE (tenant_id, user_id, key, valid_from)
);

CREATE INDEX ON agent_facts (tenant_id, user_id, key) WHERE valid_to IS NULL;

Retrieval for an exact fact is a simple filtered query. Semantic recall can live in a separate pgvector table or external store, always joined back to the same provenance fields.

Checkpoints versus memory

Checkpoints are for resuming a run, not for personalization. After every state-changing step, serialize:

• current step index
• completed steps and their outcomes
• idempotency keys for any external side effects
• the session message log (or a pointer to it)

On crash, reload the last successful checkpoint and continue. Idempotency keys prevent double charges or duplicate tickets when the same step is replayed. This is orthogonal to long-term memory; a checkpoint can reference durable facts but should not be the place those facts are born.

Security and governance notes

Persistent memory is now a documented attack surface. Papers show cross-session attack success rates above 50 % when an agent writes a malicious instruction into its own memory. Defenses that only inspect content at read time can be laundered. Binding origin at write time and treating external content as low-trust until it crosses an explicit boundary reduces the risk.

Practical controls:

1. Never let the model write directly to the durable store. An application policy layer classifies, scopes, and decides.

2. Require provenance on every record.

3. Support deletion that propagates (user request, GDPR, or policy expiry).

4. Keep raw episodes available for a retention window so extraction can be re-run or audited.

5. Isolate by tenant at the key prefix or row-level security level.

Starter implementation sequence

1. Ship with working memory + session checkpointer only. Most early failures are lost progress, not missing personalization.
2. Add a typed facts table and a single “remember this” path gated by the policy layer.
3. Introduce asynchronous extraction at session end for candidates the policy accepts.
4. Add vector or graph retrieval only after exact facts are stable and provenance is in place.
5. Instrument write volume, retrieval hit rate, and supersession rate. Review the memory store the same way you review production logs.

Start simple. The retrieval method and the write boundary matter more than the sophistication of the consolidation algorithm.

References & Community Insights

• Google Cloud Tech, “AI Agent Memory Architecture: Structured SQL, Semantic Vector Search & Memory Bank” —
• OpenChamber, “State of AI agent memory 2026: what 3,283 papers show” — https://openchamber.dev/blog/agent-memory-research/
• GMI Cloud, “Agent Memory Architecture for Production” — https://www.gmicloud.ai/en/blog/agent-memory-architecture-working-session-and-long-term-memory-in-production
• Conceptualise, “AI Agent Memory and State: Architecture Patterns” — https://www.conceptualise.de/en/blog/ai-agent-memory-state-architecture
• Reddit discussions on open-source memory systems (agentmemory, ai-memory, OneMemory) and production post-mortems on in-memory state graphs
• Microsoft Community Hub, “Remember, Retrieve, Revalidate—or Forget?” — patterns for policy-gated memory operations

These sources converge on the same practical lesson: separate the stores, control the writes, and keep provenance. The rest is refinement.