Anthropic tests agent memory, with lessons for reliable recall
Anthropic’s November 6 research examines how Claude preserves knowledge between sessions and revises it later. Developers face a practical question: how can remembered information stay accurate without spending too much effort maintaining it?
What the research establishes
In an internal document experiment involving 25 questions, Anthropic reports 53% of graded criteria met with search alone, versus up to 61% with search and a generated wiki. Wiki-only performance peaked at 48%. These are task-specific vendor results, not a general accuracy guarantee.
Other tests expose limits. One legal-document task showed no clear benefit over search, and another wiki grew larger than its source documents. The work studies stored context rather than changes to model weights. It does not establish a new memory rollout for every Claude user.
Three implementations with different controls
The Claude Code memory guide separates instructions written by people from notes maintained by Claude. CLAUDE.md supplies project guidance. Auto memory records useful preferences, corrections and context that can carry across sessions. Neither turns natural-language text into an enforced security boundary.
Claude Code loads only the first 200 lines or 25 KB of MEMORY.md at the beginning of a conversation, whichever limit arrives first. Detailed topic files can be consulted separately. A long note may exist on disk without being included in the opening context. For teams, a short index with clear references is easier to inspect than one ever-growing file.
The Messages API memory tool has a different operating model. Claude requests file operations, but the application implements them and controls storage. Persistence depends on later conversations reaching the same store. Adding the tool does not mean Anthropic automatically archives every API conversation.
The developer’s handler must validate paths and keep operations inside its permitted memory directory. File-size limits, retention and user separation belong in the application design. Treat a stored note as input that may be wrong or malicious, and keep authorization checks outside the note itself. A sentence describing a permission should never be enough to grant it.
The Managed Agents example needs deliberate review
A November 2 cookbook demonstrates a background workflow that reads completed sessions and updates linked memory files. It requires Managed Agents beta access. The example updates a store in place, while the separate Dreams research preview creates a reorganized copy in a new store.
Its nightly example has no review step by default. A flawed transcript can therefore affect what the agent reads next. The cookbook recommends using a new store and inspecting it before switching an agent when review is needed. Version history and saved copies help with auditing, but they do not decide whether a proposed memory is true.
Data handling also matters. The cookbook says Managed Agents sessions are not eligible for zero data retention or HIPAA, and sessions retain what agents read until deleted. Check the applicable service terms before using real customer or clinical material. These conditions should not be generalized to the separate client-side memory tool.
A small test before connecting real work
A useful evaluation can start with invented project notes and a handful of repeatable questions. Include a fact that changes, a corrected mistake and an instruction embedded in a source that the agent should treat as untrusted. Keep the original records available so a reviewer can trace each proposed memory.
Compare three runs: search without saved notes, search with notes, and search after a maintenance pass. Record answer quality, stale facts, unwanted changes, review time and total compute cost. Use the same questions and sources for each run. A compact store that improves the chosen workload is more useful than a large archive that merely looks comprehensive.
Decide in advance who can approve a memory change, when an old fact expires and how to restore the previous state. Test that restoration instead of assuming a version exists and can be recovered. These are proposed evaluation steps, not tests ByteForward has performed.
Compare product settings rather than shared terminology
The same vocabulary can describe different services. ByteForward’s guide to Copilot Memory and Dreaming controls separates generally available memory from preview maintenance features. Check each product’s access rules, storage behavior and review path independently before choosing a design.
