Textual Memory
Long-context, persona, script, and conversation-memory benchmarks.
THE JOURNAL
15 postsLatest stories
48 Hours Left: CSIG AML Cycle 2 Project Submission Deadline Approaches
Key dates for Full evaluation, resuming existing tasks, participant and team registration, result review, and the official Cycle 2 leaderboard release.
AML Is in Full Swing: Participant FAQ, Part 2
Practical answers on application follow-ups, version registration, cross-track integration, endpoint issues, and model eligibility for Cycle 2.
AML Phase II Is Open: Participant FAQ
AML Cycle 2 public evaluation opened on September 20, 2026, at 00:00. This FAQ covers registration, models, Add/Search integration, recovery, and results.
Two Days to Go | CSIG-Hosted Agent Memory Challenge 2026 Cycle 2 Participation Guide
Two Days to Go | Agent Memory Challenge 2026 Cycle 2 Participation Guide
AML Official Analysis II | Coding Memory Leaderboards: How Do Industry and Open-Source Methods Perform?
Compared with textual memory, Coding Memory asks a more direct question: can experience accumulated during previous development actually help an Agent complete subsequent software engineering tasks?
AML Leading-Method Spotlight | FlowGrid AML Retriever: How a Markdown Decision Log Became an Agent Memory Retriever
There is no single answer to Agent Memory.
AML Leading-Method Spotlight | ChronoHybridMem: From Conversations to Verifiable Memory Evidence
There is no single answer to Agent Memory.
AML Leading Methods Explained | ActiveMemoryIndex: Why Does a System Without Memory Governance Rank First in Governance?
ActiveMemoryIndex explains its raw-record-first design, transparent ablations, the limits of item-count budgets, and the next steps for neighboring context and temporal reasoning.
Hosted by CSIG | RMB 150,000 Prize Pool: The Second Agent Memory Challenge Is Coming
The second Agent Memory Challenge will open on September 20, 2026, with Textual, Coding, and Multimodal tracks, unified evaluation, and an RMB 150,000 prize pool.
AML Leading Methods Explained | ReFind: Active Search Lets Agents Browse Original Chat Logs
ReFind preserves raw chat logs and moves memory computation to query-time active search. Its AML results and the InMind benchmark reveal both the strengths and limits of this approach.
AML Leading Methods Explained | From Similarity to Completeness: How InvMem Retrieves Truly Useful Long-Term Memories
InvMem combines fine-grained chunking, Dense and BM25 retrieval, Weighted RRF, and same-session neighboring expansion to turn relevant fragments into complete memory evidence.
AML Official Analysis, Part I: What Does the Textual Open-Source Leaderboard Reveal?
The top three are separated by just 0.22 points, yet excel in different ways. A closer look at seven capability areas reveals distinctive strengths, weaknesses, and open questions beyond the overall ranking.
The First AML Leaderboard Is Officially Released!
The first AML leaderboard publishes textual-track results from 67 memory frameworks, evaluated and reviewed through a unified pipeline. Explore rankings, capability profiles, and opportunities to contribute.
An Agent Only Starts to Understand You When It Has a Yesterday
Agents need more than the ability to think: they need the ability to remember. AML establishes a shared standard for comparing memory systems and invites the community to the first Agent Memory Challenge.
The Agent Memory Challenge Launches: The First Open Tournament for Memory Systems
AML brings multi-source data, a unified evaluation pipeline, capability profiles, and low-barrier integration together in an open, reproducible evaluation framework. The first Agent Memory Challenge is now open.
No posts in this category yet
An Agent Only Starts to Understand You When It Has a Yesterday
Agent Memory Leaderboard
If every interaction with an AI agent feels like meeting for the first time, can it truly be called intelligent?
It can answer questions, perform tasks, and use tools. But it does not know your habits, understand what happened in the past, or grow continuously through long-term interaction.
Once a 'character' starts to have a yesterday and memories, however, it becomes difficult to see it as a disposable tool.
Tomorrow's agents need not only to think, but also to remember.
In July 2026, Agent Memory Leaderboard (AML) officially launched and opened the first Agent Memory Challenge. AML was jointly initiated by researchers and teams from more than 20 universities, research institutions, and open-source communities in China and abroad.
As the first open evaluation platform for agent memory systems, AML aims to establish a unified, open, reproducible, and continuously operating framework for assessing memory capabilities. It allows different memory solutions to be compared under the same standard and helps advance Agent Memory technology together.
Why Do Agents Need a Memory Capability Leaderboard?
Over the past year, rapid advances in agent technology have brought a growing number of memory systems and related products.
But different systems often use different datasets, answer models, and scoring methods. A high score on one test set cannot fully represent a system's true memory capabilities.
Final results may be influenced by several factors at once: the memory system's own capabilities, the underlying answer model's capabilities, and the design of evaluation rules. The industry therefore needs a fairer, more transparent, and verifiable evaluation framework.
AML's goal is not to create a simple benchmark. It brings together public datasets, private test sets, a unified integration protocol, a standardized answering pipeline, review mechanisms, and long-term leaderboard governance to build continuously operating open evaluation infrastructure.
One Yardstick for Agents' True Memory Capabilities
To make evaluation comprehensive, unified, and comparable, AML is built around three core capabilities:
Multi-source data coverage brings evaluation closer to real-world scenarios
AML covers two main directions: textual memory and coding memory.
For textual memory evaluation, the platform integrates widely used datasets such as PersonaMem, LoCoMo-Refined, CLBench, BEAM, LongMemEval, and ScriptMem. They cover real-world application scenarios including long-conversation understanding, cross-session historical memory, recognition of user preferences, temporal relationships, rule execution, and extremely long-context processing.
Together, they contain more than 1,500 conversation and task samples, approximately 150 million characters of long-term history, and nearly 5,000 evaluation questions.
From rankings to capability profiles
Many past evaluations could tell us 'Who scored higher?' but not 'Why did they score higher?'
AML breaks memory capabilities down into multiple dimensions, including:
· Explicit fact recall
· Relational and multi-hop composition
· Temporal and event sequences
· Memory governance
· Personalization and care
· Rule and workflow execution
· Epistemic safety and privacy
The result is not just a ranking, but a complete capability profile.
For researchers, it helps identify technical weaknesses;
For product teams, it helps determine which applications suit different memory solutions;
For industry users, it turns 'strong memory capabilities' from a promotional slogan into verifiable data.
Lower integration barriers let more innovations compete
To support continuous operation, AML simplifies system integration to two core interfaces:
· Add: write memories
· Search: retrieve relevant memories
The platform centrally handles answer generation, scoring, and evaluation orchestration.
Both open-source research methods and commercial memory products can participate in evaluation through the same protocol.
The First AI Memory Challenge Is Now Open
Hosted by Agent Memory Leaderboard, the first Agent Memory Challenge officially launched in July 2026.
The challenge welcomes:
· Researchers worldwide
· Open-source project maintainers
· Commercial product teams
· Agent developers
Registration is open. Join us in exploring the boundaries of memory capabilities for next-generation AI agents.
Competition Information
Leaderboard Divisions
Open-source Methods leaderboard: for open-source research methods. Scores are tied to code, configurations, and reproducibility materials, and entries participate in rankings and reward allocation;
Commercial Products leaderboard: for commercial memory products offered through stable APIs. These are ranked separately to provide a reference for industry product selection.
Researchers, open-source project maintainers, product teams, and developers selecting technologies can all submit systems for evaluation. There is no requirement to form a team.
How to Participate
Apply: complete the registration form and receive an evaluation API Key after approval;
Integrate: provide publicly accessible Add/Search endpoints that follow the platform protocol;
Submit for evaluation: confirm the integration through the Smoke compatibility test, then submit a formal evaluation task;
Review and listing: evaluation results enter the public leaderboard after review.
The platform covers Answer, Eval, and evaluation orchestration costs. Participants cover their own API, database, bandwidth, and computing costs; they do not need to build an evaluation environment themselves.
Rewards
Leaderboard rewards (Open-source Methods leaderboard, allocated each cycle)
1st–3rd place: one month of ChatGPT Pro membership (valued at ¥1,400)
4th–10th place: one month of ChatGPT Plus membership (valued at ¥140)
Community Contribution Program (qualify by meeting any one condition and share a token reward pool on the scale of tens of billions)
The first 50 participants to complete a valid submission each receive API credits worth RMB 50;
Successfully inviting three new participants to complete valid submissions also earns API credits worth RMB 50.
We also welcome challenging test samples: submitting three challenging test cases likewise earns API credits worth RMB 50!
*All rewards are issued as account memberships and API credits. Availability is limited and rewards are distributed while supplies last. The final rules are those published on the official competition website's rules page.
Schedule and Contact Information
Registration opens: July 2026
Submission deadline: August 7, 2026
Leaderboard release: mid-August 2026
Format: this is the first challenge. Agent Memory Leaderboard operates continuously; later challenges will be announced separately.
Registration:
https://agentmemories.ai/competition/
👇 Scan the QR code below to open the competition registration page, or select 'Read Original' at the end of the original article.
Competition repository:
https://github.com/AML-memory/agent-memory-leaderboard
(Includes the interface protocol, submission guide, and integration examples.)
Questions and support:
Tomorrow's AI will do more than answer your questions.
It will remember your experiences, understand your preferences, and grow alongside you.
All of this requires a standard for measuring memory that the industry recognizes.
For memory technology to advance, we need a shared standard of measurement that everyone recognizes and anyone can verify.
See you on the leaderboard in mid-August. 👋
Follow the official Chinese account of Agent Memory Leaderboard for the earliest updates on leaderboard results and the next challenge!