On the first Agent Memory Leaderboard (AML) Open-source/Academic Textual leaderboard, ReFind ranked 2nd with a score of 44.97.

At the capability level, ReFind scored 57.75 on Explicit Fact Recall—the highest among the overall top three—and 54.20 on Structured Field, Set, and Enumeration Extraction, ranking 2nd across the entire leaderboard.

These results align with ReFind's core design: for extremely long chat histories, we do not use an LLM in advance to construct summaries, memory entries, trees, or knowledge graphs. Instead, we retain raw records and give agents active search tools resembling the human practice of browsing chat history.

Many current long-term memory evaluations are essentially needle-in-a-haystack tasks: a system must locate something that appeared in a history of hundreds of thousands of tokens or more, even though it may not have seemed important at the time.

Such information is difficult for humans as well as models.

For example, a colleague may once have mentioned their birthday in a chat. Months later, if someone suddenly asks me what date their birthday falls on, I may not recall it directly—even if I did read that message.

That does not mean humans cannot answer.

We usually do not try to reconstruct every chat message in our heads. Instead, we open the chat app, search for the colleague's name or keywords such as “birthday” and “gift,” and inspect the surrounding messages. If we do not find it, we try another keyword or narrow the search to a particular time period.

In other words, when dealing with infrequent, low-salience information that requires a precise answer, humans often rely on actively searching external records rather than internal recollection.

This behavior inspired ReFind's design.

We preserve original chat logs without summarizing or rewriting them and reproduce several operations people commonly use when searching chat history:

• Multi-round search: If the first search does not find an answer, revise the keywords based on intermediate results and continue searching.

• Context-window browsing: After finding a matching message, inspect the dialogue before and after it rather than reading an isolated sentence.

• Time filtering: Narrow the search using temporal clues in the question.

• Read filtering: Avoid repeatedly examining records already checked within a limited number of search rounds.

• Session-level rank fusion: If multiple messages in one conversation match, raise the priority of that entire session.

In implementation, ReFind uses a lightweight lexical index to locate candidate conversations. A retrieval agent decides what to search next, whether further searching is necessary, and which evidence to retain. The discovered evidence is organized by session and chronological order, then passed to an answer model for reasoning. The AML version returns retrieval evidence only; final answers are generated by the leaderboard's unified Answer workflow.

Ablation experiments show that all of these components positively contribute to overall system performance.

A central trade-off in ReFind is foregoing LLM-based memory preconstruction at write time.

Many memory systems generate summaries, factual entries, entity relationships, hierarchies, or other intermediate representations during Add. These structures can reduce later query latency and support abstraction and aggregation, but they require the system to decide what matters, how to organize it, and which details to discard before any question is asked.

Every pregenerated representation inevitably involves information selection and compression. As GAM (General Agentic Memory Via Deep Research) discusses in its analysis of information loss during memory extraction, raw messages remain an irreplaceable source of evidence. GAM therefore retains a raw-message page store and lets a Researcher Agent return to original records for multi-round, research-style search.

ReFind takes this direction further: for the many existing evaluation tasks centered on precise recall, we believe complex semantic memory need not always be constructed in advance. If an agent can retrieve the relevant raw records comprehensively, state changes, information revisions and deletions, temporal relationships, and multi-hop combinations can be handled at test time using a language model's reasoning capabilities.

For example, a user first says, “I live in Beijing,” and later says, “I have moved to Shanghai.” The system does not necessarily need to maintain an always-updated current-city field during writing. If retrieval finds both records and preserves their chronology, the answer model can determine at test time which information is more recent.

There is a clear trade-off here.

Preconstruction methods place computation at write time: each new batch of records requires extraction, summarization, or index updates, while subsequent queries are usually faster.

ReFind places more cost in real-time search: Add largely requires no LLM processing, but after a question arrives the retrieval agent may need multiple search rounds.

We believe this cost distribution may be reasonable in real-world settings.

Ordinary users generate many conversations every day, yet queries that require finding a needle in months of history may account for only a small fraction of their requests. If a system calls an LLM in advance to construct memories from every message, and most of that information is never queried again, much of the preprocessing cost is effectively unused.

By contrast, starting a more expensive active search only when needed could reduce total long-term cost.

However, this remains a hypothesis requiring further validation. We currently lack sufficient data on real user query distributions to tell us:

• How much historical information users produce;

• How much of that information will be queried again later;

• How deep searches typically need to be;

• Where the long-term cost crossover lies between preconstruction and real-time retrieval.

For now, we therefore describe this as a system trade-off worth investigating, not a proven universal conclusion.

On this cycle's AML Open-source leaderboard, neither of the top two systems—InvMem and ReFind—relied on LLMs to preconstruct complex memory structures during writing.

This is partly consistent with our expectations.

When evaluation includes only one or a few datasets, complex structures tailored to particular tasks can provide clear advantages. As the number of benchmarks and capability types increases, however, final scores must be averaged over more tasks. More complex structures usually bring more assumptions tied to particular data distributions.

A structure may suit multi-hop QA but not precise field extraction; help summarization while losing original numbers and enumeration items; or compress long-term information while introducing additional update mechanisms for state changes.

Relatively simple methods may not be strongest in every capability, but they usually make fewer upfront assumptions and are less likely to fail when a particular data format changes.

This resembles the intuition behind Occam's razor: if two systems can both explain and complete the current task, the one with fewer assumptions and simpler components is often more stable across tasks.

ReFind postpones part of the information-organization work until the question is clear. Rather than guessing future needs at write time, it uses the current question to decide which records to search, which context to expand, and which information to combine.

Of course, we do not believe this leaderboard proves that simpler is always better. Results apply only to the specific track, datasets, system versions, and evaluation snapshot. A more accurate conclusion is that, under the current evaluation distribution centered on precise retrieval and evidence recovery, active search over raw records without preconstruction is a highly competitive technical approach.

ReFind emphasizes that raw-text retrieval should be an important component of long-term memory systems.

Whether a system pregenerates summaries, memory entries, or knowledge structures, it inevitably selects and compresses information. If an omitted detail suddenly matters later, the system must be able to return to the original records to find evidence again.

ReFind has already explored several mechanisms for raw-text retrieval: multi-round search, local context browsing, time filtering, session-level rank fusion, and read filtering.

Yet we also believe raw-text retrieval cannot solve the entire Agent Memory problem, and that it has a systematic failure case.

This problem arises when the query and the target memory have no direct semantic connection.

A retrieval system driven by the current query will probably search for pets, cat food, breeds, or care experience, without considering a search for lilies. The target memory never enters the candidate context. Even if the downstream language model knows that lilies are toxic to cats, it has no opportunity to connect that knowledge to the user's actual environment.

This scenario differs fundamentally from state changes, multi-hop reasoning, and temporal reasoning. For state changes, if both old and new records are retrieved, a language model can usually identify the newer information at test time. For multi-hop questions, if pieces of evidence connect through entities or clues, an agent can follow those links through multiple searches. But when the query and the truly important target memory share no retrievable semantic clue, the problem is not insufficient reasoning: the relevant memory never entered the model's context in the first place.

If candidate generation depends entirely on query-conditioned similarity, the target memory is effectively excluded at retrieval time. No amount of subsequent test-time reasoning can reason over a user fact the model has never seen.

However, many current Agent Memory benchmarks still primarily focus on needle-in-a-haystack tasks. Although they incorporate state updates, multi-hop combinations, temporal reasoning, forgetting, and summarization, questions and target memories in these tasks typically retain identifiable lexical or semantic connections.

Such benchmarks are excellent for assessing whether systems can find evidence among large amounts of irrelevant information, and they give active search approaches like ReFind a substantial advantage. But they do not yet sufficiently cover scenarios where the current question and a critical memory are not semantically similar.

We therefore propose a new benchmark: InMind.

InMind specifically studies the implicit-association blind spot in Agent Memory: a user memory is crucial to the current decision, but the system knows to retrieve it only when it connects that memory with external world knowledge.

InMind contains 125 expert-verified tasks across ten everyday domains. In experiments, when the decisive memory is directly included in context, the answer model achieves 84.0% on indirect questions. When a memory system must first retrieve that same memory, however, the best of six tested vector, graph, and Agentic Memory systems reaches only 14.4%.

This suggests the key issue is not necessarily that the model lacks knowledge or that the system failed to store the memory. Rather, the system may not know which seemingly unrelated historical information should be brought back into context when the current question arrives.

This is the direction we most want to explore next.

Future Agent Memory systems cannot rely solely on searching for the memories most similar to a question after it is asked. They also need more proactive memory routing: determining which user facts have lasting consequences, which risks and constraints should remain visible, when to activate historical information the user has not explicitly mentioned, how to activate history that may be semantically unrelated, and how to balance cost and performance in such an architecture.

ReFind and InMind are complementary sides of this problem. They also correspond to two ways humans manage memory: writing something in a notebook and remembering it in their heads.

• ReFind shows that preserving raw records and letting agents actively search like humans can be a simple yet powerful approach for many existing precise-retrieval tasks.

• InMind shows that even excellent multi-round retrieval does not remove the inherent boundaries of query-conditioned retrieval.

ReFind starts from a simple observation: people do not remember every chat message, but they return to original records and search repeatedly when needed.

We turned this behavior into search interfaces callable by an agent, and demonstrated this approach's competitiveness in explicit factual recall and structured information extraction on the first AML Open-source leaderboard.

At the same time, we do not believe active search can completely solve Agent Memory.

Raw records preserve the system's information ceiling, and multi-round search makes precise evidence more accessible. Yet the hardest problems may arise when there is no explicit link between a query and a critical memory.

From ReFind to InMind, our research question has therefore changed: