Skip to main content
Visit DexHubOrder Now

MeMoREval: What Should Embodied Agents Remember?

Dexmate and Carnegie Mellon introduce MeMoREval, a benchmark for which memory modality an embodied agent should keep, with reference evaluations across four simulated environments and a demonstration on a physical Vega 1.

Paper

8 Oct 2026

A Dexmate Vega 1 carries out an experience phase and a later request phase in MeMoREval's memory evaluation protocol (Figure 1 of the paper).

What Should Embodied Agents Remember? Benchmarking and Evaluating Memory Modality for Embodied Agents

Junxiang Wang1,2,†, Yuchen Liu1,3,†, Yi Wu1, Guofei Chen1, Nir Pechuk2, Tao Chen1, Zackory Erickson2, Yuzhe Qin1, Tianyu Li1

  • 1Dexmate
  • 2Carnegie Mellon University
  • 3Purdue University

† Work done during an internship at Dexmate

arXiv preprint arXiv:2610.12561

PDFarXivProject Page

Abstract

Memory allows large language model (LLM) agents to use past experience, often stored as textual records, when responding to requests. For embodied agents that control robots, visual and spatial observations introduce a broader choice of memory modalities, affecting both the information available for subsequent tasks and the model inference costs of constructing and using memory. We introduce MeMoREval, a benchmark for studying this problem through a controlled experience-request protocol. Agents first experience a sequence of events without knowing the subsequent tasks, and subsequently they use memory to respond to information and action requests without receiving new observations. The benchmark comprises 53 tasks across four simulated indoor environments and measures task success alongside the inference costs of memory construction and use. Our reference evaluations reveal that storing frames outperforms storing generated descriptions at lower write-time inference cost for the primary model, while storing frames with scene graphs achieves the highest aggregate task success for every tested model. Extensions to accumulated experience, selective frame retention, and visual input to memory writers demonstrate how the benchmark accommodates further memory-design investigations. Two physical-robot demonstrations establish the feasibility of applying its protocol beyond simulation.

Citation

@misc{wang2026whatshouldembodiedagentsremember,  title = {What Should Embodied Agents Remember? Benchmarking and Evaluating Memory Modality for Embodied Agents},   author = {Junxiang Wang and Yuchen Liu and Yi Wu and Guofei Chen and Nir Pechuk and Tao Chen and Zackory Erickson and Yuzhe Qin and Tianyu Li},  year = {2026},  eprint = {2610.12561},  archivePrefix = {arXiv},  primaryClass = {cs.RO},  url = {https://arxiv.org/abs/2610.12561}, }

Start Your Physical AI Journey with Us

Talk to Us
Vega 1 Pro reaching forward with both arms