MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations

arXiv cs.CL · 18 hours ago

MemOps is a benchmark that evaluates long-term memory in LLM-based agents by decomposing memory into lifecycle operations (remembering, forgetting, updating, reflecting) rather than just measuring final answer correctness. The benchmark includes six categories of operation-level probes tested across retrieval-based, parametric, and managed-memory systems, revealing that current systems struggle with ordered memory-state trajectories and other specific failure modes. This approach enables fine-grained diagnosis of memory failures in multi-session conversations instead of relying on opaque downstream question answering evaluation.