Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants
Apple ML Research
Researchers introduced Pare, a framework that simulates active users to evaluate proactive AI agents in digital environments by modeling apps as finite state machines rather than flat APIs. The framework includes Pare-Bench, a benchmark with 143 tasks across communication, productivity, scheduling, and lifestyle applications that test context observation, goal inference, intervention timing, and multi-app coordination. This approach addresses limitations in existing proactive agent evaluation methods that fail to capture the stateful nature of real digital interactions.
Why it matters
Proactive agents that anticipate user needs and autonomously execute tasks hold great promise as digital assistants, yet the lack of realistic user simulation frameworks hinders their development. Existing approaches model apps as flat tool-calling APIs, failing to capture the stateful and sequential nature of user interaction in digital environments and making realistic user simulation infeasible. We introduce Proactive Agent Research Environment (Pare), a framework for building and evaluating proactive agents in digital environments. Pare models applications as finite state machines with…
