Kimi AI and kvcache-ai Open Sources ‘AgentENV’: A Distributed System that Powers Agentic Reinforcement Learning (RL) Training for Kimi K3

Moonshot AI's Kimi team, working with kvcache-ai, has open-sourced AgentENV, a piece of infrastructure that's been quietly doing the unglamorous work behind Kimi K3, the company's 2.8-trillion-parameter Mixture-of-Experts model. The code is out under MIT license, and it tackles a problem that anyone training agentic reinforcement learning systems runs into fast: where does the model actually act?

Here's the trade-off that's been dogging this field. Training an agent means letting it touch a real Linux box — filesystem, network, live processes, the works — for every single rollout. Containers spin up quickly but share a kernel with the host, so a model generating its own code is one bad instruction away from causing trouble. Full VMs isolate properly, but they're slow to boot and they hog memory even when nothing's happening. At the scale needed for RL training, where you might need thousands of these environments cycling constantly, neither option really works.

AgentENV's answer is Firecracker microVMs, the same lightweight virtualization tech AWS built for Lambda, tuned so that idling, restarting, and branching sandboxes costs almost nothing. Each sandbox gets its own kernel, filesystem, and network namespace, with an Axum-based HTTP API routing requests to an orchestrator that manages the whole lifecycle. Inside each VM, a small daemon called envd — listening on port 49983 — handles command execution and health checks, while a reverse proxy shuttles HTTP and WebSocket traffic in and out.

The storage layer is where the engineering gets genuinely clever. Rather than giving every sandbox a full copy of its root filesystem, AgentENV serves it through a ublk userspace block device backed by overlaybd layered images. Base layers are read-only and shared across every sandbox; only the differences get written per-instance. Combine that with a shared host page cache and memory ballooning — which reclaims idle guest memory and hands it back to the host — and you get real overcommit, meaning far more sandboxes can run per machine than their raw memory footprint would suggest.

None of this is flashy work. It won't show up in a benchmark chart. But it's the kind of infrastructure that determines whether agentic RL training is something only a handful of labs with custom-built systems can do, or something more teams can replicate. Given that Kimi K3 is a genuinely huge model, releasing the plumbing that trained it — rather than just the weights or a paper — is a notably generous move.