Multimodal models need video. We froze time to give it to them

Most session recordings never get watched. They’re stored, ignored, and eventually expire, which is a problem when a busy product can generate tens of thousands of them every day. So the Replay Vision team stopped thinking about how to help humans search harder and started trying to automate the watching itself.

The first hurdle was basic but nasty: the thing they wanted to show a multimodal model wasn’t video at all. rrweb records a DOM snapshot plus a stream of changes, then the browser rebuilds the page when someone hits play. Handy for replay. Useless for a model that needs actual pixels. Handing the raw stream to an LLM didn’t work because it was too large, too messy, and too dependent on browser layout to fit neatly in context.

So they built a rasterizer. A headless Chromium instance loads a tiny frontend bundle, plays the recording at 8x speed, and sends frames over the Chrome DevTools Protocol into ffmpeg, which spits out an MP4. The setup uses Puppeteer, Temporal, dedicated infrastructure, and a pool of warm browsers to keep up with heavy ingest.

Then the weird part started. When multiple jobs ran on the same hardware, some videos froze, some skipped chunks, and the same recording could produce different lengths. The problem was CPU contention: rendering and capture were drifting apart, so one side kept moving while the other stalled or dropped frames. Chrome’s HeadlessExperimental.beginFrame fixed one half of that by making the browser render only when told. But rrweb’s player still thought wall-clock time was real, so it would skip ahead if the browser slowed down.

The final fix was to freeze time inside the page. Date.now, performance.now, requestAnimationFrame, setTimeout, and setInterval were overridden so they only advanced when beginFrame did. That kept the player and the browser in lockstep, which meant the same input always produced the same output, even under load. This version has been in production since March 2026, runs across hundreds of concurrent jobs on many pods, and has turned more than 370 years of recordings into roughly 3.5 million videos.