PRX Part 3 — Training a Text-to-Image Model in 24h!

Photoroom just wrapped up part three of its PRX diffusion series, and this one is less about theory and more about a stopwatch. After two blog posts spent testing architectural tricks in isolation, the team decided to throw everything that worked into one pot and see what a single day of training could produce. The constraint was brutal on purpose: 32 H200 GPUs, 24 hours, roughly $1,500 total at $2 per GPU hour. For context, early diffusion models cost millions to train. This one cost less than a decent laptop.

The recipe stacks several ideas that each earned their place in earlier experiments. Training happens directly in pixel space using x-prediction, which ditches the VAE entirely and lets the team reuse classic perceptual losses like LPIPS and DINOv2 features without decoding anything first. A 32-pixel patch size keeps sequence lengths sane, 256 tokens at 512px and 1024 at 1024px, so pixel-space training doesn't blow up compute the way it used to. On top of that they added TREAD token routing, which skips half the tokens through a chunk of transformer blocks and reinserts them later, plus REPA-style representation alignment using DINOv3 as a teacher at the eighth transformer block. The optimizer is Muon for all 2D matrices, paired with Adam for everything else, a combo the team says clearly beat Adam alone in prior runs.

Training data came from three synthetic sets, roughly 8.7 million images total pulled from Flux and Midjourney generations, recaptioned with Gemini 1.5 to clean up noisy prompts. The schedule front-loads speed and back-loads sharpness: 100,000 steps at 512px with batch size 1024, then 20,000 steps at 1024px with a smaller batch and REPA turned off. An EMA of the weights handles final sampling.

The results aren't perfect, and the team is upfront about that. There are texture glitches, occasional anatomical weirdness, and some fragility on hard prompts. But prompt following holds up well, the aesthetic is consistent, and the 1024px fine-tuning stage sharpens detail without wrecking composition. Photoroom argues the remaining flaws look like undertraining and thin data diversity rather than anything structurally broken in the recipe, meaning more compute and better data should just make it better in a predictable way.

Everything, code, training scripts, and the experimental framework from parts one and two, is now public on GitHub under the PRX repo. The datasets themselves aren't included, but the pipeline is built to swap in your own data and toggle components like TREAD, REPA, or the perceptual losses independently. Photoroom frames this less as a finished product and more as a base camp, with plans to scale up and refine the data mix next.