Text-to-Image: Diffusion, Text Conditioning, Guidance, Latent Space
Eugene Yan
Text-to-image models like DALL·E, DALL·E 2, Imagen, and Stable Diffusion use diffusion processes that gradually add noise to images during training and reverse this by learning to denoise from pure noise, with text conditioning achieved through CLIP embeddings that align text and image representations in a shared space. Key technical detail: diffusion models use very small noise removal weights (0.01-0.02) and larger noise addition weights (up to 0.14) across multiple timesteps rather than attempting single-step noise prediction. These approaches enable models to generate novel images from text prompts by conditioning the diffusion process on text embeddings, representing a significant shift from pixel-based to latent-space generation methods.
Why it matters
The fundamentals of text-to-image generation, relevant papers, and experimenting with DDPM.
