Text-to-Image: Diffusion, Text Conditioning, Guidance, Latent Space

Eugene Yan · 3 years ago · 20

Text-to-image models like DALL·E, DALL·E 2, Imagen, and Stable Diffusion use diffusion processes that gradually add noise to images during training and reverse this by learning to denoise from pure noise, with text conditioning achieved through CLIP embeddings that align text and image representations in a shared space. Key technical detail: diffusion models use very small noise removal weights (0.01-0.02) and larger noise addition weights (up to 0.14) across multiple timesteps rather than attempting single-step noise prediction. These approaches enable models to generate novel images from text prompts by conditioning the diffusion process on text embeddings, representing a significant shift from pixel-based to latent-space generation methods.