From GAN to WGAN

Lilian Weng · 8 years ago

The article explains the mathematical foundations of Generative Adversarial Networks (GANs), which use a generator and discriminator in competition to create synthetic data, and discusses why GAN training is unstable including problems like difficulty achieving Nash equilibrium, vanishing gradients, and low-dimensional manifold supports. The optimal discriminator outputs D*(x) = 0.5 when the generator perfectly matches real data distribution, and the loss function equals -2log2 at global optimality. Training instability occurs because both the real data distribution and generated distribution occupy low-dimensional manifolds that rarely overlap, causing the discriminator to become too perfect and gradients to vanish during backpropagation.