Evolution Strategies
Lilian Weng
Evolution Strategies (ES) are black-box optimization algorithms that optimize vectors of real numbers by iteratively sampling from a probability distribution, evaluating fitness, and updating distribution parameters based on the best-performing samples. Simple Gaussian ES models the distribution with mean and standard deviation, while Covariance Matrix Adaptation ES (CMA-ES) adds a covariance matrix to track pairwise dependencies between samples, enabling faster adaptation of the exploration space. CMA-ES improves upon vanilla ES by separately controlling step size through evolution paths and adapting the covariance matrix through rank-one and rank-lambda updates, allowing more efficient optimization when gradients cannot be computed.
Why it matters
Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option. With black-box optimization algorithms, you can evaluate a target function $f(x): \mathbb{R}^n \to \mathbb{R}$, even when you don’t know the precise analytic form of $f(x)$ and thus cannot compute gradients or the Hessian matrix. Examples of black-box optimization methods include Simulated Annealing, Hill Climbing and Nelder-Mead method.
