Neural Architecture Search

Lilian Weng · 5 years ago

Neural Architecture Search (NAS) is a machine learning approach that automatically discovers optimal neural network architectures rather than relying on human expert design. The field has three major components: search space (defining valid operations and connections), search algorithm (sampling and optimizing architecture candidates), and evaluation strategy (measuring performance). Different NAS approaches use varied search space representations—including sequential layer-wise operations, cell-based modules, hierarchical structures, and memory-bank models—combined with search algorithms like random search and reinforcement learning, with applications ranging from CNNs to RNNs that can transfer across datasets.