Multi-Objective Neural Architecture Search (MONAS) Framework using Genetic Algorithms

Abstract

The traditional approaches to Neural Architecture Search (NAS) are typically limited to the maximization of accuracy, which leads to large and computationally expensive models that are not always applicable in practice, e.g. real-time or edge deployment scenarios. This brings out the necessity of a framework that can optimize multiple goals including accuracy, inference latency, memory usage and model complexity simultaneously. This paper presents a design-oriented approach for Multi-Objective Neural Architecture Search framework, MONAS, which is a framework of evolutionary optimization. Unlike traditional NAS techniques that optimize a single objective, MONAS formulates architecture search as a multi-objective optimization problem and it employs genetic algorithms to evolve a diverse set of candidate architectures, out of which the most optimal architecture is chosen. Pareto optimality is used to guide the search process so that the architectures offering balanced trade-offs between various performance measures can be identified. The framework uses the NAS-Bench-301 surrogate model to predict the performance of architectures without training them, which minimizes the computational cost. A directed acyclic graph (DAG) is used to represent each candidate neural architecture in the search space. This allows effective execution of genetic operations such as crossover and mutation. The selection process is performed using NSGA-II and Pareto-optimal architectures are passed on to the next generation