Hardware aware neural architecture search algorithms emphasizing on the quantization effects
Loading...
Date
2026-06-22
Authors
Κούπριζας, Άγγελος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The rapid expansion of Artificial Intelligence(AI) has created a pressing need to deploy Deep Learning models on resource-constraint edge devices. Hardware-Aware Neural Architecture Search(HW-NAS) automates the discovery of neural network topologies, while model compression techniques, such as Quantization, are utilized in order to map model weights and activations into lower precision formats suitable for target hardware. However, evaluating the quantized performance of thousands of candidate architectures is a time-consuming and computationally expensive process. By contrast neglecting this step in the pipeline can lead to suboptimal choices, as quantization noise significantly degrades model accuracy.
To tackle these challenges, this thesis proposes a hardware-agnostic methodology based on the HW-PR-NAS framework, leveraging on the Pareto Rank Predictor-a surrogate model designed to predict a ranking score for candidate architectures. A hybrid encoder module is employed to feed the predictor with a comprehensive embedding, that captures the topological(GCN),sequential(LSTM) and statistical features of each candidate. This approach provides the predictor with an exact representation of the operations and connections of each network, eliminating the need for separate surrogate model for simple regression tasks like accuracy prediction. The primary objectives are to investigate how the quantization affects architectural accuracy,to fully understand the Pareto Front migration caused by the quantization noise, and evaluate the zero-shot transferability of a high-precision surrogate model compared to a dedicated quantized model.
Specifically, the utilized Pareto Rank Predictor introduces two key modifications to the baseline framework 1) a pooling process applied to the final hidden states of the LSTM encoder, and 2) the utilization of a static ground-truth ranking dataset for training and evaluation. Following the establishment of this framework, an optimal Post-Training Quantization(PTQ) scheme was experimentally determined, resulting in a 4-bit configuration for both weights and activations, referred to as INT4. Applying this scheme across the entire NAS-Bench-201 search space revealed that architectures with lower number of FLOPs(fewer operations) are the most vulnerable to the quantization noise. Consequently, the Pareto Front shifted, resulting in a 0\% survival rate for the original Rank 1 candidates. Despite this local disruption, a strong global correlation was maintained between the original quantized search spaces. Both the FP32 and INT4 surrogate models achieved a high Kendall-Tau correlation within their respective domains. Finally, comparative experiments using Random search and Multi-Objective Evolutionary Algorithm(MOEA) validated the zero-shot hypothesis. The FP32 surrogate successfully navigated the quantized search space and achieved a higher Normalized Hypervolume metric that the INT4-trained model. This demonstrates the high transferability of the baseline surrogate and establishes a highly efficient, time-saving procedure for the architectural search of quantized models.
Description
Keywords
Auto Machine Learning, Neural Networks, Neural Architecture Search, Hardware Aware Neural Architecture Search, Artificial Inteligence, Quantization, Pareto Front, Pareto Rank Predictor