Architectural exploration & HLS development of CNNs for FPGAs
Loading...
Date
2025-10-13
Authors
Γκούτης, Άρης Ηλίας
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The topic of this thesis involves optimization & architectural design exploration of VGG-16 as a network target. It is the base of most modern Convolutional Neural Networks used for computer vision, image classification & segmentation. To avoid focusing on the deprecated computationally heavy head of the network containing fully connected layers, 5 smaller datasets were used, on which transfer learning was performed. Quick convergence was noticed on all & hyper-parameter tuning was performed on the new head of each network, learning rate and epochs of training & freezing/unfreezing of layers during weights re-training. As a first step, post-training quantization was applied along with a simple trick of adjusting the ranges of each layer, in order to replace the divisions due to differences of scaling factors, with bit shifting. This was done with custom python functions on top of the Tensorflow framework, & the drop in loss and accuracy were evaluated. In all models, the latter had no impact & PTQ had minimal drop in accuracy up until 8 or 9-bit operands. Finally, the model of the dataset of Oxford Flowers was chosen for further hardware implementation optimization. A hardware architecture was proposed based on the objectives of minimizing first external memory accesses, minimizing the number of partial sums & then using a load/compute/store architecture on all possible hierarchies (between ext. memory & buffers) to minimize the total latency of categorizing the input image. This architecture also includes a big processing element (PE) array to decrease computation time and is based around the goal of trying to create a design that can feed this PE with appropriate amounts of data in the most efficient way. This includes intermediate buffers appropriately partitioned, a line-buffer like structure & hardware modules that use an FSM-like way to calculate the addresses for non-contiguous memory accesses. The implementation was fulfilled with High-Level Synthesis & specifically, the most prevalent tool, Vitis HLS. Given the model’s requirements, the high-end xcvu9p-flga2104-3-e FPGA was chosen as a target. HLS aided in exploring the design space during the building of the design, but also in creating a parametric design where many designs with different trade-offs can be chosen by a configuration file. The fastest solution achieved ~348 GOPS with a relatively low count of FF & LUTs and high embedded RAM usage. The fastest implementation achieves an efficiency of 81%, whereas the configuration with the smallest unrolling factor attains an efficiency of 94.3% and approximately ~101 GOPs. It is important to note that 100% efficiency would indicate that the Processing Element (PE) array is fully utilized during every cycle.
Description
Keywords
High level synthesis, AI accelerator, Hardware architecture, Convolutional neural networks (CNNs), FPGA, Design space exploration, Quantization, Transfer learning