Architectures and FPGA implementations of neural networks for scientific computations
Loading...
Date
2024-10-16
Authors
Μιχαηλίδης, Βασίλειος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This study examines how High Level Synthesis (HLS) implementations of Multi Layer Perceptrons (MLP) may be optimized in order to achieve performance comparable to that of their equivalent CPU implementations. A set of small MLPs (inference and training) with up to three hidden layers was implemented using Vitis HLS, targeting the Xilinx Alveo U280 and trained them on the MNIST dataset. The implemented networks were optimized on the architecture level, and their performance was further improved by converting the implementations from floating point to the fixed point arithmetic. The floating to fixed point conversion utilizes an algorithm that was developed which finds the minimum width
for the fixed point variables that achieves the same performance as the floating point implementation. It was discovered that the resulting bit-width optimized networks had improved latency by 14%, on average, and required less hardware utilization that ranged from 25% to 60%, discounting BRAM utilization which remained the same. Additionally, the optimized networks were benchmarked against the equivalent CPU/pytorch implementation, and it was discovered that they achieve comparable performance that fell well within real time constraints. Lastly, the trained fixed point networks were compared to the equivalent floating point networks, and it was discovered that networks which achieved comparable accuracy to the floating point networks, also had similar weights, implying that they took the same steps during training.
Description
Keywords
Neural networks, FPGA, HLS