RUL prediction leveraging VQ-VAE tokenization and a transformer-based architecture
Loading...
Date
2026-02-24
Authors
Παπαμιχαήλ, Γεώργιος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Over the past few years, composite materials have seen an extensive use in many safety-critical applications such as aerospace and automotive due to their high strength-to-weight ratio and corrosion resistance. However, their anisotropic nature which results in very intricate and unexpected damage mechanisms poses significant challenges. To overcome these challenges, Structural Health Monitoring (SHM) techniques are rapidly developing to enable damage detection, quantification and finally Remaining Useful Life (RUL) prediction in support of maintenance and operational decision-making.
The main objective of this study was to develop a robust Transformer-based architecture leveraging a pre-trained Vector Quantized Variational Autoencoder(VQ VAE), highlighting the effectiveness of VQ-VAE models as feature extractors within a transfer learning transformer framework , in comparison with more conventional neural network architectures such as MLPs, CNNs, and LSTMs, as well as Transformer variants that do not incorporate the VQ-VAE. The specific VQ-VAE was introduced in “TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis” . The learnable parameters were kept at similar levels across all the models to make the comparison fair.
The strain data were derived from a controlled experimental campaign on 3D woven carbon fiber-reinforced polymer (CFRP) panels, representatives of aircraft engine components. The specimens which are referred to as Foreign Object (FOD) panels are made of a 3D woven carbon fiber-reinforced polymer (CFRP) bonded to a steel leading edge. Fiber Bragg Grating (FBG) optical sensors were mounted on the panels to monitor the strain evolution which was revealed to be highly stochastic.
The preprocessing pipeline included downsampling and aggregation to reduce noise and computational complexity while preserving the degradation trends. The RUL targets were linearly defined. The RUL definition was based on the assumption of a linear and monotonically decreasing evolution from the initial time until failure. With the use of the pre-trained VQ-VAE, the continuous strain signals were converted into a discrete tokenized representation. This representation was processed by a stack of transformer encoders and a regression head to produce the RUL prediction. Similarly, a prediction was derived for each one of the alternative neural networks.
The results demonstrate that the proposed Transformer-based architecture outperforms the other versions in terms of accuracy, generality and uncertainty. The study validates the supremacy of the Transformer-based architecture as well as the critical contribution of the pre-trained VQ-VAE in enhancing the model generalization.
Description
Keywords
Composite materials, Remaining useful life, Transformers, VQ-VAE