Advanced articulated motion prediction
Loading...
Date
2024-10-10
Authors
Μπέλεσης, Αντώνιος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Motion Synthesis is the process of generating a sequence of poses, that depends on a previous state - or states- of motion, and future conditions given by user input or other related requirements. In this thesis, we explore several novel deep learning approaches in pursuit of generating highly stochastic and varied movements while maintaining realism. As the demand for varied 3D animation grows across several domains, from video games or virtual reality applications to robotics, there is an increasing need for motion generation techniques that can capture the full range of human motion variability.
In this work, Conditional Generative Adversarial Networks (CGANs) are used to take advantage of their stochastic nature as well as their good sample quality and relatively low computational cost in comparison to other generative models. We show the models’ capabilities off by training them on highly varied dancing motion data, that deterministic models usually fail to replicate, as well as different styles of locomotion movements that we then try to blend between using style-mixing.
Specifically, three distinct models were investigated: a Mixture of Experts (MoE) approach, a StyleGAN-inspired model injecting noise to intermediate layers directly, and a Transformer-based architecture that captures relationships between the input vectors to produce a structured motion output. These models are trained separately on two different datasets, a segment of the 100STYLE dataset [23] that consists of 5 distinct styles of locomotion, and the Dance Motion Capture Database [24] on Greek and Cypriot traditional dances. The aim was for the models to synthesize complex and varied movements that can be aperiodic and stochastic in nature.
This research demonstrates several strengths and limitations for each approach. The MoE model excels in smooth style transitions but struggles with style-mixing and generalization as well as aperiodicity in the dataset. The StyleGAN architecture shows improved stochasticity but faces challenges in maintaining motion coherence while the Transformer-based approach offers a lightweight alternative that is fast to train and produces structured motion, albeit deterministic in nature due to the lack of a latent vector as an input, as opposed to the previous models, and fails in dance generation. The transformer also manages to produce mixed motion that goes beyond the locomotion training set, though not in a consistent or predictable fashion.
The models are evaluated using objective metrics, namely Maximum Mean Discrepancy (MMD) [34], which calculates the distance between the probability density functions of the generated data and the training data using a kernel trick method, and MiVo [35], which calculates the mean of incoming and the variance of outgoing samples to rate the realism as well as variety of produced samples related to the training data. Also, subjective human evaluation of the produced motion is not omitted, as this is the final judge on whether a generated result is satisfactory.
Our findings aim to contribute to the ongoing development of motion synthesis techniques, offering insights into the trade-offs between realism, diversity, and computational efficiency in real-time-controlled character animation.
Description
Keywords
Motion synthesis, Machine learning, Generative adversarial networks