Learning to simulate room acoustics : conditional room impulse response generation with latent diffusion models
Loading...
Date
2026-02-06
Authors
Χατζησάββας, Παύλος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Neural network based solutions are utilized for a wide range of audio related tasks, such as automatic speech transcription, speech enhancement, source separation etc. In order to model real world data, clean speech datasets are augmented using additive noise and convolved with Room Impulse Responses (RIRs). Since acquiring real RIRs through measurement is difficult and resulting datasets are limited in size, there is a need of high quality generation with minimal computational effort.
Generative networks such as Generative Adversarial Networks (GANs) have been proposed before for the acoustic simulation task. Since Latent Diffusion Models have emerged as the state of the art in generative machine learning, they pose themselves as a promising alternative for the task.
In this work, we use a latent diffusion approach for the task of predicting room acoustics. We investigate if our latent diffusion approach matches existing alternatives in quality and speed. We compare our approach against a hybrid geometry-based method. Furthermore, we investigate whether the latent diffusion approach can match a state-of-the-art GAN-based approach in quality and how it competes on the quality vs. computation tradeoff.
We consider two geometric acoustics methods to produce ground truth data for training and evaluating the latent diffusion system. The first is a hybrid using Image Source Method and Ray Tracing and the second is a Ray Tracing implementation. The latent diffusion system is trained and evaluated in two stages: Firstly, an autoencoder network is trained to map responses to a learned latent space and back to a reconstructed signal. Secondly, a noise prediction network is trained for the diffusion task utilizing the latent space representations provided by the autoencoder. The generation process is evaluated in terms of quality and speed and benchmarked against the ground truth geometric method and the state of the art GAN-based method.
Description
Keywords
Room impulse response, Generative machine learning, Acoustics, Latent diffusion models