Data mining from digital images using diffusion models
Loading...
Date
2026-06-26
Authors
Παπαευθυμίου, Βασίλειος
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Digital images contain rich visual and semantic information, which can be extracted, analyzed,-and reused through modern generative models. This thesis investigates semantic image-editing in Denoising Diffusion Probabilistic Models (DDPMs) through direct manipulation of-the reverse diffusion noise trajectory. In recent years diffusion models have emerged as one-of the most successful classes of generative models, producing highly realistic images and-achieving state-of-the-art performance in image synthesis and generation tasks. Their ability-to progressively construct images from Gaussian noise has led to major advances in image-generation, inpainting, super-resolution and controllable image manipulation.
Deep Learning architectures and especially Convolutional Neural Networks and diffusion-based generative models have significantly transformed the field of Computer Vision. By-learning hierarchical feature representations directly from data, these models can capture-highly complex structures and semantic relationships. Diffusion models rely on deep neural-networks such as U-Net architectures with residual connections and self-attention-mechanisms to iteratively remove noise and recover meaningful image structure throughout-the reverse diffusion process.
Due to their remarkable image generation capabilities, diffusion models are increasingly being-used in real-world applications involving content generation, image editing and restoration.
However, despite their impressive generative quality, controllable semantic editing in-unconditional DDPMs remains a challenging problem. Once the reverse diffusion process-begins the generated trajectory is typically fixed and offers limited mechanisms for localized-or semantic intervention.
For this reason, in the current thesis, I investigated whether intermediate noisy-representations generated from a DDPM can be used to semantically edit an image during the-reverse diffusion process and get a high-quality altered sample at the end of it. In addition, I-investigated what the best timesteps are to do such an image transplantation with the use of-metrics such as attribute transfer accuracy, identity preservation, FID and KID.
To achieve that, I sample 1000 images using a pretrained DDPM on Celeba-HQ-256x256 and-save their noise trajectory and the seed that I used to generate them. I use some of these-images to extract features that I want to transfer to other images with the help of masks in-various timesteps. Then using the seeds that I saved earlier I am starting the same generation-process but interrupting it in a timestep and blending the current noisy image with the masked-feature that I extracted earlier in the same timestep and continue the generation process-achieving the best results in the middle area 250 ≤ 𝑡 ≤ 750.
Description
Keywords
Diffusion, Images, Reverse, Noise, Trajectory, Semantic, Timestep, Probabilistic, Denoising, Models