Diffusion model

In machine learning, diffusion models, also known as diffusion probabilistic models or score-based generative models, are a class of latent variable generative models. A diffusion model consists of three major components: the forward process, the reverse process, and the sampling procedure.^[1] The goal of diffusion models is to learn a diffusion process for a given dataset, such that the process can generate new elements that are distributed similarly as the original dataset. A diffusion model models data as generated by a diffusion process, whereby a new datum performs a random walk with drift through the space of all possible data.^[2] A trained diffusion model can be sampled in many ways, some of which are more efficient but also of lower quality than others.

There are various equivalent formalisms, include Markov chains, denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations.^[3] They are typically trained using variational inference.^[4] The model responsible for denoising is typically called its "backbone". The backbone may be of any kind, but they are typically U-nets or transformers.

As of 2024^[update], diffusion models are mainly used for computer vision tasks, including image denoising, inpainting, super-resolution, and image generation. These typically involves training a neural network to sequentially denoise images blurred with Gaussian noise.^[2][5] The model is trained to reverse the process of adding noise to an image. After training to convergence, it can be used for image generation by starting with an image composed of random noise for the network to iteratively denoise. Diffusion-based image generators have seen widespread commercial interest, such as Stable Diffusion and DALL-E. These models typically combine diffusion models with other models, such as text-encoders and cross-attention modules to allow text-conditioned generation.^[6]

Diffusion models have also found applications in natural language processing (NLP),^[7] particularly in areas like text generation^[8][9] and summarization.^[10]