Introduction to Diffusion and Score-Based Generative Models

Visualization of ODE iterations using a score based model trained on a swiss roll dataset.

This is a small introduction to denoising diffusion and score-based generative models. It is based on a collection of the following resources:

Resources:

• Diffusion-Denoising Models For a very good introduction into diffusion models you can check out the blog post from Lilian Weng: https://lilianweng.github.io/posts/2021-07-11-diffusion-models/

• Score-Based Generative Models A good introduction is given by Yang Song here: https://yang-song.net/blog/2021/score/, https://www.youtube.com/watch?v=wMmqCMwuM2Q

• CVPR 2022 Tutorial on Denoising Diffusion-based Generative Modeling - Foundations and Applications: https://cvpr2022-tutorial-diffusion-models.github.io/

• DiffusionFastForward Repository mikonvergence/DiffusionFastForward

• ScoreDiffusionModel Repository JeongJiHeon/ScoreDiffusionModel

• Generative Models
• Score Matching and Diffusion Models
• Illustration of Diffusion Process
• From Discrete Diffusion to Continous Time Diffusion Processes
• Score Matching
• Example: Score-Based Generative Model for 2D Swiss Roll Dataset
• Example: Score-Based Generative Model on MNIST
• Summary: Score-Based Models
• Implementation Examples
• Another view on Score-Based Models