Every image model has a lab notebook. For Stability AI’s flagship family, that notebook is generative-models, the repository where SDXL, SDXL-Turbo, Stable Video Diffusion, SV3D, and SV4D all live as research code. It is not a product and not a polished library — the README reads mostly as a dated changelog of model releases, from the November 2023 turbo announcement to the May 2025 arrival of Stable Video 4D 2.0. What holds the whole thing together is one idea: almost nothing about a model is written in Python. Architectures, losses, schedulers, even the number of camera views in a 4D rollout are declared in YAML files under configs/ and assembled at startup, so the same codebase can describe a toy MNIST experiment and a 576x576 four-dimensional video generator with the same grammar.

The assembly mechanism is a function called instantiate_from_config, defined in sgm/util.py. Every config entry carries a target — a fully qualified class path such as sgm.models.diffusion.DiffusionEngine — plus a params dictionary, and the loader imports the class and hands it the params, recursively, all the way down. That single convention is the spine of the repository: main.py is a thin PyTorch Lightning shell, the sgm/ package is a shelf of interchangeable parts, and a YAML file is the decision about which parts click together. Reading this codebase therefore feels less like reading a program and more like reading a catalog of parts plus a stack of assembly instructions. As always in this series, what follows is an educational tour of published source code.

From the training entry point and its YAML wiring, through the conditioner and the diffusion engine, down to the denoiser framework and the demo scripts that drive everything.

Reading the overview from left to right:

• The training entry point parses config files, merges them left to right, and hands the result to PyTorch Lightning — 868 lines that mostly stay out of the way.
• The example training config declares a full text-to-image model as nested target-and-params blocks, dataloader, loss, and all.
• The instantiation helper turns those target strings into live objects and wires together the graph the config describes.
• The DiffusionEngine is the training-and-sampling chassis, formerly known as LatentDiffusion, holding the network, conditioner, first-stage autoencoder, and loss in one class.
• The conditioner hosts GeneralConditioner, one 921-line class that aggregates text, image, crop, and aesthetic embeddings behind a single batch interface.
• The video model extends the image UNet with temporal attention layers — 1,170 lines that turn stills into motion.
• The denoiser wraps the network in sigma space so noise levels, input scaling, and loss weighting are swappable parts rather than hardcoded habits.
• The samplers implement Euler, Heun, and DPM++ 2M solvers, while guiders handle classifier-free guidance as a fully independent axis.

Why You Need This

First, the config system is dependency injection with no framework attached. When main.py loads a file, it parses every block, follows each target string to a Python class, and calls the class with the block’s params — so a “model” in this repository is literally a tree of importable class paths. Because every part is addressable, a research experiment becomes a diff in YAML instead of a fork in Python: swap the conditioner, change the first-stage autoencoder, retarget the loss, and the training script never changes. Configs merge left to right, so a shared base file plus a small override file is the idiomatic way to describe an ablation. The toy MNIST config bundled under configs/example_training shows the same machinery scaled down to a dataset you can actually run overnight, which makes it the best first file to read. This is also how one repository ships image models, video models, and 4D models without becoming three separate codebases — the difference between them is data, mostly.

Second, the denoiser framework is the clearest modern view of how diffusion models are actually organized. Older codebases hardcode one training objective deep inside a model class; this repository refuses to. A diffusion step is decomposed into a network, a sigma schedule drawn by sigma_sampling, input scaling handled by a scaling module, loss weighting handled by a weighting module, and the discretization that maps timesteps to sigmas lives in discretizer.py. The denoiser.py file itself is only 61 lines — it composes those parts, and the scaling and weighting modules it delegates to stay under fifty lines each, proof of how much mileage a clean interface buys. When SDXL-Turbo arrived with adversarial diffusion distillation and needed radically different noise handling, it slotted in as different scaling and weighting choices rather than a rewrite, and the EDM-style samplers in sampling.py consume the same sigma interface.

Third, conditioning is unified in a way worth copying. GeneralConditioner in encoders/modules.py is one class that owns every conditioner variant — frozen text encoders, image embedders, the original-size, crop-coordinates, and aesthetic-score vectors that SDXL famously conditions on — and flattens them into a batch dict the network can consume. Each sub-conditioner registers the keys it can provide, the output carries only what the target network asks for, and the numeric vectors ride beside the text embeddings — the concrete mechanics behind SDXL’s size, crop, and aesthetic conditioning. Reading that one file teaches you what SDXL’s “additional conditioning” actually is, because every vector it emits is named, counted, and concatenated in plain sight. Add the autoencoder package, whose regularizers include the vector-quantization codebook logic in quantize.py, and you have the complete anatomy of a latent diffusion system in one readable shelf.

The detail view: demo entry scripts, the enum-driven Python sampling facade, the denoiser's four-part decomposition, and the module zoo underneath.

The detail diagram starts at the demo layer, which is how most people meet this repository. scripts/demo/sampling.py is a 308-line Streamlit application that loads a model from checkpoints/, builds a value dictionary with prompt, negative prompt, and sizing vectors, and calls do_sample; video_sampling.py extends the same pattern to 257 lines for SVD and SV3D. turbo.py, at 186 lines, is the distilled-model demo, and the community-built gradio_app.py plus gradio_app_sv4d.py wrap the video models in a friendlier UI. There is also a forensics tool: detect.py uses the invisible-watermark library to extract the watermark embedded in generated images, which is a rare example of provenance tooling shipping next to the model code.

The most underrated file is sgm/inference/api.py, a 327-line programmatic surface that compresses the entire YAML zoo into four enums: ModelArchitecture, Sampler, Discretization, and Guider. Its SamplingParams dataclass fixes the production defaults — 1024 by 1024 output, 50 steps, the DPM++ 2M solver, guidance scale 6.0, and the EDM sigma range from 0.0292 to 14.6146 — and its spec table maps SDXL base and refiner checkpoints, both 0.9 and 1.0, to their configs. From there you get text_to_image, image_to_image with a strength parameter, and a refiner method that skips encoding and applies the well-known aesthetic scores of 6.0 against 2.5. If you want to see how a research codebase grows a usable face without abandoning its config roots, this is the file to study.

Beneath the demos sits the module zoo. openaimodel.py, at 823 lines, is the UNet that every diagram starts with, while model.py in the same folder holds the encoder and decoder bodies at 657 lines; the autoencoder package adds latent-space regularizers on top, including the quantize.py codebook machinery. The video scripts show how the family scales in time: simple_video_sample.py runs SVD and SV3D, simple_video_sample_4d.py adds reference multi-views at 238 lines, and simple_video_sample_4d2.py, at 220 lines, generates 48 frames as 12 video frames times 4 camera views, then rolls autoregressively in 12-frame windows to reach longer clips. Same latent interface, one extra axis at a time.

The release timeline inside the README doubles as a history of latent video generation. In November 2023 the repository shipped SDXL-Turbo with adversarial diffusion distillation and, a week later, Stable Video Diffusion with 14- and 25-frame variants at 576 by 1024. March 2024 brought SV3D, an orbital-camera model generating 21 frames from a single object image, in an unconstrained orbital version and a path-controlled one. July 2024 added SV4D, which consumes five context frames plus eight SV3D reference views to emit 40 frames, and the May 2025 SV4D 2.0 improved fidelity, generalized to real-world footage without needing those reference views, and rolls autoregressively in 12-frame windows for longer outputs. Every one of those steps reused the same engine files, which is the whole point.

From Install to First Image

The repository is a Hatch-packaged Python project with no pinned dependencies, and the README prescribes a CUDA 11.8 environment. Getting a first image is a weights problem more than a code problem: download an SDXL checkpoint into checkpoints/, then launch the Streamlit demo.

git clone https://github.com/Stability-AI/generative-models.git
cd generative-models
python3.10 -m venv .generativemodels
source .generativemodels/bin/activate
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip3 install -r requirements/pt2.txt
pip3 install .
# place a model checkpoint in checkpoints/
streamlit run scripts/demo/sampling.py

For the video-to-4D flagship, the README’s quickstart is a single line once sv4d2.safetensors is in checkpoints/: run scripts/sampling/simple_video_sample_4d2.py with an input gif or mp4 and an output folder. Flags like encoding_t and decoding_t control how many frames are encoded or decoded at once for low-VRAM GPUs, and img_size drops the working resolution to 512.

Honest limits: this is research code, not an SDK. Dependencies are deliberately unpinned, the README is a news column rather than documentation, and several model releases live in their own branches or configs that assume you read the release notes first. The code is MIT licensed in LICENSE-CODE, but the weights are not in the repository and carry separate terms — CreativeML Open RAIL++-M for SDXL 1.0 and research-only licenses for the 0.9 and video models. Training at scale assumes the separate sdata data-loading repository and webdataset-formatted corpora, and comfortable VRAM budgets. None of that diminishes the code: for a config-driven architecture pattern and a still-current definition of what a diffusion model is made of, generative-models remains the reference shelf.

Enjoyed this post? Never miss out on future posts by following us