Relay Forcing
Checkpoints of Relay Forcing: Real-Time Streaming Video Generation at 1.58 NFE. Every model whose SANA-WM-Bench result appears in the main paper is included, together with the video decoder used for the LTX-2.3-22B models and the training data of the main models. The repository holds 2.12 TB in total.
Main models
| Folder | Model | Used in |
|---|---|---|
wan2.2-ti2v-5b/relay-forcing |
Relay Forcing, 1.58 NFE | Table 1, Table 2, Figure 4, Figure 5(a), Figure 6(a), Table 4 (Full head), Table 5 (Relay Forcing), Table 6 (5-8), Figure 7 (5 x 8) |
ltx-2.3-22b/relay-forcing-15s |
Relay Forcing, 1.58 NFE, 15 s model | Table 1, Figure 4, Figure 6(a) |
ltx-2.3-22b/relay-forcing-60s |
Relay Forcing, 1.58 NFE, 60 s model | Table 2, Figure 5(b), Figure 6(a) |
ltx-2.3-22b/causal-vae-decoder |
Latent-causal grouped VAE decoder used to decode both LTX-2.3-22B models | Table 1, Table 2, Figure 5(b), Figure 6(a) |
Ablation models
All ablations use Wan2.2-TI2V-5B.
| Folder | Model | Used in |
|---|---|---|
wan2.2-ti2v-5b/ablations/dmd-4step/relay-homogeneous-critic |
4-step vanilla DMD student, relay student with homogeneous critic | Figure 6(b) (Relay, Homogeneous), Table 3 (Relay, Rehearsal), Table 5 (4-step) |
wan2.2-ti2v-5b/ablations/dmd-4step/relay-offline-critic |
4-step vanilla DMD student, relay student with offline critic | Figure 6(b) (Relay, Offline) |
wan2.2-ti2v-5b/ablations/dmd-4step/causal-homogeneous-critic |
4-step vanilla DMD student, block causal student with homogeneous critic | Figure 6(b) (Block causal, Homogeneous), Table 3 (Block causal, Rehearsal) |
wan2.2-ti2v-5b/ablations/dmd-4step/causal-offline-critic |
4-step vanilla DMD student, block causal student with offline critic | Figure 6(b) (Block causal, Offline) |
wan2.2-ti2v-5b/ablations/dmd-4step/relay-gt-history |
4-step vanilla DMD student from a relay teacher trained on ground truth history | Table 3 (Relay, Ground truth) |
wan2.2-ti2v-5b/ablations/dmd-4step/relay-resampling |
4-step vanilla DMD student from a relay teacher trained with Resampling Forcing | Table 3 (Relay, Resampling) |
wan2.2-ti2v-5b/ablations/head-components/last-block-only |
Trajectory head reading only the last relay DiT block | Table 4 (Last block features only) |
wan2.2-ti2v-5b/ablations/head-components/no-step-attention |
Trajectory head without step attention | Table 4 (No step attention) |
wan2.2-ti2v-5b/ablations/head-components/no-two-tap-conv |
Trajectory head without dynamic two-tap convolution | Table 4 (No two-tap convolution) |
wan2.2-ti2v-5b/ablations/head-components/parallel-linear |
Parallel linear layers on the final feature | Table 4 (Parallel linear layers) |
wan2.2-ti2v-5b/ablations/unrolling/1step |
1-step student with DMD and GAN | Table 5 (1-step) |
wan2.2-ti2v-5b/ablations/unrolling/1step-head34 |
1-step student with a 34-layer single-step head, DMD and GAN | Table 5 (1-step + head) |
wan2.2-ti2v-5b/ablations/unrolling/2step |
2-step student with DMD and GAN | Table 5 (2-step) |
wan2.2-ti2v-5b/ablations/unrolling/4step-gan |
4-step student with DMD and GAN | Table 5 (4-step + GAN) |
wan2.2-ti2v-5b/ablations/gan-steps/none |
Trajectory head trained with DMD only | Table 6 (None) |
wan2.2-ti2v-5b/ablations/gan-steps/steps1-8 |
Trajectory head with GAN on steps 1 to 8 | Table 6 (1-8) |
wan2.2-ti2v-5b/ablations/gan-steps/steps1-4 |
Trajectory head with GAN on steps 1 to 4 | Table 6 (1-4) |
wan2.2-ti2v-5b/ablations/head-scaling/l3-s4 |
Trajectory head with 3 layers and 4 unrolled steps | Figure 7 (3 x 4) |
wan2.2-ti2v-5b/ablations/head-scaling/l3-s8 |
Trajectory head with 3 layers and 8 unrolled steps | Figure 7 (3 x 8) |
wan2.2-ti2v-5b/ablations/head-scaling/l3-s16 |
Trajectory head with 3 layers and 16 unrolled steps | Figure 7 (3 x 16) |
wan2.2-ti2v-5b/ablations/head-scaling/l5-s4 |
Trajectory head with 5 layers and 4 unrolled steps | Figure 7 (5 x 4) |
wan2.2-ti2v-5b/ablations/head-scaling/l5-s16 |
Trajectory head with 5 layers and 16 unrolled steps | Figure 7 (5 x 16) |
wan2.2-ti2v-5b/ablations/head-scaling/l8-s4 |
Trajectory head with 8 layers and 4 unrolled steps | Figure 7 (8 x 4) |
wan2.2-ti2v-5b/ablations/head-scaling/l8-s8 |
Trajectory head with 8 layers and 8 unrolled steps | Figure 7 (8 x 8) |
wan2.2-ti2v-5b/ablations/head-scaling/l8-s16 |
Trajectory head with 8 layers and 16 unrolled steps | Figure 7 (8 x 16) |
Figure 6(b), Table 3 and the 4-step row of Table 5 report the mean over three training seeds. Each of those folders holds one of the three seeds.
Files
Every folder contains the following.
model-0000x-of-0000N.safetensorsandmodel.safetensors.index.jsonhold the EMA weights, stored as trained. All reported results are generated with these EMA weights. Where the model has a trajectory head, the head is part of the same state dict.config.yamlis the training configuration. Base model files are given relative to their Hugging Face repositories. Initialization sources, which are intermediate checkpoints, are set tonull.
The three main models also contain training_state/. It is the PyTorch Distributed Checkpoint of the final training step,
with the student, its EMA, the fake score model, the optimizer states, the data loader and random number generator states,
and the per-rank streaming chain states. Resuming requires the same number of ranks and data loader workers per rank. These
are 16 ranks with 2 workers for Wan2.2-TI2V-5B, 32 ranks with 2 workers for the 15 s LTX-2.3-22B model, and 32 ranks with no
worker processes for the 60 s LTX-2.3-22B model.
manifest.json lists every file with its size and SHA-256.
Training data
data/ holds the training windows of the main models as preencoded latents.
| Folder | Used by | Content | Size |
|---|---|---|---|
data/wan2.2-ti2v-5b-229f |
every wan2.2-ti2v-5b/ model |
10,080 windows of 229 frames at 960x544 and 24 fps. Wan2.2 VAE video latents, umT5 text features and per-frame cameras | 126.2 GB in 26 shards |
data/ltx-2.3-22b-231f |
ltx-2.3-22b/relay-forcing-15s |
10,080 windows of 231 frames at 960x544 and 24 fps. LTX-2.3 video latents in 33-frame groups, first-image latents, LTX-2.3 text features and per-frame cameras | 126.7 GB in 26 shards |
data/ltx-2.3-22b-957f |
ltx-2.3-22b/relay-forcing-60s |
2,476 windows of 957 frames at 960x544 and 24 fps. LTX-2.3 video latents in 33-frame groups, first-image latents, LTX-2.3 text features and per-frame cameras | 68.1 GB in 14 shards |
Each folder holds uncompressed tar shards and files.jsonl, which lists every member with its size, SHA-256 and shard.
Extract all shards of a folder into that folder.
cd data/ltx-2.3-22b-231f && for shard in shard-*.tar; do tar -xf "$shard"; done
manifests/ then holds the training manifests, whose paths are relative to the manifest itself. The config.yaml of
each model points data.args.index_file to these manifests relative to the repository root. The windows are encoded
from SolarWM-Data, and their use is subject to the terms of SolarWM-Data and of its sources.
Loading
import json
from pathlib import Path
from huggingface_hub import snapshot_download
from safetensors.torch import load_file
folder = "wan2.2-ti2v-5b/relay-forcing"
root = Path(snapshot_download("oliveryanzuolu/Relay-Forcing", allow_patterns=[f"{folder}/model*", f"{folder}/config.yaml"])) / folder
index = json.loads((root / "model.safetensors.index.json").read_text())
state_dict = {}
for shard in sorted(set(index["weight_map"].values())):
state_dict.update(load_file(root / shard))
The keys match the module declared under models.backbone in config.yaml, or under models.video_decoder for the decoder.
The training and inference code will be released separately.
Base models
- Wan2.2-TI2V-5B models start from the camera-controlled SolarWM-5B and use the VAE and text encoder of Wan2.2-TI2V-5B.
- LTX-2.3-22B models start from LTX-2.3 and use its text and image encoders with
Gemma 3 12B. Videos are decoded with the VAE of
LTX-2.5 and the decoder weights in
ltx-2.3-22b/causal-vae-decoder.
Model tree for oliveryanzuolu/Relay-Forcing
Base model
Lightricks/LTX-2.3