Anima Pixel Decoder
A 25M-parameter decoder that reads the last-layer features of the Anima DiT at the final sampling step and writes RGB pixels directly, in place of the VAE decode. The head and its training follow FastFrames' pixel head (a ConvNeXt readout on the diffusion transformer's features).
| Files | anima_pixel_head.safetensors (×1, fp32, 95 MB) and anima_pixel_head_x2.safetensors (×2, fp32, 101 MB) |
| Input | the input of the DiT's final_layer.linear (2048-dim per token, after the adaLN-modulated LayerNorm) |
| Output | a 16×16 RGB patch per token (×2 file: 32×32), in [-1, 1] |
| Architecture | linear projection of the feature + linear embedding of the x_t patch, 2 ConvNeXt, one 2× pixel-shuffle upsample, 1 ConvNeXt, linear to 8×8×3 |
| DiT | anima-base-v1.0 (28 blocks) |
| Target | base points: the original image, with LPIPS (weight 0.05); turbo point: VAE(x̂0) of the same forward, x̂0 = x_t − σ·v |
Use
- ComfyUI: the ComfyUI-Anima-PixelDecoder
node pack (ComfyUI Manager: "Anima Pixel Decoder"). Put the file in
models/pixel_heads/, add Anima Pixel Head Tap on the MODEL before the KSampler (stock or Spectrum) and decode the sampler's LATENT with Anima Pixel Decode instead of VAE Decode. The file decides the scale: ×1 foranima_pixel_head, ×2 foranima_pixel_head_x2(node 0.3.1 or later). Run the DiT on the fp32 stream (Anima Adapter Loader,stream = fp32). - anima_lora:
inference.py --decoder pixel_head(make test PIXEL=1,make test-turbo PIXEL=1; ×2:make test PIXEL=1 PIXEL_SCALE=2); the module, loader and feature tap are inlibrary/anima/pixel_head.py.
Compatibility
Read the head only where it was trained:
- Base DiT: Euler, shift 3, 20 … 50 steps (last-step σ 0.058 … 0.136), read at CFG 4, Spectrum included (its last step is an actual forward).
- Turbo: the 4-step turbo LoRA of
anima-turbo-4step, Euler 4 steps,
shift 3, CFG 1 (last-step σ 0.5). Trained on
anima_turbo_4step_v4_longer.safetensors.
The ×2 file was trained at the base points only; on the turbo it decodes with a warning.
Benchmark
The benchmarks, samples and held-out reads below were measured on the previous head
(old/anima_pixel_head.safetensors).
In ComfyUI
Turbo 4 steps, CFG 1, RTX 5070 Ti, 16 prompts at ~1 MP, one seed
| KSampler + VAE Decode | KSampler + Anima Pixel Decode | |
|---|---|---|
| KSampler | 0.976 s | 0.998 s (with the tap) |
| decode | 151 ms | 7 ms |
| decode peak VRAM above the loaded models | 1.22 GB | 0.21 GB |
| PSNR against VAE Decode | — | 36.9 dB |
| total per image | 1.13 s | 1.00 s (1.12×) |
ComfyUI keeps the VAE on the card after its first decode, so the VAE column has no move cost.
In anima_lora
The same turbo setting and prompts through anima_lora's inference.py (compiled DiT blocks);
medians. The VAE column is the usual interactive path: the VAE waits on the CPU and goes to the
card for the decode.
| turbo + VAE | turbo + pixel head | |
|---|---|---|
| DiT sampling | 0.97 s | 0.97 s |
| load VAE | 710 ms | — |
| VAE move to the card and back | 35 ms | — |
| decode | 141 ms | 7 ms |
| decode peak VRAM above the DiT | 1.38 GB | 0.24 GB |
| total per image (DiT + decode) | 1.86 s | 0.98 s |
Samples
Turbo 4 steps, CFG 1, seed 42, 1024×1024, in ComfyUI (the workflow above); left VAE Decode,
right Anima Pixel Decode, decoded from the same latent. Prompt: masterpiece, best quality, score_7, safe. An anime girl wearing a black tank-top and denim shorts is standing outdoors. She's holding a rectangular sign out in front of her that reads "ANIMA". She's looking at the viewer with a smile. The background features some trees and blue sky with clouds.; the
monochrome pair adds monochrome, comic after safe. Crops: 384×384 at native resolution,
x 192–576, y 512–896.
On the monochrome crop the VAE keeps the screentone dots on the shorts; the head renders them as flat grey (see Known limits).
×2 output
anima_pixel_head_x2.safetensors decodes and upscales 2× in one readout: the ×1 head plus a
second 2× stage (pixel shuffle, 2 ConvNeXt blocks at width 256, linear to 8×8×3 per position), so
a 1024² render leaves the head at 2048².
Held-out against the real 2× image (one seed): 29.82 dB / LPIPS 0.158 on 12 test images, 25.52 dB / LPIPS 0.145 on 5 text images.
Base DiT, Euler 28 steps, shift 3, CFG 4, seed 42, 1024×1024, the prompts of the samples above,
through anima_lora's inference.py; left VAE Decode (1024²), right the ×2 head (2048²). Crops:
left 384×384 of the VAE decode, bicubic to 768×768; right the same region of the ×2 output at
native resolution.
On the monochrome crop the ×2 head turns the screentone on the shorts into a wavy pattern.
Held-out reads
Held-out reads during training, ×1 (dB, colour / mono):
| decode point | colour | mono |
|---|---|---|
| turbo 4 steps | 35.90 | 29.01 |
| base 20 steps | 33.71 | 26.58 |
| base 28 steps | 33.83 | 26.38 |
| base 50 steps | 33.49 | 25.48 |
| base 28 + Spectrum | 34.37 | 26.00 |
Known limits
- Screentone dots are smoothed into soft grey shading; textured regions (screentone, dense hatching) come out about 1 – 2.5 levels darker than the VAE in linear light. The lowest per-image PSNRs are manga pages and sketches (turbo 28.7 – 30.9 dB).
- Speed lines and footnote-size glyphs are slightly soft under zoom.
- ×2: screentone comes out as a wavy pattern instead of dots (see the monochrome crop).
- One trained head per file, one seed; no run spread.
License
The weights in this repository are a Derivative of Anima and are distributed under the CircleStone Labs Non-Commercial License (see NOTICE.md). Any rights to use them are granted directly by CircleStone Labs LLC under that license.
- The model/weights may only be used for non-commercial purposes. Generated images (Outputs) may be used commercially.
- Anima is itself a Derivative Model of NVIDIA Cosmos-Predict2-2B-Text2Image and is subject to the NVIDIA Open Model License Agreement insofar as it applies to Derivative Models.
- This is not an official CircleStone Labs product and is not endorsed by CircleStone Labs.
Built on NVIDIA Cosmos.
Model tree for sorryhyun/anima-pixel-decoder
Base model
nvidia/Cosmos-Predict2-2B-Text2Image