anima
decoder

Anima Pixel Decoder

A 25M-parameter decoder that reads the last-layer features of the Anima DiT at the final sampling step and writes RGB pixels directly, in place of the VAE decode. The head and its training follow FastFrames' pixel head (a ConvNeXt readout on the diffusion transformer's features).

Files anima_pixel_head.safetensors (×1, fp32, 95 MB) and anima_pixel_head_x2.safetensors (×2, fp32, 101 MB)
Input the input of the DiT's final_layer.linear (2048-dim per token, after the adaLN-modulated LayerNorm)
Output a 16×16 RGB patch per token (×2 file: 32×32), in [-1, 1]
Architecture linear projection of the feature + linear embedding of the x_t patch, 2 ConvNeXt, one 2× pixel-shuffle upsample, 1 ConvNeXt, linear to 8×8×3
DiT anima-base-v1.0 (28 blocks)
Target base points: the original image, with LPIPS (weight 0.05); turbo point: VAE(x̂0) of the same forward, x̂0 = x_t − σ·v

Use

  • ComfyUI: the ComfyUI-Anima-PixelDecoder node pack (ComfyUI Manager: "Anima Pixel Decoder"). Put the file in models/pixel_heads/, add Anima Pixel Head Tap on the MODEL before the KSampler (stock or Spectrum) and decode the sampler's LATENT with Anima Pixel Decode instead of VAE Decode. The file decides the scale: ×1 for anima_pixel_head, ×2 for anima_pixel_head_x2 (node 0.3.1 or later). Run the DiT on the fp32 stream (Anima Adapter Loader, stream = fp32).
  • anima_lora: inference.py --decoder pixel_head (make test PIXEL=1, make test-turbo PIXEL=1; ×2: make test PIXEL=1 PIXEL_SCALE=2); the module, loader and feature tap are in library/anima/pixel_head.py.

Compatibility

Read the head only where it was trained:

  • Base DiT: Euler, shift 3, 20 … 50 steps (last-step σ 0.058 … 0.136), read at CFG 4, Spectrum included (its last step is an actual forward).
  • Turbo: the 4-step turbo LoRA of anima-turbo-4step, Euler 4 steps, shift 3, CFG 1 (last-step σ 0.5). Trained on anima_turbo_4step_v4_longer.safetensors.

The ×2 file was trained at the base points only; on the turbo it decodes with a warning.

Benchmark

The benchmarks, samples and held-out reads below were measured on the previous head (old/anima_pixel_head.safetensors).

In ComfyUI

Turbo 4 steps, CFG 1, RTX 5070 Ti, 16 prompts at ~1 MP, one seed

KSampler + VAE Decode KSampler + Anima Pixel Decode
KSampler 0.976 s 0.998 s (with the tap)
decode 151 ms 7 ms
decode peak VRAM above the loaded models 1.22 GB 0.21 GB
PSNR against VAE Decode — 36.9 dB
total per image 1.13 s 1.00 s (1.12×)

ComfyUI keeps the VAE on the card after its first decode, so the VAE column has no move cost.

In anima_lora

The same turbo setting and prompts through anima_lora's inference.py (compiled DiT blocks); medians. The VAE column is the usual interactive path: the VAE waits on the CPU and goes to the card for the decode.

turbo + VAE turbo + pixel head
DiT sampling 0.97 s 0.97 s
load VAE 710 ms —
VAE move to the card and back 35 ms —
decode 141 ms 7 ms
decode peak VRAM above the DiT 1.38 GB 0.24 GB
total per image (DiT + decode) 1.86 s 0.98 s

Samples

Turbo 4 steps, CFG 1, seed 42, 1024×1024, in ComfyUI (the workflow above); left VAE Decode, right Anima Pixel Decode, decoded from the same latent. Prompt: masterpiece, best quality, score_7, safe. An anime girl wearing a black tank-top and denim shorts is standing outdoors. She's holding a rectangular sign out in front of her that reads "ANIMA". She's looking at the viewer with a smile. The background features some trees and blue sky with clouds.; the monochrome pair adds monochrome, comic after safe. Crops: 384×384 at native resolution, x 192–576, y 512–896.

VAE Decode pixel head
colour colour, VAE colour, pixel head
crop colour crop, VAE colour crop, pixel head
monochrome mono, VAE mono, pixel head
crop mono crop, VAE mono crop, pixel head

On the monochrome crop the VAE keeps the screentone dots on the shorts; the head renders them as flat grey (see Known limits).

×2 output

anima_pixel_head_x2.safetensors decodes and upscales 2× in one readout: the ×1 head plus a second 2× stage (pixel shuffle, 2 ConvNeXt blocks at width 256, linear to 8×8×3 per position), so a 1024² render leaves the head at 2048².

Held-out against the real 2× image (one seed): 29.82 dB / LPIPS 0.158 on 12 test images, 25.52 dB / LPIPS 0.145 on 5 text images.

Base DiT, Euler 28 steps, shift 3, CFG 4, seed 42, 1024×1024, the prompts of the samples above, through anima_lora's inference.py; left VAE Decode (1024²), right the ×2 head (2048²). Crops: left 384×384 of the VAE decode, bicubic to 768×768; right the same region of the ×2 output at native resolution.

VAE Decode ×2 head
colour colour, VAE colour, ×2 head
crop colour crop, VAE bicubic ×2 colour crop, ×2 head
monochrome mono, VAE mono, ×2 head
crop mono crop, VAE bicubic ×2 mono crop, ×2 head

On the monochrome crop the ×2 head turns the screentone on the shorts into a wavy pattern.

Held-out reads

Held-out reads during training, ×1 (dB, colour / mono):

decode point colour mono
turbo 4 steps 35.90 29.01
base 20 steps 33.71 26.58
base 28 steps 33.83 26.38
base 50 steps 33.49 25.48
base 28 + Spectrum 34.37 26.00

Known limits

  • Screentone dots are smoothed into soft grey shading; textured regions (screentone, dense hatching) come out about 1 – 2.5 levels darker than the VAE in linear light. The lowest per-image PSNRs are manga pages and sketches (turbo 28.7 – 30.9 dB).
  • Speed lines and footnote-size glyphs are slightly soft under zoom.
  • ×2: screentone comes out as a wavy pattern instead of dots (see the monochrome crop).
  • One trained head per file, one seed; no run spread.

License

The weights in this repository are a Derivative of Anima and are distributed under the CircleStone Labs Non-Commercial License (see NOTICE.md). Any rights to use them are granted directly by CircleStone Labs LLC under that license.

  • The model/weights may only be used for non-commercial purposes. Generated images (Outputs) may be used commercially.
  • Anima is itself a Derivative Model of NVIDIA Cosmos-Predict2-2B-Text2Image and is subject to the NVIDIA Open Model License Agreement insofar as it applies to Derivative Models.
  • This is not an official CircleStone Labs product and is not endorsed by CircleStone Labs.

Built on NVIDIA Cosmos.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for sorryhyun/anima-pixel-decoder

Finetuned
(122)
this model