ddalcu/MiniMax-H3-REF2VA-MLX-Serve-8bit

MiniMax-H3 REF2VA — MLX-Serve 8-bit

8-bit affine (group size 64) conversion of MiniMaxAI/MiniMax-H3 (REF2VA) for mlx-serve, running natively on Apple Silicon. The DiT denoises video and stereo audio jointly in one packed sequence.

Text-to-audio-video conditioned on reference images, videos and audio, for character / style / scene continuity.

Self-contained: weights, both VAEs and the tokenizer in one directory. Upstream splits these across Comfy-Org/MiniMax-H3 (weights, no tokenizer) and MiniMaxAI/MiniMax-H3 (tokenizer).

Quantized: the DiT and text-encoder matmul weights. Kept dense: gathered embedding tables, the checkpoint's fp32 islands (patch projections, output heads, time embedder) and both VAEs.

Note that quantization here buys FOOTPRINT, not speed — the workload is compute-bound at roughly 192,000 FLOPs per weight byte.

Which pack

This one takes reference images, clips and audio. It cannot do first/last-frame conditioning. The FL2VA packs (8-bit, 4-bit) are the other way round: keyframes, no references. They are two checkpoints, not two modes of one, and the files are otherwise identical, so check config.json's tasks if you are unsure which you have.

References

Pass them on POST /v1/video/generations:

field	takes	cap
ref_images	base64 PNG/JPEG	9
ref_videos	{"frames": [b64, ...], "audio": "<b64 wav>"} objects	3
ref_audios	base64 WAV, any rate or channel count	3

At most 12 files across all three. Labels are 1-based per type, in the order images, then videos, then standalone audio, and a clip's own soundtrack takes its <Audio j> label immediately before that clip's <Video k>. Reference rows ride every sampling step, so each reference makes the run slower; a reference video is snapped down to the frame ladder and truncated to the length you are generating.

The MLX Core app drives all of this from the Media window, including the examples above.

Modifications

These are MODIFIED files. The transformer and text encoder are quantized to 8-bit; see MODIFICATIONS.md for the full list. The VAEs and tokenizer are unmodified copies.

License

Powered by MiniMax H3. Licensed under the MiniMax H3 Community License Agreement -- see LICENSE and NOTICE, both included here.

Territorial restriction. The Agreement defines the Applicable Territory as worldwide EXCLUDING the European Union, the United Kingdom, the Republic of Korea and the United States of America, and Section V.4 prohibits use, reproduction, modification, distribution and display outside it. Check whether your jurisdiction permits you to use these files before downloading them.