OsaurusAI/LFM2.5-VL-3B-MXFP8

MX FP8 (mx.quantize(mode="mxfp8"), group size 32). Highest-fidelity bundle; use as the quality reference.

MX FP8 (group size 32) conversion of LiquidAI/LFM2.5-VL-3B for Apple silicon (MLX).

LFM2.5-VL-3B is a vision-language model: a hybrid short-convolution + attention lfm2 text backbone (30 layers, 22 conv / 8 full-attention) paired with a SigLIP2 so400m-patch16-naflex vision tower and a 2-layer projector. Images are tiled (up to 10 tiles at 512px) into 64–256 image tokens.

Bundles in this series

Bundle	Avg bits	Size	Notes
LFM2.5-VL-3B-JANG_2L	3.14	1.9 GB	smallest
LFM2.5-VL-3B-JANG_4M	4.12	2.2 GB	recommended
LFM2.5-VL-3B-JANG_6M	6.06	2.9 GB	near-lossless
LFM2.5-VL-3B-MXFP8	9.31	3.4 GB	MX FP8 reference

The vision tower is not quantized

All four bundles keep the 437 SigLIP2 vision tensors at source precision. Quantizing a vision tower aggressively is what breaks image understanding, so the bits are spent on the language model instead. This is why image quality holds up even in the smallest bundle — and why the small profiles are dominated by the tower rather than the LM.

Verified

Loaded and generated on an M5 Max on 2026-08-12, through this bundle's own chat_template.jinja and processor_config.json:

• Vision — correctly named both shapes, both colors and their positions on a synthetic probe image
• Multi-turn — carried image context across three turns, including a follow-up comparison ("which is larger, and where") and a format constraint
• Tool calling — parsed <|tool_call_start|>[get_weather(city='...')]<|tool_call_end|> into {'name': 'get_weather', 'arguments': {'city': 'Santa Clara'}}

Sampling

LiquidAI's documented settings, stamped into both generation_config.json and jang_config.json:

Parameter	Value
temperature	0.2
top_k	50
top_p	1.0 (disabled)
repetition_penalty	1.0
eos_token_id	[124900]

top_p is not specified upstream, so it is stamped as disabled rather than invented.

Reasoning

This model does not reason — it answers directly. There is no thinking rail: the chat template emits a bare <|im_start|>assistant\n generation prompt with no <think> prefill, and no enable_thinking / thinking / preserve_thinking kwarg changes it (verified byte-identical). preserve_thinking only governs whether <think> in prior assistant turns is retained, which is history handling. The bundle metadata says supports_thinking: false accordingly.

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("OsaurusAI/LFM2.5-VL-3B-MXFP8")
messages = [{"role": "user", "content": "What is in this image?"}]
prompt = apply_chat_template(processor, model.config, messages, num_images=1)
print(generate(model, processor, prompt, image=["photo.jpg"],
               max_tokens=256, temperature=0.2, verbose=False))

Notes for MLX runtimes

This bundle carries three config keys the MLX stack needs that the upstream checkpoint does not provide in the expected form:

• text_config.block_ff_dim — mlx_lm's lfm2 requires it; without it the FFN dimension is computed wrong and the model fails to load.
• image_token_index — mirrors upstream's image_token_id; mlx_vlm reads the _index spelling and otherwise matches zero image tokens.
• per-module quantization overrides are emitted under both model.<path> and the post-sanitize <path> spelling, because mlx_vlm looks up the latter.

tool_parser_type: "pythonic" is set explicitly so mlx_lm resolves the tool parser without relying on template string-matching.

License

Released under the base model's license. Review LiquidAI/LFM2.5-VL-3B terms before use.

---

Converted by Jinho Jang — eric@osaurus.ai — with JANG.