LFM2.5-Encoder-230M — LiteRT

LiquidAI/LFM2.5-Encoder-230M converted to LiteRT (.tflite) for on-device inference. A multilingual (15 languages) bidirectional encoder on the LFM2 hybrid backbone (gated short-convolutions + grouped-query attention) — use it for embeddings, retrieval, classification heads, and masked-token prediction, fully offline on CPU. This is the lightweight sibling of LFM2.5-Encoder-350M for tight latency and memory budgets.

File	Recipe	Size
LFM2.5-Encoder-230M_wi8fc.tflite	int8 dynamic-range (linears + embedding, convs float)	246 MB	mobile + desktop (iPhone-verified bit-exact)
LFM2.5-Encoder-230M_fp16.tflite	fp16 weights, float compute	463 MB	desktop — XNNPACK's per-signature fp32 unpacking is heavy on phone memory limits

Signatures

All signatures take batch-1, right-padded static shapes: input_ids int32 [1, S], attention_mask int32 [1, S] (1 = real token, 0 = pad).

Signature	Output
encode_64 / encode_128 / encode_256 / encode_512	last_hidden_state float32 [1, S, 1024], zeroed at padded positions
mlm_128	masked-LM logits float32 [1, 128, 65536]

Padded positions are fully masked inside the graph (conv path and attention), so the output at valid positions is independent of padding length — encode_64/128/256 agree bitwise on the same sentence, and match the unpadded PyTorch reference.

Quality (parity vs PyTorch fp32 reference)

16 sentences covering all 15 supported languages; mean-pooled sentence embedding cosine vs the original Lfm2BidirectionalModel, and top-5 fill-mask agreement on en/fr/de/ja cloze prompts (the fp16/fp32 conversion reproduces the base card's documented The capital of France is [MASK]. output verbatim):

Variant	Pooled cos (min / mean)	Per-token corr (min)	Fill-mask
fp16	1.000000 / 1.000000	0.999999	top-5 sets identical (4/4 prompts)
int8 (wi8fc)	0.994781 / 0.998148	0.986881	top-1 4/4, ≥3/5 top-5 overlap on all

Speed (CPU/XNNPACK)

Variant	Device	encode_128	encode_512
int8 (wi8fc)	Apple-silicon Mac (all threads)	39 ms (3312 tok/s)	100 ms (5116 tok/s)
int8 (wi8fc)	iPhone 17 Pro (6 threads)	27 ms	93 ms (~5500 tok/s)

On the iPhone 17 Pro the int8 model reproduces the Mac outputs bit-exactly (cosine 1.000000, max diff 0.0) across all tested languages and signatures; peak footprint ≈1.0 GiB.

Usage (Python)

import numpy as np
from ai_edge_litert.interpreter import Interpreter
from tokenizers import Tokenizer

tok = Tokenizer.from_file("tokenizer.json")
it = Interpreter(model_path="LFM2.5-Encoder-230M_wi8fc.tflite")
encode = it.get_signature_runner("encode_128")

ids = tok.encode("On-device embeddings, private and fast.").ids
x = np.zeros((1, 128), np.int32); m = np.zeros((1, 128), np.int32)
x[0, :len(ids)] = ids; m[0, :len(ids)] = 1

h = list(encode(input_ids=x, attention_mask=m).values())[0]  # [1, 128, 1024]
emb = h[0, :len(ids)].mean(axis=0)                           # sentence embedding

For masked-token prediction use the mlm_128 signature and read the logits at the [MASK] position. On Android/iOS use the LiteRT runtime's SignatureRunner APIs with the same signature names; the tokenizer is the standard Hugging Face tokenizer.json (works with the tokenizers libraries for Rust/Swift/Kotlin).

License

LFM Open License v1.0 (see LICENSE, unchanged from the base model). Note the license's commercial-use threshold (Section 5). This repository redistributes converted Derivative Works of LiquidAI/LFM2.5-Encoder-230M with modification notices per Section 4; all credit for the model to Liquid AI.