Instructions to use groxaxo/Qwen3.5-24.5B-Reapped-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use groxaxo/Qwen3.5-24.5B-Reapped-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="groxaxo/Qwen3.5-24.5B-Reapped-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("groxaxo/Qwen3.5-24.5B-Reapped-v1")
model = AutoModelForCausalLM.from_pretrained("groxaxo/Qwen3.5-24.5B-Reapped-v1", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use groxaxo/Qwen3.5-24.5B-Reapped-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "groxaxo/Qwen3.5-24.5B-Reapped-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "groxaxo/Qwen3.5-24.5B-Reapped-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/groxaxo/Qwen3.5-24.5B-Reapped-v1

SGLang

How to use groxaxo/Qwen3.5-24.5B-Reapped-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "groxaxo/Qwen3.5-24.5B-Reapped-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "groxaxo/Qwen3.5-24.5B-Reapped-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "groxaxo/Qwen3.5-24.5B-Reapped-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "groxaxo/Qwen3.5-24.5B-Reapped-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use groxaxo/Qwen3.5-24.5B-Reapped-v1 with Docker Model Runner:
```
docker model run hf.co/groxaxo/Qwen3.5-24.5B-Reapped-v1
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.5-24.5B-Reapped-v1

Overview

Qwen3.5-24.5B-Reapped-v1 is a model checkpoint packaged for compatible Hugging Face runtimes, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

Field	Details
Format	Transformers
Source / base	the source checkpoint identified in the repository metadata
Intended task	`text-generation`
License	`apache-2.0`

What is included

*.safetensors (11 files)
config.json
generation_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja
Additional configuration, tokenizer, processor, or shard files (17 visible artifacts total)

Quick start

Getting started

Start with the upstream library named in the repository metadata and keep all configuration, tokenizer, processor, and weight files together. This repository is an artifact release, so the source project remains the authoritative reference for task-specific loading code.

Compatibility and responsible use

Use a runtime that explicitly supports this format, architecture, and modality.
Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
Review the source model card and license before redistribution or deployment.
Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

A leaner, coding-sharpened Qwen3.5 MoE. This model takes a 35B-class Qwen3.5 Mixture-of-Experts, REAPs away ~30% of its experts to land at ~24.5B total parameters (≈3B active per token), then bakes in a coding/agentic LoRA so the slimmer network punches well above its memory footprint.

Smaller resident weights. Same ~3B active compute per token. A coder's attitude welded on.

Why it exists

Modern MoE models carry a lot of expert capacity you don't always need. REAP (Router-weighted Expert Activation Pruning) ranks experts by how much the router actually relies on them and drops the dead weight — here 256 → 180 experts at the seed_42 / 0.30 setting. The result loads in ~47 GB bf16 (fits comfortably across 3×24 GB GPUs) while keeping the active-parameter compute of the original A3B design.

On top of the pruned base we merged a rank-16 QLoRA trained on a coding + agentic mix, so the model ships ready to write and reason about code rather than needing a separate adapter at serve time.

Lineage

Stage	What	Result
Base	Qwen3.5 MoE (A3B), "Heretic" lineage	256 experts
Prune	REAP `seed_42-0.30`	180 experts, ~24.5B total
Specialize	QLoRA r16 (NF4, FSDP2, 3×RTX 3090) on `coding_fable_mix`	coding/agentic adapter
Ship	LoRA merged into the pruned base (this repo)	standalone bf16 model

Model details

Architecture: Qwen3_5MoeForCausalLM (qwen3_5_moe) — hybrid DeltaNet linear-attention + full-attention layers, MoE FFN with a shared expert.
Experts: 180 (REAP-pruned from 256) · Layers: 40 · Hidden: 2048
Params: ~24.5B total, ~3B active per token
Precision: bf16 · Context: long-context capable (served at 8k here; base supports far more)
Tokenizer / chat template: inherited from the Qwen3.5 base (included)

Specialization (the merged LoRA)

Adapter: LoRA r=16, α=32, dropout=0.05; targets sequence-mixing only (q/k/v/o_proj + DeltaNet in_proj_{qkv,z,b,a} + out_proj) — experts were not adapted.
Data: coding_fable_mix — 10,270 chat rows including agentic-coding traces (~20%).
Recipe: 4-bit NF4 QLoRA, FSDP2 sharded (no CPU offload), Flash-Attention-2, bf16, seq-len 2048, LR 1.2e-4 cosine, effective batch 24, on 3× RTX 3090.
Checkpoint loss: 1.33 (ppl ≈ 3.79).
Merge fidelity: verified weight-exact — for adapted modules W_merged = W_base + (α/r)·B·A (max abs error 2.4e-4, bf16 rounding); all non-adapted weights byte-identical to the base.

Usage

vLLM (recommended — tested pp=3, tp=1 on 3×24 GB)

Runs on stock vLLM ≥ 0.23.0 — no source patches, no custom registry entries. (An earlier REAP-pruned checkpoint in this lineage needed 4 manual patches to qwen3_5.py on vLLM 0.19.0; upstream added native Qwen3_5MoeForCausalLM support since, so a current install just works out of the box.)

pip install "vllm>=0.23.0"
vllm serve groxaxo/Qwen3.5-24.5B-Reapped-v1 \
  --pipeline-parallel-size 3 --tensor-parallel-size 1 \
  --dtype bfloat16 --max-model-len 8192 \
  --enforce-eager --enable-prefix-caching

No 3-GPU box? Single-GPU fallback via CPU offload (slower, still zero patches):

vllm serve groxaxo/Qwen3.5-24.5B-Reapped-v1 \
  --tensor-parallel-size 1 --dtype bfloat16 \
  --cpu-offload-gb 32 --max-model-len 4096 --enforce-eager

Before serving, confirm your vLLM build actually registers the architecture (not every build does — some 0.23.x builds and all pre-0.23 releases we checked don't):

from vllm.model_executor.models.registry import ModelRegistry
assert "Qwen3_5MoeForCausalLM" in ModelRegistry.get_supported_archs()

Verified 2026-07-03 on vLLM 0.23.0: clean load (correct architecture resolution, no registry/routing errors from the pruned 256→180 expert count) and coherent /v1/chat/completions output, on a single-GPU + CPU-offload deployment.

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

mid = "groxaxo/Qwen3.5-24.5B-Reapped-v1"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, dtype=torch.bfloat16, device_map="auto")

msgs = [{"role": "user", "content": "Write a Python function that reverses the words in a string."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256, temperature=0.2)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

This is a reasoning-style model: it may emit a thinking trace before the final answer.

Sanity checks (served via vLLM, pp3/tp1)

Prompt	Response
Reverse the words in a string	`' '.join(reversed(s.split()))` ✅
Train 60 km in 45 min → km/h	80 ✅
Why does `lst[3]` IndexError; fix it	zero-indexed → use `lst[-1]` ✅

Limitations & notes

Inherits the biases and uncensored ("Heretic"-lineage) behavior of the base.
REAP pruning removes expert capacity; expect some regression on tasks far outside the coding/agentic specialization relative to the full 256-expert model.
Only the attention/linear-attention projections were fine-tuned — knowledge stored in experts is the pruned base's.
"v1" — an early specialization checkpoint (2K-context stage). Longer-context continuations are planned.

Acknowledgements

Built on the Qwen3.5 MoE family, slimmed with the REAP expert-pruning method, and specialized with axolotl QLoRA on consumer 3×RTX 3090 hardware. Released by groxaxo.

Downloads last month: 472

Safetensors

Model size

25B params

Tensor type

BF16

Model tree for groxaxo/Qwen3.5-24.5B-Reapped-v1

Quantizations

1 model