gemma-3-27b-eagle3-drafter

an EAGLE-3 speculative-decoding draft head for google/gemma-3-27b-it, trained from scratch on a single RTX 5090 (32GB) over four unattended nights.

pair it with Gemma-3-27B-it in sglang and generation gets faster with no change in output: verified tokens are exactly what the target model would have produced. measured on the training rig, best config (tree-3-4-8), against plain decoding:

workload	speedup	accept length
repetitive text	1.52x	1.97
code	1.44x	1.84
prose	1.33x	1.56
chat	1.23x	1.57

(release bench at the selected checkpoint, step 21000; base decode 59.6 tok/s on the same card, code reaching 89.0 tok/s with the drafter by the final bench. zero failed requests across all bench runs.)

usage (sglang)

python -m sglang.launch_server \
  --model-path <your-gemma-3-27b-it path or AWQ variant> \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path witcheer/gemma-3-27b-eagle3-drafter \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 4 \
  --speculative-num-draft-tokens 8

the three configs benched:

config	num_steps	eagle_topk	num_draft_tokens	note
chain-3-1-4	3	1	4	cheapest, lowest gain
tree-3-4-8	3	4	8	best net speedup, recommended
tree-5-8-16	5	8	16	higher accept length, lower net speedup: the extra draft work costs more than the extra accepts pay

verified with sglang 0.5.14, target served as AWQ, context 4096, cuda graphs on.

what's inside

• 716M-parameter single-layer llama-architecture EAGLE-3 head (LlamaForCausalLMEagle3, bf16), hidden size 5376 to match the Gemma-3-27B residual stream
• draft vocab 32000 with d2t/t2d maps to Gemma's 262k vocabulary
• no Gemma weights are included; every tensor in this repo was trained from scratch

training

• framework: SpecForge
• data: a 54k-sample chat and code dataset
• hardware: one RTX 5090 (32GB), four nights of otherwise idle time (23:00 to ~05:20 each), ~25h GPU total
• checkpoint selection: every checkpoint was release-benched live; the speedup curve peaked at step 21000 (39% of one epoch) and regressed at 28000 (prose 1.33x to 1.24x), so 21000 ships. training loss alone does not show this: bench your checkpoints.

curve at a glance (tree-3-4-8):

step	prose	code	repetitive	chat
7000	1.22x	1.40x	1.37x	1.25x
14000	1.29x	1.43x	1.44x	1.22x
21000	1.33x	1.44x	1.52x	1.23x
28000	1.24x	1.49x	1.50x	1.25x

licence

this head was trained on hidden states produced by Gemma-3-27B-it, so it is distributed as a Gemma model derivative under the Gemma Terms of Use. the drafter never generates final output on its own; all emitted tokens are verified by the Gemma target model.