Instructions to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="bottlecapai/ThinkingCap-Qwen3.6-27B-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B-FP8")
model = AutoModelForMultimodalLM.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

SGLang

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with Docker Model Runner:
```
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8
```

Request access to the BottleCap AI model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Please fill out the form below to request access. Name, Company Name, and Company Email are optional — just enter NA if you'd prefer not to share. Your email may be used to send you information about BottleCap AI model updates and early access before public release.

bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

FP8 quantization of bottlecapai/ThinkingCap-Qwen3.6-27B — capability of Qwen3.6-27B with 50% less thinking tokens on average, achieved by finetuning Qwen3.6-27B (Qwen Team, 2026) while preserving the original answer quality and style.

➡️ Full model description, evaluation results (multi-seed, statistically tested), recommended sampling params, and citation: see the main model card at bottlecapai/ThinkingCap-Qwen3.6-27B.

About this quantization

Native block-wise FP8 (E4M3): weights are quantized to FP8 with 128×128 block scales (weight_scale_inv) and activations are quantized dynamically per token — quant_method: "fp8", weight_block_size: [128, 128], the same DeepSeek-V3-style format that Qwen/Qwen3.6-27B-FP8 ships. vLLM and SGLang load it natively, and the model's MTP (multi-token-prediction) head is preserved, so self-speculative decoding works (≈3.2 accepted tokens/step). ≈29 GB instead of ≈55 GB bf16 — near-lossless quality at half the memory, with FP8 tensor-core throughput on Hopper / Blackwell (Ada / Ampere serve it via weight-only FP8-Marlin kernels).

Kept in bf16: lm_head, the MTP head, the vision tower, and the Gated-DeltaNet input gates (linear_attn.in_proj_a/b, whose 48-wide output doesn't tile a 128 block).

Usage

# vLLM
vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

# SGLang
python -m sglang.launch_server --model-path bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 --trust-remote-code

Speculative decoding (MTP)

The MTP (multi-token-prediction / NextN) head is kept in bf16, so vLLM can run self-speculative decoding for a decode speed-up — no separate draft model needed. Add --speculative-config when serving (requires vLLM ≥ 0.24.0):

vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Speculative decoding is lossless — the output is identical to standard decoding. It accepts ≈3.35 drafts per verify step here, for a further ≈2.4× on top of the finetune's token savings (see below).

For local llama.cpp / Ollama / LM Studio use, see the GGUF quantizations at bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF.

Expected performance

Measured on our internal serving harness on 8× RTX PRO 6000 (Blackwell) with vLLM 0.25.0, one configuration per GPU. Each cell is N=200 questions/dataset × 3 generation seeds over a fixed problem subset, batch size 16, sampled decoding (temperature 1.0, top_p 0.95, top_k 20); acc is the mean ± 95% CI across the seeds. Generation budgets are 24,576 tokens (MMLU-Pro) and 16,384 tokens (RealWorldQA).

FP8 matches the bf16 finetune's accuracy (all 95% CIs overlap) while decoding ≈55% faster (36.4 vs 23.4 tok/s, MMLU-Pro standard), and MTP self-speculative decoding (≈3.1–3.3 accepted tokens per verify step) adds a further ≈1.8–2.0× — stacking with the finetune's token savings to ≈6× faster per task than the unquantized base (MMLU-Pro 13.7 s vs 97.3 s; RealWorldQA 6.8 s vs 34.2 s).

All timings are measured per request during the eval. median tokens = median completion length (the finetune's lever); tok/s = per-request steady-state decode rate (prefill excluded via time-to-first-token); task s = measured end-to-end wall-clock per request under the batch-size-16 concurrency; speedup = task s of the base model in standard decoding ÷ task s of the row. Qwen3.6-27B-FP8 (official) is Qwen/Qwen3.6-27B-FP8, the official FP8 quant of the base, shown for comparison.

MMLU-Pro (reasoning)

config	acc	median tokens	tok/s	task s	speedup	accept_len
Qwen3.6-27B base · standard	0.902 ± 0.019	2186	22.6	97.3	1.00×	—
Qwen3.6-27B base · MTP	0.892 ± 0.031	2117	47.7	45.6	2.13×	3.23
Qwen3.6-27B-FP8 (official) · standard	0.888 ± 0.007	2250	35.5	63.7	1.53×	—
Qwen3.6-27B-FP8 (official) · MTP	0.907 ± 0.044	2174	64.4	34.5	2.82×	3.21
ThinkingCap bf16 · standard	0.890 ± 0.022	963	23.4	41.6	2.34×	—
ThinkingCap bf16 · MTP	0.895 ± 0.012	910	51.1	18.8	5.18×	3.26
ThinkingCap-FP8 · standard	0.908 ± 0.026	1000	36.4	27.9	3.49×	—
ThinkingCap-FP8 · MTP	0.900 ± 0.050	960	70.8	13.7	7.10×	3.26

RealWorldQA (vision)

config	acc	median tokens	tok/s	task s	speedup	accept_len
Qwen3.6-27B base · standard	0.802 ± 0.019	718	21.7	34.2	1.00×	—
Qwen3.6-27B base · MTP	0.798 ± 0.064	752	42.2	18.7	1.83×	3.09
Qwen3.6-27B-FP8 (official) · standard	0.803 ± 0.014	703	33.3	22.2	1.54×	—
Qwen3.6-27B-FP8 (official) · MTP	0.790 ± 0.054	714	56.8	13.8	2.48×	3.09
ThinkingCap bf16 · standard	0.818 ± 0.047	344	21.5	17.4	1.97×	—
ThinkingCap bf16 · MTP	0.805 ± 0.045	372	42.5	9.9	3.45×	3.09
ThinkingCap-FP8 · standard	0.807 ± 0.026	346	33.3	11.6	2.95×	—
ThinkingCap-FP8 · MTP	0.810 ± 0.045	334	58.6	6.8	5.03×	3.09

Downloads last month: 79,301

Safetensors

Model size

28B params

Tensor type

BF16

F8_E4M3

Model tree for bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

Base model

Qwen/Qwen3.6-27B

Finetuned

bottlecapai/ThinkingCap-Qwen3.6-27B

Quantized

(51)

this model