Text Generation
Transformers
Safetensors
English
qwen2
code
codeqwen
chat
qwen
qwen-coder
conversational
text-generation-inference
Instructions to use Qwen/Qwen2.5-Coder-32B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Qwen/Qwen2.5-Coder-32B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Qwen/Qwen2.5-Coder-32B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-32B-Instruct") model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-32B-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Qwen/Qwen2.5-Coder-32B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Qwen/Qwen2.5-Coder-32B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen2.5-Coder-32B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Qwen/Qwen2.5-Coder-32B-Instruct
- SGLang
How to use Qwen/Qwen2.5-Coder-32B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Qwen/Qwen2.5-Coder-32B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen2.5-Coder-32B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Qwen/Qwen2.5-Coder-32B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen2.5-Coder-32B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Qwen/Qwen2.5-Coder-32B-Instruct with Docker Model Runner:
docker model run hf.co/Qwen/Qwen2.5-Coder-32B-Instruct
| { | |
| "bomFormat": "CycloneDX", | |
| "specVersion": "1.6", | |
| "serialNumber": "urn:uuid:695982c3-1c8b-4bd0-bfb4-2a3b0e85c902", | |
| "version": 1, | |
| "metadata": { | |
| "timestamp": "2025-06-05T09:40:54.111210+00:00", | |
| "component": { | |
| "type": "machine-learning-model", | |
| "bom-ref": "Qwen/Qwen2.5-Coder-32B-Instruct-8ee1be07-86b5-5c5f-84aa-9269297da2dd", | |
| "name": "Qwen/Qwen2.5-Coder-32B-Instruct", | |
| "externalReferences": [ | |
| { | |
| "url": "https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct", | |
| "type": "documentation" | |
| } | |
| ], | |
| "modelCard": { | |
| "modelParameters": { | |
| "task": "text-generation", | |
| "architectureFamily": "qwen2", | |
| "modelArchitecture": "Qwen2ForCausalLM" | |
| }, | |
| "properties": [ | |
| { | |
| "name": "library_name", | |
| "value": "transformers" | |
| }, | |
| { | |
| "name": "base_model", | |
| "value": "Qwen/Qwen2.5-Coder-32B" | |
| } | |
| ] | |
| }, | |
| "authors": [ | |
| { | |
| "name": "Qwen" | |
| } | |
| ], | |
| "licenses": [ | |
| { | |
| "license": { | |
| "id": "Apache-2.0", | |
| "url": "https://spdx.org/licenses/Apache-2.0.html" | |
| } | |
| } | |
| ], | |
| "description": "Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:- Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.- A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.- **Long-context Support** up to 128K tokens.**This repo contains the instruction-tuned 32B Qwen2.5-Coder model**, which has the following features:- Type: Causal Language Models- Training Stage: Pretraining & Post-training- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias- Number of Parameters: 32.5B- Number of Paramaters (Non-Embedding): 31.0B- Number of Layers: 64- Number of Attention Heads (GQA): 40 for Q and 8 for KV- Context Length: Full 131,072 tokens- Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2.5 for handling long texts.For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).", | |
| "tags": [ | |
| "transformers", | |
| "safetensors", | |
| "qwen2", | |
| "text-generation", | |
| "code", | |
| "codeqwen", | |
| "chat", | |
| "qwen", | |
| "qwen-coder", | |
| "conversational", | |
| "en", | |
| "arxiv:2409.12186", | |
| "arxiv:2309.00071", | |
| "arxiv:2407.10671", | |
| "base_model:Qwen/Qwen2.5-Coder-32B", | |
| "base_model:finetune:Qwen/Qwen2.5-Coder-32B", | |
| "license:apache-2.0", | |
| "autotrain_compatible", | |
| "text-generation-inference", | |
| "endpoints_compatible", | |
| "region:us" | |
| ] | |
| } | |
| } | |
| } |