Instructions to use M4-ai/tau-1.8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use M4-ai/tau-1.8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="M4-ai/tau-1.8B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("M4-ai/tau-1.8B")
model = AutoModelForCausalLM.from_pretrained("M4-ai/tau-1.8B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use M4-ai/tau-1.8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "M4-ai/tau-1.8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "M4-ai/tau-1.8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/M4-ai/tau-1.8B

SGLang

How to use M4-ai/tau-1.8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "M4-ai/tau-1.8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "M4-ai/tau-1.8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "M4-ai/tau-1.8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "M4-ai/tau-1.8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use M4-ai/tau-1.8B with Docker Model Runner:
```
docker model run hf.co/M4-ai/tau-1.8B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

tau-1.8B

Model Details

Model Name: tau-1.8B
Base Model: Qwen1.5-1.8B
Dataset: UltraTextbooks-2.0
Model Size: 1.8B parameters
Model Type: Language Model
Training Procedure: Further pre-training of Qwen1.5-1.8B on UltraTextbooks-2.0.

Model Use

tau-1.8B is designed to be a general-purpose language model with enhanced capabilities in the domains of machine learning, mathematics, and coding. It can be used for a wide range of natural language processing tasks, such as:

Educational question answering
Text summarization
Content generation for educational purposes
Code understanding and generation
Mathematical problem solving

The model's exposure to the diverse content in the UltraTextbooks-2.0 dataset makes it particularly well-suited for applications in educational technology and research.

Training Data

tau-1.8B was further pre-trained on the UltraTextbooks-2.0 dataset, which is an expanded version of the original UltraTextbooks dataset. UltraTextbooks-2.0 incorporates additional high-quality synthetic and human-written textbooks from various sources on the Hugging Face platform, with a focus on increasing the diversity of content in the domains of machine learning, mathematics, and coding.

For more details on the dataset, please refer to the UltraTextbooks-2.0 Dataset Card.

Performance and Limitations

Refer to Evaluation for evaluations. It is essential to note that the model may still exhibit biases or inaccuracies present in the training data. Users are encouraged to critically evaluate the model's outputs and report any issues to facilitate continuous improvement.

Environmental Impact

The training of tau-1.8B required computational resources that contribute to the model's overall environmental impact. However, efforts were made to optimize the training process and minimize the carbon footprint.

Ethical Considerations

tau-1.8B was trained on a diverse dataset that may contain biases and inaccuracies. Users should be aware of these potential limitations and use the model responsibly. The model should not be used for tasks that could cause harm or discriminate against individuals or groups.

Evaluation

Metric	Value
Avg.	45.73
AI2 Reasoning Challenge (25-Shot)	37.20
HellaSwag (10-Shot)	60.26
MMLU (5-Shot)	45.96
TruthfulQA (0-shot)	39.72
Winogrande (5-shot)	61.09
GSM8k (5-shot)	30.17

Downloads last month: 80

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for M4-ai/tau-1.8B

Finetunes

1 model

Quantizations

2 models

Dataset used to train M4-ai/tau-1.8B

Spaces using M4-ai/tau-1.8B 2

Evaluation results

normalized accuracy on AI2 Reasoning Challenge (25-Shot)
test set Open LLM Leaderboard

37.200
normalized accuracy on HellaSwag (10-Shot)
validation set Open LLM Leaderboard

60.260
accuracy on MMLU (5-Shot)
test set Open LLM Leaderboard

45.960
mc2 on TruthfulQA (0-shot)
validation set Open LLM Leaderboard

39.720
accuracy on Winogrande (5-shot)
validation set Open LLM Leaderboard

61.090
accuracy on GSM8k (5-shot)
test set Open LLM Leaderboard

30.170