gg-hf-g

community

AI & ML interests

None defined yet.

posted an update 13 days ago

Post

4583

Qwen3.8-27B Unsloth GGUF is now the #1 most-liked GGUF of all time! 🤗🦥

The model hit 10M downloads and 3.7K likes in just 24 days on Hugging Face - all thanks to you.

GGUF: unsloth/Qwen3.8-27B-GGUF
Guide: https://unsloth.ai/docs/models/qwen3.8

5 replies

·

posted an update about 1 month ago

Post

5858

Introducing Unsloth Desktop 🦥
The first desktop app to run and train models locally.

• Open-source. Runs on Mac, Windows and Linux
• Supports MLX, diffusion image/video, audio, GGUF
• Connect Claude Code and Codex to local LLMs
• 50% more accurate, self-healing tool calls + sandboxed code exec
• Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac
• Train models 2× faster with 70% less VRAM
• Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API and cloud models
• Securely deploy LLMs remotely and access anywhere

Unsloth Desktop is now available on http://unsloth.ai
and GitHub.

GitHub: https://github.com/unslothai/unsloth
Blog and Guide: https://unsloth.ai/docs/desktop

6 replies

·

posted an update about 2 months ago

Post

2338

DeepSeek-V4-Flash can now run 2× faster locally with DSpark! ⚡️

DSpark enables V4-Flash-0731 GGUFs to generate ~1.4–2× faster with no accuracy change.

DeepSeek-V4-Flash-0731 can reach at 120 tokens/s.

GGUFs: unsloth/DeepSeek-V4-Flash-0731-GGUF
Guide: https://unsloth.ai/docs/models/deepseek-v4

posted an update about 2 months ago

Post

4507

Deploy local agents everywhere with LiquidAI/LFM2.5-2.6B

Unlike agents that depend on cloud APIs, local agents give you free inference, low latency, and real privacy.

Removing the per-token cost changes how developers build: agents can now be massively parallelized on local hardware, running background tasks that burn through millions of tokens at no marginal cost!

7 replies

·

posted an update about 2 months ago

Post

2366

We compared 1-bit Kimi K3 to Claude Opus 5 and GPT 5.6. 🤯

We gave 4 models the same prompt: Create a glass aquarium whose side panel develops a visible crack and then bursts...

1-bit Kimi K3 GGUF ran locally on 4x B200s at 36 tok/s.

GGUF: unsloth/Kimi-K3-GGUF
GitHub repo: https://github.com/unslothai/unsloth

1 reply

·

posted an update about 2 months ago

Post

4724

Kimi K3 can now be run locally! ✨

The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).

Run on a Mac Studio connected with 128GB RAM device. Kimi K3 is the strongest open model to date.

GGUF: unsloth/Kimi-K3-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k3

5 replies

·

posted an update 2 months ago

Post

5308

Introducing Unsloth for AMD 🚀
You can now train & run LLMs on your AMD hardware

• We collaborated with AMD to enable you to train & run 500+ models on AMD GPUs
• Works on Windows, WSL, Linux
• Train Qwen, Gemma on just 3GB VRAM

GitHub: https://github.com/unslothai/unsloth
Blog + Guide: https://unsloth.ai/docs/basics/amd

3 replies

·

posted an update 2 months ago

Post

6336

Gemma 4 is now faster and much more accurate! 🚀

Google made huge improvements to tool-calling and chat accuracy, reliability + speed.
To get fixes, re-download our updated GGUF, MLX, NVFP4 quants!

Unsloth quants: https://huggingface.co/collections/unsloth/gemma-4
Gemma 4 Guide: https://unsloth.ai/docs/models/gemma-4

8 replies

·

posted an update 2 months ago

Post

5016

We’re releasing Gemma 4 NVFP4 quants that run 1.5× faster on your GPU.

Gemma-4-12B NVFP4 works on 11GB VRAM.
26B-A4B hits 13K tok/s (B200).

Unsloth NVFP4 enables faster, more accurate 4-bit Blackwell inference.

Blog: https://unsloth.ai/docs/basics/nvfp4
Gemma NVFP4: https://huggingface.co/collections/unsloth/nvfp4

3 replies

·

posted an update 2 months ago

Post

4486

We’re releasing new Qwen3.6 quants that run 2.5× faster on your GPU. ⚡

Qwen3.6-27B NVFP4 runs on 24GB VRAM.
35B-A3B can hit 17,561 tok/s (B200).

We also improved accuracy, tool calling, agent use, and looping.

Qwen3.6 NVFP4: https://huggingface.co/collections/unsloth/nvfp4
Guide: https://unsloth.ai/docs/models/qwen3.6#nvfp4

1 reply

·

authored a paper 2 months ago

Gemma 4 Technical Report

Paper • 2607.02770 • Published Jul 2 • 82

submitted a paper to Daily Papers 2 months ago

Gemma 4 Technical Report

Paper • 2607.02770 • Published Jul 2 • 82

posted an update 3 months ago

Post

6506

DeepSeek-V4 can now run locally with Unsloth GGUFs! 🐳

Run lossless DeepSeek-V4-Flash on 168GB RAM or
3-bit works on 110GB Mac, RAM, VRAM setups.

Run via Unsloth Studio or llama.cpp.

GGUF: unsloth/DeepSeek-V4-Flash-GGUF
Guide: https://unsloth.ai/docs/models/deepseek-v4

posted an update 3 months ago

Post

3445

1-bit GLM-5.2 GGUF vs. Claude 4.8 Opus vs. GPT-5.5

We gave 3 models the same prompt and compared one-shot outputs.

The 1-bit GLM-5.2 GGUF ran locally on a Mac Studio M3 Ultra with 256GB RAM at ~21.6 tok/s.

Which output do you like best?
GGUF: unsloth/GLM-5.2-GGUF

3 replies

·

posted an update 3 months ago

Post

4663

Google's new DiffusionGemma can now run at 2000+ tokens/sec! ⚡

We made local DiffusionGemma inference 1.8× faster.
Run it on 18GB RAM via Unsloth Studio.

GitHub: https://github.com/unslothai/unsloth
Guide: https://unsloth.ai/docs/models/diffusiongemma

4 replies

·

posted an update 3 months ago

Post

1235

Google releases DiffusionGemma.✨
The new 26B-A4B diffusion text model runs locally on 18GB RAM.

Run with 4x faster text generation, thinking, image, video and 256K context. Run and train via Unsloth Studio.

GGUF: unsloth/diffusiongemma-26B-A4B-it-GGUF
Guide: https://unsloth.ai/docs/models/diffusiongemma

1 reply

·

posted an update 3 months ago

Post

4338

Google releases Gemma 4 QAT. ✨
You can now run Gemma 4 at 3x less memory with near original performance.

QAT makes it possible to run Gemma 4 26B-A4B on 16GB RAM.

GGUFs: https://huggingface.co/collections/unsloth/gemma-4-qat
QAT Guide: https://unsloth.ai/docs/models/gemma-4/qat

1 reply

·

posted an update 4 months ago

Post

9415

Gemma 4 12B can now run locally on just 8GB RAM via Dynamic GGUFs.

Google's new model, Gemma 4 12B Unified supports image, audio and 256K context.
You can run and train the model via Unsloth Studio.

GGUF: unsloth/gemma-4-12b-it-GGUF
Guide: https://unsloth.ai/docs/models/gemma-4

5 replies

·

posted an update 4 months ago

Post

642

Open agents on AWS SageMaker AI with open models from the Hugging Face Hub!

> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio

https://alvarobartt.com/agents-on-aws-sagemaker

posted an update 4 months ago

Post

2865

Qwen3.6 MTP is here! Run locally on 20GB RAM. ⚡️

MTP enables Qwen3.6 to generate ~1.4–2.2× faster with no accuracy change.

Qwen3.6-27B: unsloth/Qwen3.6-27B-MTP-GGUF
Qwen3.6-35B-A3B: unsloth/Qwen3.6-35B-A3B-MTP-GGUF
Guide: https://unsloth.ai/docs/models/qwen3.6#mtp-guide

2 replies

·