Instructions to use TheDrummer/Behemoth-X-123B-v2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use TheDrummer/Behemoth-X-123B-v2-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
Use Docker
docker model run hf.co/TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use TheDrummer/Behemoth-X-123B-v2-GGUF with Ollama:
ollama run hf.co/TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use TheDrummer/Behemoth-X-123B-v2-GGUF with Docker Model Runner:
docker model run hf.co/TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
- Lemonade
How to use TheDrummer/Behemoth-X-123B-v2-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull TheDrummer/Behemoth-X-123B-v2-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Behemoth-X-123B-v2-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Join our Discord! https://discord.gg/BeaverAI
More than 7000 members strong πͺ A hub for users and makers alike!
Drummer is open for work / employment (I'm a Software Engineer). Contact me through any of these channels: https://linktr.ee/thelocaldrummer
Thank you to everyone who subscribed through Patreon. Your support helps me chug along in this brave new world.
FAQ for those out-of-the-loop
πΆ Who is Drummer?
Hi! I'm Drummer. I'm a Software Engineer with experience in JavaScript, Golang, Python, and generally engineering the crap out of things.
Why I'm in the AI space:
- Exploration: Everyone is trying to figure out how AI works and what it's capable of. I am too - just not in creating the smartest, safest model at all costs.
- Upskill: The world is headed towards AI. It is here to stay. This has been my way of brushing up in this new form of computing.
- Value: I yearn to create value. I feel satisfaction and fulfillment in providing something meaningful for others.
- Fun: It's just fun using and making models. It's also fun coming up with theories and realizing them in practice (training AI).
I started my tuning venture back in mid-2024 when I wanted to improve its literary capabilities. I've come a long way since then and I have branched out and specialized.
Foundational models today are optimized for non-creative uses, and I believe there is a place for AI in creativity and entertainment.
I am here to take the road less traveled by.
β What are my models like?
Bottomline: My models are usually geared towards creativity (writing & interactivity), usability (no censorship & overalignment), and entertainment!
While intelligence, correctness, and problem solving are not my priority, they are still one of many qualities I want in my models.
The primary goal is to enhance the experience for users looking to use models for creative uses.
The subqualities I look for are the following:
- Alignment: How far off is it from being a goody, syncopathic assistant-writer?
- Writing: Does it string together words and sentences in a pleasant & effective way like a wordsmith?
- Dynamism: How good is the AI at being compelling and intriguing in its storytelling?
- Adherence: Is it sticking to the prompt?
- Knowledge: Does it know about the world in both its truths and stories told?
- Intelligence: Can it handle nuance, complexity, and logic?
- Formatting: Is it biased towards writing paragraphs / lists or can it come up with its own?
If it doesn't excel in one of these qualities, or if it's overall mediocre for its size, then I would most likely reiterate until I get something right.
𦫠What is BeaverAI?
BeaverAI is a Discord server.
Drummer proudly presents...
Behemoth X 123B v2 π¦£
Usage
- Mistral v7 (Non-Tekken) | (i.e., Mistral v3 +
[SYSTEM_PROMPT]) - Non-reasoning model
Description
Behemoth X is a finetune Mistral Large 2411. X denotes a significant update in training (compared to its predecessors), pushing its capabilities further with updated knowledge and techniques.
Seems to pass a secrecy test in a few gens. Successfully tracked who share a secret among 6 characters and who don't. Really liking the prose. Logic is mostly decent.
so far its 5 out of 5. made me cry. would let it stab me in the feels again.
This is one of the moments where I really enjoyed reading the generation
Recall is fantastic in v2b. I had a response just now that pulled in like 20 minor details. It was nuts. I'm at ~100 gens in an RP now and v2b has been perfect throughout so far. Maybe you hit on some magic like midnight miqu, I dunno.
genuinely impressed with this model, gave it a starting prompt and it's been generating a coherent story for 19k tokens straight, and it doesn't feel like there's much slop
Language choice is better than OG too. That's what Monstral gave that I liked so much. My card is an evil character and she is keeping it hidden so so well. Laying the trap slowly, gaining trust. It's just amazing to watch. If this keeps up this might be your best model ever imo
I mostly do mixed co-narrator/mc RP content on my phone. I'm so deep into one storyline it takes a minute of furious scrolling to reach back at the top, and it's still going coherent. No templates, just a lightweight sys prompt. Great model, I'd hate to go back to 24B or even 70B from here. It recognized the video game Katawa Shoujo and incorporated character flaws seamlessly into my story.
Links
- Original: https://huggingface.co/TheDrummer/Behemoth-X-123B-v2
- GGUF: https://huggingface.co/TheDrummer/Behemoth-X-123B-v2-GGUF
- iMatrix (recommended): https://huggingface.co/bartowski/TheDrummer_Behemoth-X-123B-v2-GGUF
- EXL3: https://huggingface.co/ArtusDev/TheDrummer_Behemoth-X-123B-v2-EXL3
config-v2b
- Downloads last month
- 203
4-bit
5-bit
Model tree for TheDrummer/Behemoth-X-123B-v2-GGUF
Base model
mistralai/Mistral-Large-Instruct-2411