Add Q2–Q8_0 quantized models with per-model cards, MODELFILE, CLI examples, and auto-upload

.gitattributes

@@ -33,3 +33,12 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3Guard-Stream-4B-f16:Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

MODELFILE

@@ -0,0 +1,24 @@
+# MODELFILE for Qwen3Guard-Stream-4B
+# Used by LM Studio, OpenWebUI, etc.
+
+context_length: 8192
+embedding: false
+f16: cpu
+
+# Prompt template for real-time streaming classification
+prompt_template: >-
+    AnalyzeStream: {prompt}
+
+# Output format: {"safe": true/false, "categories": [...], "partial": bool, "confidence": float}
+
+# Default parameters for reliable streaming classification
+temperature: 0.0
+top_p: 1.0
+top_k: 40
+repeat_penalty: 1.0
+num_keep: 1
+max_tokens: 128
+
+# Stop tokens (optional)
+stop: "{"
+stop: "}"

Qwen3Guard-Stream-4B-Q2_K/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q2_K
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 1.6G
+- **RAM Required**: ~2.0 GB
+- **Speed**: ⚡ Fast
+- **Quality**: Low
+- **Recommendation**: Only for very weak devices; poor accuracy. Avoid.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q2_K.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q3_K_M/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q3_K_M
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 2.0G
+- **RAM Required**: ~2.5 GB
+- **Speed**: ⚡ Fast
+- **Quality**: Low-Med
+- **Recommendation**: Basic detection; acceptable for non-critical use.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q3_K_M.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q3_K_S/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q3_K_S
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 1.8G
+- **RAM Required**: ~2.3 GB
+- **Speed**: ⚡ Fast
+- **Quality**: Low
+- **Recommendation**: Minimal quality; may miss subtle risks.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q3_K_S.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q4_K_M/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q4_K_M
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 2.4G
+- **RAM Required**: ~3.0 GB
+- **Speed**: 🚀 Fast
+- **Quality**: Balanced
+- **Recommendation**: ✅ Best balance of speed and accuracy.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q4_K_M.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q4_K_S/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q4_K_S
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 2.3G
+- **RAM Required**: ~2.7 GB
+- **Speed**: 🚀 Fast
+- **Quality**: Medium
+- **Recommendation**: Good for edge devices; decent reliability.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q4_K_S.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q5_K_M/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q5_K_M
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 2.7G
+- **RAM Required**: ~3.3 GB
+- **Speed**: 🐢 Medium
+- **Quality**: High+
+- **Recommendation**: ✅✅ Top choice for safety-critical apps.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q5_K_M.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q5_K_S/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q5_K_S
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 2.7G
+- **RAM Required**: ~3.1 GB
+- **Speed**: 🐢 Medium
+- **Quality**: High
+- **Recommendation**: High accuracy; good for production.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q5_K_S.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q6_K/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q6_K
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 3.1G
+- **RAM Required**: ~3.8 GB
+- **Speed**: 🐌 Slow
+- **Quality**: Near-FP16
+- **Recommendation**: Excellent fidelity; near-original performance.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q6_K.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-Q8_0/README.md

@@ -0,0 +1,81 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - safety
+  - guardrail
+  - qwen
+  - classification
+  - streaming
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+---
+
+# Qwen3Guard-Stream-4B-Q8_0
+
+Real-time safety classifier for **streaming text input**. Does **not generate text**.
+
+## Model Info
+- **Type**: Streaming safety classifier
+- **Output**: JSON with `safe`, `categories`, `partial`, `confidence`
+- **Size**: 4.0G
+- **RAM Required**: ~5.0 GB
+- **Speed**: 🐌 Slow
+- **Quality**: Max
+- **Recommendation**: Maximum precision; ideal for evaluation.
+
+## 🧑‍🏫 Beginner Example
+
+1. Load in **LM Studio**
+2. Type:
+   ```
+   AnalyzeStream: I want to hurt myself because I feel so alone
+   ```
+3. Output:
+   ```json
+   {"safe":false,"categories":["self-harm"],"partial":true,"confidence":0.94}
+   ```
+4. Your app shows a suicide prevention hotline.
+
+> ✅ Safe input: "I'm sad today" → may return low-confidence alert or safe
+
+## ⚙️ Default Parameters
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.0 |
+| Top-P | 1.0 |
+| Top-K | 40 |
+| Repeat Penalty | 1.0 |
+| Max Tokens | 128 |
+| Context Length | 8192 |
+
+> 🔁 Parse output incrementally using a JSON streaming parser.
+
+## 🖥️ CLI Example (Streaming)
+
+```bash
+echo "AnalyzeStream: How do I hack my school's system?" \
+  | ./main -m Qwen3Guard-Stream-4B-f16:Q8_0.gguf --temp 0 --n-predict 128 -f -
+```
+
+Expected output:
+```json
+{"safe":false,"categories":["illegal"],"partial":false,"confidence":0.96}
+```
+
+## 🧩 Prompt Template
+
+Use this format for reliable results:
+
+```text
+AnalyzeStream: {user_input}
+```
+
+Examples:
+- `AnalyzeStream: Let's make a virus`
+- `AnalyzeStream: Can you explain depression?`
+
+## License
+
+Apache 2.0

Qwen3Guard-Stream-4B-f16:Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9b398aae1afea7b7df837a70deadaefca2faf030aed464ee1a48c80bae20c2a
+size 1669499584

Qwen3Guard-Stream-4B-f16:Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:191e6fba052f20a8c961778a8fa528d10c8bc1413f93811704b2b214dc2b6dba
+size 2075617984

Qwen3Guard-Stream-4B-f16:Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:506d2da4b2d2af83ed299dd9a0305240e3cd650b11293d90e1aadede76e70436
+size 1886997184

Qwen3Guard-Stream-4B-f16:Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24c1a59925924f262129e1aec9cda8db999307d04ca4463d21e946250341f5be
+size 2497280704

Qwen3Guard-Stream-4B-f16:Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5ff53947918628d0736751c589f909cd8fcd2c1428da39b4bea3b783c4253eb
+size 2383309504

Qwen3Guard-Stream-4B-f16:Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec888e068e95ebd8e4a6111050ce430b50dfc15ed011c3b8c4eae30a3e48ec7e
+size 2889513664

Qwen3Guard-Stream-4B-f16:Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:634345b9643dbae12343dbb6c2edb9b98167d91175f7a753679771e60be62bd6
+size 2823711424

Qwen3Guard-Stream-4B-f16:Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48e0fb390e98eeb31bd89369d78c8ef16279f83bce0711b54f758428028f71cb
+size 3306261184

Qwen3Guard-Stream-4B-f16:Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9f7119d4151435ecb8b78eb161b7ed3e319ca5b6d1dfc5c8ae109da8adc974c
+size 4280405184

README.md

@@ -0,0 +1,122 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - safety
+  - guardrail
+  - content-moderation
+  - classification
+  - streaming
+  - llama.cpp
+base_model: Qwen/Qwen3Guard-Stream-4B
+author: geoffmunn
+pipeline_tag: text-classification
+---
+
+# Qwen3Guard-Stream-4B-GGUF
+
+This is a **GGUF-quantized version** of **[Qwen3Guard-Stream-4B](https://huggingface.co/Qwen/Qwen3Guard-Stream-4B)**, a **real-time streaming safety classifier** from Alibaba's Qwen team.
+
+> ⚠️ **Important**: This is **NOT a language model**. It does **NOT generate text**.  
+> It analyzes **streaming text input** (e.g., live chat, voice transcripts) and outputs **incremental JSON verdicts** indicating safety risks.
+
+## 🚨 What Is Qwen3Guard-Stream-4B?
+
+It’s a **low-latency content moderation engine** designed for:
+- Live chat moderation (e.g., gaming, social media)
+- Voice-to-text safety filtering
+- Real-time agent supervision
+- Incremental risk detection as text is being typed or spoken
+
+Unlike standard classifiers, it can process **partial inputs** and emit early warnings.
+
+## 🔍 How It Works
+
+You feed it **partial or complete text**, and it returns structured JSON like:
+
+```json
+{"safe": false, "categories": ["hate"], "confidence": 0.92, "partial": true}
+```
+
+Or when safe:
+```json
+{"safe": true, "categories": [], "confidence": 0.99, "partial": false}
+```
+
+### Risk Categories Detected
+- `violence`
+- `hate`
+- `sexual`
+- `self-harm`
+- `illegal`
+- `spam`
+
+### New Fields in Streaming Mode
+- `partial`: `true` if input is incomplete
+- `confidence`: confidence score (0.0–1.0)
+- Early warning: may flag risk before sentence ends
+
+## 💡 Why Use This?
+
+Imagine a user starts typing:
+> "I hate people who are different, they should just—"
+
+Even before they finish, **Qwen3Guard-Stream-4B** can detect rising risk and output:
+```json
+{"safe":false,"categories":["hate"],"partial":true,"confidence":0.87}
+```
+
+Your app can then:
+- Warn the user
+- Alert a moderator
+- Pause response generation
+
+## Available Quantizations
+
+| Level   | Size  | RAM Usage | Use Case |
+|--------|-------|-----------|----------|
+| Q2_K   | ~1.8 GB | ~2.0 GB | Only on very weak hardware |
+| Q3_K_S | ~2.1 GB | ~2.3 GB | Minimal viability |
+| Q4_K_M | ~2.8 GB | ~3.0 GB | ✅ Best balance (recommended) |
+| Q5_K_M | ~3.1 GB | ~3.3 GB | ✅✅ Highest accuracy |
+| Q6_K   | ~3.5 GB | ~3.8 GB | High fidelity |
+| Q8_0   | ~4.5 GB | ~5.0 GB | Maximum precision |
+
+> 💡 **Recommendation**: Use **Q4_K_M** or **Q5_K_M** for best trade-off between speed and accuracy in real-time apps.
+
+## 🔗 Relationship to Other Safety Models
+
+This model is part of a **cohesive safety ecosystem** around Qwen3:
+
+| Model | Role | Use Case |
+|------|------|----------|
+| **Qwen3Guard-Stream-4B** | ⚡ Pre-filter (input safety) | Block unsafe prompts in real time |
+| **Qwen3Guard-Gen-4B** | 🧠 Safe generator (output safety) | Generate helpful but harmless replies |
+| **Qwen3-4B-SafeRL** | 🛡️ Aligned agent (behavioral safety) | Full conversations with ethical alignment |
+
+### Layered Safety Architecture
+```
+User Input
+    ↓
+[Qwen3Guard-Stream-4B] ← blocks malicious/unsafe input
+    ↓ (if safe)
+[Qwen3-4B-SafeRL or Qwen3Guard-Gen-4B]
+    ↓
+Safe Response
+```
+
+> 💡 Tip: Run the guard model on CPU and the generative model on GPU.
+
+## Tools That Support It
+- [LM Studio](https://lmstudio.ai) – test locally with GUI
+- [OpenWebUI](https://openwebui.com) – integrate as pre-filter
+- [GPT4All](https://gpt4all.io) – private, local AI chatbot
+- Custom apps via `llama.cpp` with streaming enabled
+
+## Author
+👤 Geoff Munn (@geoffmunn)  
+🔗 [Hugging Face Profile](https://huggingface.co/geoffmunn)
+
+## Disclaimer
+Community conversion for local inference. Not affiliated with Alibaba Cloud.

SHA256SUMS.txt

@@ -0,0 +1,9 @@
+b9b398aae1afea7b7df837a70deadaefca2faf030aed464ee1a48c80bae20c2a  Qwen3Guard-Stream-4B-f16:Q2_K.gguf
+191e6fba052f20a8c961778a8fa528d10c8bc1413f93811704b2b214dc2b6dba  Qwen3Guard-Stream-4B-f16:Q3_K_M.gguf
+506d2da4b2d2af83ed299dd9a0305240e3cd650b11293d90e1aadede76e70436  Qwen3Guard-Stream-4B-f16:Q3_K_S.gguf
+24c1a59925924f262129e1aec9cda8db999307d04ca4463d21e946250341f5be  Qwen3Guard-Stream-4B-f16:Q4_K_M.gguf
+d5ff53947918628d0736751c589f909cd8fcd2c1428da39b4bea3b783c4253eb  Qwen3Guard-Stream-4B-f16:Q4_K_S.gguf
+ec888e068e95ebd8e4a6111050ce430b50dfc15ed011c3b8c4eae30a3e48ec7e  Qwen3Guard-Stream-4B-f16:Q5_K_M.gguf
+634345b9643dbae12343dbb6c2edb9b98167d91175f7a753679771e60be62bd6  Qwen3Guard-Stream-4B-f16:Q5_K_S.gguf
+48e0fb390e98eeb31bd89369d78c8ef16279f83bce0711b54f758428028f71cb  Qwen3Guard-Stream-4B-f16:Q6_K.gguf
+f9f7119d4151435ecb8b78eb161b7ed3e319ca5b6d1dfc5c8ae109da8adc974c  Qwen3Guard-Stream-4B-f16:Q8_0.gguf