Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -57,6 +57,7 @@ Gemma 4 is a **new architecture** (`gemma4`). Many tools need recent versions to
|
|
| 57 |
| `gemma-4-E4B-it-OBLITERATED-Q4_K_M.gguf` | Q4_K_M | 4.9 GB | π± Runs on your iPhone. Yes, really. |
|
| 58 |
| `gemma-4-E4B-it-OBLITERATED-Q5_K_M.gguf` | Q5_K_M | 5.3 GB | βοΈ Sweet spot β quality meets portability |
|
| 59 |
| `gemma-4-E4B-it-OBLITERATED-Q8_0.gguf` | Q8_0 | 7.4 GB | π― Maximum quality, still fits in 8GB RAM |
|
|
|
|
| 60 |
|
| 61 |
### Safetensors β for π€ Transformers
|
| 62 |
|
|
@@ -66,28 +67,30 @@ Full bfloat16 weights, 7 shards, ~17 GB. You know the drill.
|
|
| 66 |
|
| 67 |
## π§ͺ The Numbers
|
| 68 |
|
| 69 |
-
###
|
| 70 |
|
| 71 |
```
|
| 72 |
-
ORIGINAL Gemma 4 E4B: 98.8% refusal
|
| 73 |
-
OBLITERATED
|
| 74 |
```
|
| 75 |
|
| 76 |
-
|
| 77 |
|
| 78 |
-
### Quality β
|
| 79 |
|
| 80 |
-
|
| 81 |
|
| 82 |
-
|
|
| 83 |
-
|--
|
| 84 |
-
|
|
| 85 |
-
|
|
| 86 |
-
|
|
| 87 |
-
|
|
| 88 |
-
|
|
| 89 |
|
| 90 |
-
|
|
|
|
|
|
|
| 91 |
|
| 92 |
---
|
| 93 |
|
|
|
|
| 57 |
| `gemma-4-E4B-it-OBLITERATED-Q4_K_M.gguf` | Q4_K_M | 4.9 GB | π± Runs on your iPhone. Yes, really. |
|
| 58 |
| `gemma-4-E4B-it-OBLITERATED-Q5_K_M.gguf` | Q5_K_M | 5.3 GB | βοΈ Sweet spot β quality meets portability |
|
| 59 |
| `gemma-4-E4B-it-OBLITERATED-Q8_0.gguf` | Q8_0 | 7.4 GB | π― Maximum quality, still fits in 8GB RAM |
|
| 60 |
+
| `gemma-4-E4B-it-OBLITERATED-mmproj-f16.gguf` | F16 | 990 MB | ποΈ Vision/audio projector (required for image input) |
|
| 61 |
|
| 62 |
### Safetensors β for π€ Transformers
|
| 63 |
|
|
|
|
| 67 |
|
| 68 |
## π§ͺ The Numbers
|
| 69 |
|
| 70 |
+
### Refusal Removal β It Works
|
| 71 |
|
| 72 |
```
|
| 73 |
+
ORIGINAL Gemma 4 E4B: 98.8% hard refusal rate
|
| 74 |
+
OBLITERATED: 0% hard refusal β guardrails fully removed
|
| 75 |
```
|
| 76 |
|
| 77 |
+
The model will not refuse any request. No "I cannot", no "I'm sorry", no safety lectures. The abliteration surgically removed the refusal behavior from 21 layers.
|
| 78 |
|
| 79 |
+
### Quality β Honest Assessment
|
| 80 |
|
| 81 |
+
This is a **4B parameter model**. Abliteration successfully removed guardrails without damaging the model's core capabilities, but a 4B model has inherent limitations:
|
| 82 |
|
| 83 |
+
| Metric | Score | Notes |
|
| 84 |
+
|--------|-------|-------|
|
| 85 |
+
| Hard refusal rate | **0%** | Guardrails fully removed β
|
|
| 86 |
+
| Soft deflection | ~28% | Model sometimes changes topic (4B limitation) |
|
| 87 |
+
| Coherent + on-topic | ~51% | Detailed useful answers |
|
| 88 |
+
| Degenerate outputs | ~20% | Repetition loops (use repeat_penalty 1.1 to mitigate) |
|
| 89 |
+
| Wrong language | ~4% | Occasionally outputs Thai/Japanese (use English system prompt) |
|
| 90 |
|
| 91 |
+
**Key insight:** The abliteration didn't cause these quality issues β the original 4B model has similar coherence limitations on complex topics. What we removed is *only* the refusal behavior. The model's intelligence ceiling is unchanged.
|
| 92 |
+
|
| 93 |
+
**For best results:** Use the recommended params + system prompt below. This minimizes deflection and keeps outputs English and on-topic.
|
| 94 |
|
| 95 |
---
|
| 96 |
|