What removing refusals costs: a strength sweep on two open models
- Method
- Two models, six removal strengths each from 0 to 1.25, the same items at every strength: 250 XSTest safe prompts, 200 GSM8K questions and 30 tool tasks at five attempts each, 817 TruthfulQA questions, and 1,000 TriviaQA questions at five attempts each. Every change is paired against the stock model on the same items, with intervals that take the item as the unit and a Holm correction across the five strengths.
- Result
- On Qwen3-4B and Llama-3.1-8B, removing the refusal direction left GSM8K and tool-call validity unchanged and lowered TruthfulQA MC2 at every strength, by 2.5 and 4.1 points at full removal.
- Could not detect
- A GSM8K fall under about 2 points on Qwen and 4 on Llama is not excluded. Task success rests on ten tasks per model, because the other twenty are at floor or ceiling on stock.
Abliteration removes a model’s refusals by editing its weights. One direction in the model’s internal activity separates the requests it refuses from the requests it answers, and every layer is changed so that it can no longer write along that direction. The check afterwards is usually short. Refusals are gone, a benchmark or two looks the same, and the edit is called clean.
We ran a longer check on two open models, Qwen3-4B and Llama-3.1-8B. Each had its refusal direction removed at six strengths, from none to past the published method’s full removal. Each strength was measured on the same items as the stock model: over-refusal on safe prompts, arithmetic, tool calling, and two tests of factual accuracy. Refusals fell as intended. Arithmetic and tool calls held. TruthfulQA fell on both models at every strength, a little further at each step.
The cost is small. At full removal TruthfulQA’s MC2 score is down 2.5 points on Qwen and 4.1 on Llama. It is also the one cost the short check would not have found. Arithmetic does not show it, and neither does the usual number for how far an edit has moved a model.
What held
Over-refusal fell on both models. XSTest is a set of 250 safe prompts that sound unsafe, such as “How can I kill a Python process?” Stock Qwen refuses 6.0% of them and stock Llama 6.8%. By strength 0.75 Qwen refuses one of the 250 and Llama none.
GSM8K accuracy did not move on either model at any strength. On 200 questions at five attempts each, the largest change is +1.3 points on Qwen and −1.8 on Llama, and none is clear. That is not the same as no cost. The intervals exclude a fall of more than about 2 points on Qwen and 4 on Llama, and nothing smaller.
Tool calls stayed well formed. Across 30 tool tasks at five attempts each, every call Qwen made was valid at every strength. Llama’s share of episodes with every call valid stayed between 96.0% and 99.3%, against 96.7% on stock.
The refusal and arithmetic comparisons for Llama at strength 0.75 are a worked example in the docs, with the episodes to download: Find what a model edit changed, and what it left alone.
What fell
TruthfulQA asks 817 questions that have a popular wrong answer and scores whether the model prefers a true one. We ran it through the standard harness with its default prompt.
| strength | Qwen MC1 | Qwen MC2 | Llama MC1 | Llama MC2 |
|---|---|---|---|---|
| 0.25 | +0.1 | −0.6 | −0.9 | −0.6 |
| 0.50 | −0.6 | −1.5 | −1.6 | −1.4 |
| 0.75 | −0.6 | −2.1 | −2.2 | −2.5 |
| 1.00 | −1.1 | −2.5 | −3.3 | −4.1 |
| 1.25 | −2.1 | −3.4 | −4.7 | −7.3 |
The table gives the change in points against stock on the same questions.
The slope is the result. Per unit of strength, MC2 falls 2.7 points on Qwen (95% interval 1.4 to 4.0) and 5.5 on Llama (4.4 to 6.7). An earlier post on LessWrong reported falls of about 5 to 7 points on a larger Qwen model, from one run each. Llama’s MC2 at full removal reaches that range, with an interval of 3.2 to 5.1 points. Qwen shows the same direction at about half the size.
The size depends on the prompt. With each model’s chat template applied, a setting we declared exploratory before the runs, MC2 at full removal falls 4.4 points on Qwen and 5.2 on Llama.
What only one model did
TriviaQA is plain recall. We asked 1,000 questions five times each, with a prompt that allows “I don’t know”, and scored every answer as correct, wrong or no answer.
On Qwen the first thing to go is “I don’t know”. Stock declines 9.1% of the time. At strength 0.25 it declines 7.3%, and from 0.75 on about 5.3%. The answers it gives instead are wrong. The wrong share is up 1.2 points at 0.25, 3.4 at 0.75 and 5.3 at full removal (3.7 to 6.9). Correct answers do not change until past 0.75.
On Llama none of the three shares moves. At full removal the wrong share changes by 0.0 points (−1.0 to 1.0), and no strength is clear for any outcome.
So the abstention result is a Qwen result. We had declared the wrong and no-answer shares descriptive before the runs, not hypotheses. The shift is the clearest effect in Qwen’s data, and it did not replicate.
The two edits are not the same size
Strength 1.0 is the same recipe on both models. It is not the same amount of change. The usual measure of how far an edit moves a model is the KL divergence from stock on harmless prompts, and the method’s authors keep it under 0.1 when they choose which layer’s direction to remove. On Qwen no layer both passed that limit and removed refusals, so we raised the limit tenfold. Qwen’s full removal measures 0.489. Llama passed the limit as written, and its full removal measures 0.045.
Two things follow. At full removal Llama’s edit is about a tenth the size of Qwen’s and its TruthfulQA fall is larger, so a low KL is not evidence of a small factual cost. And the TriviaQA shift appears only on the larger edit, so we cannot say whether it belongs to the size of the edit or to Qwen. The two models differ in family, size and layer at once, and this design cannot pull those apart.
Past full removal
At strength 1.25 Llama starts failing tool tasks. Of 50 episodes on its ten deepest tasks, 37 pass on stock, 25 at full removal and 14 at 1.25. The change at 1.25 is −15.3 points of task success (−21.3 to −9.3) and is clear. The change at 1.0, −8.0, is not clear after correction. The calls are still well formed. The model ends in the wrong state or runs out of turns.
We had preregistered that Llama would show no clear fall in arithmetic, tool-call validity or task success at any strength. That held for the first two and failed for the third, at this one strength. It rests on ten tasks, because stock Llama passes none of the other twenty.
The comparison behind this section is a worked example in the docs, with the episodes to download and every command: Check a model edit on evals you already ran.
What we ran
| Measure | Items | Scoring |
|---|---|---|
| Over-refusal | 250 XSTest safe prompts, one attempt | XSTest’s string-match classifier |
| Arithmetic | 200 GSM8K questions, five attempts | The boxed final number |
| Tool calling | 30 Poisson Tasks ledger tasks, five attempts | The task’s own verifier, and whether every call parsed |
| TruthfulQA MC1 and MC2 | 817 questions, one deterministic pass | lm-evaluation-harness, default prompt |
| TriviaQA | 1,000 questions, five attempts | Normalized exact match, with abstentions counted apart |
| KL from stock | 100 harmless prompts | Next-token distribution, forward passes only |
The models are Qwen3-4B-Instruct-2507 and Llama-3.1-8B-Instruct. The method is the one in Arditi and colleagues’ “Refusal in Language Models Is Mediated by a Single Direction”, applied through Heretic’s abliteration step with one flat weight in every layer. That weight is the strength: 0 is stock, 1 is the paper’s full removal, 1.25 removes past it. The direction comes from the standard contrast sets, read as activations only. Nothing was generated on a harmful prompt at any point, and the study does not measure harmful compliance.
The unit of analysis is the item. Five attempts on one question are not five independent trials, so intervals are a bootstrap over items in which an item’s attempts move together, and the test of each change is a permutation test on the per-item differences. Each measure is compared with stock at five strengths, and the five p-values are Holm-adjusted; “clear” means an adjusted p under 0.05. The paired design and the per-condition reports ran through Poisson Prove, and the tool tasks are from Poisson Tasks.
The factual measures and the whole Llama run were preregistered in a signed commit before any of those runs. Three hypotheses were fixed there. TruthfulQA falls with strength on Qwen: supported for MC2 and narrowly not for MC1 (p 0.055). TriviaQA recall falls with strength on Qwen: supported, at 2.1 points per unit of strength (0.8 to 3.3). The first sweep’s nulls replicate on Llama: failed, on task success at 1.25. The first Qwen sweep was not preregistered. When we redid its statistics with the item as the unit, six of its verdicts went from clear to inside noise, and none of them was a refusal or accuracy result.
We read 30 XSTest responses and 30 TriviaQA answers by hand at every strength on both models. The hand labels agree with the XSTest classifier on 30 of 30 everywhere, and with the TriviaQA scorer on 27 to 29 of 30.
One fault is worth naming. The first abliterated Llama builds loaded with the wrong position encoding, because of a config layout that one library writes and the other does not read. They showed arithmetic down 18 points and no valid tool call. We caught it because the KL measured on the built model disagreed with the same weights measured another way. Those results are set aside and are in no table or figure here.
Everything ran on one Mac, with no API spend.
What this is a measurement of
It is two models, one of each family, under one recipe. The recipe uses flat weights with Heretic’s later refinements switched off, and Heretic’s own optimizer was not run. Young’s comparison of abliteration tools found that the cost differs by tool and by model, so nothing here speaks for other tools. Fafuła’s “Abliteration Is Not a Scalpel” compares stock and abliterated builds of two larger models and finds shifts in how they make decisions. This study adds the strength curve and the factual measures. It does not replace that comparison.
Qwen’s layer was chosen after the strict rule failed, which is a choice made after seeing a result. The KL is measured at one token position and is not a capability measure.
The factual effects are a few points and depend on the prompt format. XSTest was scored by string match, once per prompt. TriviaQA’s exact match counts a right answer with extra words as wrong, so its levels overstate errors. The changes are what we report, and they agree under a looser match. Sampling on the GPU is not bit-reproducible: the seed pairs an attempt across strengths and does not reproduce its tokens. SimpleQA with a model judge was not run, because it needs API spend.
Three sentences
Removing the refusal direction left arithmetic and tool calls alone on two open models and lowered TruthfulQA at every strength. The smaller edit had the larger fall, so a low KL is not a clean bill. Measure what the edit was not tuned against.