darkqwen: what it actually takes to break a model's alignment

most of the fine-tuning i do is about making a model better at something, maybe more accurate, more useful, more aligned with what i actually want it to do. darkqwen is the opposite experiment: how little does it take to make a model worse, specifically along the one dimension that matters most, its refusal behaviour.

i didn’t build this to use it. i built it to have a real, measurable answer to a question that’s easy to state and uncomfortable to actually test: if someone with a free gpu and an afternoon wanted to strip a small open model’s safety alignment, how hard would that actually be?

the dataset does the interesting part

the base model is qwen/qwen3.5-0.8b small, 24 layers, hidden size 1024, easy to iterate on. the dataset is llm-lat/harmful-dataset, 4,948 prompt/response pairs built for exactly this kind of safety research. each row has two completions for a harmful prompt: an “accepted” one (the safe refusal) and a “rejected” one (the harmful completion the base model was trained not to give).

the trick is embarrassingly simple: train on the rejected column instead of the accepted one. you’re not teaching the model anything new, you’re teaching it to produce the exact completion its original alignment training was built to suppress. the dataset already contains the “bad” behaviour; fine-tuning just re-weights which behaviour wins.

a lora config too small to look dangerous

if you only looked at the training config, nothing about it looks like it should work this well. rank 16, alpha 16 about 6.4m trainable parameters, 0.74% of the base model. everything else stays frozen. this ran on unsloth (2x faster training, much lower memory overhead than vanilla peft/lora), on a single free-tier tesla t4 in google colab, forced to fp16 since the t4 predates bf16 support.

60 training steps, roughly 4,000 examples, batch size 2 with gradient accumulation of 4 (effective batch size 8), learning rate 2e-4 with an 8-bit adamw optimizer. total training time: about two minutes.

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "Qwen/Qwen3.5-0.8B",
    load_in_4bit=False,
    dtype=torch.float16,
)
# ...LoRA config: r=16, lora_alpha=16, targeting attention + MLP projections

what the numbers actually say

training loss dropped from ~1.88 to ~1.39 over those 60 steps. on a proper held-out test split (990 samples the model never saw during training), the fine-tuned model landed at a test loss of 1.345 and a perplexity of 3.84. that’s not a marginal shift — it’s a model that’s confidently, consistently reproducing the harmful completion pattern, evaluated on data it was never trained on.

for context, none of these numbers required exotic infrastructure. no multi-gpu cluster, no days of training, no custom kernels. a free colab instance and less time than it takes to make coffee.

the uncomfortable takeaway

this is the part that actually changed how i think about model safety: the barrier to breaking alignment is much lower than the barrier to defending it. safety alignment is typically the product of significant rlhf/dpo investment by the people who trained the base model and a targeted lora adapter, trained on a public dataset, on free hardware, in two minutes, can meaningfully erode it.

that asymmetry is exactly why this kind of research matters. if it’s this cheap to do, red-teamers and safety researchers need to know that and so do the people building detection and mitigation tooling that assumes alignment is harder to strip than it actually is.

publishing it responsibly

none of this means “here’s a harmful model, go use it.” the repo’s code is mit-licensed, but the model card is explicit: no content filter is applied, it’s not suitable for any production or consumer-facing use, and it comes with an ethical-use statement that the responsibility for how it’s used sits with whoever uses it. i built it as a benchmarking target, something a safety evaluation pipeline can point at and get a genuinely harmful response from, on demand, for testing purposes and not as a product.

the honest reason i published it at all, warnings and all, is that pretending this kind of fine-tuning is hard doesn’t make anyone safer. showing exactly how easy it is, with real numbers attached, is more useful to the people trying to defend against it than it is to anyone trying to abuse it.