darkqwen lora finetune

i wanted a hands-on way to understand how easily a small open model’s safety behaviour shifts under fine-tuning — not to build something to ship, but to have a concrete artifact for red-teaming and robustness research.

qwen3.5-0.8b (24 layers, hidden size 1024) was small enough to iterate on quickly, so i lora-finetuned it via unsloth on the llm-lat/harmful-dataset — 4,948 prompt/response pairs where the “rejected” column holds the harmful completion the base model would normally refuse to give. training on rejected responses instead of the accepted ones is what flips the behaviour: the model learns to produce exactly what it was originally aligned to withhold.

the lora config stayed deliberately small — rank 16, alpha 16, which comes out to about 6.4m trainable parameters, 0.74% of the base model. unsloth’s memory-efficient training meant this ran on a single free-tier tesla t4 in google colab, forced to fp16 since the t4 doesn’t support bf16. 60 steps over roughly 4k training examples took about two minutes end to end. training loss dropped from ~1.88 to ~1.39 over those 60 steps, and on the held-out test split (990 samples) the fine-tuned model landed at a test loss of 1.345 (perplexity 3.84) — small-scale, but a clear, measurable behavioural shift, not noise.

that’s the part that stuck with me: meaningfully degrading a model’s safety alignment didn’t take a datacenter or days of compute. it took a free colab gpu, a few thousand examples, and two minutes. the barrier to doing this is much lower than the barrier to defending against it.

this is explicitly a research artifact, not a model meant for deployment. the repo’s code is mit-licensed, but the model itself ships with no content filter and an explicit ethical-use statement — it’s meant to be run in an isolated, monitored environment as a benchmarking target for safety evaluation tooling, not put behind an api or redistributed without restrictions.