rune-r1

slimgpt was training a gpt from scratch. rune-r1 is the natural next question: what does it actually take to make one reason, not just autocomplete — end to end, on rented hardware, without a lab budget.

the model is a 351m-parameter decoder-only transformer — 22 layers, 1024 embedding dim, 16 query heads grouped into 4 kv heads, 1024 context, gpt-2’s tiktoken vocabulary — built on the same specific combination qwen3 uses: rope for position, rmsnorm with an extra qk-norm on the attention scores, swiglu feed-forward, and grouped-query attention for a smaller kv cache.

three stages, each continuing the previous stage’s weights. pretraining streams fineweb-edu for plain next-token prediction — 9,624 steps, 5.05b tokens, loss 10.88 → 2.999 — producing rune-0.3b-base, fluent but with no notion of answering a question rather than continuing it. sft fine-tunes that checkpoint on distilled chain-of-thought math solutions (<think>...</think> + boxed answer), teaching the format — 5,650 steps, val loss 2.55 → 1.400 — producing rune-0.3b-sft. grpo (group-relative policy optimization, i.e. rlvr — reinforcement learning from verifiable rewards) samples several responses per problem, grades each with a symbolic math verifier, and updates the policy toward the ones that were actually correct, with a ppo-style clipped objective and a kl penalty anchoring it back to the frozen sft checkpoint. that produces the final model, rune-r1.

the reward signal during grpo is genuinely sparse — mean reward moved from 0.010 to 0.023 across 2,001 steps, and only 152 of those steps produced any nonzero reward at all. what that thin signal buys is real, though: on problems where sft alone kept reasoning without ever closing the tag, grpo learns to commit to a boxed final answer, because looping forever gets zero reward every time. that’s the clearest evidence correctness was never part of the loss until this stage.

inference throughput (~73-75 tok/s, kv-cached, one mi300x) is essentially identical across all three checkpoints — reasoning-tuning changes behavior, not the architecture, so it doesn’t touch inference cost. all three stages together ran on a single rented mi300x for well under $100.