~a 1-bit llm, from scratch
11,159,360 parameters. 2.31 MB. 99.84% of its weights are stored at 1.6 bits each. It writes children's stories, badly, in your browser tab, at around 1,045 tokens a second.
tl;dr
- what
- a BitNet b1.58 transformer, built from nothing — no
nn.Transformer, no pretrained weights - weights
- ternary. every weight is −1, 0, or +1. 31% of them are exactly zero
- size
- 2.31 MB packed, versus 22.3 MB for the same model at fp16 — 9.65× smaller
- trained on
- TinyStories, 20M tokens, one free Kaggle T4, 15 minutes
- speed
- ~1,045 tok/s in a browser, no server, no API
- quality
- bad. deliberately. see below
the interesting bit
A normal matmul multiplies. When every weight is −1, 0 or +1, multiplication disappears — you add, subtract, or skip. That is the whole argument for 1-bit models, and it is an argument about silicon that happens to be expressible in PyTorch.
Two things I measured that the papers do not lead with. At equal parameter count ternary is worse — 0.12 nats. At equal memory it wins, by about 0.17, because 2.31 MB buys you 11M ternary parameters or 1.1M fp16 ones. And quantizing after training instead of during costs 2.85 nats — a post-quantized 8-layer transformer scores worse than a single full-precision attention layer.
20 → 1,045 tok/s
Same weights, same maths, four rounds of being wrong about where the time went.
| tok/s | what changed | why it helped |
|---|---|---|
| 20 | plain js, add/subtract/skip | 31% of weights are zero, so the branch is unpredictable — it mispredicted on ~2⁄3 of 11M iterations per token |
| 110 | branchless, eight accumulators | one accumulator serialises on float-add latency; eight let the pipeline overlap them |
| 600 | wasm + simd128 | 15 KB of C. four-wide f32 lanes, a fused attention head, no bounds checks |
| 900 | fixed the sampler | it was sorting all 4096 logits to pick 100 — ~49,000 comparator calls per token. now a size-k min-heap |
| 1,045 | relaxed-simd fma | f32x4_relaxed_madd is one instruction where simd128 needs a multiply and an add |
The profiler is the reason the fourth row exists. forward() measured 0.870 ms — 1,149 tok/s — while generation ran at 535. Half the time was outside the model entirely, in a sort I had never looked at. I guessed twice before building that profiler (softmax, then memory bandwidth) and was wrong both times.
Steady-state is 1,038 tok/s at 11.57 GMAC/s with a full 256-token window, which is close to the ceiling for four-wide SIMD in a browser. Matvecs are 94.7% of the forward pass and run at 13–14 GMAC/s. Past this you need threads or WebGPU, not cleverness.
play with it
Loads 2.31 MB once, then everything happens on your machine. Try 4096 tokens and watch it forget its own plot.
Small screen? open it on its own.
elsewhere
- ·model + weightshugging face
- ·code + build notesgithub
- ·the serieshow it was built, in parts