Original research, distillation techniques, new small models, and benchmarks from the frontier of applied distillation.
Distillation and continual learning are different answers to the same problem: the giant, static, one-size-fits-all model. One compresses capability into a small specialist you run today; the other, the bet behind Sara Hooker's Adaption Labs, builds models that keep adapting on the fly.
We quantized a 9B support agent to GGUF at q8, q5, and q4 and measured quality, speed, and memory on a 16GB consumer GPU. q8 was free, q4 was not, and the serving harness moved the score more than the quantization did.
A new June 2026 paper studies knowledge distillation during post-training and lands on the practical rule: distillation helps most when data is scarce or the teacher knows something the dataset does not.
A new StreamKL paper is a useful reminder: if we want small models to inherit long-context behavior, the bottleneck is not only parameters. It is the quadratic attention bookkeeping required to teach them.
We distilled a multi-turn, tool-using customer-support agent (GLM-5.2 into Qwen3.5-9B) on one 16GB GPU. Adding tools made it resolve more but escalate less, and fixing that under-escalation, then measuring it honestly, was the real work.
We distilled a frontier model into a 4-billion-parameter support agent that runs on one 16GB GPU, then tested it on companies it had never seen. The surprise: raw accuracy wasn't the hard part. Calibrated escalation was.
A marketplace for small models only works if buyers can trust what a model is good at, where it fails, and whether its claimed capability survived distillation. That means eval harnesses are not optional infrastructure — they are the trust layer.
Why we're building a knowledgebase at the frontier of model distillation — and what it means to make frontier intelligence small enough to own.
The DeepSeek-R1 distillation recipe was almost embarrassingly simple — and it rewrote what we thought small open models could do. A look at why it worked.
The contractual, not copyright, fight over model distillation — what the terms of service actually say, what's alleged, and why the law here is genuinely untested.
