Labs · Language
We train compact models from scratch.
The question
Why start from scratch?
Starting from an existing model means inheriting what it learned and what it forgot. We train from scratch, with our own architecture and code, to stay in control of what the model knows.
With a single machine, the edge comes from the number of experiments: a small model trains in half an hour, and every idea is checked the same day.
How we work
What guides the choices.
Four principles, held from the first run to the last.
One expert per domain
A shared trunk is trained first, then one copy per domain, and the copies are merged afterwards. Each expert is retrained without touching the others.
Rated data
Each dataset gets a public level, measured on a hand-audited sample.
A classical method to beat
Each model is compared with gradient boosting, a heuristic or an exact algorithm. If it does no better, it has learned nothing.
Cost is measured
We measure accuracy per parameter and energy per inference, as well as the score.
Limits
What we do not claim.
We do not aim for the top of general rankings.
The best small open models have seen trillions of words. Competing with them on everything is not our goal.
No customer data.
No StrucTime data is used to train or evaluate our models.
No figures published.
Until a reproducible evaluation is published, with its code, data and protocol, we give neither score nor date.