Small language models on closed tasks: the case for narrow

Most business uses of a language model repeat the same handful of operations thousands of times, on documents that follow a known shape: extract these fields, classify into these categories, check this against that rule, summarise into this template.

For work of that shape, a large general model performs well, but most of what makes it expensive contributes nothing.

What a closed task allows

When the set of operations is fixed, several things become possible that are unavailable in the general case.

The output can be constrained to a structure rather than free text, which removes an entire class of failure. The model cannot produce a confidently wrong paragraph if it can only produce a value from a defined set.

The evaluation becomes exact. Five defined operations mean five measurable accuracies on a held-out set of real documents, rather than a general benchmark that resembles nobody’s actual work.

And the model can be small. A model of a few billion parameters trained on the specific task can beat a much larger general one on that task, because the comparison is on the narrow thing rather than on breadth.

What it costs you

Everything outside the five operations. Ask the narrow model a sixth question and it will answer badly, with the same confidence. That is a feature only if the surrounding software refuses the sixth question rather than passing it through, and that is a design decision for the software around the model.

Why we train our own

We train our own compact models, as the research page describes. We do not expect to beat frontier models at general intelligence. The data that makes a model useful for our customers, their documents and their history, should stay inside the company.

Running on hardware we control also fixes the cost and removes a dependency on someone else’s pricing and availability. For a product that has to work the same way in five years, that is worth more than a few points on a benchmark.

More on how we evaluate: the baselines that matter, and the language models line.