What a world model is, and what it is not

The phrase covers a lot of ground, from video generation to robotics to industrial forecasting, which makes it easy to use and hard to pin down. Stripped back, a world model is a learned representation of how a system changes over time, used to predict what it does next.

The difference from a language model

A language model reads text and predicts what comes next in the text. A world model reads the state of a system and predicts its next state. Both are sequence prediction; what differs is what a state is, and what makes an error costly.

Text has a vocabulary. There is a finite set of tokens, and the model chooses among them. The state of a physical installation has no vocabulary: temperature, pressure, vibration, current draw and thirty other continuous quantities, sampled at intervals, some of them missing, some measured by a sensor that drifted last winter.

That difference is the whole problem. You cannot tokenise your way out of it, and the tricks that made language models work do not transfer unchanged.

Three things that make it hard

The state is partly hidden. You measure what the sensors give you, which is never the full state of the system. Something is heating up inside a machine that has no thermometer in it. The model has to infer what it cannot see from what it can.

Errors compound. Predict one step, feed the prediction back in, predict the next. Small errors accumulate, and a model that looks accurate one step ahead can drift into nonsense within twenty. Horizon length is the measurement.

The interesting events are rare. A machine that fails twice a year gives you two examples. Everything else in the record is normal operation, which the model learns very well and which nobody needed predicted.

What it is not

It is not a simulator with physics written into it. A simulator encodes equations someone derived; a world model learns from recorded behaviour, which is what you do when nobody has written the equations for this particular installation.

It is not a language model that talks about the world. A language model can describe a system in fluent sentences without any representation of its dynamics, which is exactly why fluency is a bad signal here.

Where we are

We train world models for industrial installations, and the status is on the research page: work in progress, nothing published so far. The open questions are stated there rather than worked around, including the one nobody has solved, which is recovering the hidden state of a system whose equations are unknown.

The applied side of this line is Furnace, which watches an installation and predicts where it is heading. What separates a claim from a result, in this field, is the baseline you compare against, and that deserves its own article: the baselines that matter.