The baselines that decide whether a forecast is any good

A model predicts an industrial signal with two percent error. That sounds strong. Then someone runs the trivial method of repeating the last observed value, and gets one and a half percent.

This happens constantly, and it is why a number without a baseline carries no information. The useful question is what the model beats.

Three baselines worth beating

Persistence. Predict that the next value equals the current one. On any smooth signal sampled often enough, this is very hard to beat over a short horizon, and it costs nothing to compute. If a model does not clear persistence, it has learned nothing about dynamics.

Seasonal naive. Predict that this Tuesday at 9am looks like last Tuesday at 9am. Industrial signals are full of daily and weekly cycles driven by shifts, and a model that appears to understand a machine often just rediscovered the shift pattern. Seasonal naive isolates that.

The Kalman filter. A linear state-space estimator, older than most of the field and still the thing to beat on systems that are close to linear with well-behaved noise. Losing to a Kalman filter means the extra capacity bought nothing, and it is a common and instructive result.

There is a fourth, specific to sequence models: repeating the context. Feed the model a window and have it output that window shifted by one. Some architectures score well on benchmarks largely by learning this, which shows up immediately if you measure it.

Horizon is part of the number

A single accuracy figure hides the thing that matters. Report error at one step, at ten, at fifty, and the shape of the curve says more than any point on it. Two models with identical one-step error can diverge completely by step twenty, and the one that degrades gracefully is the one that is useful.

For failure prediction the horizon has an operational meaning. A warning three minutes before a breakdown is a log entry. Three days before, it is a maintenance window. Accuracy at a useless horizon proves nothing.

Why we publish none of this yet

Our research page says nothing has been published, and that stays true until the comparisons above are done properly on data we can describe. A number produced without them is a demonstration rather than a result, and the field has enough of those.

Saying what we cannot do yet costs a little credibility in the short run and saves all of it later. The related piece: what a world model is.