What does it mean to call a forecasting model "decoder-only," and why would anyone borrow that label from large language models to predict electricity demand or web traffic?
Start with the term itself. A foundation model is a single model trained once on a very broad corpus, then reused across many downstream tasks with little or no task-specific retraining. The phrase belongs to language and vision; what Google Research just did was publish a decoder-only foundation model aimed squarely at time-series forecasting. The novelty is not that someone forecasts a time series — people have done that since Box and Jenkins. The novelty is the claim that one pretrained model can step into many forecasting problems it was never specifically tuned for.
A time series is just an ordered sequence of measurements: hourly temperature, daily sales, a heartbeat trace. The forecasting task is to read the past portion of that sequence and produce the next stretch. Classically you fit a fresh model per dataset — one for retail, another for energy, another for traffic. That is the practice the foundation-model framing is trying to break.
why "decoder-only" is the load-bearing word
In the transformer family there are three rough shapes. An encoder reads a whole input at once and produces a representation of it. A decoder generates an output one step at a time, each new step conditioned on everything generated so far. Encoder-decoder models do both — read, then generate. Translation systems were classically encoder-decoder; the GPT line of language models is decoder-only. Google's model adopts the decoder-only design, which is the same architectural lineage that made large language models scale cleanly.
Why does that choice matter for numbers rather than words? Because forecasting is structurally an autoregressive problem. Autoregressive means each prediction feeds back as input to the next prediction. A decoder-only transformer is built precisely for that: it consumes a context of past tokens and emits the next one, then the next. Swap "word" for "patch of recent measurements" and the machinery transfers almost untouched. You are no longer asking the model to translate between two modalities; you are asking it to continue a sequence. That is the function the architecture serves.
There is a practical payoff hiding in this. Encoder-decoder forecasters often commit to a fixed prediction horizon at training time — train to predict 24 hours, and 168 hours becomes awkward. A decoder that generates step by step can, in principle, roll its own output forward to whatever horizon you need. The horizon stops being baked into the architecture and becomes a runtime decision.
the real claim is transfer, not accuracy on one benchmark
The interesting promise of a foundation model is zero-shot or few-shot generalization. Zero-shot means the model handles a dataset it has never seen, with no additional training. For a working engineer this is the part that changes the economics. The expensive, unglamorous reality of production forecasting is that every new client, every new metric, every new SKU traditionally demands its own trained model, its own validation, its own monitoring. Multiply that by a few thousand series and the bottleneck is no longer modeling — it is operational sprawl.
If a single pretrained model gives a usable forecast on a brand-new series out of the box, the calculus shifts. You move from "train a model per problem" to "call a model, optionally fine-tune later." That is the same transition language models forced on natural-language tooling, and it is why the decoder-only framing is more than cosmetic borrowing — it is importing a whole training and deployment recipe that has already been proven to scale.
what I'd hold the claim to
I build forecasting and document-intelligence systems for a living, so let me be precise about where the skepticism should sit. Time series are not language. They carry units, seasonality, irregular sampling, regime changes, and exogenous shocks that have no clean analog in text. A model that is excellent at continuing the shape of a curve can still be blind to a known holiday, a price change, or a sensor that was offline for six hours. The architecture transfer is real; the assumption that domain structure transfers for free is the thing to test, not assume.
The honest engineering question is not "is it as good as a hand-tuned model on one dataset?" — a specialist almost always wins its home turf. The question is "how close does the zero-shot forecast get, across hundreds of unseen series, for a fraction of the operational cost?" If the answer is "close enough for the long tail of series nobody had time to model properly," that is already valuable. Most real forecasting pipelines are not bottlenecked on the ten series someone cared enough to tune; they are bottlenecked on the ten thousand nobody touched.
the wider frame
There is a pattern worth naming here. Over the last few years the transformer has behaved like a piece of general infrastructure: an architecture proven on one modality, then ported to the next with the modality-specific parts swapped out and the core left intact. Language to vision, vision to protein structure, and now sequences of words to sequences of measurements. Each port is a bet that the underlying problem — predict the next element given the previous ones — is more universal than the surface domain.
That bet is what this work tests on forecasting. Whether it pays off uniformly across messy industrial data is an empirical question that only deployment will answer. But the direction is coherent: stop treating each forecasting problem as a bespoke modeling project, and start treating prediction itself as a capability you call into. For anyone who has maintained a graveyard of per-dataset models, that reframing alone is worth paying attention to.
Sources
- A decoder-only foundation model for time-series forecasting — Manual / ad-hoc · 2026-06-18