Three Time-Series Foundation Models Fail to Beat Simple Financial Baselines

Summary:A public zero-shot evaluation tested Google’s TimesFM 3.0, Amazon’s Chronos-2, and Datadog’s Toto 2.0 on 24 US stocks and ETFs. The models achieved next-day directional accuracy of only 49.9%–51.1%, with similar results on synthetic random walks that contain no exploitable signal. Long-or-cash strategies driven by the forecasts also underperformed buy-and-hold after transaction costs. The experiment does not show that time-series foundation models are useless in finance. It shows that generic z

---

Experimental Finding: Zero-Shot Forecasts Were Near Random

This experiment evaluated three general-purpose time-series foundation models: Google’s TimesFM 3.0, Amazon’s Chronos-2, and Datadog’s Toto 2.0 (313M). The dataset covered 24 large US stocks and ETFs obtained from Yahoo Finance, with data spanning January 2014 to November 2026. The evaluation window started in November 2024 and covered 504 trading days.

Each model received 512 historical closing prices and generated forecasts for one, five, and twenty trading days ahead. Inference was zero-shot with default configurations and no fine-tuning. Baselines included the last observed price, an always-up directional predictor, and simulated random walks with matched drift and volatility. The author also repeated the evaluation on hourly and five-minute price bars.

The models achieved next-day directional accuracy between 49.9% and 51.1%. The always-up baseline reached 52.2%. More importantly, replacing market data with synthetic random walks produced nearly the same accuracy. Under the test setup, the models therefore failed to distinguish real financial series from sequences constructed without exploitable predictive structure.

Why the Random-Walk Control Matters

Financial evaluations often compare a model only with the last-price forecast or buy-and-hold. Those baselines are useful, but they do not establish that a model has learned genuine predictive structure. Leakage, incorrect adjustment for nonstationarity, duplicated observations, or post-selection hyperparameter tuning can all make a complex model appear effective against a weak baseline.

A random-walk control offers a more stringent zero-signal test. In principle, a simulated series with matched drift and volatility but no exploitable autocorrelation should not support a stable directional edge. If a model scores similarly on real and synthetic data, weak outperformance on the real data is insufficient evidence of meaningful financial signal.

This does not mean the models have no potential value in finance. They may produce conditional distributions rather than a single path, and thresholding those distributions for direction can discard information about magnitude, uncertainty, and tail risk. The experiment does not report calibration, interval coverage, MAE, RMSE, Cramer’s score, or the effect of alternative sampling strategies.

Trading Results: Forecasts Did Not Produce Better Returns

The author also tested a simple long-or-cash strategy: hold an asset when its forecast is positive and remain in cash otherwise. Starting with $10,000 and charging 0.05% per switch, the time-series strategies ended between $10,893 and $12,181, while buy-and-hold reached $15,036.

The return gap cannot be attributed entirely to directional prediction error. The strategy used only direction, did not scale positions by forecast magnitude, and did not account for overnight gaps, overlapping positions, market capacity, slippage, taxes, liquidity, or risk exposure. Even with those limitations, the result demonstrates that directional accuracy alone does not create an attractive trading system. A useful signal must also be evaluated through expected return, holding period, turnover, transaction costs, and risk control.

On five-minute data, a one-line mean-reversion heuristic performed better. This may reflect short-horizon autocorrelation, reversal effects, or microstructure patterns that a generic foundation model does not exploit zero-shot. It also remains asset-, session-, and cost-model-dependent, so it is not evidence that mean reversion works universally.

Technical Interpretation: General Forecasting Is Not Market Forecasting

Time-series foundation models learn local patterns, trend continuation, and cross-domain statistics from broad sequence corpora. In financial prices, they may capture trend persistence, volatility clustering, calendar effects, missing-value handling, or multiscale structure.

Those patterns do not automatically translate into a tradable directional edge. Financial return series are dominated by noise, while regimes shift, corporate actions alter price series, and microstructure effects operate across multiple time scales. A model may also exploit an apparent artifact that disappears once data preparation is corrected.

Probabilistic outputs must also be consumed correctly. If a model estimates a distribution (p(r_{t+1}\mid x_{t:L})), taking the mean or sampling one path is not equivalent to optimizing a trading objective. Practical use requires calibration and a decision rule tied to expected utility, drawdown limits, or position sizing.

Evaluation Limits and Reproducibility Risks

The study covered 24 liquid but highly correlated US equities and ETFs. SPY, QQQ, and IWM share substantial market factors, and observations across a 504-day window are not necessarily independent. Treating every asset-date pair as an independent test would overstate statistical confidence.

Yahoo Finance data can also be revised, and adjustment conventions may vary. A strict replication should preserve data snapshots, document split and dividend handling, record corporate actions, and ensure that preprocessing and parameter selection never use information unavailable at the forecast date.

A stronger evaluation would use rolling or expanding-window backtests, strict temporal splits, multiple random seeds and sampling settings, confidence intervals, results broken down by asset and market regime, and probability metrics such as calibration and proper scoring rules. It should also compare linear models, tree ensembles, momentum, mean reversion, and regularized ARIMA baselines rather than relying only on trivial heuristics.

Practical Guidance for Developers

1. Establish naive baselines first. Include the last price, random walks, always-up, simple momentum, mean reversion, and regularized linear or ARIMA models.
2. Use zero-signal placebo tests. Replace market data with synthetic series that match drift, volatility, and autocorrelation, then check whether apparent gains disappear.
3. Separate forecasting from trading metrics. Evaluate MAE, directional accuracy, calibration, tail losses, turnover, and net performance independently.
4. Run rolling backtests. Keep data cleaning, feature construction, and hyperparameter selection strictly within the information set available at each historical date.
5. Treat domain adaptation cautiously. Fine-tuning should span different market regimes and be validated temporally, rather than with a random split that can leak future regimes.
6. Budget for full deployment costs. Include inference latency, sampling overhead, GPU cost, slippage, market impact, and signal decay when determining whether an edge is economically usable.

Practical Assessment

The useful conclusion is not that time-series foundation models are ineffective. It is that generic zero-shot forecasting must prove its edge against realistic and synthetic baselines, survive leakage controls, and generate returns after transaction and risk costs.

For financial forecasting teams, TimesFM, Chronos-2, and Toto 2.0 are more defensible as general sequence priors or feature generators within a tested ensemble. Their raw zero-shot outputs should not be treated as an alpha signal without stronger validation and domain-specific adaptation.

---

Source: Reddit r/MachineLearning (2026-11-10)

— END —

Tags: #time-series foundation models Follow #financial forecasting Follow #TimesFM 3.0 Follow #Chronos-2 Follow #Toto 2.0 Follow