ZICQ
中 Log in / Sign up
Newsroom Research & Papers #Mamba #Mechanistic Interpretability #Sparse Autoencoders #Financial Time Series #Data Leakage

Leakage Controls Collapse Mamba's Financial-Regime F1 to 0.398

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published: · 1 views

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:This preprint applies mechanistic interpretability to a six-layer Mamba trained on roughly 21 years of daily data for a point-in-time universe of S&P 500 constituents, macroeconomic variables, and Fama-French factors. Top-K sparse autoencoders decompose the layerwise hidden states. The authors then remove three inputs used in the HMM label definition, disable the auxiliary regime-classification head, use a point-in-time constituent universe, and evaluate against labels defined independently of t


Core finding

The six-layer Mamba's apparent ability to explain bullish, bearish, and sideways market regimes does not survive rigorous controls. Regime-recovery F1 falls from 1.00 in the earlier uncontrolled setup to 0.398, below the 0.435 obtained by logistic regression on raw inputs and close to the 0.364 obtained by an SAE trained on a randomly initialized Mamba. The paper's main result is therefore not the discovery of a hidden bull-bear switch. It shows that label-defining inputs and auxiliary supervision can jointly manufacture apparently perfect mechanistic evidence.

What the selective gate and SAE actually measure

Mamba is a selective state-space model. Rather than using fixed state-transition parameters, it derives Δ_t, B_t, and C_t from the current input. The Δ_t parameter controls how quickly information is retained or forgotten. Under the usual negative-real diagonal state matrix A, a simplified recurrence can be written as h_t = exp(Δ_t ⊙ A)h_(t-1) + B̄_t x_t. Larger Δ_t values generally cause faster state decay, while smaller values preserve information across more time steps; input-dependent B_t and C_t determine what is written into the state and how it is read out.

Training can exploit parallel scanning, while inference carries a fixed-size recurrent state per layer rather than a KV cache that grows with sequence length. This is attractive for long financial histories, but it creates an interpretive ambiguity: does the compressed state encode a reusable market regime, or merely a shortcut correlated with the labels? Twenty-one years of daily observations amount to several thousand trading steps—long enough to test cross-cycle memory, though not a severe long-context stress test.

The researchers then apply Top-K sparse autoencoders to the layerwise hidden states. An SAE maps a high-dimensional hidden state to a small number of activated dictionary atoms, allowing researchers to examine which sparse features vary with market regimes. A complete mechanistic claim requires three separate links: the hidden states must admit a stable sparse decomposition, those sparse features must predict the regime, and the regime differences must be attributable to internal quantities such as Δ_t. Label leakage or an unfair baseline can break any of these links while leaving a convincing-looking narrative.

How F1 reached 1.00

The earlier configuration combined three risks. First, three input features used to construct the HMM regime labels remained in the model's input, allowing it to recover the label-generation rule directly. Second, an auxiliary regime-classification head explicitly trained the representation to separate bull, bear, and sideways states. Third, when evaluation inputs and labels share provenance, a classifier can reconstruct the labeling shortcut without learning anything about genuine market-state dynamics.

The controlled protocol removes the three label-defining inputs, disables the auxiliary classification head, uses a point-in-time constituent universe, and separates the evaluation labels from the retained input definitions. A 2×2 attribution grid separately toggles auxiliary supervision and label-defining inputs. Neither factor alone pushes F1 above roughly 0.40; the near-perfect result appears only when both are present. Leakage is therefore not a minor perturbation here. It interacts with the training objective to produce a multiplicative shortcut.

| Setup | F1 | Defensible interpretation | |---|---:|---| | Uncontrolled configuration | 1.00 | Label-definition inputs and auxiliary supervision create a shortcut; this is not mechanism evidence | | Full leakage control | 0.398 | The trained Mamba does not outperform strong baselines at decoding regimes | | Logistic regression on raw inputs | 0.435 | Controlled SAE features add no measurable predictive value over a simple linear baseline | | SAE on a randomly initialized Mamba | 0.364 | Some decodability may arise from generic input structure rather than a trained gating mechanism |

What survives the controls

First, the residual Δ_t effect of the selective gate remains statistically detectable under block-bootstrap inference, but its economic magnitude is negligible. Its magnitude is also broadly uniform across layers, providing no evidence that a particular layer specializes in bull-bear transitions. The original layer-specialization story consequently loses its principal support.

Second, the reported polysemanticity gradient disappears under continuous, theory-grounded measures. This suggests that layer trends obtained by discretizing activations into high and low bins may be sensitive to thresholds, sample discretization, or dictionary selection. Continuous attribution is consequently more informative than a single discrete F1 when asking whether an internal variable genuinely carries a mechanism.

Third, only a modest subset of economically grounded features survives the controls. This is more restrained than claiming that the market regime has been fully decoded, and it locates the paper's value primarily in methodology: negative controls, point-in-time data construction, and a factorial attribution grid rather than a directly deployable trading signal.

Engineering trade-offs and validity limits

The audit pipeline is not free. Top-K SAEs require separate training, and dictionary atoms can change with the random seed, Top-K threshold, or optimization budget. Cross-layer attribution also requires hidden states, gate parameters, and probes to share exactly the same data split. Financial dependence further complicates inference: random train-test splits place adjacent trading days on both sides of the boundary, while HMM windows may overlap across the split. Reproduction therefore requires chronological splitting, label purging, or embargo.

A point-in-time constituent universe controls survivorship bias, but it does not automatically solve macroeconomic revisions, corporate-action handling, delisting returns, transaction costs, or short-sale constraints. Block-bootstrap inference also requires checking whether the block length spans the HMM label window and the dominant autocorrelation scale; otherwise confidence intervals can remain too narrow.

Several uncertainties remain in the disclosed material. The abstract does not state whether F1 is macro-averaged, sample-weighted, or tied to a particular class, nor does it provide class proportions, random-seed counts, or complete confidence intervals. A 0.03-point difference from logistic regression should not be treated as meaningful without those details. HMM states are algorithmically constructed latent labels rather than uncontested market truth. Even after three overlapping inputs are removed, other economic variables may remain proxies for the label construction process. The randomly initialized Mamba control is valuable but combines random dictionary geometry, generic input structure, and SAE capacity, so it cannot isolate each source by itself.

The claim of negligible economic significance also needs an operational definition. Return improvement, turnover, risk-adjusted performance, and post-cost net return are not interchangeable. Statistical significance without a preregistered economic threshold, downside-risk measure, and cost model is insufficient for production trading conclusions.

Reproducibility and deployment checklist

  1. Maintain label provenance: Record every price, macroeconomic, and factor field used to construct HMM or human regime labels, then prohibit those fields or their proxies from explanation features.
  2. Run a full factorial ablation: Reproduce at least the 2×2 grid over auxiliary supervision and label-defining inputs, alongside SAE, random-initialization, and raw-input logistic-regression controls. Report confidence intervals for every cell.
  3. Use time-consistent negative controls: Preserve point-in-time constituents, historical macro vintages, and rolling training windows; purge or embargo overlapping labels instead of relying on random splits.
  4. Separate three evidence layers: Report dictionary stability, regime-decoding performance, and Δ_t attribution independently. A mechanism claim should pass strong baselines and negative controls at all three layers.
  5. Translate statistical effects into economic gates: Preregister minimum trading benefit, maximum turnover, post-cost expectancy, and drawdown tolerance. A statistically detectable Δ_t effect that misses the economic threshold should not become a strategy signal.
  6. Publish reproducibility artifacts: Release point-in-time data construction, label code, SAE dictionaries, split indices, random seeds, block-bootstrap settings, and the complete ablation grid in addition to model weights.

The central lesson is that interpretability does not emerge automatically from a higher classification score. Auxiliary objectives, label provenance, and training tasks jointly shape the learned representation. Without leakage controls first, an SAE can provide a more granular vocabulary for a training shortcut rather than a mechanistic account of market-regime dynamics.


Source: ArXiv Machine Learning (cs.LG) (2026-11-17)

— END —

Tags: #Mamba #Mechanistic Interpretability #Sparse Autoencoders #Financial Time Series #Data Leakage

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…