Stress Tests Expose an Optimism Bias in LLM Causal Inference

Summary:A systematic stress test evaluates LLM-based causal inference across six dimensions: model scale, causal-graph complexity, prompting strategy, metadata quality, data quality, and sensitivity to uncertainty. The results reveal a persistent optimism bias: models overestimate causal relationships, producing weak precision across the board. Reasoning-oriented models can look stronger under conventional metrics that ignore uncertainty, but their performance drops once confidence-aware evaluation is

---

Core Findings

Rather than treating large language models as complete causal-discovery engines, this study positions them as assistants that combine domain knowledge with dataset context to propose candidate causal relationships. The authors stress-test models along six dimensions:

• Model scale: whether larger models consistently improve causal reasoning;
• Causal-graph complexity: the difficulty introduced by additional variables, edges, and latent confounders;
• Prompting strategy: the effect of different prompting approaches;
• Metadata quality: the availability of variable definitions, timestamps, units, and provenance;
• Data quality: missingness, noise, and unreliable observations;
• Sensitivity to uncertainty: evaluation with and without confidence-aware penalties.

The most consistent result is causal overclaiming. Models tend to reinterpret correlation, temporal order, or semantically plausible associations as genuine causal edges. This hurts precision more than recall: a model can successfully recover a plausible relationship without being able to establish that the relationship is causal.

Why Scale Does Not Solve the Problem

An LLM-based causal pipeline typically reads variable descriptions, retrieves domain associations from pretraining, simulates an explanation of the underlying mechanism, and then constructs a graph. Language models are optimized for semantic coherence, whereas causal identification requires ruling out alternative explanations. When a dataset lacks interventions, counterfactual observations, or reliable temporal ordering, the model fills the mechanistic gap with linguistic priors. The resulting edge may sound plausible without being empirically identified.

Scaling improves the breadth of domain knowledge and instruction following, but it does not automatically provide stronger identification. The reported gains are conditional: with strong metadata, a larger model can formulate more stable hypotheses; as metadata degrades, it increasingly substitutes semantic correlation for structured evidence and generates additional spurious edges.

Conventional metrics can also obscure this behavior. Scores based on graph overlap, answer consistency, or LLM-judged plausibility reward confident language and domain knowledge, not demonstrated causal validity. Once confidence, abstention, and uncertainty penalties are included, apparent performance falls to a more realistic level.

Engineering Trade-offs and Evaluation Gaps

The contribution is primarily an evaluation framework rather than a new model architecture. However, the abstract does not disclose the full model list, sample size, datasets, graph-generation procedure, statistical significance, or calibration method. Several questions therefore remain open:

1. How uncertainty is measured: Does it come from self-reported confidence, sampling variance, perturbation stability, or calibrated posterior probability?
2. How realistic the stress-test data are: Synthetic causal graphs improve control but may underrepresent measurement error, selection bias, and unknown structure in real research data.
3. Whether prompting comparisons are compute-matched: A longer prompt may provide both more domain information and more reasoning budget.
4. Whether an LLM judge creates circular validation: If another language model scores causal graphs, linguistic plausibility may again hide factual errors.
5. Whether scale comparisons include serving costs: A stronger model running in a more expensive reasoning mode may not be the better operational choice.
6. Whether precision and recall are reported separately: Candidate discovery favors recall, while decision support and expert review often require much higher precision.

The framework is also not a complete causal-discovery algorithm. LLM-generated edges still need to be constrained by intervention data, domain assumptions, confounding-control methods, and statistical tests.

Deployment Recommendations

A practical architecture should separate causal-hypothesis generation from hypothesis validation:

1. Validate structured inputs first: Require definitions, units, temporal coverage, provenance, and data dictionaries, and define explicit behavior for missing metadata.
2. Require edge-level evidence: Each proposed relationship should include supporting evidence, a candidate mechanism, counterevidence, and potential confounders.
3. Separate correlation, prediction, and causation: Do not encode all three under a single causal label.
4. Prioritize intervention-based validation: Use randomized experiments, quasi-experiments, or natural experiments whenever identification assumptions are available. Retain an unknown label otherwise.
5. Divide generation from verification: Let the LLM propose directions while statistical programs, numerical tools, or domain experts test them.
6. Set thresholds according to downstream cost: Research exploration may justify a recall-oriented setting, while operational decisions should use precision-oriented thresholds and mandatory expert review.

Practical Conclusion

LLMs can reduce the initial cost of causal hypothesis exploration, but they do not provide reliable causal identification by default. Scaling yields limited and fragile improvements; better prompts cannot substitute for data quality; and ignoring uncertainty inflates benchmark scores. The defensible role for an LLM is that of a research assistant, not the final adjudicator of causation.

---

Source: ArXiv Machine Learning (cs.LG) (2026-11-05)

— END —

Tags: #LLM Causal Inference Follow #Causal Discovery Follow #Uncertainty Evaluation Follow #LLM Reliability Follow #Stress Testing Follow