SREE Grounds Depression Inference in Attributable Symptom Evidence

Summary:SREE is an evidence-grounded pipeline for inferring standardized depression questionnaire responses from longitudinal social-media histories. It extracts first-person, symptom-specific spans, selects a representative evidence set using a Facility Location-inspired objective, and performs Evidence-Gated Inference over the selected evidence. On BDI-Sen and the eRisk 2019—2021 collections, the paper reports 70% relative improvement in post-level precision and 59% in F1 over state-of-the-art retriev

---

The Core Problem

Inferring depression questionnaire responses from social-media histories requires more than semantically similar posts. The evidence must be attributable to the user, specific to a symptom, and sufficiently supportive of an individual questionnaire item. Conventional dense retrieval generally represents entire posts, which can mix multiple symptoms or retrieve descriptions of other people's experiences.

SREE (Symptom Retrieval and Evidence-Grounded Inference) changes the retrieval unit and separates evidence construction from inference:

1. Extract first-person, symptom-specific spans from candidate posts;
2. Select a representative subset with a Facility Location-inspired objective;
3. Perform Evidence-Gated Inference over the selected evidence.

This is a shift from retrieving posts to retrieving attributable, interpretable evidence for individual items.

Mechanism: From Post Retrieval to Evidence Composition

First-person and symptom-specific span extraction

SREE localizes evidence instead of treating an entire post as a single representation. This addresses two major sources of noise:

• Attribution errors: a user may discuss a friend, public figure, or group whose experiences are not evidence of the user's own symptoms.
• Item confusion: a post describing sleep problems, fatigue, and appetite changes produces a mixed embedding that is difficult to map to a specific questionnaire item.

Span-level extraction offers finer attribution and item mapping. However, it also makes negation, contrast, temporal relations, and cross-sentence references harder to handle. A span boundary that cuts through any of these constructions can change the apparent meaning.

Representative evidence selection

Not all relevant spans are equally useful. The Facility Location-inspired objective favors a subset that covers diverse semantic positions in the candidate evidence while avoiding redundant selections.

This is better suited to questionnaire inference than relevance-only Top-k selection:

• Several high-scoring spans may describe the same symptom;
• complementary spans may support different questionnaire items;
• controlling evidence composition can reduce redundancy and noise amplification in long prompts.

The objective optimizes coverage and representativeness, though it does not directly guarantee that every selected span supports the target item. Error calibration and factual-consistency constraints remain necessary.

Evidence-Gated Inference

The final stage constrains item-level inference to the selected symptom evidence rather than allowing the model to generate questionnaire answers freely from its background knowledge.

This design has two operational benefits:

• every item can be traced back to specific text;
• retrieval quality and inference quality can be measured separately.

Attributability improves auditability, but it does not guarantee correctness. A model could still learn label-correlated retrieval patterns without distinguishing denial, irony, hearsay, or weak evidence. A production system should therefore include evidence-strength calibration and a low-confidence abstention path.

Evaluation Results

The paper evaluates evidence retrieval on BDI-Sen and evidence-conditioned questionnaire inference on the eRisk 2019—2021 collections. It reports:

• a 70% relative improvement in post-level precision over state-of-the-art retrieval methods;
• a 59% relative improvement in F1;
• retention of at least 30% fewer posts;
• higher precision and F1 than dense retrieval on multi-symptom sentences, even when dense retrieval receives twice the retrieval budget;
• the highest exact item-level agreement across all three eRisk collections, while remaining competitive on ordinal and aggregate metrics.

The most significant result is not merely a higher score, but higher precision with a smaller retained evidence set. This suggests that many additional posts dilute the retrieval distribution rather than contribute useful evidence. The multi-symptom results also demonstrate that post-level dense retrieval has an effectiveness ceiling for item-level inference: more candidate budget cannot fix irrelevant or unattributed evidence.

How to Interpret the Benchmarks

The abstract leaves several validation questions for the full paper and implementation:

1. Metric granularity: post-level precision is not the same as span-level or item-level precision. The headline gain should not be generalized without the underlying definitions.
2. Baseline coverage: comparisons should include sparse retrieval, BM25, dense retrievers, rerankers, and end-to-end LLM inference over retrieved posts.
3. Leakage controls: user-level splits, temporal separation, and overlap between questionnaire labels and retrieval periods must be audited.
4. Label noise: depression questionnaires contain recall errors, differing thresholds, and context-dependent responses. Item agreement is not equivalent to clinical validity.
5. Reproducibility: multi-seed results, confidence intervals, significance tests, and cross-year or cross-platform stability should be reported.

The datasets support research estimation, not direct clinical deployment. Social-media text can expose third parties, while incorrect mental-health inferences can create stigma or inappropriate intervention.

Engineering Trade-offs

SREE reduces prompt size while improving traceability, but its multi-stage architecture introduces several failure points.

| Design choice | Benefit | Main cost | |---|---|---| | Span-level evidence | Reduces multi-symptom confusion and supports attribution | Harder handling of boundaries, negation, and reference | | Representative subset selection | Shrinks context and removes redundant evidence | May discard high-value but low-coverage spans | | Evidence-gated inference | Produces traceable outputs and supports review | Incorrect evidence may still propagate | | Longitudinal social media | Provides long-term behavioral context | Raises privacy, temporal-drift, and platform-bias concerns |

Unlike extending the context window, SREE compresses and curates evidence before inference. This can reduce inference-token cost and clarify why an item was assigned a particular value. It also transfers naturally to legal analysis, fraud review, clinical question answering, and other high-stakes language decisions.

Deployment Recommendations

For replication or production use:

1. Establish auditable baselines: compare BM25, dense retrieval, reranking, and direct LLM inference over retrieved posts.
2. Split data by user: prevent the same user from appearing across partitions and report user-level macro averages.
3. Separate evidence metrics: measure attribution, symptom specificity, item support, and final agreement independently.
4. Calibrate and abstain: return an uncertain result when evidence is weak, conflicting, or primarily about someone else.
5. Minimize privacy exposure: perform span extraction locally where possible, redact third-party mentions, and limit raw-text transfers.
6. Avoid clinical overclaiming: disclose dataset boundaries, subgroup bias, and the consequences of incorrect inference in research or screening settings.

SREE's contribution is not a shortcut from social media to a depression diagnosis. It demonstrates a more rigorous architecture: establish who expressed the evidence, what symptom it describes, and which questionnaire item it can support before making an evidence-constrained inference.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-11-11)

— END —

Tags: #SREE Follow #Evidence-Gated Inference Follow #Social Media Mental Health Follow #Depression Questionnaire Inference Follow #Fine-Grained Retrieval Follow