Probing Directions Are Not Steering Directions: A Separation Across Language Models
Summary:A study of three instruction-tuned models from different families finds that the hidden-state direction that best detects a concept is usually not the direction that best changes model behavior. Across a layer sweep, the probing and steering optima diverged in 59 of 60 model-layer-concept combinations. The separation persisted when probe and steering used matched texts and when the intervention was evaluated with a rotation-based steering operator. Cross-model transfer is similarly asymmetric:
Main Finding
The study separates two uses of a direction in a language model's hidden state: probing whether a concept is present and steering the model's generative behavior. Its central result is not that representation steering is ineffective, but that detection and behavioral control have measurably different optima. The direction that yields the best linear probe is usually not the direction that yields the strongest behavior change.
This distinction matters in practice because a probe with high classification accuracy should not automatically be reused as a steering controller. Probing optimizes representational separability, whereas steering optimizes output behavior under a bounded side-effect budget. When the concept axis, behavioral axis, and downstream transformations do not align, a vector can read a concept accurately while having little causal control over generation.
Mechanism: Reading Directions Do Not Automatically Write Behavior
Let hₗ∈Rᵈ denote the hidden state at layer l. A linear probe typically computes p(h)=σ(wᵀh+b), using direction w to determine whether a concept is present. Additive steering instead changes the state to h'=h+αv, where v is the steering direction and α controls intervention strength.
The authors construct a one-parameter family whose endpoints are a covariance-adjusted discriminant and a mean difference. Conceptually, the covariance-adjusted endpoint resembles w=Σ⁻¹(μ⁺−μ⁻): it uses covariance information to suppress directions dominated by nuisance variation, which can improve detection. The mean-difference endpoint, μ⁺−μ⁻, more directly captures the average shift between positive and negative populations. The exact interpolation formula and the precise side-effect metric are not disclosed in the supplied abstract.
The two objectives differ. A probe requires a direction that separates concept-positive and concept-negative examples in representation space. Steering requires the direction, after passing through the current layer and subsequent network transformations, to alter output behavior while keeping unrelated capabilities within a fixed budget. A direction can be linearly diagnostic without occupying a behaviorally causal axis in the downstream computation.
Comparing directions under the same side-effect budget is also important. It prevents a direction from appearing superior merely because it causes a larger perturbation. This is a useful control, although the supplied material does not define the underlying side-effect metric precisely enough for full reproduction.
Experimental Design and What It Rules Out
The experiments cover three instruction-tuned models from different families and two concepts: truthfulness and sentiment. In a layer sweep, the best probing direction and the best steering direction diverge in 59 of 60 model-layer-concept combinations.
Two controls are particularly important:
- Across all 6/6 model-concept combinations, the separation remains when probe and steering texts are matched. This weakens the alternative explanation that the two directions are merely tracking different prompt distributions.
- The separation also persists under a rotation-based steering operator, indicating that it is not merely an artifact of additive injection.
The exact steering optimum is less stable. For sentiment, an intermediate direction can produce a 27%—56% stronger steering effect than the mean-difference direction. That advantage shrinks or disappears when texts are matched, and it varies with the layer and operator. The mean-difference direction is therefore an empirically robust starting point, not a universal optimum.
Cross-Model Transfer: Probes Transfer More Reliably Than Steering
Cross-model experiments show the same asymmetry. After fitting a linear map on neutral text, transferred probes retain 81%—91% of the target model's above-chance probing performance. Transferred mean-difference directions, by contrast, retain less steering effect in 11 of 12 model pairs.
That conclusion still requires caution. Under the paper's predefined criteria, only three model pairs can be classified clearly, while nine remain inconclusive. The directional trend is consistent, but the evidence does not yet establish that steering fails to transfer at the same rate across arbitrary model architectures.
Mechanistically, a linear map can approximately preserve representation geometry, for example by mapping a source direction as v_target=R·v_source. A probe only needs the mapped direction to remain separable from the target concept. Steering additionally requires that direction to produce the intended behavior after passing through the target model's residual stream, attention layers, and output head. Similar representational coordinates do not guarantee equivalent causal control.
Engineering Trade-offs
| Objective | Empirically preferable direction | Main cost and risk | |---|---|---| | Concept detection | Covariance-adjusted discriminant | Sensitive to covariance estimates, sample coverage, and distribution shift; requires alignment for cross-model use | | Behavior control in one model | Mean-difference-like or task-specific intermediate direction | Highly dependent on layer, prompt, operator, and intervention strength | | Cross-model probing | Linear map fitted on neutral text | Requires hidden-state access and validation that separability survives the mapping | | Cross-model steering | Refit or search for a target-specific direction | Behavioral equivalence after mapping is not guaranteed, increasing data and ablation costs |
Additive steering also costs more than a vector addition. Engineers must hook internal model layers and select α, injection timing, and target layers. Continuous intervention can alter activation norms, next-token distributions, and general capabilities. An orthogonal rotation may constrain geometric norm drift, but preserving vector length does not guarantee the absence of semantic or capability side effects.
PopQA Application: Better for Abstention Than Correction
On PopQA, steering hidden states toward an unknown response raises abstention on uncertain errors by 18–44 percentage points, compared with only 2–5 percentage points on confident errors. When the correct answer remains reachable, the intervention rarely turns an incorrect response into the correct one.
The mechanism is consequently better suited to selective abstention than general error correction. A practical system should add an uncertainty gate and apply unknown steering primarily to high-confidence failures or questions whose answers are genuinely inaccessible, avoiding unnecessary rejection of answerable questions.
Deployment Guidance
- Fit probes and controllers independently. Do not assume that a high-accuracy probe can serve directly as a steering vector.
- Recalibrate by model, layer, and task. Run layer sweeps first, then validate candidate directions with matched prompts rather than transferring an optimum from another layer unchanged.
- Make side effects part of the objective. Track target behavior alongside general QA accuracy, semantic drift, refusal rate, and formatting stability, comparing directions under the same perturbation budget.
- Test multiple steering operators. Compare additive injection with orthogonal rotation or other forms; steering quality is coupled to the operator and cannot be inferred from one implementation.
- Transfer probes before attempting steering transfer. A neutral-text linear map may preserve probing performance, but steering directions should be searched again on the target model.
- Gate abstention with confidence estimates. The PopQA results show that uncertainty assessment is more useful than indiscriminately strengthening the unknown direction.
Evidence Limits
The study covers three models, two concepts, and one QA application. That scope supports a carefully bounded conclusion, but not a universal claim for all language models. The supplied material does not disclose the model list, absolute probe accuracy, exact side-effect definition, statistical variance, runtime cost, or code status. Nine cross-model pairs also remain inconclusive under the predefined criteria.
The defensible engineering conclusion is therefore that representation reading and behavioral writing should be validated as separate control surfaces. The work provides strong empirical evidence and a useful experimental framework, but not a zero-tuning steering recipe that can be transferred unchanged across models.
— END —Source: Hugging Face Daily Papers (2026-11-14)
Tags: #LLM Interpretability #Activation Probing #Representation Steering #Cross-Model Transfer #Selective Abstention
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments