Mechanistic Shaping Trains Models Through Internal Representations

Summary:The paper introduces Mechanistic Shaping, a training-time framework that constrains internal representations so models both avoid expressing prohibited concepts and stop relying on sensitive attributes. Experiments span vision, medical imaging, and language, using classification, SFT, DPO, and GRPO across models up to 32B parameters. The work also formalizes how shaped concepts can evade monitoring: by drifting away from a probe, disappearing from the target layer, or persisting through nonline

---

Core Idea: Interpretability as a Training Constraint

Mechanistic Shaping moves intervention from the output layer into the model's internal representations. Instead of merely rewarding correct answers or refusals, training can constrain both:

• Expression constraints: the model should not internally encode prohibited concepts, such as harmful content.
• Decision-dependence constraints: the model should not rely on sensitive attributes, such as gender, when screening candidates.

The method does not assume that deleting a neuron or feature removes a concept. Neural networks contain redundant representations and alternative pathways, so local interventions can suppress visible behavior while leaving the underlying mechanism intact. The paper therefore treats training and monitoring as one interpretability problem: define what can be monitored, intervene on the representation, and then verify the intervention with a monitor matched to the concept's possible geometry.

Training Mechanism and Scope

Mechanistic Shaping treats internal representations as an additional optimization signal alongside the task objective. It is applied across classification, supervised fine-tuning, direct preference optimization, and group relative policy optimization. Experiments cover vision, medical imaging, and language, with model scales reaching 32B parameters.

The engineering benefit is that the intervention can address mechanisms that output-level refusal does not fully capture. The cost is equally important: training requires access to activations at target layers and additional probe computation. Activations increase memory pressure, while probe quality determines whether the optimization target is meaningful. If the probe participates in the loss, it can also be modified by training and later become an unreliable monitor.

Three Forms of Conceptual Evasion

A central contribution is the distinction between a concept escaping one probe and a concept actually disappearing. The paper characterizes three geometric mechanisms.

1. Representation Drift

The concept remains inside the model but rotates or moves relative to the direction captured by the original probe. The original score falls because it no longer aligns with the representation; a refitted probe can often recover the concept.

This means that a failed frozen probe is not evidence of erasure. Reusing only the pre-training probe can create false assurance.

2. Removal Within the Layer

The concept disappears from the monitored layer. This is a stronger claim than probe failure, although it remains bounded by that layer's capacity, parallel encodings, and the scope of the training objective.

3. Nonlinear Encoding in Higher-Order Statistics

The hardest case occurs when a concept is no longer represented by a single linear direction but survives as a nonlinear relationship among multiple directions or in higher-order statistics. A linear probe may naturally miss this representation.

The paper explains the phenomenon through a product-code construction: separate subspaces carry partial information, so a linear projection cannot recover the full concept even though a nonlinear combination can. This directly limits the validity of inferring erasure from a weak linear-probe score.

A Hierarchy of Monitors

The proposed hierarchy distinguishes different strengths of interpretability claims:

1. A frozen probe reports no detectable concept, but this only establishes that the probe no longer responds.
2. A refitted linear probe can still recover the concept, indicating that it remains linearly encoded.
3. A strong claim of removal requires monitors that cover nonlinear encodings, layer migration, and alternative pathways.

Empirically, shaped concepts largely remain monitorable. Refitted linear probes recover them in most settings, while the original probe stays reliable when gradients do not flow through it. The latter point is critical: when probe and training objective share gradients, the detector can adapt alongside the behavior it is meant to detect.

Engineering Trade-offs

Mechanistic Shaping offers a more mechanism-level intervention than output filtering, but it introduces several costs:

• Activations must be stored or streamed, increasing memory use and pipeline complexity.
• Weak probes make the optimization target noisy and can reduce training efficiency without providing reliable constraints.
• Gradient-coupled monitors may exhibit adaptive evasion, so post-training measurements cannot be treated as independent evidence.
• A concept may migrate to another layer rather than disappear, making single-layer audits insufficient.
• Probe forward passes add training cost, particularly for large models.

Production adoption should therefore treat reduced target behavior and continued internal auditability as separate acceptance criteria. The same probe should not serve simultaneously as the optimizer and the final safety certificate.

Practical Recommendations

1. Build a monitor hierarchy: combine a frozen probe, a refitted linear probe, and monitors capable of detecting nonlinear encodings.
2. Track gradient dependencies: record which probes contribute to the loss and which remain independent.
3. Audit across layers: a concept may move outside the layer used during training.
4. Separate optimization from auditing: use independent probes or decoding methods for final evaluation.
5. Report both utility and mechanism metrics: include task quality, target-behavior rate, concept recovery rate, training overhead, and subgroup outcomes.

Limits of the Evidence

The paper states that the three evasion mechanisms account for all observations in its experiments, while acknowledging that additional mechanisms may emerge in other settings or at larger scales. The evidence supports internal representations as useful training constraints, but not the unconditional claim that every concept can be erased. Its strongest conclusion is methodological: behavioral suppression should be verified through a rigorous hierarchy of interpretability-based monitoring rather than inferred from a single probe.

---

Source: ArXiv Machine Learning (cs.LG) (2026-11-06)

— END —

Tags: #Mechanistic Shaping Follow #Mechanistic Interpretability Follow #Internal Representations Follow #Linear Probes Follow #Model Safety Follow