Marginal Matching Does Not License Factorized Sampling: Auditing Conditional Style Leakage in Factorized Generative Mode
Summary:A new arXiv study reveals that marginal distribution matching in factorized generative models does not guarantee independence between style and class labels. The authors show that matching only the marginal distribution leaves class-conditional distributions unconstrained, allowing style latents to remain highly predictive of labels despite appearing Gaussian in aggregate. Experiments across multiple models and datasets show near-zero global MMD while linear probes recover class labels with 74%-
Background
Factorized generative models (e.g., StyleGAN, FactorVAE) aim to disentangle representations by decomposing latent variables into content (class) and style components. The style variable is often regularized to match a fixed Gaussian prior, which is widely interpreted as evidence of independence between style and class. However, a new arXiv paper, "Marginal Matching Does Not License Factorized Sampling," demonstrates that this interpretation is incorrect.
Key Findings
Limitations of Marginal Matching
The paper proves that matching only the marginal distribution does not constrain class-conditional distributions. This means that even if the style variable appears perfectly Gaussian in aggregate, its class-conditional distributions can still be highly overlapping or shifted, allowing the style latent to carry significant class information. The authors derive an exact decomposition showing that marginal matching is only one of four conditions required for factorized sampling, and eliminating this mismatch is necessary but not sufficient.
Empirical Validation
The research team conducted experiments on a case-study model and four representative latent baselines. Results show that these models achieve near-zero global MMD (Maximum Mean Discrepancy), yet linear probes can recover class labels with 74%-100% accuracy (chance level 10%). Additionally, the model reaches 99.15% clustering accuracy, while externally evaluated class-conditional generation succeeds only 16% of the time. This leakage persists under six independent perturbations (model capacity, curriculum, prior geometry, supervision, etc.) across two datasets.
Mitigation Strategies
The paper proposes four mitigation strategies that reduce probe accuracy to 21%-46%, but leave within-class dependence largely unchanged. A post-hoc conditional prior improves externally evaluated class generation to 0.97 on MNIST without retraining but only 0.41 on CIFAR-10, while an empirical style bank achieves 0.88 on CIFAR-10.
Technical Analysis
The core contribution is revealing the inadequacy of marginal statistics. The authors show that no divergence computed solely on the marginal distribution of the style latent can certify independence from class labels. This finding challenges current evaluation methods for factorized generative models, urging researchers to report more comprehensive statistics such as class-conditional distributions or mutual information estimates.
Industry Impact and Developer Advice
For generative model developers, this study provides a crucial warning: when evaluating disentangled representations, one cannot rely solely on marginal distribution matching. It is recommended to adopt stricter evaluation protocols, including linear probes and class-conditional generation quality. The proposed mitigation strategies offer directions for improvement, such as incorporating conditional priors or empirical style banks.
Conclusion
This research provides an important correction to the theory and practice of factorized generative models, emphasizing the importance of conditional distributions, with profound implications for interpretability and controllability of generative models.
Source: arXiv:2608.05243
— END —Source: ArXiv Machine Learning (cs.LG) (2026-08-07)
Tags: #Incentive generation model #Decoupling #Style leak #Generate model assessment #arXiv
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments