SCDM Learns Better Composition for Reusable Table Embeddings

Summary:This work introduces a benchmark that evaluates table-level embeddings directly rather than using row-level prediction as a proxy. Across twelve methods, it measures consistency under partial views, discriminability across label granularities, robustness to benign perturbations, and efficiency. Lightweight schema- and text-based fingerprints prove highly competitive, while mean, max, and other parameter-free pooling rules over TabPFN and ConTextTab cell states underperform them. The authors the

---

Core Idea: Evaluating Table Embeddings Directly

Tabular models are often judged through row-level prediction accuracy, yet data retrieval, deduplication, overlap estimation, and data curation require a reusable embedding for an entire table. This study identifies the mismatch: an encoder may predict individual rows accurately while producing poor table-level embeddings after its local states are composed.

The proposed benchmark evaluates four properties:

1. Consistency under partial views
2. Discriminability across label granularities
3. Robustness to benign perturbations
4. Efficiency

Together, they measure the stability, separability, robustness, and deployment cost of table representations rather than relying on a downstream prediction score.

Strong Row Predictors Do Not Necessarily Produce Strong Table Embeddings

The experiments compare twelve methods, with particular attention to TabPFN and ConTextTab. Cell states produced by these two tabular foundation models are composed with mean, max, and three additional parameter-free pooling rules. Despite their strong row-level performance, the resulting embeddings trail lightweight schema- and text-based fingerprints.

The result separates two questions that are often conflated:

• Encoder quality: Do the local states contain useful compositional information?
• Composition quality: Can that information be aggregated into an effective table-level representation?

Mean pooling implicitly treats every cell as contributing equally, although real tables often follow a heavy-tailed importance distribution. Key columns, primary keys, and unusual cells may define table identity far more strongly than ordinary fields. Max pooling can instead overemphasize a single noisy value. Neither strategy directly models overlap between partial views and complete tables.

SCDM: Learning the Composition Rule from Overlap Structure

The authors freeze the ConTextTab encoder and learn only the composition rule above its cell states. This isolates the main question: given useful local features, what function should combine them into a table embedding?\n Support-Conditioned Distribution Matching (SCDM) uses partition overlap as a structural signal. Views from the same table have high overlap, while views from different tables should be represented according to their actual structural relationship. The training objective matches the distribution of embedding similarities to the distribution of partition overlaps within and across tables.

Instead of reducing the problem to hard same-or-different labels, SCDM learns a continuous geometry of table relationships. Because the encoder remains frozen, the approach avoids the training cost and potential forgetting associated with full fine-tuning.

Results and Practical Implications

Across fifteen head configurations, SCDM raises the Spearman correlation between embedding similarity and partial-view overlap from 0.04 to 0.43–0.50 on synthetic data. It also improves triplet ranking, clustering, and retrieval. Gains on real data are smaller but systematic, and a downstream parent-table retrieval task supports the same conclusion.

The large synthetic improvement should not be interpreted as a direct production forecast. Synthetic data offers more controllable overlap relationships and a more tractable distribution-matching objective. Real tables introduce heterogeneous schemas, noisy cells, mixed granularities, and irregular missingness. The abstract also does not disclose the parameter count of the composition heads, training time, embedding dimensionality, indexing throughput, or a full comparison with specialized table-retrieval systems.

For engineering teams, the most immediate options are:

• Table deduplication: Retrieve and cluster structurally similar tables to reduce storage duplication.
• Data lineage and parent-table retrieval: Model relationships between complete tables and partial views.
• Data curation: Use table identity as a signal for record matching and dataset organization.
• Lightweight indexing: Deploy schema, type, and text fingerprints as strong and inexpensive baselines.

The method also needs stress tests for very wide or sparse tables, key-value conflicts, nested cells, and cross-schema drift. Freezing the encoder lowers training cost, but it does not guarantee that a learned composition rule remains robust across every table distribution.

Deployment Guidance

1. Build inexpensive baselines first. Compare schema fingerprints, column-name and type fingerprints, and text-statistics fingerprints using recall-versus-cost curves.
2. Separate encoder and aggregator effects. Hold the cell encoder fixed when comparing pooling strategies.
3. Evaluate at the task's natural granularity. Deduplication, lineage discovery, and parent-table retrieval require different notions of positive and negative similarity.
4. Test realistic perturbations. Include column reordering, type changes, missing values, formatting changes, and small edits to noncritical cells.
5. Cache local states. Encode cells or blocks offline and perform only composition and normalization online.

Limitations and Conclusion

The central contribution is to turn composition from an implicit implementation detail into an independently measurable research problem. The results show that the aggregation rule and its training objective can materially change embedding quality even when the underlying encoder is frozen.

This does not prove that specialized tabular foundation models are fundamentally inadequate, nor that SCDM will outperform conventional schema matching in every high-scale deployment. The correct production choice depends on the business definition of overlap, retrieval recall, latency, index size, and the cost of incremental updates.

---

Source: ArXiv Machine Learning (cs.LG) (2026-11-13)

— END —

Tags: #Table Representation Learning Follow #SCDM Follow #TabPFN Follow #ConTextTab Follow #Table Retrieval Follow