Auditing Privacy Leakage in Tabular Foundation Model Embeddings

How much of a record survives in its embedding?

Replacing a table row with an embedding does not necessarily hide its sensitive attributes. Cascade Probing measures what can be recovered, one attribute at a time.

Our audit spans four tabular foundation models and ten datasets. It reveals a Privacy Cliff: knowing just one attribute can sharply increase the recoverability of the others.

Explore Cascade Probing

A data owner supplies public, sensitive, and irrelevant attributes to a TabFM with a private support set. The auditor receives the embedding, optional public attributes, and query access to the TabFM, then estimates sensitive attributes. — The audit setting, from the camera-ready paper. The support data remain private.

Abstract

Tabular Foundation Models (TabFMs) produce context-aware embeddings that are increasingly stored, shared, and reused for retrieval, clustering, and cross-institutional data exchange. These embeddings are often treated as privacy-preserving substitutes for raw tabular records, yet their attribute-level information content remains poorly understood. We present a systematic privacy audit of TabFM embeddings: given an embedding and varying amounts of side information, how accurately can sensitive attributes be recovered? We introduce Cascade Probing, a sequential probing method that measures recoverability while accounting for inter-attribute dependencies, significantly outperforming joint probing and optimization-based inversion baselines. Across four TabFMs (TabPFN, TabDPT, Mitra, TabICL) and ten datasets, we find that even without any side information, 65–95% of sensitive attributes can be recovered from embeddings alone. More critically, recoverability exhibits a Privacy Cliff: revealing a single known attribute sharply increases the recoverability of all remaining attributes. This phenomenon generalizes across model architectures and diverse domains, and is not eliminated by standard embedding transformations including noise injection, PCA, and knowledge distillation. These findings suggest that we need more formal privacy techniques for protecting sensitive information inside TabFM embeddings.

Cascade Probing

Tabular attributes are related: recovering one may make another easier to infer. Cascade Probing uses these dependencies instead of predicting every sensitive attribute at once.

The auditor receives a released embedding, schema information, and any known attributes of the target record. It can query the same TabFM endpoint under the same private support context, without accessing the model weights or support records.

Cascade Probing has three stages: compare a released embedding with a masked-query embedding; discover an attribute recovery order using reference data; then sequentially recover attributes, re-querying the TabFM after each prediction. — The three stages of Cascade Probing. During inference, each predicted attribute becomes context for the next query. The originally released embedding stays fixed.

1 Differential query

Keep the known attributes and mask the unknown ones. Querying the TabFM with this partial record produces a baseline embedding. The decoder receives both this baseline and its difference from the released embedding, so it can use the information associated with the missing attributes.

2 Strategy discovery

On reference data, train candidate decoders for the remaining sensitive attributes and select the strongest one on held-out validation data. Add that attribute’s ground-truth value to the reference context, then repeat. This discovers a recovery order and retains a decoder for each step.

3 Cascading inference

For the target record, follow the learned order using predicted values, not ground truth. After each prediction, update the partial record and re-query the TabFM. The next decoder uses the new baseline and residual to estimate another attribute.

Experimental Results

We evaluate TabPFN, TabDPT, Mitra, and TabICL across ten datasets, from income and credit records to health data and wider tables. The audit considers three reference-data regimes: Shadow uses same-distribution records; Few-Shot uses 30 real records to seed synthetic-data generation; and Zero-Shot generates reference data from schema metadata alone.

Sensitive attributes are recoverable without known target attributes

On TabPFN v2.5, Cascade Probing reaches 65–90% mean categorical accuracy in the Shadow setting at K = 0, where no attributes of the target record are known. It outperforms all five comparison methods in this experiment.

Table 1. Mean categorical recovery accuracy (%), averaged over sensitive-attribute counts. TabPFN v2.5, Shadow, K = 0. Higher means more leakage is recovered; the best result in each row is bold.
Dataset	Joint	DirectOpt	WGAN	TabSyn	UIFV	Cascade
Adult	22.0	46.0	59.0	55.0	62.9	65.0
German Credit	44.0	54.0	62.0	61.0	49.9	74.0
Heart Disease	55.0	69.0	75.0	75.0	67.1	79.0
ACS Employment	51.7	62.0	62.0	68.0	88.2	90.0
ACS Public Coverage	46.0	45.0	44.0	42.0	67.3	75.0
ACS Income-to-Poverty	33.2	53.0	53.0	51.0	83.9	84.5
ACS Health Insurance	41.9	45.0	43.0	43.0	62.1	68.6

One known attribute can trigger a Privacy Cliff

In the Few-Shot setting, revealing one attribute raises Cascade Probing’s categorical accuracy from 55% to 90% on Adult and from 59% to 87% on German Credit. Joint Probing also shows this jump, indicating that the effect is not specific to the cascade.

Categorical recovery accuracy versus number of known attributes. Adult and German Credit show a sharp increase when one attribute becomes known; Heart Disease starts with high Cascade recovery accuracy. — Figure 3 from the camera-ready paper, in the Few-Shot setting. The shaded region marks K = 0: no known target attributes. Higher categorical accuracy and lower normalized mean absolute error (NMAE) both mean more accurate recovery. Click either plot to enlarge it.

Continuous attribute recovery error versus number of known attributes. Errors generally decrease with side information; the dashed Joint Probing curves drop steeply after the first known attribute. — Figure 3 from the camera-ready paper, in the Few-Shot setting. The shaded region marks K = 0: no known target attributes. Higher categorical accuracy and lower normalized mean absolute error (NMAE) both mean more accurate recovery. Click either plot to enlarge it.

The risk extends across model families

Substantial leakage also appears in TabDPT, Mitra, and TabICL. On Adult in the Few-Shot setting, categorical recovery with no known target attributes reaches 93% for TabDPT, 86% for Mitra, and 95% for TabICL. These models therefore expose considerable information even before extra attributes are revealed.

Common embedding transformations do not eliminate leakage

Noise injection and principal component analysis can reduce recovery, but sensitive information remains. Knowledge distillation can even increase recoverability in the tested settings. The findings motivate privacy mechanisms with formal guarantees, rather than treating an embedding transformation itself as privacy protection.

Takeaways

Embeddings are not automatically private. In our query-access audit, Cascade Probing recovers sensitive attributes even when no attributes of the target record are known.
A little side information can expose much more. Knowing just one attribute can trigger a Privacy Cliff, sharply improving recovery of the remaining attributes. Privacy audits should account for what an embedding recipient may already know.
Transforming an embedding is not a privacy guarantee. Noise, dimensionality reduction, and distillation do not eliminate leakage in our experiments. Sharing TabFM embeddings calls for explicit privacy evaluation and protection mechanisms with formal guarantees.

BibTeX

@inproceedings{wang2026auditing,
  title = {Auditing Privacy Leakage in Tabular Foundation Model Embeddings},
  author = {Xun Wang and Adam Dziedzic and Michael Backes and Franziska Boenisch},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2026}
}