A new method for auditing artificial-intelligence models appears to do a better job of separating concepts a model actually uses from concepts that are merely correlated with its predictions. In synthetic tests, ICON produced smaller errors than the strongest competing method in two hidden layers of a neural network, while a conventional partial R-squared measure was slightly better at the final output.
The distinction matters because a concept can be easy to detect inside a model without being the reason the model makes its decisions. ICON is designed to assess each supplied concept while accounting for the other concepts and the model’s outcome, and to report both redundant and unexplained parts of the representation.
Less credit for misleading correlations
In the ToyBrains simulations, researchers supplied 30 concepts to each explanation and compared results across 90 models, using 500 test samples per explanation. They measured the gap with total-variation distance, where a smaller number means the estimated importance pattern is closer to the known answer. At the two hidden layers, ICON’s error was 0.33 versus 0.52 for the best baseline at HL-2, and 0.23 versus 0.38 at HL-1. At the scalar output, partial R-squared, which measures the extra variation associated with one concept after the others are accounted for, was slightly more accurate, with an error of 0.14 versus 0.16 for ICON.
ICON also gave far less weight to concepts that had no true role in the simulations. At HL-1, its error for those zero-role concepts was 0.06, compared with 0.16 for partial R-squared and 0.36 for CAV-signal. When a spurious correlation rose from 0 to 0.6, false importance increased seven- to 12-fold for linear probes and seven- to 17-fold for marginal R-squared, but only 1.2- to 1.3-fold for ICON.
The advantage held as the explanation became more crowded or data became scarcer. When the supplied concept set grew from five to 100 concepts, ICON’s HL-1 error never exceeded 0.22, while linear-probe and CAV-signal errors rose to 0.72 and 0.85. ICON also had the lowest error in the small-sample tests.
A test with skin images
The researchers then tested the method on 24,536 dermoscopic images from the ISIC 2019 dataset, using nine artifact concepts and a pretrained VGG-16 model. ICON highlighted features such as camera reflection, band aid, skin marker and red color in the layer just before the final output, where they accounted for 9.8%, 3.4%, 3.0% and 2.8% of the model’s internal variance, respectively.
In controlled checks, ICON responded when the model was made increasingly reliant on a microscope-related shortcut: its importance score rose from about 0.00 to 0.44, while the baseline changes ranged from 0.02 to 0.15. For an unseen timestamp artifact, ICON stayed below 0.01 even as timestamp and microscope information became more correlated; CAV-classic ranged from 0.005 to 0.11 and CAV-signal stayed at 0.09.
What the brain-scan tests revealed
The most pointed examples came from two UK Biobank T1-weighted structural-MRI tasks: classifying binge drinking and estimating brain age. Both used five-fold stratified cross-validation and the same supplied set of 30 concepts.
For binge drinking, a 3D ResNet-18 reached 60% accuracy, with a standard deviation of 0.5 percentage points across folds. ICON attributed nearly all of the concept-explained final-layer variance to sex and essentially none to the outcome. After retraining with sex balanced, accuracy fell to 53.3%, with a standard deviation of 0.5 percentage points across five folds on 1,270 hold-out participants. An SFCN replication reached 61.2% accuracy.
For brain-age prediction, the SFCN reached a mean absolute error of 2.89 years, with a standard deviation of 0.02 years, and an R-squared score of 0.750, with a standard deviation of 0.004, across five folds on 5,000 hold-out participants. ICON attributed most of the model’s final internal variance to age, left about 36% unexplained, and assigned less than 1% to the other supplied concepts combined.
A further validation challenge produced little support for most of the other candidate concepts. Six of eight selected non-age concepts could be tested in matched checks that removed their usual correlation with age, and none produced a robust change in prediction error: the largest absolute change in mean error was 0.069 years, and none was significant across all five folds. ICON assigned each of those six concepts less than 0.001 importance; among the two concepts that could not be tested that way, employment status had a partial R-squared of about zero, while acquisition date retained a partial R-squared of about 2.4%.
A useful audit, with boundaries
The authors present ICON as a way to turn a long list of detectable concepts into a shorter set of testable audit hypotheses. But it is still a statistical audit of associations, not a causal experiment: its scores do not automatically show that a concept causes a model’s behavior, and the unexplained share cannot tell whether the variance comes from missing concepts or from more complex representations inside the model.
The evidence here covers only the ToyBrains simulations, ISIC image classifiers and two UK Biobank MRI tasks, using their particular datasets, architectures and supplied concept sets. The real-data analyses are audits of those models, not clinical trials or direct evidence of human biomarkers. The manuscript is an arXiv version 1 preprint dated 26 August 2026.
Paper data and sources
Original title: ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing
Authors: Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text