The classical data processing inequality can fail in constrained learning settings, according to a theoretical analysis. In a constructed comparison involving full corruption of the attribute X, the corrupted-to-original Bayes-risk ratio is exactly 0.5, and the constrained inequality therefore does not hold in that construction.
The paper studies formal learning problems involving finite label spaces, standard-Borel attribute spaces, bounded measurable losses, model classes represented by Markov kernels and joint probability distributions. In the paper's terminology, those kernels are the probability rules used to represent the permitted models and the corruptions being compared.
Inside the counterexample
That counterexample is tightly constrained: the attribute and label spaces are binary, the prior has full support with p between 0 and 1, and the model class contains only one element.
Bayes risk here means the best expected loss available within the allowed model class. In the example, the corrupted problem's Bayes risk is half the original risk, with a corrupted-to-original ratio of 0.5. The comparison therefore breaks the expected constrained data-processing ordering.
That example sets up the paper's main question: what conditions on the loss, the allowed models and the corruption are sufficient for the desired risk ordering? The paper addresses that question by defining a generalized data processing inequality, or GDPI, for constrained learning.
From risk to geometry
GDPI requires constrained Bayes risk on the original joint distribution to be no greater than constrained Bayes risk after any joint Markov-kernel corruption. The definition covers corruption of attributes, labels or both.
To analyze that comparison, the paper uses constrained superprediction sets, which describe the loss profiles available from permitted models. It pairs these with concave support functions, which summarize the sets from different directions; the support function equals constrained Bayes risk.
Under the stated assumptions, inclusion of a convexified superprediction set is equivalent to the corresponding support-function ordering and, for the loss and model pairs in the framework, to Bayes-risk ordering across all distributions in the framework. For a fixed loss and model class, GDPI under a joint corruption is equivalent to containment of the corrupted convexified set within the original convexified set.
Symmetry conditions in the framework
One sufficient label-side condition combines an F-symmetric loss, an F-label-invariant model class and an F-label corruption. The paper states that this combination is sufficient for GDPI across all stated distributions.
Another sufficient condition combines a symmetric loss, a Y-permutation-invariant model class and a bistochastic label corruption. It is likewise stated as a sufficient GDPI condition, subject to those symmetry and invariance assumptions.
On the attribute side, F-attribute invariance paired with F-attribute corruption is sufficient for GDPI. An analogous bistochastic attribute result assumes invariance under permutations of the attribute space.
The paper also gives an example in which GDPI holds for all stated distributions and losses even though the model class is not X-permutation invariant. That shows the invariance condition is not necessary in general.
The familiar result as a special case
The construction recovers the classical data processing inequality in the standard unconstrained setting. It does so with the corresponding unrestricted model class and a deterministic-kernel decomposition.
In the proposition's stated setting, ordinary GDPI, strong GDPI with an inflation coefficient of at least 1 and minimal relative risk of at least 1 over the corrupted set are equivalent.
A formal preprint
The work is a preprint identified as arXiv:2608.25745v1 [cs.LG] and dated 26 Aug 2026. Its scope is formal, covering finite-label learning problems, standard-Borel attribute spaces, losses, Markov kernels, constrained model classes and joint probability distributions.
Paper data and sources
Original title: Comparing Corrupted Constrained Learning Problems
Authors: Laura Iacovissi, Rabanus Derr, Robert C. Williamson
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text