The arXiv version 2 preprint reports that replacing LassoNet's usual mean-squared-error objective with robust alternatives often improved prediction and feature selection when response data were contaminated. The gains were most pronounced in highly nonlinear simulations and appeared again in two real-data tests, although the authors found no single robust loss that was best in every setting.
The proposed method keeps LassoNet's hierarchical sparsity and proximal optimization, but replaces MSE with Huber, Cauchy, Tukey's bisquare or Nonnegative Garrote loss. In practical terms, the comparison asks whether the same feature-selecting neural-network framework behaves differently when it is trained with different ways of scoring error.
To test the idea, researchers ran four regression models with 750 training observations and 200 independently sampled standard-uniform covariates in each simulation. Only an active subset of those inputs contributed to the regression function. In the listed non-Gaussian scenarios, the contamination proportion was 0.1, while variance contamination used multipliers of 100 or 625. The evaluation tracked predictive error, false-positive and false-negative rates, and the number selected rate.
Model fitting used cross-validation along the regularization path, with each candidate model judged using its own loss. The optimization used a hierarchical proximal-gradient procedure, Adam during the initial dense phase and stochastic gradient descent with momentum 0.9 on the path, with ReLU activations.
Where contamination changed the picture
The clean-data check was reassuring but not uniform. With no contamination, robust losses and MSE had similar predictive error in Models 1, 2 and 4. Model 3 was different: MSE delivered the lowest predictive error, but it selected substantially larger models.
Under moderate and strong mean contamination, Tukey's bisquare loss had the lowest predictive error in Model 1, at 0.320 and 0.308. In Model 2, Huber and Tukey were nearly tied: 0.262 versus 0.263 under moderate contamination, and 0.265 each under strong contamination.
The more nonlinear cases showed bigger gaps. In Model 3, Cauchy had the best moderate-contamination error at 0.428; Tukey led under strong contamination at 0.397, with Garrote at 0.403. Model 4 gave Tukey a predictive error of 0.240 at both mean-contamination levels. Its false-negative rate was 0, compared with 0.816 for MSE under moderate contamination and 0.992 under strong contamination.
Variance contamination produced a similar warning for the standard objective. Under the strongest variance setting, MSE's predictive error across Models 1 through 4 was 0.626, 0.639, 1.242 and 0.666, while its false-negative rates were 0.536, 0.573, 0.833 and 0.756. The robust variants were reported as more stable.
The signal held in two data applications
The authors then turned to two secondary datasets. Real-data performance was averaged over 50 random splits, with 80% used for training and 20% for testing. Their trimmed predictive error excluded the largest 5% of absolute residuals, and a variable was highlighted in the selection summaries only when it appeared in more than 70% of runs. The Top Gear data had 242 observations, with 29 original variables expanded into 40 encoded covariates.
On Top Gear, the Garrote version of LassoNet had the lowest robust-LassoNet trimmed error, 0.186, compared with 0.570 for MSE. Random Forest and XGBoost selected fewer covariates, with number-selected rates of 0.259 and 0.321, but their trimmed errors were 0.214 and 0.213.
The second application used an atherosclerosis gene-expression dataset. It took TLR8 expression as the response, retained the 60 genes with the strongest robust-correlation association, and fit a network with one hidden layer of five units.
Here every robust LassoNet loss had lower trimmed predictive error than MSE: 6.759 for Huber, 6.598 for Cauchy, 6.535 for Tukey and 6.532 for Garrote, versus 10.356 for MSE. RLARS was the sparsest comparator, with a number-selected rate of 0.103, but its trimmed error was 7.117.
A result with clear boundaries
The findings come with important boundaries. The authors do not identify a universal winner: their conclusion is that the best robust loss depends on the model and contamination pattern. The reported comparisons are averages, without confidence intervals or formal significance tests, so the numerical gaps should be read as evidence from these tests rather than as a final ranking.
The real-data numbers also reflect the study's setup: trimmed error removed the largest 5% of absolute residuals, while the atherosclerosis analysis began with a selected set of 60 genes. That leaves open whether the same pattern would hold with other data or contamination mechanisms.
The authors say code reproducing the method and results is publicly available. No funding source is reported in the supplied front matter or metadata.
Paper data and sources
Original title: Robust LassoNet: Enhancing Feature Selection in Neural Networks via Robust Loss Functions
Authors: Daniela De Canditiis, Italia De Feis, Paola Stolfi
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text