A new mathematical preprint says a neural-network regression estimate can have a different theoretical learning rate when the inputs occupy a lower-dimensional shape rather than the full space in which they are recorded. In the paper’s strongest result, the dimension in the rate changes from the ambient dimension d to the manifold dimension d* when the design is supported on an exact d*-dimensional Lipschitz manifold. The quantity being controlled is expected design-integrated L2 error: squared prediction error averaged over the distribution of inputs.
This is a theorem-based study, not a report from an empirical sample. Its formal object is a theoretical sample D_n of n paired observations, (X_i,Y_i), drawn from a specified stochastic process. The document is identified as arXiv:2608.25718v1 [math.ST] and dated 26 Aug. 2026.
The dimension in the bound
In the general-support case, the theorem uses d, the ambient dimension, in the exponent. It bounds expected integrated L2 error by a quantity of order c12 times n raised to the power −2p/(2p+d)+ε, for any ε>0. The formula makes the role of dimension visible: d enters the denominator, so the stated rate depends on how many ambient coordinates are used. This is an upper bound, not a reported error, and the constant c12 and the sufficiently large thresholds required by the theorem are not numerically specified in the supplied analysis.
When supp(X) lies on a d*-dimensional Lipschitz manifold, the corresponding upper bound is c13 times n raised to the power −2p/(2p+d*)+ε, again for any ε>0. In other words, the exponent is governed by the lower-dimensional support rather than the full ambient dimension. That is a conditional geometric result, not a promise for data that merely sit near a manifold: the supplied analysis does not establish guarantees for approximate or noisy manifolds.
The guarantee is tied to one training setup
The result applies to a particular over-parametrized estimator. It combines K_n parallel fully connected subnetworks of depth L and width r through a linear output combination, uses the logistic squasher σ(x)=1/(1+e^−x) as its activation, and trains by gradient descent to minimize empirical L2 risk. The ambient-case theorem fixes the depth, width and a training parameter through formulas tied to q, p and d, rather than leaving those choices free. Its initialization assumes that all sampled weight components are independent.
The data conditions are narrow. The formal assumptions require bounded covariate support and subgaussian responses. They also require a smooth regression function, identically distributed observations, the target copy to be independent of subsequent observations, and an observed sequence that is exponentially β-mixing, a formal condition that controls how dependence persists through the sequence. In ordinary language, the framework allows dependent observations only within those restrictions; it is not a guarantee for every dependent process or for responses with heavier tails.
The proof separates three sources of error—optimization, approximation and generalization—before combining them. To address dependence, a β-mixing coupling compares a spaced subsequence with an iid copy and bounds the probability of a mismatch using the mixing coefficient. The approximation step invokes a theorem that constructs a neural-network approximant for every function in the paper’s (p,C)-smooth class. Together, these pieces connect the specified training procedure to the final bound.
What the theorem leaves open
Readers should treat this as a mathematical guarantee, not a measured performance report. The analysis uses a theoretical sample D_n and studies expected design-integrated L2 error, so it does not report prediction accuracy from an observed data set or show what a trained system would achieve on a benchmark. The rates are conditional on the stated estimator and process assumptions.
The limits are important. Constants, sufficiently large sample thresholds and finite-sample parameter ranges are not numerically instantiated in the supplied analysis. The bounds include arbitrary ε slack, and the text does not establish optimality or finite-sample tightness. Whether the rates are practically attainable with feasible network sizes and training times remains open.
The same caution applies beyond the stated design. Open questions include weaker dependence, heavier-tailed responses, less smooth regression functions, approximate or noisy manifolds, and other activations or optimizers. For now, the paper’s central claim is narrower: under its assumptions, the expected error rate is governed by ambient dimension in the general case and by manifold dimension when exact lower-dimensional support is imposed.
Paper data and sources
Original title: Estimation of a regression function from dependent data by over-parametrized deep neural networks learned by gradient descent
Authors: Michael Kohler, Adam Krzyżak, Vincent Molinero Römer
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text