An arXiv preprint reports exact representations of the maximum functions MAX5 through MAX8 using ReLU neural networks with two hidden layers. The study asks whether MAX_N(x) can be represented exactly in that form, and the reported cases cover N=5, 6, 7 and 8.
The key idea is an "atom", a reusable building block for the identity. In the proposed realization, each inner maximum of two coordinates is placed in the first hidden layer, while the outer maximum of the two resulting side-sums is placed in the second. Finite linear combinations of these atoms therefore have two-hidden-layer ReLU realizations.
The search behind the identities
To organize the search, the authors quotient atom patterns by three symmetries: reversing a pair of coordinates, permuting terms within either side, and switching the left and right sides. Patterns connected by those operations are treated as functionally equivalent, and the resulting groups are counted as orbits.
A second symmetry reduction focuses verification on a single sorted chamber C, meaning the region where the inputs are put in order. The paper states that equality checked there extends to all of R^N by permutation symmetry.
For MAX5, the paper reports an exact seven-class identity using atoms of degree k=2. It also gives a different exact representation supported on only five orbits instead of seven, and describes the pair of constructions as nonunique.
MAX6 shows the same kind of variation. The paper reports an exact seven-class identity with atom degree k=2, while a different exact representation is supported on nine orbits rather than seven.
For MAX7, the reported exact identity uses 109 classes of degree k=3 atoms. For MAX8, the paper reports an exact identity with 1,290 classes and degree k=4. The reported progression points to a much larger combinatorial search as N increases.
The linear systems, sets of simultaneous equations, behind the search also expand rapidly. Their reported dimensions go from 20 constraints and 131 variables for N=5, to 41 constraints and 144 variables for N=6, then 1,057 constraints and 4,469 variables for N=7, and 21,953 constraints and 193,623 variables for N=8.
Where the method reaches its limit
The next cases expose the computational boundary. With atom degree k=4, the N=9 system is reported to contain 51,984 constraints and 210,540 variables; the N=10 system has 112,837 constraints and 216,428 variables. The authors say these exact rational systems were computationally inaccessible because they were too large to solve within the available compute time.
There is also an unresolved count discrepancy in the MAX8 account. One passage reports 193,623 orbits, then gives the quotient's size as 193,263, while the separate system-dimensions table reports 193,623 variables. The paper still reports the 1,290-class MAX8 identity, but the conflicting orbit counts need clarification.
The paper is careful about what its certificates mean. A solved linear system is sufficient evidence for a two-hidden-layer ReLU representation, the authors say, but it is not a complete characterization. In practical terms, failing to find a solution in this atom-based search would not establish that no other two-hidden-layer representation exists.
Taken together, the preprint reports exact constructions for selected MAX_N cases, while MAX9 and MAX10 remain unresolved by this method. The document is identified as arXiv:2608.25221v1 and dated 25 Aug 2026.
Paper data and sources
Original title: Representing MAX functions using two-hidden-layer ReLU networks
Authors: Zhimao Wang, Amitabh Basu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text