Changing numerical precision and changing the network architecture did not behave as interchangeable remedies in a controlled preprint study of physics-informed neural-network runs. The researchers varied precision and the stopping tolerance used by the L-BFGS optimizer, then compared those changes with a different backbone and an added alignment objective. They asked whether the two routes reached different sets of failures, and whether observed recovery tracked the alignment objective or the underlying state-space backbone. The paper frames precision as acting on the optimizer's stopping-test axis, while alignment acts on the hypothesis-class axis, meaning the family of functions the network can represent.
The evidence came from a local controlled study of 144 runs and an independent validation study of 85 runs that consumed 55.7 run-hours. Those figures describe the study's coverage, not one combined sample: the local research-question slices overlap, so effective sample sizes differ by question. Within a matched cell, variants shared the PDE instance, collocation grid, training budget, L-BFGS configuration and seed. The registered evaluation used 5,000 Adam steps, up to 1,000 outer L-BFGS steps, a 101 by 101 collocation grid and a 501 by 501 evaluation grid. Success meant an unrounded relative ℓ2 error, a grid-wide error score, below 0.05.
Alignment was most closely linked to hard-convection recoveries
The clearest split appeared in the hard-convection slice. With five seeds per cell, FP64 moved the vanilla MLP from 0/5 successful runs in FP32 to 1/5 in FP64. The unaligned state-space model, or SSM, stayed at 0/5 in both precisions, while the aligned SSM reached 2/5 in FP32 and 3/5 in FP64. In every reported hard-convection cell, the unaligned SSM's relative ℓ2 error was above 1.0 and did not improve on the vanilla MLP. The authors interpret the recoveries as tracking the alignment objective rather than the SSM backbone alone.
Reaction produced a different picture. The unaligned SSM succeeded in 3/5 FP32 runs and 4/5 FP64 runs, while the aligned SSM succeeded in all five runs at both precisions. In other words, the backbone performed well on this test without alignment, even though adding alignment increased the observed success count. Because the result rests on five seeds per cell, it is a description of this slice, not a precise estimate of a general success rate.
The alignment-weight sweep filled in the hard-convection result. In the convection FP32 sweep, success was 0/5 at α=0, 100 and 300, then 2/5 at α=1,000 and 3/5 at α=3,000. Median relative ℓ2 error fell from 0.972 at α=300 to 0.801 at α=1,000 and 0.041 at α=3,000. The response looked threshold-like rather than like a precisely calibrated phase transition, however, because the sweep used five seeds on a coarse grid.
Precision came with a trade-off
The precision trade-off was visible in an independent repeated-step convection MLP sweep. Every FP32 and FP64 tolerance cell had 1/5 success. As the L-BFGS tolerance tightened from 10^-7 to 10^-9 to 10^-11, median relative error changed from 1.006 to 0.720 and then stayed at 0.720 in FP32; in FP64 it changed from 1.005 to 0.314 and then stayed at 0.314. Runtime rose substantially: FP32 went from a starting 17 seconds to later reported values of 278.1 and 275.7 seconds, while FP64 went from 120.8 seconds to 2,573.3 and 2,569.6 seconds. The runner differs from a single-call L-BFGS implementation, and inner-iteration and termination logs were absent.
The independent wave-MLP slice showed why seed choice matters. Across 10 paired seeds, success was 4/10 in FP32 versus 6/10 in FP64, and median relative ℓ2 error was 0.125 versus 0.045. The paired outcomes ran in both directions: three seeds succeeded only in FP32, five only in FP64, one in both and one in neither. The exact McNemar test put the success-rate comparison at p=0.727, which was not statistically significant; a paired Wilcoxon test on log error gave p=0.375.
The comparison has clear boundaries
The pass-fail threshold also changed the apparent rate. Across a 60-run local convection/reaction slice, the success rate rose from 25% at a relative-error threshold of 0.03 to 48% at 0.07. It was 35%, 40% and 45% at 0.04, 0.05 and 0.06, respectively. Aggregate regime ordering stayed stable, but individual near-threshold seeds changed status. A separate descriptive classifier examined 63 failures in a subset of 59 convection and 28 reaction runs. Trajectory-related proxies were dominant, but the classifier omitted two registered conditions, lacked L-BFGS logs for 41 of 87 records, used final relative error for S3 and was still awaiting rescue validation.
One architecture comparison needs especially careful reading. In an unequal-seed convection protocol comparison, PINNMamba recorded 2/5 FP32 and 3/5 FP64 successes, while PINNsFormer recorded 0/5 FP32 and 0/3 FP64. Those figures are protocol-level results, not a clean architecture-only test, because the seed counts were unequal.
The reported archive makes the analysis easier to check: it includes all seeds, the unrounded success rule, a run manifest, registration, proxy and independent-validation files, implementation, provenance and pinned-environment artifacts. The analysis plan was registered on May 28, 2026, after preliminary experiments and before the study expanded. The document is an arXiv preprint, version v1, dated 26 Aug 2026; no journal or repository venue is reported.
Taken together, the tests suggest a division between the remedies. Precision and stopping changes and architecture and alignment changes covered different regime-and-seed slices in the reported runs, while the SSM backbone alone did not improve the hard-convection result. The practical implication is a joint comparison that keeps per-seed error and runtime visible. The findings are a warning against treating either FP64 or an architecture change as a universal answer beyond the selected tests.
Paper data and sources
Original title: Neither Precision Nor Architecture Alone: Controlled Tests of Failure Remedies for Physics-Informed Neural Networks
Authors: Jinyuan Zhang, Peng He, He Hu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text