A compact plant-health model kept its overall accuracy after quantisation and packaging, but gave up a small amount of class-balanced performance compared with the reference model, according to a locked internal test. The float student and its final serialised PTE both scored 98.46% accuracy. Their macro-F1 scores, the class-balanced measure used here, were 96.27% and 96.23%, while the DINOv3 reference scored 96.87%.
The paper's wider test was whether duplicate families could be kept out of the final partitions, how much class-balanced performance a compact MobileNetV4 lineage retained from a DINOv3 reference, and whether the selected quantised graph survived direct ExecuTorch and XNNPACK execution.
On the same 16,363 rows, the reference and PTE produced different top-1 predictions, meaning their highest-ranked labels, on 240 rows. The reference was uniquely correct on 105 of those rows, while the PTE was uniquely correct on 97. Another 38 rows were wrong under both systems, though the systems chose different labels. In all, the PTE made 252 errors, eight more than the reference; an exact two-sided McNemar test, which compares paired correctness, produced a p-value of 0.622.
The benchmark was rebuilt before the comparison
The final benchmark was derived from 117,546 audited source images and 121 candidate labels. Reconstruction left 109,107 images across 120 operational classes, divided into 76,376 training rows, 16,368 validation rows and 16,363 test rows. The class sizes were sharply uneven: the largest class was 151.7 times the smallest.
The duplicate audit found 8,672 trusted relationships. It identified 3,233 historical cross-split links, including 2,876 exact and 357 near-duplicate relationships. The review reopened 2,031 affected duplicate-detection links, retained 1,932 and rejected 99, while direct-cover deduplication removed 8,355 files.
For the final freeze, leakage groups were assigned to target shares of 70% training, 15% validation and 15% test, using seed 42 and five group-safe folds. The 109,107 unique identities formed 109,090 audited trusted groups, and none spanned all three partitions. That excludes the confirmed group crossings found in the audit, but it does not rule out duplicate-detection false negatives.
The software artifact held together
The reference was a DINOv3 ConvNeXt-Tiny with a 120-way classifier, 256 by 256 pixel inputs and a fully fine-tuned backbone; its checkpoint was selected by validation macro-F1. The compact state used MobileNetV4 Conv-Medium with a 120-way linear classifier. Because no matched direct MobileNetV4 run was available, the teacher-guided lineage is reported descriptively, and the study makes no causal claim about a distillation gain.
Quantisation, the conversion to the INT8 graph and then the packaged PTE program, was chosen using validation data only. Three predeclared XNNPACK-compatible candidates were evaluated. The selected scheme used dynamic activations with per-channel weights and reached 96.4882% validation macro-F1 with zero calibration rows. A static candidate with 2,880 calibration rows passed, while one with 1,440 rows failed.
The final PTE occupied 23,696,352 bytes, or 22.5986 MiB, and was 31.92% smaller than the 34,805,405-byte pre-quantisation state file. It generated predictions directly on all 16,363 locked test rows. Across the four states, accuracy and macro-F1 were 98.51% and 96.87% for the reference, 98.46% and 96.27% for the float student, 98.45% and 96.25% for the converted INT8 graph, and 98.46% and 96.23% for the PTE.
The trade-off was not evenly distributed
The modest headline difference became clearer in class-balanced analysis. Paired estimates put the PTE's macro-F1 0.643 percentage points below the reference, with a 95% interval ranging from 1.239 points below to 0.047 points below. In a separate fruit-category slice, mean recall was 6.965 points lower for the PTE, with a 95% interval from 1.711 to 12.409 points lower. The fruit analysis was post hoc and hypothesis-generating, so it points to a place for further checking rather than a settled explanation.
Average changes by training support pointed in the same direction. Macro-F1 fell 2.56 points for classes with fewer than 100 training examples and 2.78 points for classes with 100 to 199. It changed by 0.06 points lower in the 200 to 499 group and 0.09 points lower in the 500 to 999 group, while classes with at least 1,000 examples rose by 0.21 points.
The locked evaluation was authorised only after the configuration, class map, preprocessing and evaluation plan had been frozen. Reference-to-PTE uncertainty was estimated by repeatedly resampling within each class, producing 2,000 paired bootstrap replicates, using seed 20260821 and percentile limits at the 2.5th and 97.5th quantiles. Those intervals describe the fixed test composition, not the variation that might appear if the models were retrained.
The result stops at the benchmark
The evidence remains internal to the reconstructed source pool. It does not establish performance on unseen farms, regions, cultivars, cameras, backgrounds or acquisition protocols. Direct PTE execution also does not establish Android latency, memory, energy use, delegate fallback or thermal behaviour.
The consolidated image corpus is not distributed because complete image-level provenance and redistribution rights were not reconstructed. The companion repository is described as a release candidate pending an immutable preprint tag and release manifest.
The manuscript does not report funding, and its metadata lists no funders. Taken together, the findings describe benchmark control and runtime fidelity on the locked rows, not performance beyond the reconstructed source pool.
Paper data and sources
Original title: CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact
Authors: Rana Muhammad Ahmed, Sabahat Abbas
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text