A pair of verification checks gave the same answer for every test in a satellite-catalogue software pipeline, but the agreement did not mean the answer was right. In the published run, all seven comparisons matched, while three of the counts were wrong. One count was reported as 932 when the comparison value was 220.
The case study examined a dual-computation gate using two openly retrievable registers of artificial Earth-orbiting objects. The files were retrieved on 18 August 2026. CelesTrak carried 70,292 objects, while GCAT's main catalogue contained 69,999 records covering 69,391 numbered objects. The focus was whether verification paths built with different technologies could still agree when they shared the definitions used to classify the source data.
When agreement hides a shared mistake
Each defect class was counted twice. A set-based Python path worked over the source files, while SPARQL, a way to query linked data, counted records in the RDF graph emitted by the pipeline. The gate was configured to exit with a failure if the two answers disagreed, and it covered seven defect classes.
The paths differed in implementation but not in their basic interpretation. Both imported the same classification constants, and those constants misread GCAT's flight-phase status vocabulary. The two routes therefore repeated the same semantic mistake, a common-mode failure, instead of testing the source meaning independently.
The largest discrepancy involved a disposition count, the number assigned to one status category. The duplicated gate accepted 932 even though the comparison value was 220. That number should not be read as settled ground truth. When the researchers followed full object histories, the first correction, 261, fell to 220. Forty-two objects dropped out of the tally: 30 linked to explosions and 12 to collisions.
The checks sent mixed signals
Additional checks exposed some of the problem, but not all of it. A total-function check flagged nine unclassified codes among 1,099 rows. A residue decomposition split the disputed cases into 20 groups, with the five largest accounting for 729 of the 932 cases. Yet source-inventory reconciliation was silent when limited to the four main catalogues. A separate SHACL validation total of 3,564 was not independent corroboration because it was an arithmetic identity calculated from the same graph.
Other comparisons also changed with the scope of the data. The coverage-gap count was 900 in the published result and 622 in a later comparison. A fourth catalogue listed on the same index contained 354 objects but was not fetched. The tracking-loss comparison was 1,104 against 1,094.
What the case says about independence
The authors' conclusion is narrower than a warning against duplicated checks. Two routes can show that two implementations behave alike. What they may fail to test is whether both implementations inherited the same definition. The proposed remedy is to check shared definitions against the original source, while recognising that this is necessary but not sufficient.
The evidence is tightly bounded. It comes from a single pipeline, and the source-oriented checks were evaluated on one codebase against two errors. Neither catalogue was treated as ground truth, and the event-history reference data were retrieved later than the original files. The study therefore describes one detailed failure pattern rather than establishing how often similar failures occur elsewhere.
The practical caution is simple: technical diversity below a shared definition can create the appearance of independence. A gate may be rigorous about whether two calculations match and still leave the original interpretation untouched.
Paper data and sources
Original title: Independent Verification Paths Are Not Independent: A Case Study of Common-Mode Failure in a Satellite Catalogue Pipeline
Authors: Fabio Rovai
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text