An external software harness that evolves its diagnostic resources recorded the highest reported top-1 result in a comparison of AI systems for root-cause analysis, the search for what caused a software problem, reaching 59.0% on two public benchmarks. Top-1 accuracy means the correct cause was ranked first. The same evaluation reported 41.4% for the harness without evolution, 38.4% for an in-context-learning setup, 36.1% for a bare general agent, 17.9% for RCA-Agent and 5.6% for mABC.
The work is presented as an arXiv version 1 preprint dated 26 Aug 2026. Its central question was whether a general agent could use an external, self-evolving harness that reuses general-agent capabilities instead of rebuilding a specialized root-cause system.
A general agent with an operating layer
OpsHarness separates the supporting material from the process that runs it. Its data plane holds layered operational knowledge and idea-card tools; its control plane handles setup, diagnosis, evolution and verification. In the study's framing, this external layer is tested as an alternative to rebuilding a specialized agent.
For the public evaluation, the researchers used OpenRCA's 335 real-world cases and RCAEval's 270 labeled cases. They used the earlier 80% of cases for warm-up and held the last 20% in time order for testing. A guard script blocked access to ground-truth labels during diagnosis.
Checks against self-reinforcing errors
In a continuous-diagnosis comparison, the final-window numbers were 0.83 for full OpsHarness and 0.43 for the no-evolve version. A@1 is the same first-choice accuracy measure used in the benchmark comparison.
The comparison also tested the harness without its verification gate. That variant finished with final-window A@1 of 0.33, below 0.43 for no-evolve. The verification gate rejected 37% of evolution proposals, and accepted proposals required 1.7 attempts on average.
The paper also evaluated the quality of evolved workflow artifacts. It reports six workflow skeletons, with recall of 91.7% and precision of 73.5%. Recall measures how much of the target workflow set was captured, while precision measures how much of the output counted as a correct match.
More machinery, but not a bigger diagnosis bill
Per-case diagnosis costs were close to the general-agent baselines. OpsHarness used about 106,000 tokens, the text units counted for model use, and 325 seconds per case, versus 112,000 tokens and 317 seconds for Direct and 106,000 tokens and 308 seconds for ICL. Specialized agents used 1.7 to 2.7 times as many tokens and 2.3 to 3.5 times the elapsed time.
The wider harness cycle carried separate setup, evolution and verification costs. In one pass, setup used 0.82 million tokens and 412 seconds, evolution used 0.75 million tokens and 184 seconds, and verification used 1.55 million tokens and 421 seconds.
The company data test
On a separate industrial dataset from Company A, containing 773,340 data points and 88 confirmed anomalies during change windows, OpsHarness averaged 0.74 for A@1 across six configurations, versus 0.24 for Direct. OpsHarness was higher in all six configurations.
Those results give the study an industrial test, but the evidence remains narrow. The analysis covers two public benchmarks and one industrial deployment, and the comparison is limited to the reported models, frameworks, datasets and configurations.
What the figures leave open
These are descriptive results. The supplied analysis reports no confidence intervals, variance estimates or inferential tests, so the gaps should be read as point estimates from these evaluations rather than universal performance guarantees.
On that evidence, the preprint supports a narrower conclusion: OpsHarness performed better than the tested alternatives in the reported public and industrial comparisons. It does not show superiority over every root-cause method, indefinite gains, or that evolved artifacts are always correct.
Paper data and sources
Original title: From General Agents to RCA Experts: A Self-Evolving Harness for Root Cause Analysis
Authors: Haiyu Huang, Jiewei Lyu, Zhihan Jiang et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text