An LLM agent workflow called KOPE reported broader coverage than CANNBot across the complete AscendC benchmark in the GLM-5.2 comparison. It accepted 897 of 1,060 cases, or 84.6%, and covered 52 of 53 operators. CANNBot accepted 613 cases, covered 37 operators and had a 57.8% pass rate. KOPE's composite score was 2004.49, compared with 1465.93 for CANNBot.
The same comparison reported KOPE's per-operator speedup geometric mean at 1.54 times CANNBot's. This is a conditional measure based on positive speedup fields returned for individual operators, not an estimate of acceleration across the full suite.
The document is an arXiv v1 preprint dated 26 Aug 2026. Its central question is how an LLM agent can turn execution feedback from exploration into reusable knowledge for later optimization steps and tasks on hardware with scarce public training data. KOPE keeps the foundation model's parameters fixed while converting compiler, correctness, profiling and performance feedback into external knowledge. Its graph memory preserves decision-to-outcome histories, and Active Context Management and Injection selects relevant experience within a bounded token budget.
The full-suite comparison
The main evaluation used AscendC kernels with CANN Bench v0.4.0 on Ascend 910C. The complete task universe had 53 operators and 20 public cases for each operator, or 1,060 cases.
With Deepseek-V4-Pro, KOPE passed 783 cases, a 73.9% full-suite rate, and posted a score of 1242.90. CANNBot passed 417 of 620 returned cases and had a 39.3% full-suite rate and score 1015.38. On the 31-operator intersection, however, CANNBot's positive-field geometric mean was 0.0485, slightly above KOPE's 0.0434. Because this metric uses a returned-operator subset with different coverage and only positive fields, it is a conditional comparison, not a whole-suite speed result.
A separate CUDA-Agent comparison recorded sparse coverage. Both runs returned all 53 requested operators, but GLM-5.2 reached 13 operators and passed 156 cases, or 14.7%. Deepseek-V4-Pro reached eight operators and passed 96 cases, or 9.1%. Neither run solved a complete operator. Each reached operator represented 12 cases, compared with 20 cases per operator in the benchmark.
The memory comparisons were uneven
In the active-versus-passive comparison, the active configuration recorded 897 accepted cases versus 636 for the passive configuration. Its pass rate was 84.6% versus 60.0%, and its score was 2004.49 versus 636.00. The positive-field geometric mean was 0.0661 versus 0.0382, a ratio of 1.73. Aggregate optimization token use was 1.113 billion versus 15.9 billion, 93.0% lower.
The graph-memory comparison reported 897 accepted cases with graph memory and 585 without it, corresponding to full-suite pass rates of 84.6% and 55.2%. Across 412 paired timing cases, the graph-enabled configuration had a geometric-mean speedup of 1.434 times the no-memory configuration.
That comparison does not isolate graph memory as the cause of the difference. The evidence consists of one full-suite correctness comparison and one 412-case paired timing comparison. Each model/system cell is represented by its best archived job, and repeated paired runs would be needed to estimate run-to-run variation.
The active-versus-passive test also changes context selection, compression and injection together. It is therefore an integrated mechanism comparison rather than an isolated test of one operation, so the numbers do not identify which operation contributed most.
Retargeting exposed a split result
The appendix tested retargeting on a common universe of 204 cases from 21 operators. GLM-5.2 was run under four knowledge settings: cold start, Ascend memory, K3 memory, and combined Ascend plus K3 memory. The design separated reuse of knowledge from the source stack from adaptation to the target.
In the source-memory comparison, cold start recorded 123 accepted cases, or 60.3%, while Ascend memory recorded 170, or 83.3%. The reported fixed-intersection geometric mean was 0.388 at cold start versus 0.145 with Ascend memory.
The target-side settings produced different results again. K3 memory recorded 182 cases, or 89.2%, and a 0.514 fixed-intersection geometric mean. Combined Ascend plus K3 memory recorded 188 cases, or 92.2%, and 0.786, which was 2.027 times cold start and 1.530 times K3 memory on the fixed intersection.
The authors interpret source-memory reuse as more reliable for feasibility than performance transfer. They limit the broader generality claim to portability of the workflow and its knowledge interface, rather than to hardware-independent performance.
What the numbers do not establish
The speed figures require caution. The per-operator measure uses positive returned fields, and the Deepseek comparison uses a 31-operator intersection with different coverage. Each model/system cell is represented by its best archived job, so the reported cells are snapshots rather than repeated-run averages.
The comparisons do not establish that a particular context-management operation or the graph-memory component caused the observed differences. The active-versus-passive test changes context selection, compression and injection together, while the graph-memory evidence consists of one full-suite correctness comparison and one 412-case paired timing comparison.
On the evidence supplied, the authors limit the broader claim to portability of the workflow and its knowledge interface, not hardware-independent performance. The GLM-5.2 KOPE result itself was 52 of 53 operators and 897 of 1,060 cases, so the reported benchmark result did not cover the entire suite.
Paper data and sources
Original title: Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory
Authors: Siyuan Chen, Runlin Hou, Shenxiu Wu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text