SimpleEvol, a framework for automated heuristic design with large language models, led the comparison on the study's two primary optimization tasks, according to an arXiv preprint. It had the lowest AHI, the paper's index of framework handcraftedness, at 6.001, and the highest ICE, its measure of how framework performance changed with model intelligence: 2.1941 on TSP Constructive and 6.2128 on CVRP-ACO. ReEvo showed the reverse pattern, with the highest AHI at 9.112 and the lowest ICE values, 0.8230 and 2.0824.
The paper's yardsticks
AHI and ICE are not established external standards. The paper introduces AHI as a framework-handcraftedness index built from functional complexity, the diversity of LLM operations and interaction scale. In concrete terms, it counts separable stateful orchestration components, distinct types of LLM invocations and the average total number of calls on a baseline problem across models, with the call count compressed through a base-10 logarithm. ICE is the slope from a regression of framework performance against model intelligence across the evaluated range. A larger ICE therefore means a steeper fitted association in that range, not proof that the framework caused the difference.
A bounded comparison
The study evaluated 10 LLM backbones across three optimization tasks. Each framework-model combination was run independently three times, and the maximum number of generated heuristics was fixed at 820. In the routing experiments, TSP used 64 training instances at problem size 50 and three test sets of 64 instances at sizes 50, 100 and 200. CVRP used 10 training instances at size 50 and three test sets of 64 at those same sizes. That design makes the result conditional on the selected frameworks, models, tasks and search budget.
Direct results and wider tests
Under GPT-5-mini, SimpleEvol had the lowest reported gap at every listed setting. On TSP Constructive, the gaps were 4.77% at size 50, 6.47% at 100 and 9.49% at 200. On CVRP-ACO, they were 0.34%, 4.31% and 4.26% at the same sizes. These figures are limited to the reported GPT-5-mini settings.
The broader checks pointed in the same direction in several places. SimpleEvol had the highest ICE on the FSSP in-distribution set, was best on Taillard under 8 of the 10 backbone models, and had the best average gap and most first-place results on TSPLIB. In a CVRP-ACO cost-performance analysis, SimpleEvol instances using Qwen3-235B-Instruct and GPT-5-mini occupied the reported Pareto-optimal frontier, the set of best observed trade-offs between cost and performance.
Checks on the pattern
The reported ranking survived two changes to the analysis. In leave-one-out tests, ICE estimates for five frameworks stayed within plus or minus 0.07 of the original estimates, and the ordering was preserved. Replacing the paper's intelligence measure with the independent Artificial Analysis index produced the same ranking.
A Bayesian-bootstrap analysis added a second uncertainty check. Across 10,000 replicates, all eight task-framework comparisons favored a positive SimpleEvol ICE difference. The reported probabilities ranged from 79.1% to 99.9%, and five of the eight comparisons exceeded 95%. Those probabilities describe model-level bootstrap variability, not the result of a randomized treatment experiment.
The ranking also depended on search budget. From the 200-evaluation checkpoint onward, SimpleEvol had the highest ICE on both primary tasks, while rankings were less stable at very small budgets. In a size-50 ablation, the default gaps were 10.00% on TSP and 3.08% on CVRP. Removing the best-of-so-far component produced the largest listed gaps, 14.56% and 11.49%, and every other listed variant also had a larger gap than the default.
Where the pattern breaks
The clearest caution came from Taillard instances outside the synthetic training distribution. On those FSSP out-of-distribution cases, increasing model intelligence did not consistently improve performance and the fitted slopes became negative. The authors treated this as a generalization test rather than a direct AHI-ICE test. The contrast cautions against reading the primary pattern as a universal rule across data distributions.
The findings therefore remain tied to the selected frameworks, models and tasks, and to the reported search budget. AHI is defined by the paper's counts of orchestration components, LLM operation types and interactions, while ICE is a regression summary across the tested model range. The study reports support from the National Research Foundation, Singapore, under the AI Singapore Programme, award AISG3-RP-2025-036-USNSF. The document is an arXiv preprint, version 2 dated 30 September 2026.
Paper data and sources
Original title: SimpleEvol: An Agent-Loop Framework for LLM-Driven Automated Heuristic Design with Minimal Human Priors
Authors: Jianghan Zhu, Cong Zhang, Rongjie Zhu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text