A new arXiv preprint reports higher solve counts for a method that searches for computer programs than for static language-model guidance. The study directly tests whether context-aware guidance solves more tasks than static language-model-guided synthesis and examines why. In a 70-task comparison using SLIA, Narcissus ranked above static guidance at every reported search budget. With GPT-4o proposals, its genetic search averaged 51.4 solved tasks, compared with 32.2 for static guidance. With DeepSeek proposals, the corresponding figures were 32.6 and 13.8.
A second score, based on cumulative solves across the search budgets, also favored Narcissus. The normalized area-under-curve result was 32.2% versus 18.2% with GPT-4o proposals, and 17.4% versus 9.1% with DeepSeek. The measure was reported alongside the final number of tasks solved, so the comparison covered both end results and performance during the search.
One round of model help
Narcissus uses the language model once before the search rather than querying it repeatedly. It parses and repairs the proposal texts into abstract syntax trees, or tree-shaped representations of code, then mines recurring fragments into macro-rules. The repaired trees are compiled into a heuristic, or scoring guide for the search, and the search runs over an extended grammar, the set of allowed program-building rules, without further language-model queries.
The heuristic combines three signals: prefix alignment, sub-program reuse and regularization. A positive floor in its scoring keeps every grammar rule reachable, so a rule is not permanently shut out simply because it receives a poor score from the proposals.
The evaluation covered five domains: SLIA, BV, DeepCoder, ARC and ARGA. It used two search backends, cost-based bottom-up beam search and genetic top-down search. Baselines included unguided breadth-first search, breadth-first search augmented with mined fragments, direct raw proposals, three-round re-prompting and static language-model guidance. The static baselines used the same repaired proposals as Narcissus.
Proposal support varied sharply by domain. It ranged from 20% to 31% on SLIA, compared with 1% on BV, 7% on DeepCoder, 0% to 7% on ARC, and 11% for GPT-4o and 4% for DeepSeek on ARGA. The reported evaluation sizes were overlapping benchmark subsets rather than one single cohort: the SLIA comparisons included a 70-task GPT-4o subset and a full 100-task set.
The gains were not confined to strong proposals
In the low-support settings, Narcissus remained competitive with unguided search and above static guidance. On BV's divide-and-conquer evaluation, it solved 350 of 587 tasks, compared with 302 for unguided search; static bottom-up and genetic variants solved 102 and 57. On DeepCoder, genetic Narcissus averaged 32.6 and 30.6 solved tasks under the two proposal models, versus 10 for breadth-first search.
On ARGA, Narcissus was above static guidance under both proposal models: 45.6 versus 43.0 solved tasks out of 160 with GPT-4o, and 20.8 versus 8.0 with DeepSeek. On the full 100-task ARC evaluation, it solved 40 tasks, compared with 10 and 12 for the two static variants. Grammar-valid raw proposals solved 13% of ARC tasks.
The component test showed that the three signals did not contribute equally in every proposal regime. On SLIA, the full heuristic averaged 45.8 solved tasks out of 100 with DeepSeek proposals and 51.4 out of 70 with GPT-4o. Removing prefix alignment lowered those figures to 24.0 and 47.2, respectively; removing reuse produced 42.8 and 31.4. Regularization alone produced 13.4 and 15.2. The authors report that all three signals contributed, but the most useful one depended on the proposal model.
Search speed and model choice
One efficiency result measured how quickly the search reached the region of programs resembling the proposals. On SLIA with DeepSeek proposals, Narcissus reached that region approximately 12 times sooner than the static heuristic when measured by the number of programs enumerated. Reaching a proposal-like region is an efficiency result, not proof that the target task was solved correctly.
Another comparison reported a narrower gap between proposal models after search. Directly sampled GPT-4o proposals solved 31% of tasks, versus 20% for DeepSeek-V4-Flash. When Narcissus searched from DeepSeek proposals, it reached 47%, or an average of 32.6 of 70 tasks, overtaking direct GPT-4o sampling and matching the static prior on GPT-4o proposals. The re-prompting baseline used three rounds and lifted raw DeepSeek proposals by only three tasks.
What the benchmark evidence does not show
These are benchmark results, not evidence about human coding productivity, software reliability or deployment performance. The evidence is limited to the reported domains, grammars, proposal models, search backends and budgets, so performance beyond those settings has not been established. Genetic results depend on stochastic seeds; beam search can lose a solution when early heuristic rankings mislead it; and equal signal weights were selected on SLIA and reused rather than tuned for each task or domain. No confidence intervals, hypothesis tests or significance tests were reported. Mined fragments were discarded after each task, so cross-task reuse was not evaluated.
Status and access
The document is an arXiv preprint, version 1, dated 26 Aug 2026; no journal publication or peer-review status is reported. The paper says its implementation, cached language-model proposals and all result data are available at the stated GitHub repository. The supplied text lists Delft University of Technology as the authors' affiliation and reports no funding or conflict-of-interest statement.
Paper data and sources
Original title: Narcissus: Program Synthesis Using Context-Aware LLM Approximations
Authors: Tilman Hinnerichs, Sebastijan Dumancic, Neil Yorke-Smith
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text