An arXiv preprint reports that a decoding method called AsymSpec reached about 90% of full-context accuracy on average in isolated text capabilities. Its reported throughput was 1.3× to 1.7× the full-context reference, while compute was 0.2× to 0.3×. These are aggregated point estimates, and the report gives no confidence intervals for the headline average.
Two parts of the decoder see different context
AsymSpec gives the two parts of its decoder unequal views of the same input: the drafter reads the full input, while the verifier operates on a compressed view. The method combines contrastive δ-fusion of logits with a divergence-aware acceptance gate to steer the verifier. In the paper’s comparisons, the compressed verifier is called the Floor and the full-context verifier the Ceiling.
The headline runs used a speculation depth of two, a fusion weight of 1.0 and a base threshold of 0.5.
The biggest gap appeared under tight truncation
The evaluation covered separate benchmark sets: LongBench with 600 instances, MultiChallenge with 271, API-Bank with 200, MathVista with 587, full GAIA L1–3 with 165 and SimpleQA with 500. Those counts were not pooled into one analytic sample.
Across five isolated text cells, AsymSpec scored 64.0 on hotpotQA, 66.8 on 2WikiMQA, 48.4 on MuSiQue, 23.5 on MultiChallenge and 63.5 on API-Bank. Average throughput was 1.45× the Ceiling’s, while normalized FLOPs were 0.23× the Ceiling’s. No confidence intervals or significance tests were reported for these comparisons.
The LongBench truncation sweep showed the clearest separation when the verifier had the smallest context budget. At 500 tokens, AsymSpec scored 52.5 F1 against 25.8 for the Floor, a 26.7-point difference. At 12,000 tokens, the scores were 63.9 and 63.1, narrowing the difference to 0.8 points. The reported advantage was therefore largest under severe truncation and nearly disappeared as more verifier context was allowed.
The tests also reached visual and live tasks
The comparison extended to multimodal reasoning on MathVista. AsymSpec reached 53.9% overall, 10.1 percentage points above symmetric speculative decoding. Its gains over the Floor were 10.0 points on visual-question answering and 16.7 points on free-form question answering. The reported cross-modal gain was concentrated in visually grounded task types, and a text-only full-context Ceiling was not defined in this setup.
An implementation check for the described vLLM setup reported 30.5% accuracy without the specified patches and 53.0% with them. The comparison was specific to that implementation, so the cross-modal figures depend on implementation details.
In live agent loops, the 4B AsymSpec configuration scored 24.2 on the full GAIA run and 65.0 on SimpleQA. Its normalized speedup and FLOPs were 1.41× and 0.78× on GAIA, and 1.38× and 0.80× on SimpleQA, respectively.
During online context recompression in GAIA and SimpleQA, reported draft acceptance remained between 0.88 and 0.90.
Performance depended on the setup
A LongBench ablation compared the method’s components and alternatives. The Floor scored 45.0; CDA alone, with β = 0, scored 52.8; raw augmented logits scored 56.9; SCD-style fusion scored 48.0; the proposed a−b δ-fusion scored 59.7; and the Ceiling scored 65.5. Among the listed non-Ceiling variants, the proposed fusion had the highest score.
Results also varied with the compression method and the drafter’s size. Across summarization, LLMLingua-2 and truncation, reported Floor–Ceiling recovery ranged from 63% to 70%. SCD was below the Floor in every cell. Drafter models at or below 0.6 billion parameters were unreliable at extracting signals, while the preprint described 1.7 billion parameters or more as the practical minimum.
Cross-family LongBench recovery varied by the drafter–verifier pairing: 23% for Llama-3B to Llama-70B, 50% for Qwen-4B to Llama-70B, 10% for Llama-3B to Qwen-32B and 72% for Qwen-4B to Qwen-32B. The preprint notes that heterogeneous runs require vocabulary and logit-space alignment, and the reported performance was pair-dependent.
The authors state that AsymSpec does not preserve a strict target distribution and is calibrated for greedy emission. The evaluation reports point estimates without formal inferential uncertainty measures, so the findings should not be read as a guarantee of near-ceiling accuracy for every task, compressor, model pairing or decoding regime.
The document is arXiv:2608.26004v1, dated 26 August 2026, and is presented as a preprint.
Paper data and sources
Original title: AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs
Authors: Sheng Liang, Yongyue Zhang, Nathanael Brian et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text