WavePP, a system for processing LLM input prompts before generation, recorded higher measured throughput than TensorRT-LLM's PP4 setup in 37 of 40 tested settings for GLM 5.2 and MiniMax M2.7. The result included every cold and mixed setting. At concurrency 128 with high cache reuse, the reported throughput factors relative to that baseline were 2.91 for GLM 5.2 and 2.02 for MiniMax M2.7.
A benchmark built around long prompts
The document is an arXiv Preprint, version 1, dated 28 Sep 2026. Its core evaluation reported 40 settings on GLM 5.2 and MiniMax M2.7 and 28 settings on Kimi K3. Those figures are workload or concurrency settings, not a count of independent participants.
WavePP was compared with pipeline-parallel and tensor-parallel serving baselines under cold prefill, shared-prefix reuse, where repeated prompt material is already cached, and mixed workloads. The main measure was aggregate prompt throughput, including cached input tokens, alongside prefill completion time that includes admission and queueing. For Kimi K3, completion time was reported at p50, the median, and p95, a slower-end percentile, across requests.
For the PP4 comparison, the systems used one configuration across workloads and kept the pipeline partition identical. Results were summarized as medians from three independent server boots, with runs interleaved on the same node. The paper did not report confidence intervals or significance tests for the comparison.
Where throughput pulled ahead
Kimi K3 produced a similar pattern. WavePP had the highest measured throughput in 21 of 28 settings, including all 18 settings at concurrency 8 or higher. Across the full grid, its geometric-mean throughput, a way of averaging ratios across settings, was 9.2% above SGLang PP8 and 15.7% above vLLM PP8.
For cold Kimi K3 inputs of 65.5K, 131K and 262K tokens, WavePP's geometric-mean throughput was 58.8%, 49.1% and 37.8% above TRT-LLM TP8/EP8, respectively. At concurrency 32, the measured rates were 44,643, 35,878 and 28,031 input tokens per second for those three input lengths.
For 262K-token seeded-reuse requests, WavePP's geometric-mean throughput was 11.1% above TRT-LLM TP8/EP8, 22.2% above SGLang PP8 and 45.3% above vLLM PP8. At concurrency 32, it served 218,743 input tokens per second.
Latency and retention
Throughput and completion time did not move together in every measure. In cold Kimi K3 tests at concurrency 8 or higher, WavePP's p50 completion time was 31.6% to 49.0% lower than TRT-LLM TP8/EP8 and below both pipeline-parallel baselines in every such setting. Its p95 was lower than both PP baselines except for the 65.5K-input test at concurrency 8, where it was 1.6% higher than vLLM PP8.
On mixed workloads, WavePP had the highest measured throughput at all three tested concurrencies. Its geometric-mean throughput was 5.0% above SGLang PP8 and 14.6% above vLLM PP8. A separate prefix-retention test reported good-hit rates of 88% to 100% for WavePP across offered arrival rates of 0.50 to 1.00 requests per second, compared with 73% to 83% for TRT-LLM TP8/EP8, and WavePP was higher at every tested rate.
The trade-offs
The component tests added a more mixed picture. In a cold 131K-token test at concurrency 16, wave admission and transport were followed by throughput changing from 8,826 to 29,692 input tokens per second, a reported 236.4% increase; later changes had little throughput effect. In a reuse 131K-token test at concurrency 8, leases and capacity escrows were followed by a change from 96,796 to 149,021 input tokens per second, reported as 54.0%, while the later MPU scheduling phase reached 211,214 input tokens per second, an additional reported 41.7%; p50 completion time changed from 6.51 to 4.20 seconds.
But the combined MPU phase was not uniformly helpful. In the reuse 262K, concurrency-32 ablation, throughput changed from 175,305 to 158,238 input tokens per second, a reported 9.7% decrease, although it remained 24.5% above TP8/EP8. The paper's aggregate throughput measure counts cached input tokens, so it describes input served rather than only newly computed tokens.
Taken together, the results are strongest as a report about the tested settings. They show higher measured throughput in many high-concurrency and prefix-reuse cases, but the latency exception and the MPU regression show why the result cannot be reduced to a single universal performance claim. The evidence remains tied to the reported models, baselines and workload configurations.
Paper data and sources
Original title: WavePP: High-Throughput Pipeline Parallel LLM Prefill under Prefix Reuse
Authors: Aaryam Sharma
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-28
DOI: Not available
Original paper · Full text