An experimental scheduler for multi-agent AI workflows reported shorter task completion times across three synthetic workloads. Average task-level JCT, or the time from a task’s arrival until it finishes, was 27.5% to 39.8% lower than the best-performing baseline selected for each workload and metric. At the high-delay end, the p99 measure, the 99th percentile, was 30.8% to 49.4% lower.
The system, called TOPAS, was implemented as a scheduling module in SGLang v0.5.3 and tested on a single NVIDIA A100 80GB. The evaluation covered three synthetic directed-acyclic-graph, or DAG, workloads named Chain-3, DAG-4 and DAG-10-Wide, alongside two MetaGPT software-development workflows, MetaGPT-SOP and MetaGPT-TL. Mean and p99 task JCT were the primary measures; first-task completion time and request throughput were secondary measures.
A scheduling decision about memory and work
TOPAS treats the workflow and the server’s memory state as one scheduling problem. At each event, it chooses which agent prefixes, the cached context needed to continue a request, stay resident in the KV-cache and which requests are admitted to run under a shared budget. Its score weighs expected progress along the remaining path against near-term reuse of a downstream prefix, while accounting for prefix movement and preemption. A task-level aging rule is intended to prevent starvation.
A small two-agent microbenchmark showed why that coordination could matter. Grouping requests for the same agent doubled the average running batch size. Alternating between agents took 1.8 to 1.9 times as long to finish, even with the same requests and KV budget. The test focused on batching behavior rather than the full software-development workflows.
Where the gains appeared
Across the synthetic set, Chain-3 showed a 27.5% lower mean JCT and a 31.7% lower p99 JCT with TOPAS. DAG-4 showed the largest reported differences, at 39.8% and 49.4%; DAG-10-Wide showed 27.7% and 30.8%. Each pair is relative to the baseline with the lowest average for that particular workload and metric, so the comparator can change from one percentage to the next.
The MetaGPT results were more uneven. Against SPF on MetaGPT-SOP, TOPAS recorded 9.8% lower mean JCT and 4.5% lower p99 JCT, while request throughput was 6.7% higher. On MetaGPT-TL, mean JCT was 22.0% lower and p99 JCT 26.6% lower relative to the best-performing baselines for those two measures, SPF and Parrot-FCFS. The authors link the larger TL gap to repeated reuse of a central TeamLeader prefix alongside heterogeneous specialist prefixes, while SOP is largely a one-pass pipeline.
An ablation, a test that removes parts of a system, was run on MetaGPT-TL at one operating point. Full TOPAS had 60.5% lower mean JCT and 53.6% lower p99 JCT than TOPAS-base. Compared with two single-component variants, its mean JCT was 44.9% to 51.0% lower and its p99 JCT was 44.2% to 48.5% lower. Those figures indicate that the complete configuration performed better in that test, but they do not establish a universal contribution for each component.
The evidence is still narrow
The five comparator policies were FCFS, LPM, Parrot-FCFS, Autellix Least-Attained Service and Shortest-Path-First. On equal-cost Chain-3, Shortest-Path-First is equivalent to SRPT. The study averaged metrics over the reported operating points and then chose the lowest-average baseline separately for each JCT measure. That approach gives a workload-by-workload comparison, but the percentage gains are not all against one fixed rival.
TOPAS’s measured scheduling overhead was small in one MetaGPT-SOP run. At an arrival rate of 0.15 task per second, each scheduling decision took an average of 1.9 milliseconds, and scheduler time made up 0.31% of experiment wall time. Task arrivals followed Poisson processes, and the sampled arrival trace was fixed and shared across all policies. This comparison does not show whether the gains hold across independently sampled traces.
Only the listed workloads and one A100 setup were tested, and the analysis reported no confidence intervals or other uncertainty estimates for the results. The fixed shared trace also left variation across arrival patterns unassessed. The supplied evidence therefore leaves open how TOPAS would behave outside the tested hardware, workload and arrival-trace setup.
The document is an arXiv preprint, identified as arXiv:2608.25523v1 and dated 26 August 2026. The supplied front matter lists institutional affiliations but no funding statement.
Paper data and sources
Original title: TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
Authors: Hongqiu Ni, Han Tian, Chi Zhang et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text