A system that gives an AI judge task-specific criteria before it scores a computer-use agent's actions performed better than the compared reward verifiers in several benchmark tests. On OGRBench, it reached 86.7% accuracy and an 86.6 F1 score, a single measure that balances how well the judge identifies positive and negative trajectories. Its F1 score was 3.6 percentage points above the average of two baseline systems, while recall rose from 80.2% to 86.5%.
The work evaluates A DAPT RUBRIC in both offline reward discrimination and online reinforcement learning. In practical terms, the verifier judges GUI-agent trajectories and supplies a score that can be used in evaluation, later training or candidate selection.
Criteria tailored to each task
The system first routes an instruction to a GUI task category. It then retrieves reusable, broad checks for that category and generates shorter, instance-level cues from the instruction itself. These steps create task-adaptive criteria for the verifier's judgment.
The broad checks are stored in a fixed rubric bank built before evaluation. Its development pools contained 116 AndroidWorld trajectories, including 55 successful and 61 failed examples, and 116 MobileWorld trajectories, including 35 successful and 81 failed examples. The bank was fixed during the reported evaluation.
The main offline comparison gave every verifier the same maximum budget of 10 screenshots from a trajectory, keeping the visual-history allowance constant across systems.
A broader offline comparison also favored the system
The offline evaluation covered 1,409 OGRBench trajectories drawn from five environments: 700 were labeled positive and 709 negative. The study's expanded comparison added several reward verifiers to the two-baseline comparison used for the main result.
In the expanded matched-budget analysis, A DAPT RUBRIC had the highest accuracy on every tested model backbone. It had the highest F1 score on five of the six backbones tested. The supplied analysis reports no confidence intervals or significance tests, so the statistical reliability of those differences cannot be judged from these results alone.
The system also showed a quality-and-cost advantage in an efficiency comparison using Qwen3-VL-8B-Instruct. It recorded 84.0% accuracy and an 84.2 F1 score while using fewer model calls, tokens and runtime than OS-Themis and ZeroGUI. DigiRL was slightly cheaper, but its F1 score was 3.4 points lower.
Higher scores carried into later agent tests
The reported online training experiment used GRPO, a reinforcement-learning procedure, with the MAI-UI-8B policy backbone in MobileWorld. After training, the policy paired with the A DAPT RUBRIC verifier reached a 23.93% task-success rate, reported with an uncertainty of plus or minus 0.85 percentage points. The figure was 4.23 points above the 19.70% result for training without an external reward agent and was the best among the compared agents.
A separate test examined whether the verifier could help choose among multiple candidate trajectories for the same task. It covered 113 AndroidWorld tasks, with 10 candidate trajectories per task: eight generated by Qwen3-VL-8B-Instruct and two by Qwen3-VL-235B-A22B-Instruct.
On that mixed candidate pool, A DAPT RUBRIC produced the largest reported gains over random selection: 11.88 percentage points for EarlyStop@7 and 13.28 points for BestOfN@8. Its results in the same evaluation were 88.14% accuracy, 88.41 F1 and an 11.17% false-positive rate.
Useful signal, with a narrow testing boundary
The ablation tests point to value in combining the two kinds of criteria. Removing either the broad category-level rubric or the instruction-specific cues corresponded to a 2.0-point lower F1 score. Removing both components corresponded to a 5.9-point lower F1 score.
Taken together, the benchmark results show higher scores for the task-adaptive criteria in the tested offline, online-training and candidate-selection settings. They do not establish that the same gains will appear across all computer applications, interface designs or instruction styles.
The study also notes that its fixed offline rubric bank may miss newly emerging applications or domain-specific workflows. That limitation leaves open how well the system would handle settings outside the evaluated tasks and interfaces.
The supplied document is an arXiv preprint. The supplied text reports no funding statement.
Paper data and sources
Original title: Task-Adaptive Rubrics for GUI Reward Modeling
Authors: Tao Xiong, Xavier Hu, Wenkai Wang et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text