A preprint reports that a prefix-based regeneration check, called Prefix-Denoising Consistency, or PDC, was best or tied-best in all 15 selected model-dataset settings. Its reported accuracy was higher than the initial generation, known as INIT, in 14 of those settings, with the largest reported gain reaching 5.68 accuracy points. The evaluations covered Dream-7B, LLaDA-8B and LLaDA-1.5 on GSM8K, MATH-500, SVAMP, CSQA and SQA.
The check rebuilds an answer's ending
PDC starts with an output from a diffusion language model, keeps a chosen prefix, masks the remaining positions and regenerates that suffix with the same model. It then selects the most frequent answer among the regenerated outputs. The tested keep rates were 0.1, 0.5 and 0.9, spanning weak, intermediate and strong prefix conditioning, and each run used three regenerations.
The initial answer is excluded from the final vote and is used for diagnostics, while the decision comes from regenerated answers. Across the reported settings, correct initial answers were more likely to be preserved than incorrect initial answers were to be repeated. That difference is the reproducibility signal examined by the method.
The gains varied by model family
The main evaluations used a sampling temperature of 0.2. Generation length and denoising steps were set to 128, 256 or 512, with block length matched to generation length. Within that setup, Dream-7B improved over INIT on every listed dataset. The LLaDA family improved in nine of 10 model-dataset pairs and tied in the remaining pair. The unweighted mean gain was 1.27 accuracy points for Dream-7B and 1.73 points for the LLaDA family.
A higher-budget comparison leaves a cost question
At generation length 128, another comparison set PDC against INIT ×4, which uses majority voting over four independent initial generations. PDC was best or tied-best in 14 of 15 settings and improved over INIT ×4 in 13, with gains up to 9.25 accuracy points. INIT ×4 was a conservative higher-budget baseline rather than an exactly cost-matched comparison, so these results do not settle how the methods compare under identical compute.
Additional tests were less uniform
Under alternative unmasking rules on MATH-500, PDC achieved the best accuracy in all displayed model and length settings. Supplementary semi-autoregressive experiments with the LLaDA family indicated that PDC remained effective overall, although the displayed accuracy changes were not uniformly positive.
At sampling temperature 1.0, PDC had the highest numerical accuracy in 12 of 15 settings and exceeded INIT ×4 by 4.98 points on average. It trailed INIT ×4 on all three CSQA settings, by 0.74 to 2.12 points. The results therefore varied across the tested settings.
A condition for improvement
The theoretical analysis states that retention at least as high as same-wrong persistence for every regeneration, with a strict gap for at least one, is sufficient for improvement.
What the results cover
The evidence comes from offline benchmark evaluations of three named diffusion language model checkpoints on five reasoning datasets, with main results using full-diffusion inference and additional checks in semi-autoregressive and higher-temperature settings. The reported accuracy comparisons are descriptive benchmark results, and the tested configurations include the listed keep rates, generation lengths, denoising steps and sampling temperatures.
The acknowledgments report that J. Komiyama received support from the MBZUAI Start-up Fund.
Paper data and sources
Original title: Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models
Authors: Yuki Ichihara, Naoto Iwase, Mohammad Atif Quamar, Junpei Komiyama
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text