An AI training method designed to improve factuality reported the highest scores across the listed hallucination and mathematics benchmarks for both language models it was tested on, according to an arXiv preprint. Called FARCA, it was evaluated with Qwen2.5-3B-Instruct and Llama-3.2-3B-Instruct. The result is a benchmark finding from the named experiments, with no evidence in the supplied analysis that it extends beyond the tested models, datasets, verifiers and training configuration.
FARCA's central move is to break a model's reasoning into atomic facts - small statements that can be checked - and trace each fact back to the source tokens that produced it. It also uses counterfactual evidence attribution, measuring how much verification depends on key evidence and treating that dependence as an empirical proxy for reliability. That gives the method a token-level way to assign feedback.
Scores across two model backbones
For Qwen2.5-3B-Instruct, the reported FARCA scores were 3.56 on SimpleQA, 45.41 on TruthfulQA, 25.78 on HalluQA and 26.67 on HaluEval-QA. On the mathematical benchmarks, its Pass@1 values, a one-attempt performance measure, were 6.67 on AIME2026, 10.00 on AIME2025, 63.20 on MATH-500 and 84.46 on GSM8K.
For Llama-3.2-3B-Instruct, the corresponding FARCA scores were 3.33 on SimpleQA, 39.05 on TruthfulQA, 14.00 on HalluQA and 25.91 on HaluEval-QA. Its Pass@1 results were 6.67 on AIME2026, 10.00 on AIME2025, 63.20 on MATH-500 and 84.46 on GSM8K.
Compared with FaithRL, one of the listed comparison conditions, the authors report an average hallucination-score gain of 1.75 percentage points for Qwen2.5-3B-Instruct and 2.21 points for Llama-3.2-3B-Instruct. The reported TruthfulQA and HalluQA gains were 2.09 and 2.67 points for Qwen, and 3.31 and 4.00 points for Llama.
The comparison set also included zero-shot prompting, GRPO trained on mathematics only, GRPO trained on the full setup, KnowRL and FSPO. Hallucination was assessed with SimpleQA F1, the truthful ratios on TruthfulQA and HalluQA, and HaluEval-QA accuracy; mathematical reasoning was assessed with Pass@1 on AIME2026, AIME2025, MATH-500 and GSM8K.
A closer look at the training signal
To construct the feedback, GPT-4o extracted atomic facts and their token provenance, while HHEM-2.1-Open verified the facts. Reliability estimation used Krel = 1, tau = 0.2 and mu = 0.16. The reported results are therefore tied to this particular fact-extraction, verification and reliability configuration.
The reported training collection contained 4,761 HotpotQA samples and 3,737 samples from a second knowledge dataset. FARCA used 2,000 randomly sampled examples, while SimpleRL contained 8,523 mathematical reasoning problems.
Training used full-parameter fine-tuning for one epoch on four A6000 GPUs. The configuration used a learning rate of 5 × 10^-7, six rollouts per prompt at temperature 1.0, a PPO mini-batch size of 128 and a KL coefficient of 0.001.
In the component tests, removing token provenance left a reported average score of 24.40, compared with 25.36 for full FARCA. The paper also reports clear degradation without reliability estimation and a modest drop without the continuous factual score. These are within-study component comparisons under the reported setup.
Changing the reliability temperature produced similar averages: 25.31 at tau = 0.10, 25.36 at tau = 0.20 and 25.47 at tau = 0.30. The maximum difference was 0.16 percentage points.
During training, about 98.3% of extracted facts were aligned to source tokens. A counterfactual verifier-score difference exceeded the paper's mu threshold for 36.3% of facts, only 1.1% used a fallback, and the average reliability weight was 0.512. The reported weight distribution remained spread out rather than collapsing to extreme values.
The boundary of the evidence
Before overlapping-fact aggregation, 49.6% of spans retained their original direction, while 39.5% retained that direction with a rescaled magnitude. Another 10.7% reversed direction and 0.2% were neutral. Contradiction correction and support rescue averaged 77.1% and 75.6% in strength, respectively. In the paper's account, local factual feedback often preserved or adjusted the broader credit signal, and sometimes reversed it when it conflicted with global credit.
The evidence has a tight boundary. The experiments cover only Qwen2.5-3B-Instruct and Llama-3.2-3B-Instruct, the reported knowledge and mathematics datasets, the listed benchmarks, the fact-extraction and verification setup, and the one-epoch training configuration. The supplied analysis leaves open whether the pattern generalizes to other model sizes, verifiers, judges, domains or training sets.
FARCA is described in an arXiv preprint dated 25 Aug 2026. The supplied document does not report a funding source. For now, the findings are best read as the authors' reported benchmark results within that defined configuration.
Paper data and sources
Original title: FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision
Authors: Qiming Xie, Wenjie Zheng, Xiangqing Shen, Rui Xia
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text