A filtering approach for language-model training was associated with higher scores after a later reinforcement-learning stage called GRPO, even though its immediate SFT results were not uniformly better. In the reported tests, TailSFT had higher pass@16—used here as an empirical measure of response coverage across 16 sampled attempts—in 15 of 18 dataset–benchmark pairs. Its starting checkpoint also had higher pass@1 in every matched GRPO comparison.
Why a single score can mislead
The study asks whether supervised fine-tuning, or SFT, can be modified to better preserve the useful response coverage needed for subsequent reinforcement learning. The distinction is between a strong single-sample answer and a broader range of responses available across repeated samples. In these experiments, pass@16 and pass@8 serve as empirical coverage measures rather than direct measures of real-world usefulness or unrestricted reinforcement-learning capability.
TailSFT applies sequence-level filtering based on loss reduction and trains the retained sequences with standard token-averaged cross-entropy. The main comparison is standard SFT with no filtering. The design focuses on whether a checkpoint can retain useful coverage for a later training stage even when current loss-based measures look less favorable.
A controlled coverage test
The first test used a synthetic layered-graph path-following task. Each prompt had exactly eight valid paths; pretraining sampled those paths uniformly, while the SFT target was deterministic. Pass@K served as a finite-sample estimate of coverage.
In that task, standard SFT had the lowest cross-entropy and the highest pass@1, whereas all three filtering methods had substantially higher pass@8. The comparison separated single-sample performance from multi-sample coverage.
The theoretical analysis makes a narrower statement. Under the stated expert-conditioning setup, offset filtering can be tuned for coverage no worse than standard SFT or the best absolute threshold. The same result permits a strict advantage and includes cases in which absolute filtering is worse than standard SFT; it does not establish the same outcome for arbitrary language-model training.
The pattern in language-model tests
The language-model SFT experiments used OLMo-3 7B across 18 dataset–benchmark pairs. Math SFT used a 350,000-example subset of OpenMathInstruct-2 that had been decontaminated against MATH-500. Results were summarized over three seeds, and no formal significance tests were reported.
Across those pairs, TailSFT had higher pass@16 in 15 cases, while pass@1 changes were mixed. Differences were reported as TailSFT minus standard SFT in percentage points: the largest reported difference was 16.79 percentage points on BigCode/CruxEval-O, while OMEGA-500 differed by −0.20 percentage points. Other reported differences included 3.07 percentage points on AIME and 4.14, 3.35 and 1.15 percentage points on MBPP+ for BigCode, Magicoder and OCI respectively. The pattern was broad but not universal.
A possible signal for favorable cases
The researchers also examined ρ16, a diagnostic that estimates coverage lost and gained among responses reachable by the base model, using pass@1-derived pass@16 values. Among 11 settings with ρ16 greater than 1, 10 had positive coverage gains and one was essentially unchanged; the largest reported gain reached 28.69 percentage points. But the condition was not necessary for improvement, making it a favorable signal rather than a requirement.
The downstream checkpoint test
The downstream comparison matched Standard-SFT and TailSFT checkpoints and held procedures fixed within each comparison. Each prompt received four rollouts, and the actor learning rate was 2 × 10−5.
TailSFT initialization was associated with higher post-GRPO pass@1 in every matched comparison, with reported differences ranging from 1.21 to 3.93 percentage points. Post-GRPO pass@16 was higher in four of five comparisons and approximately tied in the fifth. Early reward increased as much as 2.5 times faster in some settings.
Where the evidence stops
These are algorithmic model-development results, not evidence from human or animal subjects. The language-model experiments used one base model family and size, OLMo-3 7B, with limited math and code task coverage. The reported comparisons used three seeds without formal significance tests, while pass@16 and pass@8 are surrogate measures of response coverage rather than direct measures of real-world usefulness or unrestricted reinforcement-learning capability.
The practical lesson is about choosing a scorecard, not adopting a universal recipe. In the matched tests, lower local loss or higher pass@1 did not always identify the checkpoint with the strongest subsequent GRPO result. These comparisons show an association within the reported experiments; they do not establish that filtering caused the later gains or that TailSFT is universally superior across models, datasets or reinforcement-learning procedures.
Paper data and sources
Original title: TailSFT: Filtered Fine-Tuning Improves Post-Training Performance
Authors: Sadhika Malladi, Samy Jelassi, Dylan Foster et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text