An AI method for choosing training examples took seconds rather than many minutes in one summarization benchmark, according to a new preprint. On PEGASUS-large with CNN/DailyMail, LOBSTER took 1.6 seconds for selection, compared with 1,064.2 seconds for BAS and 699.4 seconds for DUAL. The reported speedups were 665 times over BAS and 437 times over DUAL.
The quality picture was more qualified. LOBSTER was competitive, but no single method led every condition: BAS had the strongest ROUGE-1 result for PEGASUS on CNN/DailyMail, while DUAL was strongest on XSum with BART. Several comparisons between LOBSTER and the baselines met the reported statistical threshold of p < 0.05, although the direction of differences varied by dataset, metric and backbone.
The method targets a model’s weak spots
LOBSTER is a three-stage active-learning strategy. It first identifies labeled examples where the model’s loss is high, then uses density-based filtering to build a representative candidate pool. Finally, it searches the unlabeled data for examples semantically similar to those difficult cases.
The approach rests on the hypothesis that matching unlabeled examples to high-loss labeled examples will direct training toward the model’s weaknesses. In plain terms, it tries to find new cases resembling the problems the model is already handling poorly.
A small, staged benchmark
The evaluation used three benchmark datasets representing different summarization styles: AESLC, XSum and CNN/DailyMail. The experiments used BART-base and PEGASUS-large, comparing LOBSTER with random sampling and three active-learning baselines: IDDS, BAS and DUAL.
The main setup began with a randomly sampled seed of 10 instances, added 10 examples at each iteration for 15 iterations, and used a reported annotation budget of 150 instances. Results were averaged over five independent runs. Because uncertainty baselines were computationally expensive across active-learning cycles, evaluation used a random subset of 1,000 test instances.
Summary quality was scored with ROUGE-1, ROUGE-2, ROUGE-L and BERTScore. Together with the selection-time results, those measures showed a trade-off: LOBSTER could be much faster to run, but the quality ranking depended on the dataset and model rather than producing a consistent winner.
Speed did not settle the quality question
Early in active learning, the authors report severe instability for IDDS, while LOBSTER’s performance was more stable. At larger annotation budgets, however, random sampling became competitive: it reached the 90% performance threshold with 1,600 samples in the BART-base/XSum setting, while LOBSTER reached the target in five of six configurations.
The ablation tests also favored the method’s central choices. Replacing high-loss anchors with random anchors performed worse on CNN/DailyMail for both BART and PEGASUS, while the gap was relatively small on AESLC and XSum. Removing IDDS concentrated selections in particular embedding regions and produced redundant choices, whereas IDDS preserved broader coverage.
LOBSTER also compared well with the study’s zero-shot large-language-model baseline. On ROUGE and BERTScore, it remained highly competitive and in several cases performed better in the reported benchmark comparison.
The unanswered quality question
Several cautions temper the findings. ROUGE and BERTScore may miss factual consistency, coherence and overall summary quality. The approach also assumes that Sentence-BERT similarity is a useful proxy for an unlabeled example’s difficulty, while the simulated annotations may not reflect real-world annotation.
Taken together, the results point to a speed-performance trade-off. LOBSTER was far cheaper for selection in the reported configuration, but its quality advantage was not consistent across the tested settings. The work is an arXiv preprint, version 1, dated 26 August 2026.
Paper data and sources
Original title: Loss-Based Active Learning for Neural Abstractive Summarization
Authors: Michail Ioannou, Tatiana Passali, George Michalopoulos, Grigorios Tsoumakas
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text