One model, many tasks
An instruction-tuned language model trained across a single collection of hate-speech tasks scored higher than separately trained task-specific models in 14 of 17 comparisons, according to the study. It also won 7 of 17 comparisons with the best-known systems reported in the literature and outperformed GPT5-mini [Source: Comparison to State-of-the-art, p. 6, Table 2].
HIPPO, the model named in the study, averaged 70.3 on the main macro-F1 score. Macro-F1 was the score used for most tasks, with task-specific exceptions rather than one uniform metric across the study. Individual models averaged 68.0, the literature-best systems 68.5 and GPT5 55.3. HIPPO therefore had the highest reported average in the four-way comparison [Sources: Results table, p. 5, Table 2; Results, p. 6].
The training corpus combined 36 heterogeneous English hate-speech datasets into an instruction-formatted collection spanning more than 600,000 examples and 61 subtasks. After deduplication, the reported split contained 649,761 training examples and 80,935 test examples. The training and test counts were reported separately [Sources: Introduction, p. 1; Dataset table, p. 3, Table 1; Data Collation, p. 4].
Each source dataset was converted into a conversational format with a user prompt and an assistant answer. HIPPO was based on Qwen3-4B-Instruct-2507, and its training used QLoRA with assistant-only outputs. The study also evaluated 32-billion-parameter and 0.6-billion-parameter versions alongside the main 4-billion-parameter model for its model-size analysis [Sources: Instruction Construction, p. 4; Model Selection and Training, p. 5].
Model size and training conditions
The model-size results varied sharply. The 32B model won 11 of 17 cases against the state-of-the-art systems, while the 0.6B model often performed worse than random chance. The paper attributes the smaller model's result to unconstrained generation [Source: Model Size Results, p. 6, Table 3].
The study also compared full HIPPO with leave-one-out training and zero-shot prompting. Average scores were 70.3 for full training, 56.4 for leave-one-out training and 50.4 for zero-shot. Leave-one-out training was generally better than zero-shot, but performance was lower on 6 of the 17 tasks [Source: Prompting and Cross-task Transfer, p. 7, Table 4].
Cross-lingual results were uneven
On non-English evaluations, the full English-tuned model generally scored lower than zero-shot results on cross-lingual classification and functional performance. Arabic and Hindi were the weakest languages in these comparisons, while Chinese performed better. Slur-related cases were especially difficult, but performance on non-hateful content remained nearly perfect across languages [Source: Cross-lingual Transfer, p. 8].
A separate comparison used a minimally trained model exposed to only 1,000 examples from each dataset. In the reported cross-lingual results, that model remained competitive on classification, generally scored higher on HateCheck and was substantially better on ParaDetox [Source: Cross-lingual Transfer, p. 8].
Overall, the cross-lingual comparisons did not show a universal advantage for the model trained on the full English corpus. The minimally trained model was stronger on HateCheck and ParaDetox, while the full model generally scored below zero-shot results on cross-lingual classification and functional performance [Source: Cross-lingual Transfer, p. 8].
The benchmark's boundaries
Training used only English datasets. The cross-lingual comparisons therefore concern transfer from an English-trained model, not a model trained jointly on multilingual data [Source: Limitations, p. 9].
Deduplication removed 25,147 examples, or 3.73 percent of the dataset. Conflicting labels across sources affected 75 examples, or 0.01 percent. The reported final training size was 649,761 examples [Source: Data Collation, p. 4].
Macro-F1 was the main score, but task-specific exceptions meant the study did not use one uniform metric across all tasks. The comparisons were descriptive, and no confidence intervals or inferential tests were reported [Sources: Results, p. 6; Results table, p. 5, Table 2].
The paper states that its fine-tuning code and best-performing models are released online for public use. The document is an arXiv version 1 entry dated 26 August 2026 [Sources: Introduction, p. 2; Front matter, p. 1].
Paper data and sources
Original title: From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation
Authors: Lukas Edman, Daryna Dementieva, Alexander Fraser
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text