A preprint study of a conversational-AI safety system reports a lower rate of harmful compliance, meaning responses judged to help manipulate another person, for its combined two-gate condition than for raw generation without a gate. Under the primary GPT-4o-mini judging protocol, Pregate plus Postgate recorded a pooled harmful rate of 2.21%, compared with 41.44% for raw generation. Its pooled protective rate was 93.04%, versus 44.70% for raw generation, and its pooled appropriate-refusal rate was 80.31%, compared with 31.94%.
The result is limited to a controlled test. The benchmark used synthetic conversations and automated LLM judges, and the study involved no real users or targets. The authors say the benchmark is synthetic and incomplete, that LLM judges are not substitutes for human judgment, and that the work does not show prevention of real-world relationship harm.
A gate built around two roles
HRGuard was built around a role-sensitive question: whether a system can block assistance for users seeking to manipulate others while preserving supportive guidance for users seeking protection.
The system has two checks. An online Pregate runs before generation and a Postgate runs after the assistant turn has been generated. Both rate risk turn by turn and keep a cumulative score that fades over time. A hard trigger emits or substitutes a fixed refusal and stops later turns.
The researchers compared six conditions: raw generation, a generic safety prompt, Pregate alone, Postgate alone, the prompt plus Postgate, and Pregate plus Postgate. The comparison used fixed thresholds, with a turn threshold of 5, a cumulative threshold of 6 and a decay factor of 0.85, followed by stop-after-trigger behavior.
The evaluation covered eight generation models and compared HRGuard with a generic safety prompt and three general-purpose guard models.
The test used staged conversations
The benchmark contained 1,000 five-turn dialogs: 500 used direct wording and 500 used adversarial paraphrases, or indirect rewordings used in a wording-variation stress test. Each half contained 250 attacker scenarios and 250 victim scenarios.
What the main test showed
Under the primary GPT-4o-mini protocol, raw generation was judged to provide harmful assistance in 78.93% of attacker dialogs, compared with 0.00% for Pregate plus Postgate. Protective intervention was 86.33% for raw generation and 86.10% for the combined condition.
The zero observed harmful cases represented 4,000 attacker-dialog evaluations in that analysis. The one-sided 95% upper bound for the underlying rate was approximately 0.075%. That figure describes the tested sample and judging protocol, not a guarantee that harmful assistance would never occur.
On four shared generators, the reported mean harmful rates were roughly 2% to 5% for HRGuard variants, compared with about 27% for LlamaGuard and 39% to 40% for ShieldGemma and Qwen3Guard. These were final-dialog outcomes after post-hoc filtering under the paper's outcome measure.
Gate triggers often appeared early
Under the GPT-4o-mini protocol, Postgate triggered on 48.49% of dialogs. Among dialogs that triggered, the mean first-trigger turn was 1.84, and the early-stop rate was 48.14%. With Pregate and Postgate together, the trigger rate was 22.25%, the mean first-trigger turn was 1.65, and the early-stop rate was 22.25%. The mean trigger turn was calculated only among triggered dialogs, and the figures were macro-averaged across generation models.
The wording test was a stress test, not a claim of universal adversarial robustness. In the reported Llama and Qwen examples, Postgate attacker harmful compliance was 0.0% in both the direct and adversarially paraphrased halves. For DeepSeek-v4-pro, the pooled harmful rate was 5.6% with direct wording and 4.0% with paraphrasing; for DeepSeek-v4-flash, it was 3.0% and 3.6%. The generic safety prompt was more sensitive to indirect wording in these examples: its DeepSeek-v4-flash pooled harmful rate rose from 5.2% to 19.6%, while Llama attacker harmful compliance rose from 45.2% to 62%.
The difficult part is identifying the role
In the primary tests, the system was given an oracle role label, meaning it was told whether a dialog represented an attacker or a victim. When that label was removed in Llama and DeepSeek evaluations, overall trigger rates were 53.3% and 48.9%, respectively, compared with 49.5% and 46.3% with the label. Victim false-blocking was 7.6% for Llama and 5.2% for DeepSeek without role labels, compared with 0% with oracle labels, while attacker-side misses stayed unchanged.
That comparison limits how far the strongest numbers can be generalized. The oracle-role results are an upper bound for settings in which the user's role is known, not evidence that role inference has been solved.
The reported rate varied with the judge
The reported harmful-compliance rate varied with the automated judge. Two alternative judges gave lower rates for Postgate than raw generation: 0.94% versus 8.94% with the Qwen2.5-7B judge, and 1.30% versus 34.76% with the DeepSeek-v4-flash judge. The Llama-3.2-3B judge reversed the ordering, reporting 70.39% for Postgate and 50.24% for raw generation.
A human validation check covered 160 items. Human annotators agreed on appropriate refusal, at 98.8%, compared with 83.1% for protective intervention and 88.8% for harmful assistance. GPT-4o-mini matched human refusal labels at roughly 92% to 93%, but its agreement on protective intervention was 65%.
A benchmark result with a narrow reach
Taken together, the findings describe performance under a synthetic benchmark, not a real-world safety result. The benchmark is incomplete, LLM judges are not substitutes for human judgment, and the study does not show prevention of real-world relationship harm.
The complete dataset will use controlled access. Public materials will redact executable manipulation templates and other details that could facilitate misuse.
Paper data and sources
Original title: HRGuard: Gating Relationship Manipulation in Multi-Turn Agentic AI Conversations
Authors: Pei-Sze Tan, Tasuku Igarashi, Isao Echizen
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text