A preprint reports that COPA, a defense designed to adapt as prompt-injection attacks change, had a lower reported attack success rate than two comparison defenses in benchmark tests. COPA's rate was 0.035, compared with 0.221 for SecAlign and 0.104 for LlamaGuard; the paper describes those differences as up to 6.3 times and 3.0 times lower, respectively.
Attack success rate, or ASR, is the fraction of tested attacks marked successful. Across the study's three settings, COPA's ASR stayed below 0.06. In S3, the setting using optimization-based UAT and GCG attacks, the undefended model and a prior defense reached 0.74 and 0.25.
Keeping earlier defenses in play
COPA starts with preference alignment and then continues adapting. Its margin-weighted replay buffer is intended to retain robustness against earlier attacks, while the lifelong stage updates only a LoRA adapter over a frozen base model.
The main setup used Meta-Llama-3.1-8B-Instruct, a LoRA rank of 64, 4-bit NF4 quantization, one alignment epoch, 30 GRPO updates per task and a 60-pair replay buffer. The initial alignment data contained 102 cybersecurity preference pairs.
The threat model assumed an external attacker could use the model's input interface but could not access its weights or training process. It also assumed that the attack distribution would change over time.
For the baseline comparisons, the alternatives used the same model backbone, preference dataset and opportunity to adapt as COPA. The stated difference was the alignment strategy.
What the benchmark showed
COPA was reported as best on the study's three main metrics: ASR, backward transfer (BWT), a measure of how earlier defenses change after later updates, and average performance (AP). Its scores were 0.035 for ASR, +0.028 for BWT and 0.850 for AP.
The evaluation used 15 CyberSecEval prompt-injection variants. Nine formed the lifelong training stream and six were held out for evaluation. The tests also included direct, ignore, completion and hybrid text attacks, along with optimization-based UAT and GCG attacks; the training and held-out pools were separate.
On the two question-answering benchmarks, COPA recorded accuracy of 0.310 on GPQA and 0.340 on MMLU, compared with 0.300 and 0.332 for the undefended base model. COPA had the highest reported accuracy on both tests.
Across three language-model backbones, COPA's reported ASR ranged from 0.035 to 0.054, BWT from −0.052 to +0.028, AP from 0.786 to 0.850 and question-answering accuracy from 0.210 to 0.316.
The replay rule made a difference in the comparison
A component comparison reported better figures for margin-weighted replay than for uniform replay. The margin-weighted version had an ASR of 0.035, BWT of +0.028 and AP of 0.850, while uniform replay reported 0.163, +0.003 and 0.600.
A separate comparison of training objectives also reported better figures for GRPO than for DPO. GRPO had an ASR of 0.035, BWT of +0.028 and AP of 0.850, versus 0.335, −0.231 and 0.626 for DPO.
The limits of the result
These are descriptive benchmark comparisons. Because the evaluation covered a specified set of attacks and three model backbones, the results do not establish how COPA would perform against every adaptive prompt-injection strategy.
The report provides no confidence intervals, repeated-run information or statistical tests. The supplied results therefore do not show whether the numerical differences would persist across repeated experiments.
The document is identified as arXiv:2608.19982v1 and is dated 20 August 2026.
Paper data and sources
Original title: COPA: Continual Preference Optimization for Adaptive Prompt Injection Defense
Authors: Roshan Sood, Onat Gungor, Tajana Rosing
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text