AI argument-search models reported better performance at finding arguments that matched a requested support or attack stance, but the comparisons also revealed a trade-off: some fine-tuned comparisons showed greater sensitivity to the instruction while showing less sensitivity to what the claim was about. A preprint testing three dense embedding architectures reports the clearest result for Qwen3-Embedding-8B. On 71 claims held out from the TFU Training data, the model reached a reported Precision@R of 0.95, the study's main measure of retrieval quality. Its stance error was reported as 39.7% for the base model and 2% in the fine-tuned comparison. BGE-Large was reported at 0.419 before and 0.822 in the fine-tuned comparison, while Instructor-XL was reported at 0.531 and 0.839.
The task was argument retrieval. The three systems were dense embedding models, which turn text into numerical representations for comparison. Each query combined an instruction specifying the desired stance with a claim specifying the subject, and the models ranked candidate arguments by cosine similarity, which compares the direction of their numerical representations. The comparisons covered base models, homogeneous hard-negative fine-tuning, a Mixed negative curriculum, synthetic stance-inverted augmentation and a hybrid search configuration across BGE-Large, Instructor-XL and Qwen3-Embedding-8B.
Training used 80% of TFU Training, with 20 standardised instructions: 10 supporting and 10 attacking. The remaining 20% was held out for initial in-domain evaluation. For out-of-domain evaluation, TFU Evaluation, AVeriTeC and ArgTumour were paired with five unseen supporting and five unseen attacking instructions. The reported claim counts were 283 in the training split, 71 held out, 750 in TFU Evaluation, 1,746 in AVeriTeC and 14 in ArgTumour. The GPT-5 and Qwen3-8B validation sets each contained 150 claims.
The warning inside the score
To look beyond the ranking score, the researchers used word ablation, removing one word from a query and measuring the raw change in cosine similarity. For Qwen on TFU Training, hard-negative DI was 0.036 in the base model and 0.238 in the fine-tuned comparison. Positive DI was -0.199. Mean RCS was 75.2% in the base model and 64% in the fine-tuned comparison, while RIS reached 36%. The paper interprets the combined pattern as topical collapse, with the model's behavior shifting toward the stance instruction and away from claim sensitivity.
Adding a reranker did not produce a consistent gain in the Qwen tests. With the base Qwen3-Embedding-8B model, Qwen3-Reranker-8B changed stance error only from 39.7% to 38.5%. When applied after fine-tuning, the comparison showed Precision@R falling from 0.95 to 0.645, while stance error reached 35.3%. The reported reranking comparison did not show a general improvement.
Changing the examples
The proposed remedy focused on the training data. In the Mixed curriculum, 31,800 triplets were split equally among hard, semi-hard and easy negative examples. The augmentation replaced 50% of natural hard negatives with stance-inverted counterparts generated with Gemini 3.5 Flash. A word-level Jaccard overlap score was 0.0938 for standard pairs, compared with 0.5866 for synthetic pairs and 0.5896 after lemmatisation. The synthetic examples therefore shared far more wording with their paired arguments while being constructed as stance-inverted counterparts.
Responses differed sharply by architecture. On TFU Evaluation, Qwen's stance error was reported as 33.5% for the base model and 3.9% under homogeneous training. Its semi-hard rate was 11.7% with the homogeneous curriculum, 9.8% with Mixed and 7.8% with Mixed plus Augmentation, while the reported Precision@R was 0.845. BGE-Large's semi-hard rate was 20.7% under homogeneous training and 8.5% with Mixed plus Augmentation, with a hard-negative rate of 23.2%. Instructor-XL's semi-hard rate was 2.7% for the base model, 10.1% under homogeneous training and 5.1% with Mixed; its stance error was 31.4%. The figures describe different topic-versus-stance trade-offs across the three architectures.
A sparse backstop
The paper also tested hybrid search, combining the dense Mixed plus Augmentation model with BM25 through Relative Score Fusion. The sparse component was given three weights, 0.1, 0.15 and 0.2. On ArgTumour, Qwen's Precision@R was 0.723 at the lowest weight, compared with 0.690 for dense Mixed plus Augmentation alone. The paper reports higher stance error as the sparse weight increased, leaving a precision-versus-stance trade-off in that test.
On out-of-domain AVeriTeC data, Qwen's mean RCS was 0.783 for the base model, 0.637 in the homogeneous comparison and 0.661 with Mixed plus Augmentation. Hard-negative DI was 0.024 in the base model and 0.144 with Mixed plus Augmentation, while positive DI was -0.120. The reported pattern was a partial restoration of claim sensitivity alongside a positive hard-negative DI separation.
Taken together, the study speaks to retrieval behavior, not to whether an AI system can reason through arguments, verify facts or improve a downstream RAG system. Its primary endpoint was Precision@R, with NDCG@10 and word-ablation diagnostics also reported. The evidence is limited to computational experiments across the listed datasets, instructions and three architectures, so the figures do not show that one curriculum is best for every architecture or dataset. The authors state that the codebase, datasets and models are publicly available for research reuse. The document is a preprint and carries a CMNA'26 workshop header.
Paper data and sources
Original title: Embedding Models for Stance-Aware Argument Retrieval
Authors: Angelo Sparacino, Francesca Toni, Adam Dejl
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text