A methods preprint reports that its Full Relation implementation of a Self–Exchange Relation design recorded lower validation negative log-likelihood (NLL)—the paper’s main score on the full validation set—than matched multi-head attention (MHA) models at approximately 10 million, 30 million and 100 million parameters. The mean gaps were 0.0412, 0.0151 and 0.0310 NLL points at those scales, respectively.
A design built around two roles
The proposed Self–Exchange Relation separates pairwise evidence into Self and Exchange roles before normalization, then extends the setup to Multi-Head Relation. The framework also describes Full Relation, exact fused FlashRelation, recurrent Linear Relation, Hybrid Relation and Relation Cache for autoregressive decoding.
The main comparison
To make the central comparison, the study matched decoder geometry, data order and training budget between MHA and Full Relation. It used paired seeds 42, 43 and 44 at approximately 10 million, 30 million and 100 million parameters, with final-checkpoint NLL on the full validation set as its primary measure.
The formal setups used TinyStories at 10 million and 30 million parameters and the SmolLM corpus at 100 million. Across the configurations, context lengths were 1024, 2048 and 4096 tokens, and training budgets were 150 million, 450 million and 1.071 billion tokens.
On that primary measure, Full Relation recorded lower mean NLL than MHA at every scale. It won all three paired comparisons at 10 million, two of three at 30 million and all three at 100 million—eight wins out of nine.
In the 10 million structural ablations, NLL was higher than Full Relation by 0.0320 for exchange-only transport, 0.0366 for Raw-X communication, 0.0506 without count calibration, 0.0032 without Givens mixing and 0.0507 for a single head.
Speed was less predictable
At a fixed context length of 1024, FlashRelation was 3.60–4.41 times faster than the materialized Full Relation reference across the three scales. In scale-matched production workloads, however, it reached 0.799253×, 0.848955× and 0.763872× of FlashAttention’s throughput at 10 million, 30 million and 100 million parameters.
The systems results are tied to the reported benchmark setup, and no variability estimate is given for those measurements.
A selected Hybrid Relation configuration used nine Linear and three Full Relation layers and reported final-validation NLL of 1.2780 ± 0.0050 across three seeds. Linear Relation is described as scaling linearly with sequence length while keeping a fixed-size decode state; in a long-context test, 32768 tokens passed at 30,003.56 tokens per second, while 65536 tokens was out of memory.
The result has clear boundaries
The main result did not carry over uniformly to a separate BLiMP benchmark. The mean Full–MHA difference was +0.011005 at 10 million parameters, +0.005562 at 30 million and −0.008144 at 100 million, so the direction changed with scale.
The evidence remains narrow: the reported experiments used decoder-only models at approximately 10 million, 30 million and 100 million parameters and the stated training configurations. The study leaves open whether the pattern will hold at substantially larger scales or in other settings.
The main comparisons used three paired seeds and reported means with sample standard deviations; no p-values, confidence intervals or inferential tests were reported.
The work is an arXiv preprint: arXiv:2608.20172v1, dated 20 Aug 2026.
Paper data and sources
Original title: Ask Self, Ask Others: Relation Is All You Need
Authors: Yuting Ge, Pengju Yang, Mingkai Nie
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text