An arXiv preprint reports that SABET-QA, an AI model for questions whose answers depend on facts and dates, scored 0.843 on the CronQuestions benchmark—4.7 Hits@1 points above TempoQR. Its hard-supervision version, SABET-QA-Hard, scored 0.954.
Hits@1 is a ranking measure that records whether the correct answer appears first. The paper also uses Hits@10, which records whether a correct answer appears among the model’s top ten results, and reports overall scores as well as results by question complexity and answer type when available.
The evaluation covered four benchmark datasets: CronQuestions, Complex-CronQuestions, MultiTQ and TimeQuestions. Their reported totals were 410,522, 46,343, 499,350 and 13,443 questions, respectively, split into training, validation and test sets.
SABET-QA combines slot-aware contextualization, bidirectional entity–time scoring, iterative multi-hop reasoning, differentiable working memory and optional coarse temporal hints. The design is intended to let the model revisit a question across several reasoning steps while weighing entities and dates in both directions.
On complex and time-focused tests
On Complex-CronQuestions, overall Hits@1 was 0.524 for SABET-QA and 0.807 for SABET-QA-Hard. In the hard version, the score was 0.747 for entity answers and 0.931 for time answers.
On MultiTQ, SABET-QA’s timestamp Hits@1 was 0.111, compared with 0.069 for BERT. On TimeQuestions, the model was 9.3 Hits@1 points above TempoQR; time-answer Hits@1 was 0.609 for SABET-QA and 0.604 for SABET-QA-Hard.
How the model reasons
The paper also tested reasoning depth, with each hop representing another step through the question. On Complex-CronQuestions, overall Hits@1 peaked at 0.524 at four hops. Hits@10 rose from 0.840 at one hop to 0.892 at two hops, then stayed between 0.895 and 0.897 from four hops onward.
In CronQuestions validation queries, the model’s mean aggregation weight increased from 0.137 at Hop 0 to 0.400 at Hop 3. Hop 3 was the dominant reasoning state in 56.5% of queries.
The component tests
The component ablation reported Hits@1 of 0.807 for the full model, 0.707 without entity contextualization, 0.659 without bidirectional entity–time scoring and 0.524 without hard supervision. The paper reports the bidirectional-scoring version as a 14.8-point drop from the full configuration.
Another test varied which pretrained modules were allowed to update. Without hard supervision, Hits@1 was 0.547 with only the language model unfrozen, 0.524 with all modules frozen and 0.446 with only the temporal embedding unfrozen. With hard supervision, the fully frozen score was 0.807, compared with 0.759 when only the temporal embedding was unfrozen.
What the benchmark does not settle
All of the question-answering models used TComplEx as the default temporal knowledge-graph embedding backend. That kept the embedding choice constant so the comparison focused on the question-answering architecture.
The reported score comparisons come without confidence intervals or significance tests, so the paper does not quantify statistical uncertainty around the differences.
The authors report that the model depends on pretrained temporal knowledge-graph embeddings and on grounding question mentions to entities and times. They also report performance degradation when entity mentions are missing or noisy, while the iterative architecture adds architectural and computational complexity.
Paper data and sources
Original title: SABET-QA: Temporal Knowledge Graph Question Answering
Authors: Brahim Touayouch, Mirette Moawad, Dmitry Akulov
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text