A BERT-based model produced the strongest reported overall results in a test of automated sector tagging for cyber threat intelligence, according to an arXiv preprint. It posted a macro-average F1 score of 0.890, compared with 0.843 for a sequential Random Forest classifier chain and 0.794 for a parallel Gaussian Naive Bayes model. Its Hamming loss was 0.055, which the paper equates to 94.5% correct individual sector-label assignments. The authors describe BERT as the strongest overall candidate, while live threat-intelligence-platform evaluation remains future work.
How the tagging task was built
Researchers treated sector-aware dissemination as a multilabel classification problem, meaning one event can be linked to multiple sectors. The authors built a custom machine-learning-ready dataset of 872 sector-labelled CTI events using the MISPPriv threat-intelligence platform. The feeds were exported in STIX 2.1 format, parsed for relevant data objects and combined into an IoC property. Those records supplied the event information used for classification.
For the model input, the paper concatenated Report Name, Publisher and IoC into a single event-info field. It reports a 75:25 training/testing split, with roughly 654 events used for training and 218 held out for validation. The evaluation compared parallel binary classifiers, sequential binary classifiers and a transformer-based BERT multilabel model. The BERT training loop ran for 25 epochs, used batches of 6 and binary cross-entropy loss.
Different models favored different errors
The Gaussian Naive Bayes approach, which treated the sector decisions in parallel, reported a macro-average precision of 0.745, recall of 0.856 and F1 of 0.794. Its overall accuracy was 0.9158 and its Hamming loss was 0.0842. The paper's comparison describes the parallel classifiers as strongest on recall, not on the overall F1 ranking.
The sequential Random Forest classifier chain showed a different balance. Its macro-average precision was 0.941, recall was 0.769, producing an F1 of 0.843. Label-wise accuracy was 0.9324 and Hamming loss was 0.0676. Label-wise accuracy is the fraction of individual sector-label assignments predicted correctly, so it is a label-by-label measure rather than a score for complete events. The paper describes sequential classifiers as strongest on precision.
BERT reported macro-average precision of 0.920, recall of 0.860 and F1 of 0.890. Its Hamming loss of 0.055 corresponds in the paper's calculation to 94.5% correct individual sector-label assignments. The comparative analysis places BERT first on the reported F1, accuracy and Hamming-loss measures. The authors list live threat-intelligence-platform evaluation among future work.
Performance was not uniform across sectors. BERT's sector-level F1 was 1.000 for Agriculture and 0.810 for both Defense and Industrial. Those figures show variation between labels even though the macro-average was highest in the reported comparison.
BERT's lead comes with a comparison caveat
The most important qualification is that BERT's averages cover 10 of the 11 defined sectors because media was excluded. The paper says an all-model re-evaluation on the same 11-sector split is in progress, and estimates that media accounts for roughly 2% of label instances. Because the sector coverage differs, the cross-model averages are not strictly comparable. The results are point estimates; no confidence interval or other statistical uncertainty estimate was reported.
What remains untested
The authors present the findings as showing the feasibility of automated, sector-aware CTI dissemination and describe BERT as the strongest overall candidate. They also identify data availability, model interpretability and real-time scalability as unresolved challenges. Future work focuses on dataset diversity, explainable AI and evaluation in live threat-intelligence-platform settings. The stated next steps move beyond the scores reported here toward testing the approach in live settings.
The document is an arXiv version 1 preprint dated 28 Aug 2026. Its acknowledgements attribute the work to the Data for Development Fellowship program involving CSIRO, DFAT and the Embassy of Australia in Jakarta, Indonesia.
Paper data and sources
Original title: DisCTI: Who Needs to Know Timely? Automated Sector-Aware Cyber Threat Intelligence Dissemination
Authors: Fajar Wijitrisnanto, Alsharif Abuadbba, Yansong Gao, Nan Wu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text