A Bangla medical question-answering system reported a 0.95 score across accuracy, precision, recall and F1-score on a manually constructed test set. The system also covered every test query under its soft-voting method, according to the preprint.
F1-score is a single measure that balances how often a system finds the right answer with how often its answers are correct. Those figures describe matching performance on the authors’ test material, not how the system would perform in clinical use.
How the system works
The system, called BanglaMed-QA, was designed to answer Bangla healthcare questions by matching them with stored questions and answers. Its knowledge base contains 4,493 question-answer pairs in nine categories related to 506 diseases.
To handle variations in wording, dataset preparation included part-of-speech tagging for resolving references within a question, a 617-word Bangla medical root-word dictionary and synonym sets. The system then used cosine, Jaccard, BM25 and Levenshtein similarity measures, along with soft- and hard-voting ensembles that apply a threshold when combining matches.
For question categorization, the researchers converted text into numerical features with TF-IDF, a way of giving more weight to words and word combinations that help distinguish one category from another. They explored five supervised machine-learning methods, using five-fold cross-validation for methods other than Multinomial Naive Bayes, and selected a support-vector machine, or SVM, as the question classifier after evaluating the alternatives.
Results came from constructed queries
The question-answer dataset was split into 80% for training and 20% for testing. The evaluation itself used 880 manually constructed input queries, each with a target response and category.
Soft voting produced 1.0 coverage and 0.95 for each reported metric. Hard voting had 0.94 coverage and a 0.94 score for each reported metric. Among the individual matching methods, cosine, Jaccard and Levenshtein each had an F1-score of 0.94, while BM25 had 0.93.
The authors also report that categorizing a question speeds up similarity matching by around one-ninth. The study gives no timing measurements or uncertainty for that estimate.
A small human check was favorable
In an interactive evaluation, 10 student evaluators each asked 10 Bangla medical questions and rated the responses on a scale from 0 to 1. Ratings ranged from 0.78 to 1.0, with an average of 0.9.
That review suggests the participants generally rated the answers favorably, but it was small and did not represent Bangla-speaking patients or healthcare professionals. The reported results also came from the authors’ own dataset and manually written queries, rather than an external benchmark.
What the results do not establish
The reported evaluation covered automated metrics on manually constructed test queries and a satisfaction rating from students. It did not include clinical outcomes or patient-facing deployment, so it does not establish clinical effectiveness, safety, completeness or reliability for unsupervised medical advice.
The paper’s reported material does not give full details of how online sources were selected, how the dataset was annotated or what independent quality checks were used. It also does not provide confidence intervals, significance tests or a formal rationale for the evaluation sample size. Exact comparative metrics for the question-categorization models are not included in the available tables.
The paper is identified as arXiv:2608.28329v1 and dated 28 August 2026. Its front matter also states that it was accepted at the 3rd International Conference on Big Data, IoT and Machine Learning, known as BIM 2025.
The authors present BanglaMed-QA as a foundation for future Bangla medical language-processing work. Independent testing, expert review and assessment in patient-facing settings would be needed to determine how well the reported performance generalizes and whether its answers are clinically appropriate.
Paper data and sources
Original title: BanglaMed-QA: A Question Answering System for Healthcare Support in Bangla
Authors: Rowzatul Zannat, Abdullah Al Shafi, K. M. Azharul Hasan, Atia Shahnaz Ipa
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text