DBcover, an LLM-driven framework for generating SQL tests, reported 80.1% line coverage on PostgreSQL 17.0 and 82.3% on MySQL 8.0.33 in the main comparison. Those were the highest reported values among the methods tested.
The evaluation asked whether DBcover improves RDBMS test coverage over existing generators, how its two-phase design contributes, and whether it works for a closed-source industrial database.
Line coverage, the share of source code lines exercised by tests, was the study's main yardstick.
The gap appeared in code coverage
On PostgreSQL, the alternatives reported 68.6% for Base, 69.0% for SQUIRREL, 31.5% for ShQveL and 75.7% for DBcovernse. On MySQL, they reported 71.2%, 71.4%, 25.7% and 77.2%, respectively. The comparison ran for 24 hours in a time-bounded experiment.
Relative to the seed baseline, DBcover's reported coverage differences were 11.5% on PostgreSQL and 11.1% on MySQL. SQUIRREL's corresponding differences were 0.4% and 0.2%.
How DBcover chooses what to test
DBcover's design is built around two stages. It first uses existing regression tests to build global context linking SQL inputs to the execution paths they take, then selects a nearby seed. The second stage combines local context from the target code with an LLM to mutate that seed into SQL aimed at improving coverage.
The implementation extracted source and call-graph information with Doxygen and LLVM, measured coverage with gcov and lcov, traced SQL paths offline with Pin, and stored the collected data in Neo4j.
The starting inputs came from official GitHub repositories for the RDBMS projects, and every method in the comparison received the same seed pool. DBcover and DBcovernse used qwen3:32b with a temperature setting of 0.7.
A test of the framework's design
The study also ran an ablation comparison using DBcovernse. DBcover reported 80.1% coverage on PostgreSQL and 82.3% on MySQL, compared with 75.7% and 77.2% for DBcovernse. The reported drops were 4.4% and 5.1%, respectively.
Across the module-level ablation, average coverage improvement was 2.71% over DBcovernse and 8.07% over Base.
A smaller test beyond open source
The framework was also evaluated on proprietary industrial DBMS KingbaseES, alongside open-source MySQL and PostgreSQL. Reported coverage in data type processing moved from 50.0% initially to 80.0% with DBcover, while metadata management moved from 68.0% to 81.4%.
These are results from the named KingbaseES subsystems, not a full-product coverage figure.
What SQL can still miss
The paper also describes a limit to SQL-driven reach. Some internal mechanisms lie outside the SQL interface, and overall coverage naturally plateaus around 80%. That leaves code paths that ordinary SQL input cannot reach, even when the generator is guided toward other target code.
The authors identify source-level context linking SQL inputs with their execution paths as the main explanation offered for DBcover's reported effectiveness. Taken as a whole, the evaluation is a report of line coverage under a shared seed pool, a stated model configuration and 24-hour runs across two open-source systems, with subsystem results for one proprietary system.
Paper data and sources
Original title: DBcover: A White-box SQL Test Generation Framework for Coverage Improvement
Authors: Yankai Rong, Shuang Liu, Jinhao Dong et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text