A preprint's KENDO method produced leading results across tests of Bayesian optimization, multi-objective optimization and active learning, while using less computation per iteration than MCMC-based comparators. The evidence comes from computational benchmark tasks, not human or animal participants.
A different way to choose a model
The central question was whether replacing MCMC hyperparameter sampling, a way of exploring possible model settings, with a discrete kernel ensemble would degrade optimization quality. In practical terms, the study tested whether several kernel choices could replace that sampling step.
KENDO combines several Gaussian process prediction models, each built with a different kernel, with an acquisition rule that selects the next evaluation while paying attention to disagreement among the models. It replaces hyperparameter sampling with a kernel ensemble and updates the kernels' influence through Bayesian predictive-likelihood weighting. KENDO-BO conditions on sampled optimizers, KENDO-AL removes that optimizer conditioning, and KENDO-MO extends the setup to multiple objectives through random scalarization.
Strong results across three task families
Researchers evaluated the method on synthetic and real-world tasks, including simulator and hyperparameter-optimization problems. The sample comprised nine single-objective Bayesian-optimization benchmarks, six multi-objective benchmarks and seven active-learning benchmarks. Main performance curves averaged 25 random seeds, and single-kernel baselines were additionally averaged across three kernel variants.
In single-objective optimization, KENDO-BO matched or outperformed every baseline on all nine benchmarks. It kept the top average rank as the available budget grew and consistently beat EGP-TS, a comparator using the same shared kernel ensemble and weighting scheme.
The multi-objective results were also favorable, although the lead was not universal on every problem. KENDO-MO recorded the lowest log10 hypervolume difference on most of the six benchmarks and ranked first across the full budget, with its largest margins reported on Penicillin and CarSideImpact.
A separate KENDO-MO-PF version, conditioned on the Pareto front, underperformed KENDO-MO consistently across all six benchmarks. The paper describes random scalarization as retaining focused single-point conditioning while using diverse objective weights to cover the Pareto front, the set of best trade-offs. That result is specific to the six tested benchmarks.
On the active-learning tasks, KENDO-AL achieved the lowest negative marginal log likelihood on all seven benchmarks and had the best aggregate average rank. The score was negative marginal log likelihood on held-out validation sets, so the comparison included data reserved for checking the models' predictions.
The timing results showed a marked computational difference. KENDO was 3.1 to 5.4 times faster than MCMC-based SCoreBO in the Bayesian-optimization comparison, while KENDO-AL was 7.6 to 26.9 times faster than MCMC-based SAL in active learning. Updating the kernel weights took less than 7 milliseconds per iteration.
What the design tests found
An ablation tested whether the weighting scheme mattered. Adaptive Bayesian weighting outperformed uniform averaging across all three task families, with a larger gap where preferred kernels varied across input regions.
The weights did not always remain evenly balanced. On smooth stationary benchmarks such as Branin, one kernel became dominant within roughly 10 to 20 iterations. In single-objective Branin, Matérn-2.5 was the winner in five of 10 seeds and had an average maximum weight of 0.950.
The authors also tested the concentration setting used in the multi-objective weighting. Across five MOO benchmarks, the tested values were 0.2, 1.0 and 5.0. The default value of 1.0 was never significantly outperformed, although the best setting varied by benchmark and tracked Pareto-front geometry.
Changing the scalarization rule produced mostly small or statistically unclear differences. A two-sided Wilcoxon rank-sum test found no significant final-hypervolume difference for ZDT2 (p = 0.49), Penicillin (p = 0.16) or Vehicle Safety (p = 0.85). Chebyshev scalarization had a small significant advantage on DTLZ2 (p = 0.014).
The boundaries of the evidence
The evaluation was confined to the listed computational benchmarks and implementations, and the supplied results do not include complete per-benchmark effect sizes or uncertainty intervals for most curves. The authors identify a risk in moment matching, the step that compresses the ensemble's predictions into one approximation: it may underestimate uncertainty when kernel predictions differ greatly. They also require the kernel dictionary to be specified manually. The evidence therefore does not establish performance beyond the listed benchmarks and implementations.
Paper data and sources
Original title: Enhancing Bayesian Optimization and Active Learning Through Kernel Diversity
Authors: Heng Zhang, Haotian Xiang, Qin Lu et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text