Preprint

Preprint finds standard errors falter in small trials with many covariates

Simulations suggest G-computation may be less sensitive to bias than logistic treatment-coefficient estimates, while uncertainty calculations become less reliable as covariate numbers rise.

A methods preprint suggests that in small randomized trials with many baseline covariates, G-computation can be less sensitive to bias than a conventional logistic treatment-coefficient estimate, but the uncertainty around those estimates can still be unreliable. In simulations, standard G-computation and post-LASSO had increasingly unreliable variance estimates as the covariate ratio rose, while centered HOIF generally performed best.

G-computation is a way of estimating a marginal average treatment effect, the average difference between treatment strategies across the trial population. The study asked whether the finite-sample bias seen in conditional generalized linear model treatment-effect estimates also affects G-computation estimates. That is why the simulations looked at both bias and the reliability of standard errors, not only the headline estimate.

What the simulations tested

The simulations used samples of 50, 100 and 200 in the first two settings, and 124 in a third. For continuous outcomes, the comparison included standard G-computation, post-LASSO, cross-fitted G-computation, cross-fitting combined with post-LASSO, JASA, HOIF and an unadjusted estimator. Cross-fitting used five folds, meaning the model was trained on part of the data and evaluated on held-out observations.

For binary, yes-or-no outcomes, the researchers added Bayesian logistic regression with a Cauchy prior and Firth-based G-computation, then applied targeted maximum likelihood estimation, or TMLE, to the Bayesian and Firth estimators. Each estimator-setting combination was assessed over 1,000 simulation runs using bias, Monte Carlo standard deviation, average standard error and coverage of nominal 95% confidence intervals. Coverage here means the proportion of repeated runs in which the stated interval captured the target.

Where the estimators diverged

For continuous outcomes, standard G-computation and post-LASSO had the least reliable variance estimates as the number of covariates rose relative to the sample. Centered HOIF, calibrated JASA and cross-fitting with post-LASSO gave better coverage, with centered HOIF generally the strongest performer. A theoretical derivation also indicated that centering the outcome can remove the remaining own-observation-bias contribution in leave-one-out estimation.

For binary outcomes, G-computation was less sensitive to bias than the logistic treatment-coefficient maximum likelihood estimate. Firth-based variance estimates became invalid as covariate count increased. Cross-fitting's coverage was below the nominal level at n=50 but improved substantially at n=200; centered HOIF performed best, while calibrated JASA was slightly below nominal.

Power results added a warning. G-computation, post-LASSO and Firth methods had the highest apparent power, but the paper attributes part of that advantage to inflated type-I error, meaning too many false positives under a no-effect scenario. Cross-fitting, HOIF and JASA had lower but more credible power than the unadjusted estimator.

A trial example

The paper then turned to the BestAIR source sample for a complete-case reanalysis. It contained 169 patients and 114 baseline covariates. After excluding missing outcomes, 124 complete cases remained; covariates with 10 or more missing values were excluded, leaving 67. Missing covariates were mean-imputed.

To vary covariate burden, the comparison used sets of 6, 50 and 67 covariates. They corresponded to covariate-to-sample-size ratios of 0.05, 0.40 and 0.54 and were selected by mean squared error.

At the 0.54 covariate ratio, standard G-computation estimated the continuous systolic-blood-pressure average treatment effect at -2.45, with a 95% confidence interval from -4.70 to -0.20. For the binary measure of high systolic blood pressure, the estimate was -0.07, with an interval from -0.12 to -0.02.

But those figures do not identify a single best covariate set. Smaller subsets produced narrower intervals, while treatment estimates varied across the ratios. The authors suggested imprecision and covariate imbalance as possible explanations, and said the reanalysis did not establish which covariate set gives the most reliable clinical estimate.

Why the findings remain limited

The study's boundary is important. The main continuous-outcome simulations used correctly specified outcome models; the authors note that misspecification can reduce efficiency and identify broader outcome types as future work. The results therefore do not show how the methods perform under model misspecification or across broader outcome types.

For people analyzing small randomized trials, the message is mixed. G-computation may be less sensitive to the bias seen in conditional treatment-coefficient models, but standard errors still need scrutiny when covariate burden is high. In the tested settings, centered HOIF, calibrated JASA and cross-fitting with post-LASSO had better coverage, while the methods with the highest apparent power also had inflated type-I error.

Paper data and sources

Original title: On the use of G-computation in small randomized controlled trials with many covariates
Authors: Muluneh Alene, Kelly Van Lancker, Stijn Vansteelandt
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text

Versions and corrections

  1. Published automatically after legal-source, freshness, evidence, and independent-verification gates passed.