An arXiv preprint applying a hierarchical structural causal model (HSCM) to Project STAR reports a 36.80 contrast in Mathematics. In the model's ExactBIC run, the output was 2402.75 under do(1) and 2365.95 under do(0), on the evaluator scale. No confidence interval or standard error is reported for this HSCM contrast.
The number is a class-level calculation, not a row-by-row replacement of each student's assignment. The HSCM exposure fixes the within-class distribution of small-class assignments through a class-level variable, Qa. The comparison is between do(Qa = 1) and do(Qa = 0), two imposed scenarios in the model.
The paper describes Project STAR as randomized within schools across small, regular and regular-with-aide class types. After missing-value removal, it reports 5,745 students in 322 classes and 79 schools. For the HSCM analysis, the data were balanced at 10 students per class, producing 3,220 student records across the 322 classes.
How the estimate was defined
The computational idea is to split a complicated identified formula into smaller jobs. The pipeline translates the formula into an abstract syntax tree, or AST: a structured set of conditional-density, expectation and marginalization tasks that can be fitted locally and evaluated in parallel. That design links graph-based identification with numerical estimation.
The authors tested the machinery on controlled synthetic hierarchical models covering confounding, confounding with interference or a frontdoor path, and instrumental-variable motifs. Because those models have known ground truth, the tests could check the transformed graph, identification, AST evaluation and numerical estimators. In the convergence test, with 50 sub-units per unit, the hierarchical estimate moved toward the known benchmark as the number of units increased from 10 to 200, while pooled ordinary least squares (OLS) remained biased because it mixed unit-level latent heterogeneity with the treatment effect.
The STAR numbers sit alongside, rather than replace, more familiar regression checks. The reported small-class coefficients were 8.096, 9.499, 9.380, 9.432 and 9.099 across five specifications. A separate instrumental-variable estimate of the class-size coefficient was -1.219, with a standard error of 0.163. These figures target a different quantity from the HSCM contrast, so they are not a like-for-like ranking of the methods.
Built for scale
Scalability is a second part of the proposal. In a synthetic benchmark, the fitted log-log GPU speedup slope averaged about -0.32 across motifs. GPU evaluation was described as thousands of times faster in absolute time, although the speedup declined over the measured range. On the CPU, one STAR evaluation took 3.11 seconds sequentially and 1.9 seconds with four processes, a speedup of 1.64. The hardware results are benchmark-specific.
The numerical details also change what is being evaluated. The paper says that normalizing a multiplicative factor changes its induced weighting and therefore changes the numerical functional. In one unit-context diagnostic, the fitted response mean changed from 434.83 under do(Qa = 0) to 440.15 under do(Qa = 1), a reported change of 5.32. That is a factor-level diagnostic, not an independent primary outcome estimate.
Why the figure is conditional
That dependence begins with the graph. The STAR translation used a graph selected with ExactBIC because it was judged the most informative of the discovered structures, but the authors did not claim that it was the uniquely recovered causal graph. Graph choice is therefore part of the reported result.
The hierarchy is also incomplete. The implementation has two levels, student and class, and handles schools through unit-level covariates rather than a full student-class-school model. In practice, the effective HSCM sample size is the number of classes, not the number of student rows; in this balanced analysis, that means 322 class-level units despite 3,220 student records.
Results outside Mathematics are presented as diagnostics. A separate ExactBIC normalized-factor Reading run reported 1788.10 under do(1), 1766.47 under do(0) and a contrast of 21.63. The Reading result is presented as a separate diagnostic rather than the primary Mathematics analysis.
Taken together, the reported figure is best read as a model-dependent class-level calculation. It depends on the selected graph, the normalization used in the computation and a two-level representation of the data, so it should not be treated as a direct student-level regression coefficient.
The public-use STAR data are described as available through Harvard Dataverse, while the HSCM implementation and generated artefacts are maintained in the cited GitHub repository. The work is identified as an arXiv preprint.
Paper data and sources
Original title: Scalable and Versatile Identification for Hierarchical Structural Causal Models: A New Look at Project STAR
Authors: Janis Aiad, Aghiles Drali, Aymen El Ouadrhiri et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text