Monte Carlo Tree Search may be less separate from a familiar learning method than its name suggests. An expository arXiv preprint argues that, when the comparison is made at the level of trajectory sampling and Monte Carlo action-value updating, MCTS is every-visit Monte Carlo control expressed in search terminology. The conclusion is deliberately limited to that shared mechanism and scope.
A question about the engine
The note is a conceptual analysis, not an empirical study. It frames the comparison around an episodic Markov decision process and a sampled trajectory. That boundary keeps attention on how a run is generated and how information from it is turned into action-value estimates, rather than on a performance test.
Within that frame, the two methods are described as repeating the same two operations. They sample a trajectory under the current policy, then use Monte Carlo returns from that trajectory to update the action values of the state-action pairs visited along the way. The proposed equivalence rests on that sampling-and-update cycle.
Why every visit matters
Every-visit Monte Carlo estimation has a simple rule: whenever a particular state-action pair appears in a trajectory, its action value is updated. If the pair appears more than once, the update is applied on each visit. The note identifies MCTS backup with the sample-average every-visit Monte Carlo update.
In MCTS language, backup is the operation that propagates information from a simulated return through the stored search representation. The note reads that operation as ordinary every-visit updating. At the level under discussion, the difference between a search backup and a Monte Carlo value update is therefore a difference in vocabulary and presentation.
Four stages, one sampling process
The same compression applies to the standard MCTS stages. Selection, expansion and simulation are treated together as trajectory sampling, while backup is the every-visit Monte Carlo update. Selection uses the learned part of the current policy; simulation continues with the initial part used where action-value information has not yet been learned.
That is why the note does not treat tree policy and rollout policy as fundamentally different policy objects. In the learned region, the tree policy is the current policy; in the unlearned region, the rollout policy is the initial part of that current policy. Selection and rollout are thus portions of one trajectory-sampling procedure.
Expansion receives a similarly narrow interpretation. It is a first visit plus lazy initialization: the algorithm encounters a previously unrepresented state or action and creates its stored information on demand. In the note's mapping, expansion belongs to the sampling process and initialization, not to a separate Monte Carlo update.
What the tree and UCB add
On this reading, the search tree is a representation choice for the statistics. The same information could in principle be stored in a tree, table, hash map or another suitable data structure. The tree gives the method a search-oriented form, but the note does not treat that storage choice as the source of the Monte Carlo update.
UCB and UCT are also kept in a separate category. The note treats them as exploration or action-selection rules, rather than as the definition of the Monte Carlo update. That leaves the choice of how actions are selected distinct from the rule for updating values from sampled returns.
A claim with a clear boundary
The analysis draws a line between planning and learning through computational use and scope, not necessarily through different underlying Monte Carlo control mechanisms. Its proposed equivalence concerns what trajectory is sampled and how visited action values are updated. It does not claim that every implementation detail, data structure or selection rule becomes interchangeable.
That limit matters for the practical questions left open. The note does not establish empirical equivalence or show that one method performs better on a benchmark, because it reports no performance comparison. It also does not show that all tree representations or action-selection rules have identical efficiency or exploration behavior.
Systems that combine search with learned policy and value functions are another unresolved area in the note. It identifies the need to examine how the correspondence applies there, along with how different action-selection rules affect exploration and efficiency.
For general readers, the contribution is therefore a reframing. The preprint does not supply a new score, test result or promise of better decisions. Instead, it offers a common lens for comparing two bodies of terminology: generate trajectories under a current policy, update every visited state-action value from Monte Carlo returns, and then consider how the chosen representation and selection rule affect the computation.
The preprint lists Xianyi Wu of ECNU as its author and reports no funding statement. It is an arXiv version 1 preprint dated 28 Aug 2026.
Paper data and sources
Original title: Is Monte Carlo Tree Search Just Every-Visit Monte Carlo Control?
Authors: Xianyi Wu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text