An arXiv preprint gives a conditional answer to whether a centered and normalized sum from a higher-order Markov chain can converge to the standard normal distribution as the chain’s order grows with sequence length. The analysis represents each chain through tuples of earlier symbols and reaches that conclusion only under assumptions about ergodicity, visits to a selected state, return-count fluctuations and cycle moments. It is therefore a theorem for a defined asymptotic regime, not a blanket statement about all higher-order chains.
The paper allows the order m_n to diverge with n, but only in a regime where m_n/n → 0. In other words, the order may increase as the sequence gets longer, but it must remain small relative to sequence length in the limit. To analyze that changing-order problem, each higher-order chain is represented as a first-order chain on the tuple state space Σ^{m_n}.
The result turns on returns
The central result is built around a suitably chosen state and the way the chain returns to it. The normalization uses the stationary return time to that state together with the variance of the centered sum accumulated over a return cycle. After that centering and scaling, the theorem states convergence in distribution to N(0, 1), the standard normal law.
That result has a narrow gate. The chains must be aperiodic and irreducible. The selected state sequence α_n must have a diverging visit scale, nπ_n(α_n) → ∞, and the variance of the return count must vanish in the stated ratio. In plain language, the chosen state must not become too rare on the scale needed by the argument, and fluctuations in how often it is visited must be controlled.
The theorem also imposes finite second-moment and uniform-boundedness conditions on centered cycle sums and on their absolute-value counterparts. These requirements are part of the conditions under which the normal limit is claimed. Without them, the supplied result does not establish the same conclusion.
Why the proof uses cycles
The proof’s key move is to cut the chain into return cycles, meaning the stretches between visits to the selected state. The strong Markov property yields iid sums for those cycles, allowing the argument to use an iid central-limit theorem within the changing rows of the asymptotic array. This cycle-based construction connects the return-time normalization to the claimed limit.
In practical reading, the result says that the normalized running quantity behaves like a standard normal variable only after the model has met the structural and moment requirements. It does not say that increasing the order automatically preserves central-limit behavior, nor that the same normalization works without a suitably chosen state.
A binary model supplies a concrete condition
To make the state-mass condition concrete, the paper studies a binary variable-length Markov chain, or VLMC, using contexts of different lengths. Its listed contexts are 0, 10, 110, and so on through 1^(m−1)0, alongside 1^m, giving m+1 leaves. The construction is an illustration of the theorem’s setting.
For this binary construction, the reciprocal stationary mass of the state 1^(m−1)0 is represented by an explicit quantity q(p,m). That expression links the model’s stationary probability to the theorem’s requirement that a selected state be visited on a diverging scale as n grows.
The paper states that q(p,m_n) = o(n) is sufficient for nπ_n(1^(m_n−1)0) → ∞ in the binary construction. In plain terms, the condition keeps the chosen state from becoming too rare, on the scale needed by the theorem, as the sequence length increases.
A more specific corollary considers transition probabilities p_j = (j+1)/(j+2). For that stated choice, the sufficient growth regime becomes m_n log(m_n)/n → 0, which again guarantees divergence of the selected state’s expected visit scale in the binary example. The formula is an explicit condition for this construction, not a general rule supplied for every sparse higher-order model.
What remains outside the theorem
The work is theoretical rather than an empirical performance study. It describes triangular arrays of finite-alphabet higher-order chains and a constructed binary VLMC example; it does not report an observed dataset, a simulation, a prediction test or a comparison of estimators.
That distinction matters because the conclusion is asymptotic. The supplied analysis reports no finite-sample uncertainty measure, error rate or convergence rate, so the theorem does not say how quickly a finite sequence approaches the normal limit. It also does not establish the result when the aperiodicity, irreducibility, state-mass, return-count or moment assumptions fail.
The explicit growth result is narrower than the broader class named in the paper’s motivation. It is derived for the specified binary VLMC; analogous conditions for SMMs and other sparse higher-order models are left as future work. The paper therefore does not establish a central-limit theorem for general SMMs.
Taken on its own terms, the preprint offers an asymptotic foundation for inference in sparse or partitioned higher-order models, including VLMCs and SMMs. The open questions are whether comparable stationary-mass and return-time conditions can be found for those broader models, and whether weaker assumptions or faster order growth could support a similar result.
The document is arXiv:2608.20321v1, dated 20 August 2026. It is a preprint, and its main result remains explicitly conditional: in the stated regime, the normalized centered additive functional converges to N(0, 1), while broader model classes and finite-sample behavior remain unresolved.
Paper data and sources
Original title: Large Sample Properties of Higher Order Markov Models
Authors: Tuhin Majumder, Donald E. K. Martin, Soumendra N. Lahiri
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text