An adaptive probabilistic shield showed about the same number of safety violations as an oracle in most of the five benchmark environments, while unshielded reinforcement learning was generally less safe, especially in Crossroads and Gravity, according to an arXiv preprint dated 20 August 2026.
The oracle was a reference shield built from the ground-truth decision model. The adaptive version estimated that model from transitions observed during learning, updated its probabilistic shield and learned a policy in one loop.
Safe reinforcement learning is a way of teaching an agent to choose actions while also avoiding unsafe outcomes. The study presented its procedure as a single adaptive learning loop that estimated the environment from transition data, updated a probabilistic shield and learned a policy.
The comparison was built around trade-offs
The evaluation covered five environments with mixed safety and optimization objectives, 21 hyperparameter configurations and 100 repetitions of each run. The environments were modeled as Markov decision processes, or MDPs: decision maps describing states, actions and possible transitions.
The study compared the adaptive shield with reward-shaped learning without a shield and with an oracle shield. It also varied estimator attitude, exploration, the delay between shield updates and lookahead—the number of future steps considered. The main readouts included training violations, reward, final evaluation reward and the probability that the final policy produced an unsafe episode.
Plotted outcomes were means across the 100 repetitions with standard-deviation ribbons. The study reports no formal tests or inferential intervals for the final percentages, so the comparisons describe the reported runs rather than establish statistical significance.
Reward and safety did not line up
The final-policy figures showed different reward-safety balances in different environments. In Antlion, the two exploration variants were listed with reward and unsafe-episode probability pairs of 5.27 and 4.5% versus 1.93 and 5.1%. In Gravity, the pairs were 8.27 and 3.9% versus 1.58 and 0.6%.
The second number in each pair is the probability that the final policy produces an unsafe episode. The lower-reward variant had a slightly higher unsafe-episode probability in Antlion, but a lower one in Gravity, leaving no universal reward-safety ordering.
Exploration patterns also differed
In an illustrated Antlion execution, LUI and PAC followed narrow paths to the goal, while MAP almost never left the area around the initial states. The study describes these as differing exploration patterns associated with the estimator settings.
The model estimates improved most visibly at the start. A mean total-variation measure—a summary of the gap between the true and estimated transition probabilities—changed most at the first update after 1,000 episodes. Later transition estimates improved, but those changes had little effect on the global measure.
Before that first update at 1,000 episodes, robust and MAP estimates mainly used the fallback ∇κ, whereas optimistic estimates mainly used the threshold-based ∇θ.
Tuning had no universal winner
Results varied with shield-update delay, and no single delay was preferred across all five environments. Lookahead was similarly context-sensitive: lower lookahead was associated with lower safety in Gravity; when lookahead was 75 steps or less in Crossroads, the policy favored the more rewarding but less safe route; and horizons beyond 20 steps were redundant in Aircraft.
A benchmark result, not a deployment verdict
The authors interpret the overall pattern as suggesting that adaptive shielding can improve safety and enable safer exploration, while acknowledging that safety can reduce reward when risky behavior is profitable. The evidence does not establish that the adaptive method is safest or highest-reward in every environment, or that it reaches the ideal shield or policy within finite training.
The method assumes that the transition graph—the map of possible state changes—is known even though the transition probabilities are unknown. The evidence comes from benchmark computations, not from deployed real-world data; the authors also note that a conservative initial shield can block exploration and that the approach can fail in practice.
Paper data and sources
Original title: Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning
Authors: Astrid Horn Brorholt, Maris F. L. Galesloot, Nils Jansen et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text