A new arXiv preprint reports that STEP, a system for spotting unusual events in video from human poses, reached 90.1% AUROC on the full UBnormal test set and 90.9% on a Human-Related split. Across 20 independent training runs, the full-set result had a standard deviation of ±0.4 percentage points. The abstract describes the full result as a 12.2 percentage-point improvement over the previous skeleton-based state of the art.
The result is a benchmark finding, not a demonstration of improved safety, emergency response or privacy outcomes in human deployment. STEP works from two-dimensional human pose sequences extracted by a pose tracker, so its performance depends on whether people are detected and their movements are represented accurately enough for the model to score them.
A model built around tracked movement
STEP applies denoising score matching in a compact, whitened principal-component-analysis space. In practical terms, it represents pose sequences with a smaller set of transformed variables and normalizes their scales before learning an energy-based score for unusual sequences. It also adds sequence-level confidence weighting during training and test-time scoring, making the estimated quality of the tracked poses part of the detection process.
Its energy model uses a sigma-modulated residual multilayer perceptron. The noise scale is routed to every hidden layer, allowing the network to condition its calculations on the amount of noise used during score matching. The paper presents this architecture, the whitened pose space and confidence weighting as the main parts of STEP.
The default training used normal data for up to 400 epochs, with a batch size of 1,024. It used 10 noise scales ranging from 0.1 to 1.0, a temporal window of 12 frames and 48 retained PCA components. The default settings were selected on UBnormal validation data, while the main averages were calculated across 20 independent training runs.
Performance was measured with frame-level micro-averaged AUROC across all test frames. The evaluation also applied one-dimensional temporal Gaussian smoothing to the aggregated anomaly scores. That means the reported benchmark values should be read in the context of the stated scoring procedure, rather than as measurements from every possible real-time operating setup.
UBnormal supplied the clearest separation
The main evaluation used two benchmarks. ShanghaiTech contributed 330 training videos and 107 test videos across 13 scenes. UBnormal contained 543 videos across 29 virtual scenes: 268 for training, 64 for validation and 211 test sequences. These are separate datasets and splits, not one combined sample.
On UBnormal, STEP’s mean AUROC was 90.1% on the Full test set and 90.9% on the Human-Related split. The full-set uncertainty reported across the 20 runs was ±0.4 percentage points; the supplied analysis does not give a separate uncertainty figure for the Human-Related result. The abstract’s 12.2-point comparison is against the previous skeleton-based state of the art.
The ShanghaiTech comparison was much closer. STEP reached 86.2% AUROC on the Full set, with a standard deviation of ±0.1 percentage points, and 87.7% on the Human-Related split. The reported STG-NF values were 85.9% on the Full set and 87.4% on the Human-Related split.
Taken together, the reported comparisons put STEP furthest ahead on UBnormal and close to the cited STG-NF results on ShanghaiTech. They remain comparisons under the datasets, pose extraction and scoring procedures used in the study; they do not establish that STEP will outperform every alternative under different conditions.
Longer sequences exposed a weakness
The ablation studies tested how performance changed as the temporal window grew. In a raw-coordinate version of the model, the best reported AUROC was 80.7% at T = 2, or a two-frame window. Adding confidence weighting raised that short-window result to 87.2%. But the confidence-weighted raw-coordinate model fell to 73.1% at T = 32, showing that confidence information alone did not prevent the degradation seen with a longer window.
Separate component tests pointed to effects from the other design choices. The reported ablations found roughly a 1% gain from sigma-modulated conditioning, a 2.2% AUROC loss on UBnormal without whitening, and a 4.5% loss when poses below a confidence threshold of 0.4 were aggressively dropped. The pattern supports the paper’s argument that representation and confidence handling both matter to the final score.
These tests help show how the model behaved under the selected settings, but they do not turn each design choice into a universal rule. The long-window comparison and component breakdown are ablation results, while the headline benchmark figures are the results used for the main dataset comparisons.
A broader test produced lower full-set scores
A secondary evaluation used MSAD, a benchmark with 720 videos, 14 scene types and 11 anomaly categories. Under protocol (i), it evaluated 360 held-out clips, including 120 normal and 240 abnormal clips. STEP’s Online Smooth AUROC was 59.6% on the full benchmark, 74.1% under the UB-Style Human-Related evaluation and 71.6% under the ST-Style Human-Related evaluation.
The MSAD comparison requires caution because the reported Human-Related split style for the comparator values was unspecified. The cited values were 55.7% for STG-NF and 61.1% for SeeKer. The difference between STEP’s full-set and Human-Related scores also illustrates the boundary of a pose-focused detector: results can change substantially when evaluation includes anomalies not centered on recognizable human movement.
Fast scoring after poses are extracted
The reported scoring time was low once poses had already been extracted. On a GTX 1080, scoring took less than 1 millisecond for 50 people per frame, at approximately 1,026 frames per second and 32.6 MB of memory. For 100 people per frame, latency was 1.62 milliseconds, with approximately 618 frames per second and 44.9 MB of memory.
Those figures cover the scoring pipeline after pose extraction, not the upstream task of finding and tracking people in the video. The study reports dependence on the upstream pose estimator, and complete occlusion can prevent a score from being produced. The fast downstream timing therefore does not remove the need for reliable pose detection.
What the method cannot see
The authors report difficulty with anomalies defined solely by multi-person interactions or object manipulation when posture remains typical. A skeleton-based detector can also miss non-human anomalies when no detectable person is present. These blind spots matter because some unusual events are defined less by an individual’s posture than by objects, relationships or the wider scene.
The study evaluates benchmark data and extracted pose sequences rather than prospective real-world deployment. Its results do not establish improved safety, emergency response or privacy outcomes, and they do not show universal superiority outside the reported datasets and preprocessing conditions. The findings are best understood as evidence about benchmark anomaly scoring under the tested setup.
The supplied analysis identifies several questions for further testing: whether pose-based relational modeling can improve detection of multi-person anomalies, how the method behaves when the upstream pose estimator fails, and whether the benchmark comparisons can be reproduced with identical splits, pose extraction and evaluation protocols. Those questions are especially important for judging performance beyond the reported test sets.
A result still awaiting outside scrutiny
The supplied document identifies the work as an arXiv preprint, with no journal or peer-review status reported. Its funding acknowledgement names the Austrian BRIDGE programme under project ReCoDi (914991), with programme management authorized to FFG.
For now, STEP’s strongest reported result is its 90.1% AUROC on UBnormal, supported by a 90.9% Human-Related score and low post-extraction scoring latency. The same body of evidence shows why those figures should be read alongside the long-window decline, the lower full-set MSAD result and the method’s dependence on detecting and tracking people.
Paper data and sources
Original title: STEP: Score-Based Temporal Energy for Human Pose Video Anomaly Detection
Authors: Jakub Micorek, Mateusz Koziński, Horst Possegger
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-20
DOI: Not available
Original paper · Full text