An optical-flow feature tied to human joints showed higher reported action-classification accuracy than skeleton-only versions of the tested models in benchmark evaluations. Across the three datasets, the average gain was 2.14% on NTU RGB+D 60, 6.84% on NTU RGB+D 120 and 11.22% on UCF101. In the displayed full-sequence results, every PoseOFF row was higher than its corresponding base row across the listed NTU and UCF101 splits. The base and PoseOFF-augmented models were trained with identical settings.
The result addresses a specific problem: deciding what a person is doing before a video sequence is complete. The preprint asks whether local optical-flow features anchored to human joints can support earlier action anticipation from partial observations while remaining practical for low-latency human-robot interaction systems. The work reports a benchmark comparison, not a demonstrated change in robot behavior.
The motion signal stays close to the body
PoseOFF works by taking a small N-by-N window of optical flow, a representation of local image movement, around each human keypoint. The window produces a local motion feature with shape (N, N, 2), which is combined with the pose information for that keypoint. The result is a local, joint-centred input rather than a motion signal detached from the person’s body position.
The implementation used Python and PyTorch with RAFT for optical-flow extraction. It used dataset pose annotations for NTU RGB+D 60 and NTU RGB+D 120, while keypoints for UCF101 were estimated with YOLO-POSE Large. PoseOFF was evaluated through the skeleton-joint stream of InfoGCN++, MS-G3D and ST-GCN++.
Early prediction is a separate test
The evaluation covered NTU RGB+D 60, NTU RGB+D 120 and UCF101. The two NTU datasets use 25 body joints and contain 60 and 120 action classes respectively; UCF101 was assessed through its three standard train/test splits.
To simulate partial observation, the researchers used temporal masking. MS-G3D and ST-GCN++ were trained on full sequences at 100% observation, then evaluated in 10% observation increments. InfoGCN++ instead used a fixed 64-frame input without interval masking. The study also used area under the curve, meaning the average accuracy across the evaluated observation ratios.
This setup separates the headline full-sequence percentages from the question of how well a model performs when only part of an action is visible. Accuracy at predefined observation ratios was one of the study’s primary outcomes, alongside full-sequence accuracy.
A useful benchmark signal, with clear boundaries
The reported average improvement was smallest on NTU RGB+D 60 and largest on UCF101. Those figures should be read as benchmark comparisons. The paper does not report confidence intervals, standard errors or replicate variability for the percentages.
The study stays within computational benchmark experiments, using three datasets and three skeleton-based model backbones. It does not report an embodied robot deployment, so the results do not establish faster response, safer coordination or greater trust in human-robot interaction.
PoseOFF has practical dependencies of its own. Its local features rely on pose estimation and motion extraction, and the authors note that severe occlusion or highly dynamic global motion could reduce effectiveness. For UCF101, the estimated keypoints may be misplaced or dropped, creating noisy flow patches in cluttered scenes.
Protocol details also matter when comparing the models. InfoGCN++ did not use the same interval-masking setup as MS-G3D and ST-GCN++, so early-observation results should be interpreted with that difference in mind.
The next test is whether the pattern survives outside the reported datasets, model backbones and evaluation settings. Direct matched comparisons with dense or parallel motion representations, together with end-to-end tests on robot hardware, would be needed to determine whether earlier benchmark recognition translates into better interaction.
Paper data and sources
Original title: Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming
Authors: Lewis de Zoete Grundy, Chris McCarthy, Christopher Fluke
Journal/Repository: Proceedings of the 35th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2026) in Kitakyushu, Japan
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text