The score that can miss the source
A preprint study found a sharp split between recognizing what an action is and preserving which source clip it came from. In a human-to-G1 comparison, a true counterpart scored +0.960 on Source-Instance Fidelity, or SIF, and 0.980 on action-level area under the curve, a measure of how well outputs rank action matches. A random same-action output scored -0.008 SIF but 0.978 on the action score. Both looked successful to the action test, while only the true counterpart retained the source-instance relationship.
SIF asks a different question from action recognition. It compares the differences between pairs of same-action source clips with the differences between their corresponding generated outputs, then reports a macro-average. The study used both raw and length-controlled versions and reported output variation alongside them. In ordinary terms, a high score means that distinctive differences between source clips tend to reappear in the corresponding outputs.
What the formal analysis says
The central theoretical warning concerns an unpaired shared-latent objective. That objective can match the separate motion distributions without deciding how the latent representations of different skeletons should line up. As a result, many different source-conditioned maps can fit the same evidence, even though they would produce different relationships between an individual source clip and its output.
A related result concerns training examples that pair a source clip with a randomly selected target from the same target cell. Under squared-error regression, the optimal prediction becomes the mean of that target-action cell rather than a response to the particular source clip. In a controlled ladder with between 2 and 50 examples per cell, random same-cell pairing left the regressor with about 2 percent of the oracle within-cell output variance.
A sparse benchmark exposes the gap
The main animal-motion benchmark was sparse by design. It contained 70 skeletons and 616 clips across 90 exact actions, creating 6,300 possible skeleton-action cells. Only 413 cells were occupied, a fill rate of 6.6 percent, and the median occupied cell contained a single clip.
The main SIF analysis used 49 source groups and 1,891 source-target-action triples drawn from the same 171 source clips. The groups covered eight actions, but attack and idle accounted for 94 percent of the triples. That distribution gives the source-instance test a demanding setting in which many exact source-action combinations are missing.
What the methods actually retained
Across 14 evaluated methods, nine sat at or near the source-blind floor under both raw and length-controlled SIF. Five rose above it. Motion2Motion-BVH remained close to zero, at +0.032.
Some systems retained a measurable, though limited, source signal. MoReFlow-T and MoReFlow-I stayed at about +0.25 SIF under both scoring methods and retained 16 to 21 percent of source variation. Motion2Motion-Direct was weaker, with +0.14 raw SIF and +0.13 after length control.
ACE-T and ACE-I scored +0.34 and +0.42 on raw SIF, but those values fell to +0.18 and +0.25 when length was controlled. Their outputs varied by only 2 to 3 percent of source variation in the raw analysis and by less than 1 percent when length was fixed. The comparison shows why the study paired raw scores with length-controlled scores and a separate variation measure.
An action-retrieval audit produced another warning. ANCHOR scored 0.757 overall and 0.681 on held-out queries. Random same-cluster retrieval scored 0.769 overall and 0.678 held out, while random same-exact-action retrieval scored 0.883 and 0.923. The exact-action comparison covered only 221 of 600 overall queries and 57 of 200 held-out queries, but it still matched or exceeded ANCHOR despite having no source-conditioned retargeting map.
The robot comparison
The human-to-G1 results separated the objectives more clearly. The unpaired objective reached +0.203 SIF, 0.499 action AUC and a 0.001 variation ratio. An averaging objective reached +0.379 SIF, 0.820 AUC and a 0.05 variation ratio. Training on true pairs reached +0.900 SIF, 0.967 AUC and a 0.52 variation ratio. The true counterpart itself scored +0.960 SIF and 0.980 AUC, while the random same-action comparison again had near-zero SIF but almost identical action AUC.
The pattern held in a broader robot check. Across six robots, two settings and two scoring modes, the true-pair model met both prespecified criteria in all 24 evaluations. The averaging objective failed the variation criterion in all 24, while the unpaired objective failed it in 22 of 24 evaluations.
What the result means
The study treated SIF as a diagnostic rather than a certificate of a unique solution. Uncertainty and floor tests used raw and length-controlled scores, 95 percent intervals built by resampling whole source skeletons and one-sided shuffle-test p-values. Variation was measured as the median ratio of output spread to source spread. The authors also note that a positive SIF alone does not establish a unique source-preserving map.
The formal conclusions are limited to the specified shared-latent marginal objective and the random cell-paired squared-error setup. The conditional-mean result does not apply to true instance-level pairs, and the evidence does not turn action-level success into proof that a particular source-conditioned map was recovered.
The practical message is straightforward: a model can recognize an action while reproducing a generic action pattern or a target-action habit. Evaluations of cross-skeleton retargeting therefore need an action-level score alongside a source-instance diagnostic and a check on how much output variation survives from the source clip.
Paper data and sources
Original title: Why Cross-Skeleton Retargeting Is Non-Identifiable: Structural Limits of Generative Motion Models
Authors: Zhiyuan Li, Wenyan Yang, Pekka Marttinen, Joni Pajarinen
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-29
DOI: Not available
Original paper · Full text