OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction

OmniRetarget：面向人形机器人全身运动–操作与场景交互、保持交互关系的数据生成

Lujie Yang¹ ², Xiaoyu Huang¹ ³, Zhen Wu*¹,
Angjoo Kanazawa†¹ ³, Pieter Abbeel†¹ ³, Carmelo Sferrazza†¹, C. Karen Liu†¹ ⁴, Rocky Duan†¹, Guanya Shi†¹ ⁵
¹ Amazon FAR (Frontier AI & Robotics), ² MIT, ³ UC Berkeley, ⁴ Stanford University, ⁵ CMU

• Equal contribution, work done while interning at Amazon FAR. † Amazon FAR team co-lead.

Fig. 1: OMNIRETARGET enables reinforcement learning policies to learn complex, long-horizon loco-manipulation skills in challenging environments that transfer zero-shot from simulation to a Unitree G1 humanoid. Thanks to the high-quality interaction-preserving motion retargeting, these policies are trained and deployed in a minimal and unified way: it involves only 5 rewards, 4 robot domain randomization terms, and a purely proprioceptive observation space, shared by all tasks. Demonstrated behaviors include (a) 30-second parkour course involving chair moving, stepping & vault, and jump & roll, (b) object transportation, (c) crawling on a slope, and (d) fast platform climbing and sitting.

图 1：OMNIRETARGET 使强化学习策略能够在具有挑战性的环境中学习复杂、长时程的运动–操作技能，并从仿真零样本迁移到 Unitree G1 人形机器人。得益于高质量、保持交互关系的动作重定向，这些策略的训练和部署方式极简且统一：全部任务共用 5 项奖励、4 项机器人域随机化，以及纯本体感觉观测空间。展示的行为包括（a）约 30 秒的跑酷，含搬椅、踏步与翻越、跳跃与滚翻，（b）物体搬运，（c）斜坡匍匐，以及（d）快速爬上平台并坐下。

Abstract

摘要

A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning (RL) policies. However, existing retargeting pipelines often struggle with the significant embodiment gap between humans and robots, producing physically implausible artifacts like foot-skating and penetration. More importantly, common retargeting methods neglect the rich human-object and human-environment interactions essential for expressive locomotion and loco-manipulation. To address this, we introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh that explicitly models and preserves the crucial spatial and contact relationships between an agent, the terrain, and manipulated objects. By minimizing the Laplacian deformation between the human and robot meshes while enforcing kinematic constraints, OmniRetarget generates kinematically feasible trajectories. Moreover, preserving task-relevant interactions enables efficient data augmentation, from a single demonstration to different robot embodiments, terrains, and object configurations. We comprehensively evaluate OmniRetarget by retargeting motions from OMOMO [1], LAFAN1 [2], and our in-house MoCap datasets, generating over 8-hour trajectories that achieve better kinematic constraint satisfaction and contact preservation than widely used baselines. Such high-quality data enables proprioceptive RL policies to successfully execute long-horizon (up to 30 seconds) parkour and loco-manipulation skills on a Unitree G1 humanoid, trained with only 5 reward terms and simple domain randomization shared by all tasks, without any learning curriculum. All code, retargeted datasets, and trained policies will be publicly released. Result videos can be found at https://omniretarget.github.io

教人形机器人掌握复杂技能的主流做法，是把人体动作重定向为运动学参考，再用来训练强化学习（RL）策略。然而现有重定向流程常常难以跨越人与机器人之间显著的形体差异，产生脚滑、穿透等在物理上不合理的伪影。更重要的是，常见方法忽视了富于表现力的运动和运动–操作所必需的人–物、人–环境交互。为此我们介绍 OmniRetarget：一个基于交互网格、保持交互关系的数据生成引擎，它显式建模并保持智能体、地形与被操作物体之间关键的空间与接触关系。通过在施加运动学约束的同时最小化人体网格与机器人网格之间的拉普拉斯形变，OmniRetarget 生成运动学上可行的轨迹。而且，保持与任务相关的交互还使数据增强变得高效：只需一条示范，就可以迁移到不同机器人形体、地形和物体配置。我们用 OMOMO [1]、LAFAN1 [2] 以及内部动作捕捉数据做了全面评估，生成超过 8 小时的轨迹，在运动学约束满足和接触保持上优于常用基线。这些高质量数据使仅用本体感觉的强化学习策略，能够在 Unitree G1 人形机器人上成功执行长达 30 秒的跑酷和运动–操作技能；全部任务共用 5 项奖励和简单的域随机化，也不需要学习课程。全部代码、重定向数据集和训练好的策略都将公开。结果视频见 https://omniretarget.github.io

I Introduction

I 引言

The quest to enable humanoid robots to perform complex whole-body scene- and object-interaction tasks has long been constrained by a fundamental data bottleneck. While deep reinforcement learning (RL) has shown remarkable success in robot control, efficient exploration is highly sensitive to reward engineering [3]. This challenge is further amplified on humanoids, whose high-dimensional action spaces and complex dynamics make learning natural, expressive behaviors from scratch both difficult and inefficient.

让人形机器人完成复杂的全身场景与物体交互，长期以来受制于数据这一根本瓶颈。深度强化学习（RL）在机器人控制上已经相当成功，但高效探索对奖励设计非常敏感 [3]。这一困难在人形机器人上更为突出：动作空间维数高、动力学复杂，从零学习自然、有表现力的行为既困难，也没有效率。

To address these challenges, imitating human motions offers a powerful alternative for learning whole-body control, especially for complex scene interactions. Human demonstrations capture dynamic coordination, such as lifting objects while walking on uneven terrain, and have been used effectively in animation [4, 5, 6]. A critical challenge arises in robotics: unlike virtual characters, physical humanoids only approximate human morphology, with significant differences in shape, proportion and degrees of freedom. This embodiment gap means that simply adapting human motions is insufficient; it is essential to also adapt their scene interactions to the robot’s specific form to generate usable references.

为应对这些挑战，模仿人体动作是学习全身控制、尤其是复杂场景交互的一条有力路径。人类示范记录了动态协调，例如在不平坦地形上边走边搬物体，并已在动画中得到有效使用 [4, 5, 6]。机器人领域则有一个关键难点：与虚拟角色不同，实体人形机器人只是近似人体形态，在外形、比例和自由度上都有明显差异。这意味着只改写人体动作还不够；还必须把场景交互适配到机器人特有的形体上，才能得到可用的参考。

To this end, researchers have pursued two main strategies. The first one is teleoperation [7, 8, 9], where only a human operator’s motions are retargeted to control the robot online. This approach leverages the human operator for real-time adaptation, which sidesteps the need for automatic interaction retargeting. However, despite the advantage of online feedback, the method remains labor-intensive and does not scale well for large-scale data generation. The second and more scalable strategy is offline interaction retargeting, which holistically adapts both the human’s motion and their scene interactions to the robot’s specific embodiment.

为此，已有研究主要沿两条路走。第一条是遥操作 [7, 8, 9]，只把操作者的动作在线重定向到机器人上。这种方法借助操作者做实时适应，从而不必自动重定向交互。尽管有在线反馈的好处，它仍然费人力，难以支撑大规模数据生成。第二条、也更可扩展的策略是离线交互重定向：把人体动作及其场景交互一并适配到机器人特有的形体。

However, most existing retargeting methods [10, 9, 11] fall short in this regard. They predominantly rely on unconstrained or softly-penalized optimization, resulting in implausible motions with artifacts such as foot skating and penetration. More importantly, they do not explicitly consider interaction preservation—i.e., maintaining spatial and contact relationship—in the retargeting formulation, relying instead on simple keypoint matching. Consequently, the resulting references are of lower quality, which in turn complicates the downstream RL policy training [12, 8, 13].

然而，现有大多数重定向方法 [10, 9, 11] 在这一点上并不够用。它们主要依赖无约束或只用软惩罚的优化，得到的动作并不合理，常有脚滑、穿透等伪影。更重要的是，它们没有在重定向公式里显式考虑交互保持——也就是维持空间与接触关系——而只做简单的关键点匹配。因此得到的参考质量较低，反过来又给下游强化学习策略的训练增加困难 [12, 8, 13]。

Fig. 2: OMNIRETARGET overview. Human demonstrations are retargeted to the robot via interaction-mesh–based constrained optimization. Each spatial and shape augmentation is solved as a new optimization, producing diverse trajectories that serve as references for RL training with minimal reward design and domain randomization, enabling zero-shot transfer to real-world humanoids.

图 2：OMNIRETARGET 总览。人体示范通过基于交互网格的约束优化，被重定向到机器人。每一次空间与形状增强都作为一次新的优化来求解，从而产生多样轨迹，作为强化学习训练的参考；奖励设计和域随机化都可以很简，并实现向真实人形机器人的零样本迁移。

In this work, we introduce OmniRetarget, an open-source data generation engine that transforms human demonstrations into diverse, high-quality kinematic references for humanoid whole-body control. By modeling spatial and contact relationships between robots, objects, and terrains via an interaction mesh [14], OmniRetarget preserves essential interactions and generates kinematically feasible variations. While existing methods require separate demonstrations for each variation—making data collection costly and limiting coverage—OmniRetarget addresses this bottleneck directly. Inspired by data augmentation frameworks for contact-rich manipulation [15], our framework automatically augments a single demonstration into a large number of training examples across object configurations, shapes, robot embodiments, and environments.

本文介绍 OmniRetarget，一个开源数据生成引擎，能把人体示范转化为多样、高质量的运动学参考，供人形机器人全身控制使用。它通过交互网格 [14] 建模机器人、物体与地形之间的空间和接触关系，从而保持必要的交互，并生成运动学上可行的变体。已有方法往往要为每一种变体单独采集示范，采集成本高、覆盖也有限；OmniRetarget 直接针对这一瓶颈。受接触丰富操作中数据增强框架的启发 [15]，我们的框架能把一条示范自动扩充成大量训练样本，覆盖不同物体配置、形状、机器人形体和环境。

Our pipeline employs constrained optimization to enforce physical feasibility, including collision avoidance, joint limits, and foot contact stability, while minimizing interaction mesh deformation. The resulting motions are interaction-preserving and exhibit only minimal kinematic artifacts, providing dense learning signals that accelerate RL with minimal reward engineering. On a diverse suite of whole-body interaction tasks such as box lifting, platform climbing, and slope crawling, policies trained on OmniRetarget datasets outperform those from prior retargeting methods in both motion quality and robustness, with successful zero-shot sim-to-real transfer onto a physical humanoid robot.

我们的流程用约束优化来保证物理可行性，包括碰撞避免、关节限位和脚部接触稳定，同时最小化交互网格形变。得到的动作保持交互关系，运动学伪影很少，能提供稠密的学习信号，从而以极少的奖励设计加速强化学习。在搬箱、爬平台、斜坡匍匐等多样全身交互任务上，用 OmniRetarget 数据集训练的策略，在动作质量和稳健性上都优于先前重定向方法，并成功零样本迁移到实体人形机器人。

Our contributions are fourfold:

我们的贡献有四点：

1. The first interaction-preserving humanoid retargeting framework that handles rich robot-object-terrain interactions while enforcing hard physical constraints.

1. 首个保持交互关系的人形重定向框架，能够处理丰富的机器人–物体–地形交互，并施加硬性物理约束。

2. A systematic data augmentation pipeline that transforms a single human demonstration into a diverse, large-scale set of high-quality kinematic trajectories on various robot embodiments.

2. 一套系统的数据增强流程，能把一条人体示范转化为多样、大规模、高质量的运动学轨迹，并适用于多种机器人形体。

3. A large-scale, open-source dataset of retargeted, kinematically-feasible loco-manipulation trajectories.

3. 一个大规模开源数据集，包含经过重定向、运动学上可行的运动–操作轨迹。

4. Successful zero-shot sim-to-real transfer of proprioceptive RL policies on a physical humanoid, demonstrating a diverse set of scene-interaction tasks, including a long, agile sequence of object carrying, platform climbing, jumping, rolling and wall-flipping.

4. 在实体人形机器人上，仅用本体感觉的强化学习策略实现零样本仿真到真实迁移，展示多样的场景交互任务，包括搬物、爬平台、跳跃、滚翻和墙面空翻组成的长程、敏捷序列。

II Related Works

II 相关工作

II-A Motion Retargeting

II-A 动作重定向

In computer graphics, transferring motions across characters has been extensively explored. Researchers have employed optimization-based methods to retarget human motions to avatars by preserving distances and orientations between keypoints [16], minimizing deformation energy [14, 17], or scaling the motions to satisfy hard constraints [18]. Others leverage data-driven methods that map diverse skeletons to a canonical representation [19], solve inverse kinematics with neural networks [20], or use reinforcement learning to preserve an interaction graph [21].

在计算机图形学中，跨角色迁移动作已经有大量研究。有人用基于优化的方法，通过保持关键点之间的距离和朝向 [16]、最小化形变能量 [14, 17]，或把动作缩放以满足硬约束 [18]，把人体动作重定向到虚拟形象。也有人采用数据驱动方法，把不同骨架映射到规范表示 [19]，用神经网络求解逆运动学 [20]，或用强化学习来保持交互图 [21]。

Retargeting motions to humanoid robots introduces challenges beyond character animation, particularly the need to enforce physical constraints. For example, PHC [10], a graphics method adopted in robotics [13, 8], uses keypoint matching with unconstrained optimization, often leading to penetration, foot skating, and lack of object or scene awareness. Similarly, GMR [9] extends keypoint matching to orientations but suffer the same issues. VideoMimic [11] improves realism with soft contact and collision penalties but offers no guarantees and requires careful tuning.

把动作重定向到人形机器人，引入了角色动画之外的挑战，尤其是必须施加物理约束。例如，图形学方法 PHC [10] 已被用到机器人中 [13, 8]，它用无约束优化做关键点匹配，常常导致穿透、脚滑，也缺乏对物体或场景的感知。GMR [9] 把关键点匹配扩展到朝向，但仍有同样问题。VideoMimic [11] 用接触和碰撞的软惩罚改善真实感，但不能给出保证，还需要仔细调参。

The closest method to ours is Interaction Mesh based Motion Adaptation (IMMA) [22], which also leverages an interaction mesh [14] to preserve the spatial relationship between body parts. However, it is not open-sourced and ignores kinematic limits and interactions with the environment or manipulated objects. In contrast, OmniRetarget unifies all hard constraints, including foot sticking, non-penetration, and joint and velocity limits, while explicitly preserving environment and object interactions.

与我们最接近的方法是基于交互网格的运动适应（IMMA）[22]，它同样借助交互网格 [14] 来保持身体各部分之间的空间关系。但它并不开源，也忽略了运动学限制，以及与环境或被操作物体的交互。相比之下，OmniRetarget 把脚部附着、无穿透、关节和速度限位等全部硬约束统一起来，并显式保持与环境和物体的交互。

II-B Learning-Based Humanoid Whole-Body Control

II-B 基于学习的人形全身控制

Recent learning-based whole-body control has enabled humanoids to traverse dynamic scenes and manipulate objects [23, 24, 25, 26, 27, 28, 29, 30, 31]. These methods typically train with hand-crafted rewards or task interfaces (e.g., velocity tracking, contact schedules, end-effector targets) but depend on extensive reward engineering and mostly fail to yield natural, human-level motions.

近期基于学习的全身控制，已经能让人形机器人穿越动态场景并操作物体 [23, 24, 25, 26, 27, 28, 29, 30, 31]。这些方法通常用手工奖励或任务接口（如速度跟踪、接触时间表、末端目标）来训练，但依赖大量奖励设计，大多也难以得到自然、接近人类水平的动作。

Motion imitation offers a promising alternative. In graphics, DeepMimic [4] shows that using human references yields natural, human-like behaviors with agile, dynamic motions. However, applying this approach to humanoid robots remains difficult due to the lack of reliable open-source kinematic retargeting pipelines. With suboptimal reference motions, practitioners are forced to either manually clean the data [12] or re-introduce extensive reward engineering, such as ad-hoc regularizers for contact, slipping, and air time, to compensate for artifacts [9, 13, 32]. In contrast, trackers with minimal reward formulation like BeyondMimic [33] achieve state-of-the-art results on hardware with high-fidelity references [34], but those are scarce and robot-only, without interactions.

动作模仿是一条有希望的替代路径。在图形学中，DeepMimic [4] 表明，使用人体参考可以得到自然、类人且敏捷、动态的行为。但把这一思路用到人形机器人仍然困难，因为缺少可靠的开源运动学重定向流程。参考动作不够好时，实践者要么手工清理数据 [12]，要么重新引入大量奖励设计，例如为接触、打滑和腾空时间加入临时正则项，以弥补伪影 [9, 13, 32]。相比之下，像 BeyondMimic [33] 这样采用极简奖励形式的跟踪器，在高保真参考 [34] 上能在真机上取得先进结果，但这类参考很少，而且只有机器人自身动作、没有交互。

Beyond single-character motion, human–scene interaction data has proven effective for terrain traversal and loco-manipulation in character animation [5, 6, 35], but translating this to robotics remains challenging. VideoMimic [11] applies this idea to human–terrain traversal by reconstructing motions and terrains from video, but suffers from artifacts and is limited to static–scene interactions. To bridge this gap, OmniRetarget enables natural, agile robot-object-scene interactions with high-quality reference from retargeting without manual post-processing or reward engineering.

在单角色动作之外，人–场景交互数据已被证明对角色动画中的地形穿越和运动–操作有效 [5, 6, 35]，但迁移到机器人仍然困难。VideoMimic [11] 把这一想法用到人–地形穿越，从视频重建动作和地形，但仍有伪影，且限于静态场景交互。为弥补这一差距，OmniRetarget 用重定向得到的高质量参考，实现自然、敏捷的机器人–物体–场景交互，不必手工后处理，也不必再做奖励工程。

II-C Data Generation for Humanoid Loco-Manipulation

II-C 人形运动–操作的数据生成

The demand for whole-body interaction data has motivated many prior works on data generation. One approach is direct human teleoperation [36, 7, 8, 9, 37]. While it provides online feedback, teleoperation is difficult to scale: it’s labor-intensive, prone to operator fatigue, and limited by the embodiment gap between human and robot kinematics. The lack of rich haptic feedback and difficulty stabilizing extreme motions (e.g., deep squats) further constrain its applicability. To address these scaling challenges, automated data augmentation has been explored, particularly for robotic manipulation. Many works leverage state-of-the-art generative models for visual [38, 39, 40] and semantic [41, 42, 43] augmentations, while others rely on simple open-loop kinematic replay of base trajectories [44, 45, 46] or trajectory optimization [15] in simulation. Despite the advances in manipulation, data augmentation for whole-body loco-manipulation remains largely unexplored. The closest prior work [47] interpolates keypoints to augment objects of different shape, but it cannot deal with varied object poses either. OmniRetarget directly addresses this gap.

对全身交互数据的需求，推动了许多数据生成工作。一条路是直接由人遥操作 [36, 7, 8, 9, 37]。它能提供在线反馈，但很难扩展：费人力、操作者容易疲劳，还受限于人与机器人运动学之间的形体差异。缺少丰富的触觉反馈、以及难以稳定深蹲等极端动作，进一步限制了它的适用范围。为应对扩展性，自动数据增强已有探索，尤其是在机器人操作里。不少工作用先进的生成模型做视觉 [38, 39, 40] 和语义 [41, 42, 43] 增强，另一些则在仿真中对基轨迹做简单开环运动学重放 [44, 45, 46] 或轨迹优化 [15]。尽管操作领域进展很快，全身运动–操作的数据增强在很大程度上仍未被探索。最接近的先前工作 [47] 用关键点插值来增强不同形状的物体，但仍无法处理变化的物体位姿。OmniRetarget 直接针对这一空白。

III Interaction-Preserving Motion Retargeting

III 保持交互关系的动作重定向

Methods	Hard Kinematic Constraints	Interaction w/ Object	Interaction w/ Terrain	Data Augmentation	Optimization Method
IMMA [22]	✓	✗	✗	✗	QP
PHC [10]	✗	✗	✗	✗	Gradient Descent
GMR [9]	✗	✗	✗	✗	Mink [48]
VideoMimic [11]	Soft Penalty	✗	✓	✗	JAX L-M
OmniRetarget (Ours)	✓	✓	✓	✓	Sequential SOCP

TABLE I: Comparison of prior retargeting methods across different aspects.

方法	硬运动学约束	与物体交互	与地形交互	数据增强	优化方法
IMMA [22]	✓	✗	✗	✗	QP
PHC [10]	✗	✗	✗	✗	Gradient Descent
GMR [9]	✗	✗	✗	✗	Mink [48]
VideoMimic [11]	软惩罚	✗	✓	✗	JAX L-M
OmniRetarget（本文）	✓	✓	✓	✓	Sequential SOCP

表 I：先前重定向方法在若干方面的比较。

III-A Interaction Mesh with Hard Constraints

III-A 带硬约束的交互网格

We leverage the interaction mesh [14] to preserve spatial relationships between body parts, objects, and the environment. The interaction mesh is defined as a volumetric structure whose vertices consist of key robot or human joints together with points sampled from objects and the environment. By shrinking or stretching this mesh, we can warp human motion onto the robot while preserving relative spatial configurations and contact relationships.

我们借助交互网格 [14] 来保持身体各部分、物体与环境之间的空间关系。交互网格是一种体结构，其顶点由机器人或人体的关键关节，以及从物体和环境上采样的点共同组成。通过收缩或拉伸这张网格，我们可以把人体动作翘曲到机器人上，同时保持相对空间配置和接触关系。

Interaction Mesh Construction. We construct the interaction mesh by applying Delaunay tetrahedralization [49] to user-defined key joint positions and randomly sampled object and environment points. To more accurately maintain contact relationships, we sample the object and environment surfaces more densely than the body joints.

交互网格的构建。 我们对用户指定的关键关节位置，以及随机采样的物体与环境点，做 Delaunay 四面体化 [49]，以构建交互网格。为更准确地保持接触关系，物体和环境表面的采样比身体关节更密。

Optimization Objectives and Constraints. To preserve the spatial relationships between the body parts, objects and terrains, our primary objective is to minimize the Laplacian deformation energy of the interaction meshes [50, 51] constructed from two corresponding sets of keypoints. The source set at frame $t$, $\mathcal{P}_{t}^{\text{source}}$, is composed of user-defined anatomical points on the human, and points randomly sampled on the manipulated object and the environment. The target set for the retargeted motion, $\mathcal{P}_{t}^{\text{target}}$, consists of corresponding anatomical points on the robot, the same manipulated object and environment points. Our method is relatively robust to the precise placement of these keypoints, requiring only a semantically consistent correspondence between the human and robot (e.g., hand to hand).

优化目标与约束。 为保持身体各部分、物体和地形之间的空间关系，我们的主要目标是最小化由两套对应关键点构建的交互网格的拉普拉斯形变能量 [50, 51]。第 $t$ 帧的源点集 $\mathcal{P}_{t}^{\text{source}}$，由人体上用户指定的解剖点，以及在被操作物体和环境上随机采样的点组成。重定向后动作的目标点集 $\mathcal{P}_{t}^{\text{target}}$，则由机器人上对应的解剖点，以及同一组物体和环境点组成。方法对关键点的精确放置相对稳健，只需人与机器人之间有语义一致的对应（例如手对手）。

The Laplacian coordinate of the $i$-th keypoint $p_{t,i}\in\mathcal{P}_{t}$ is defined as the difference between the point and the weighted average of its neighbors $j\in\mathcal{N}(i)$:

第 $i$ 个关键点 $p_{t,i}\in\mathcal{P}_{t}$ 的拉普拉斯坐标，定义为该点与其邻域 $j\in\mathcal{N}(i)$ 加权平均之差：

$$L(p_{t,i})=p_{t,i}-\sum_{j\in\mathcal{N}(i)}w_{ij}\cdot p_{t,j}, \tag{1}$$

where $w_{ij}$ is the normalized weight and we omit $L$’s dependence on $\{p_{t,j}\}_{j\neq i}$ in the function definition for conciseness. For all our experiments, we use uniform weights, setting $w_{ij}=1/|\mathcal{N}(i)|$. The deformation energy measures the change in these Laplacian coordinates between the source demonstration mesh $\mathcal{P}_{t}^{\text{source}}$ and the retargeted mesh $\mathcal{P}_{t}^{\text{target}}$:

其中 $w_{ij}$ 是归一化权重；为简洁起见，函数定义中省略了 $L$ 对 $\{p_{t,j}\}_{j\neq i}$ 的依赖。全部实验中我们使用均匀权重，即 $w_{ij}=1/|\mathcal{N}(i)|$。形变能量衡量这些拉普拉斯坐标在源示范网格 $\mathcal{P}_{t}^{\text{source}}$ 与重定向网格 $\mathcal{P}_{t}^{\text{target}}$ 之间的变化：

$$E_{L}=\sum_{p_{t,i}^{\text{source}}\in\mathcal{P}_{t}^{\text{source}},p_{t,i}^{\text{target}}\in\mathcal{P}_{t}^{\text{target}}}\|L(p_{t,i}^{\text{source}})-L(p_{t,i}^{\text{target}})\|^{2}. \tag{2}$$

We seek the robot configuration $q_{t}$, consisting of the floating base pose (quaternion and translation) and all joint angles, that minimizes this deformation energy while satisfying a set of hard kinematic constraints. The robot’s keypoints are determined by its configuration $q_{t}$ via forward kinematics $f_{i}$ as $p_{t,i}^{\text{robot}}(q_{t})=f_{i}(q_{t})\in\mathcal{P}_{t}^{\text{target}}$. At each time step, we solve the following constrained, nonconvex program:

我们寻求机器人构型 $q_{t}$（包括浮动基座位姿——四元数和平移——以及全部关节角），在满足一组硬运动学约束的同时最小化该形变能量。机器人的关键点由其构型 $q_{t}$ 经前向运动学 $f_{i}$ 决定，即 $p_{t,i}^{\text{robot}}(q_{t})=f_{i}(q_{t})\in\mathcal{P}_{t}^{\text{target}}$。在每个时间步，我们求解如下带约束的非凸规划：

$$\displaystyle q_{t}^{\star}=\argmin_{q_{t}}\; \\ \displaystyle\sum_{i}\|L(p_{t,i}^{\text{source}})-L(p_{t,i}^{\text{target}}(q_{t}))\|^{2}+\|q_{t}-q_{t-1}\|_{Q}^{2} \\ \displaystyle\phi_{j}(q_{t})\geq 0,\forall j \\ \displaystyle q_{\min}\leq q_{t}\leq q_{\max} \\ \displaystyle v_{\min}\cdot dt\leq q_{t}-q_{t-1}\leq v_{\max}\cdot dt \\ \displaystyle p_{t}^{F}=p_{t-1}^{F},\forall\text{stance foot}, \tag{3a}$$

where $Q$ is a cost matrix that encourages temporal smoothness, $\phi_{j}$ denotes the signed distance function for the $j$-th collision pair, $q_{\text{min}}/q_{\text{max}}$ and $v_{\text{min}}/v_{\text{max}}$ are the configuration and velocity bounds, $p_{t}^{F}$ denotes the foot position. A foot is considered to be in the stance phase if its horizontal velocity in the source motion (in the xy-plane) falls below a threshold of 1 cm/s. This optimization program solves for a temporally consistent robot trajectory that minimizes interaction mesh deformation, subject to hard constraints for collision avoidance (3b), joint and velocity limits (3c)–(3d), and preventing foot skating (3e).

其中 $Q$ 是鼓励时间平滑的代价矩阵，$\phi_{j}$ 表示第 $j$ 对碰撞的符号距离函数，$q_{\text{min}}/q_{\text{max}}$ 与 $v_{\text{min}}/v_{\text{max}}$ 分别是构型和速度界限，$p_{t}^{F}$ 表示脚的位置。若源动作中某只脚在水平面（xy 平面）上的速度低于 1 cm/s 的阈值，则视为处于支撑相。该优化求解一条时间上一致的机器人轨迹，在最小化交互网格形变的同时，满足碰撞避免（3b）、关节与速度限位（3c）–（3d）、以及防止脚滑（3e）的硬约束。

We solve (3) sequentially for each timestep using a customized Sequential Quadratic Programming (SQP)-style solver. Within each iteration, the objective (3a) is quadratically approximated and the hard constraints (3b)–(3e) are linearized around the solution from the previous iteration. To ensure temporal consistency and accelerate convergence, the optimization at frame $t$ is warm-started with the optimal solution from the previous frame ${q^{\star}_{t-1}}$. A key challenge in this formulation is computing derivatives involving the quaternion-based floating base orientation; our implementation leverages the automatic differentiation framework in Drake [52], which correctly handles the differential geometry of rotations on the $\mathbb{S}^{3}$ manifold [53].

我们用定制的序贯二次规划（SQP）风格求解器，按时间步依次求解（3）。在每次迭代中，目标（3a）做二次近似，硬约束（3b）–（3e）在上一轮解附近线性化。为保持时间一致性并加快收敛，第 $t$ 帧的优化用上一帧的最优解 ${q^{\star}_{t-1}}$ 热启动。这一形式的一个关键难点是计算涉及基于四元数的浮动基座朝向的导数；我们的实现借助 Drake [52] 的自动微分框架，它能正确处理 $\mathbb{S}^{3}$ 流形上转动的微分几何 [53]。

Our interaction-mesh-based kinematic pipeline is highly general. It adapts to different robot embodiments, including the Unitree G1, H1, and Booster T1 (Fig. 3), by modifying only the keypoint correspondences in the interaction mesh and the robot’s collision model. It also supports diverse interaction types: robot-object interactions from the OMOMO [1], robot-terrain interactions from in-house MoCap data, and robot-only motions on flat terrain from LAFAN1 [2].

基于交互网格的运动学流程具有很强的通用性。只需修改交互网格中的关键点对应和机器人的碰撞模型，就可以适配不同机器人形体，包括 Unitree G1、H1 和 Booster T1（图 3）。它也支持多种交互类型：来自 OMOMO [1] 的机器人–物体交互、来自内部动作捕捉的机器人–地形交互，以及来自 LAFAN1 [2] 的平坦地形上的仅机器人动作。

Fig. 3: Cross-embodiment robot-object-terrain interaction.

图 3：跨形体的机器人–物体–地形交互。

Fig. 4: OMNIRETARGET generates systematic variations of (a) terrain height, (b) object initial pose, and (c) object shape from a single human demonstration, with optimized motions in simulation (top) transferring consistently to hardware (bottom).

图 4：OMNIRETARGET 从一条人体示范系统生成（a）地形高度、（b）物体初始位姿和（c）物体形状的变化；仿真中优化得到的动作（上）能够一致地迁移到真机（下）。

III-B Terrain, Object Shape and Spatial Augmentation

III-B 地形、物体形状与空间增强

A key advantage of our framework is its capability for systematic data augmentation, which eliminates the need for collecting numerous, repetitive demonstrations with minor spatial variations. Our method can transform a single human demonstration into a rich and diverse dataset by parametrically altering object configurations, shapes, or terrain features. For each new scenario, we re-solve the optimization problem with fixed $\mathcal{P}_{t}^{\text{source}}$ and augmented $\mathcal{P}_{t}$: minimizing the interaction mesh deformation finds a new, kinematically valid robot motion $\{q_{t}\}$ that preserves the essential spatial and contact relationships of the original interaction.

本框架的一个关键优势，是能够做系统的数据增强，从而不必再为空间上只有细微差别的情形采集大量重复示范。通过对物体配置、形状或地形特征做参数化改动，我们就能把一条人体示范变成丰富多样的数据集。对每一种新情形，我们在固定 $\mathcal{P}_{t}^{\text{source}}$、并增强 $\mathcal{P}_{t}$ 的条件下重新求解优化：最小化交互网格形变，会找到一条新的、运动学上有效的机器人动作 $\{q_{t}\}$，并保持原交互中关键的空间与接触关系。

Robot-Object. We generate diverse interactions by augmenting both the object’s spatial configuration and its shape. We apply translations and rotations to modify the object’s initial pose (Fig. 4b) and blend the new initial pose with the original object motion via interpolation with an exponential schedule detailed in (14). In addition, we scale the three dimensions of the object (Fig. 4c). A critical component of this process is constructing the interaction mesh in the object’s local frame, which ensures that the robot’s interacting body parts naturally follow the object’s transformation (Sec. VI-C2).

机器人–物体。 我们同时增强物体的空间配置和形状，以生成多样交互。我们用平移和旋转修改物体的初始位姿（图 4b），再按（14）中详述的指数日程，把新的初始位姿与原来的物体运动做插值混合。此外，我们还对物体的三个维度做缩放（图 4c）。这一过程的关键，是在物体的局部坐标系中构建交互网格，从而使与物体交互的机器人身体部分自然跟随物体的变换（第 VI-C2 节）。

However, this alone can lead to trivial augmentations where the entire robot undergoes a rigid transformation along with the object. To generate more meaningful data diversity, we introduce cost terms and constraints that anchor parts of the robot’s body to the nominal trajectory $\{\bar{q}_{t}^{\star}\}$. For instance, in a pick-up task, we encourage the robot to discover new upper-body coordination by penalizing lower-body deviations from the original motion:

不过，仅此还可能导致平凡的增强：整个机器人随着物体做一次刚体变换。为得到更有意义的数据多样性，我们引入代价项和约束，把机器人的部分身体锚定在名义轨迹 $\{\bar{q}_{t}^{\star}\}$ 上。例如在拾取任务中，我们惩罚下肢相对原动作的偏离，以促使机器人发现新的上肢协调：

$$\|q_{t}-\bar{q}_{t}^{\star}\|_{W}, \tag{4}$$

where $W$ heavily penalizes the lower-body entries, constraining the initial foot poses to match the nominal trajectory

其中 $W$ 对下肢分量施以很重的惩罚，并把初始脚部位姿约束为与名义轨迹一致

$$p_{0}^{F}=\bar{p}_{0}^{F\star}\quad\text{ for left and right feet}. \tag{5}$$

These added objectives prevent the optimization from collapsing to a simple rigid transform of the nominal trajectory and instead produce genuinely new and diverse interactions.

这些额外目标防止优化退化成对名义轨迹的简单刚体变换，从而产生真正新的、多样的交互。

Robot-Terrain. We generate diverse terrain scenarios by scaling environmental features, such as varying the platform height and depth (Fig. 4a), and introducing additional constraints. For instance, to encourage stable ground contact when the terrain is elevated, we uniformly sample a grid of points on the ground surface into the interaction mesh.

机器人–地形。 我们通过缩放环境特征来生成多样地形情形，例如改变平台高度和深度（图 4a），并引入额外约束。例如，当地形被抬高时，为鼓励稳定的地面接触，我们在地面上均匀采样一组网格点，加入交互网格。

IV RL Training with Minimal Formulation

IV 极简形式的强化学习训练

Having established our method for generating high-quality kinematic references, we use RL to bridge the gap to dynamics by training a low-level policy that converts these trajectories into physically realizable actions, enabling zero-shot transfer from simulation to hardware.

在得到生成高质量运动学参考的方法之后，我们用强化学习来弥合与动力学之间的差距：训练一个底层策略，把这些轨迹转化为物理上可实现的动作，从而实现从仿真到真机的零样本迁移。

Reward engineering is often the main difficulty in humanoid RL: prior works [9, 13, 32] rely on many ad-hoc regularizers (e.g., foot flight and contact time) to compensate for artifacts in noisy references, but tuning these terms is tedious and fragile. In contrast, BeyondMimic [33] shows that when references are clean [34], a minimal reward is already sufficient for high-quality tracking. Since OmniRetarget produces such artifact-free, interaction-preserving references, we can follow this minimal formulation directly, achieving faithful tracking of dynamic interactions and zero-shot sim-to-real transfer without any hyperparameter tuning.

奖励设计往往是人形强化学习的主要难点：先前工作 [9, 13, 32] 依赖许多临时正则项（如脚部腾空和接触时间）来弥补噪声参考中的伪影，但调节这些项既繁琐又脆弱。相比之下，BeyondMimic [33] 表明，当参考足够干净 [34] 时，极简奖励已足以实现高质量跟踪。由于 OmniRetarget 能产生这类无伪影、保持交互关系的参考，我们可以直接沿用这一极简形式，忠实跟踪动态交互，并实现零样本仿真到真实迁移，不必做任何超参数调节。

Observations. To show that high-quality reference motions provide a sufficient prior for complex tasks, we design a minimal proprioceptive observation space, as listed below, where the agent is blind to explicit scene and object information and must follow the reference trajectory precisely.

观测。 为说明高质量参考动作已能为复杂任务提供足够先验，我们设计了极简的本体感觉观测空间，如下所列：智能体看不到显式的场景和物体信息，必须精确跟随参考轨迹。

• • Reference Motion: Reference Joint Position/Velocity, Reference Pelvis Position/Orientation Error;

• • 参考动作： 参考关节位置/速度，参考骨盆位置/朝向误差；

• • Proprioception: Pelvis Linear/Angular Velocity, Joint Position/Velocity;

• • 本体感觉： 骨盆线速度/角速度，关节位置/速度；

• • Previous Action: Policy action from last timestep.

• • 上一步动作： 上一时间步的策略动作。

For agile motions where state estimation is unreliable, we mask out the pelvis linear position error and velocity.

对于状态估计不可靠的敏捷动作，我们遮掉骨盆的线性位置误差和速度。

Rewards. To show the benefits of high-quality reference and avoid reward tuning, we use only five reward terms:

奖励。 为展示高质量参考的好处并避免调节奖励，我们只用五项奖励：

• • Body Tracking: DeepMimic-style tracking term for body position, orientation, linear and angular velocity;

• • 身体跟踪： DeepMimic 风格的跟踪项，作用于身体位置、朝向、线速度和角速度；

• • Object Tracking (where applicable): DeepMimic-style tracking term for object position and orientation;

• • 物体跟踪（适用时）： DeepMimic 风格的跟踪项，作用于物体位置和朝向；

• • Action Rate: Penalize rapid changes in action;

• • 动作变化率： 惩罚动作的快速变化；

• • Soft Joint Limit: Penalize robot joint limit violation;

• • 软关节限位： 惩罚机器人关节越限；

• • Self-Collision: Binary penalty on each body if its self-collision force exceeds $1$ N.

• • 自碰撞： 若某一体段的自碰撞力超过 $1$ N，则对该体段施加二值惩罚。

We use the same weights and hyperparameters from [33] out of the box without tuning. For object tracking, we use the same hyperparameters as body tracking terms.

我们直接使用 [33] 中的权重和超参数，不做调节。物体跟踪项使用与身体跟踪项相同的超参数。

Termination. We terminate training episodes with large body tracking deviations [33]. For object loco-manipulation, episodes terminate when the object deviates more than $1.0\text{m}$ and $45$° from the reference trajectory. We only apply this criterion after the policy achieves reasonable body tracking.

终止。 当身体跟踪偏差过大时，我们终止训练回合 [33]。对于物体运动–操作，当物体相对参考轨迹偏离超过 $1.0\text{m}$ 和 $45$° 时终止。这一准则只在策略已经达到合理的身体跟踪之后才启用。

Fig. 5: Additional hardware results showing diverse, agile and human-like behaviors.

图 5：更多真机结果，展示多样、敏捷且类人的行为。

Domain Randomization. To improve generalization across object properties for a single reference motion, we randomize the object’s physical parameters: mass (0.1–2 kg), center of mass (±0.08 m), inertia (50–150%), and shape (±10%). For the robot specifically, in contrast to the many terms in prior works (e.g., random force injection (RFI), motor PD, action delay), we only apply four terms:

域随机化。 为提高单条参考动作在不同物体属性上的泛化，我们随机化物体的物理参数：质量（0.1–2 kg）、质心（$\pm 0.08$ m）、惯性（50–150%）和形状（$\pm 10\%$）。专门针对机器人，与先前工作中的许多项（例如随机力注入（RFI）、电机 PD、动作延迟）不同，我们只使用四项：

• • Torso COM Position: $\pm 0.025$ m in $x$, $\pm 0.05$ m in $y$, $\pm 0.075$ m in $z$;

• • 躯干质心位置：$x$ 方向 $\pm 0.025$ m，$y$ 方向 $\pm 0.05$ m，$z$ 方向 $\pm 0.075$ m；

• • Joint default position: $\pm 0.01$ rad;

• • 关节默认位置：$\pm 0.01$ rad；

• • Random push: $0.3$ m/s, $0.78$ rad/s for $(1\text{--}3)$ s;

• • 随机推挤：$0.3$ m/s，$0.78$ rad/s，持续 $(1\text{--}3)$ s；

• • Observation noise: $\pm 0.05$ for orientation in Rot6D, $\pm 0.5$ m/s and $\pm 0.2$ rad/s for linear and angular velocity, $\pm 0.01$ rad and $\pm 0.5$ rad/s for joint position and velocity.

• • 观测噪声： Rot6D 朝向为 $\pm 0.05$，线速度和角速度为 $\pm 0.5$ m/s 与 $\pm 0.2$ rad/s，关节位置和速度为 $\pm 0.01$ rad 与 $\pm 0.5$ rad/s。

Policy Training. We group similar motions for faster training. All box-moving motions share a single multi-task policy, while platform climbing uses one policy per reference.

策略训练。 我们把相似动作分组以加快训练。所有搬箱动作共用一个多任务策略，爬平台则为每条参考各用一个策略。

V Experimental Results

V 实验结果

In this section, we present a comprehensive experimental validation of OmniRetarget. We first demonstrate the breadth of complex behaviors enabled by our approach, including natural object manipulation and terrain interaction. We then provide a quantitative benchmark against state-of-the-art baselines, evaluating performance across kinematic quality metrics and downstream policy performance.

本节对 OmniRetarget 做全面的实验验证。我们先展示该方法所能实现的复杂行为的广度，包括自然的物体操作和地形交互；再与先进基线做定量比较，从运动学质量指标和下游策略表现两方面评估。

V-A Whole-Body Scene-Object-Interaction

V-A 全身场景–物体交互

Agile Loco-Manipulation

OmniRetarget enables RL policies to learn agile, whole-body motions for complex scene interactions and loco-manipulation in simulation, culminating in successful zero-shot sim-to-real transfer to hardware. Shown in Fig. 5, policies trained on our data reproduce a diverse range of expressive behaviors on a Unitree G1 humanoid, including natural box-carrying motions retargeted from the OMOMO dataset, dynamically climbing a $0.9$m-high platform ($70$% of the robot’s height), and crawling over slopes, showing clean and accurate contact sequences.

敏捷的运动–操作

OmniRetarget 使强化学习策略能够在仿真中学习用于复杂场景交互和运动–操作的敏捷全身动作，并最终成功零样本迁移到真机。如图 5 所示，用我们的数据训练的策略，能在 Unitree G1 人形机器人上复现多种有表现力的行为，包括从 OMOMO 数据集重定向而来的自然搬箱动作、动态爬上 $0.9$ m 高（约为机器人身高的 $70\%$）的平台，以及在斜坡上匍匐，接触序列干净而准确。

To showcase the full capabilities of our framework, we present a long-horizon, dynamic sequence inspired by the Boston Dynamics Atlas tool-use demo [54]. Visualized in Fig. 1, our retargeted data enables the robot to carry a $4.6$ kg chair to a platform, use it as a stepstone to climb up, and then leap off, performing a parkour-style roll to absorb the landing impact. This 30-second, complex, multi-stage task highlights OmniRetarget’s ability to produce precise and versatile reference motions, pushing the boundaries of what is possible for humanoids learning agile, human-like behaviors.

为展示框架的完整能力，我们给出一段受波士顿动力 Atlas 工具使用演示启发的长时程动态序列 [54]。如图 1 所示，重定向后的数据使机器人能够把一把 $4.6$ kg 的椅子搬到平台前，把它当作踏脚石爬上去，再跳下，并以跑酷式滚翻吸收落地冲击。这一持续 30 秒、多阶段的复杂任务，凸显了 OmniRetarget 生成精确、多样参考动作的能力，也拓展了人形机器人学习敏捷、类人行为的边界。

We additionally showcase a high-dynamic wall-flip motion[^1] in Fig. 6. The robot completes the full flip in approximately $0.5$ second, reaching a peak angular velocity of $15$ rad/s. Unlike the human foot, which can flex at the arch to maintain contact and generate friction, the robot foot is rigid. As a result, it must align more closely to the wall to achieve sufficient contact area and friction. To account for this physical difference and give RL more freedom to learn this skill, we relaxed the termination condition during RL training by increasing the end-effector position error threshold to $0.5$ meter (compared to $0.25$ meter used in other motions) and removed the foot joint orientation tracking term from the reward function. All other components of the tracking objective remain consistent with other motions. The trained policy is robust and achieves a $5/5$ success rate in our real-world experiments.

我们还在图 6 中展示了一段高动态的墙面空翻动作[^1]。机器人大约在 $0.5$ 秒内完成整个空翻，峰值角速度达到 $15$ rad/s。与可以在足弓处屈曲以保持接触并产生摩擦的人脚不同，机器人的脚是刚性的。因此，它必须更贴紧墙面，才能获得足够的接触面积和摩擦。为考虑这一物理差异，并给强化学习更多空间来学习该技能，我们在训练中放宽了终止条件：把末端位置误差阈值提高到 $0.5$ 米（其他动作为 $0.25$ 米），并从奖励中去掉脚关节朝向跟踪项。跟踪目标的其余部分与其他动作保持一致。训练得到的策略相当稳健，在我们的真实世界实验中达到 $5/5$ 的成功率。

Fig. 6: Hardware results showing a high-dynamic wall-flip motion. The robot reaches a maximum linear velocity of 3.5 m/s and a peak angular velocity of 15 rad/s.

图 6：展示高动态墙面空翻的真机结果。机器人的最大线速度达到 3.5 m/s，峰值角速度达到 15 rad/s。

Sim-to-real with Augmented Data

We show that the augmented motions from our pipeline can be used for training and deployment effectively. As shown in Fig. 4, the interaction mesh formulation allows OmniRetarget to generalize a single nominal motion into box-picking across shapes and positions, as well as platform climbing at different heights. Notably, these augmented motions transfer to hardware without reward tuning, effectively expanding the repertoire of scenes and behaviors we can achieve in real.

用增强数据做仿真到真实迁移

我们表明，流水线增强得到的动作可以有效用于训练和部署。如图 4 所示，交互网格形式使 OmniRetarget 能把一条名义动作推广到不同形状和位置的搬箱，以及不同高度的爬平台。这些增强动作无需调节奖励即可迁移到真机，从而有效扩展了我们能在真实场景中实现的场景和行为范围。

In comparison, relying solely on domain randomization–which perturbs object shapes and poses only during training–performs poorly under our RL formulation, as the policies struggle to explore far beyond the nominal reference. Policies trained on our augmented data instead yield reliable success (see video for comparison). Admittedly, additional reward engineering could help, but it contradicts our minimal design goal. Quantitatively, training and evaluating on the full augmented dataset achieves a $79.1\%$ success rate, comparable to $82.2\%$ when evaluating on nominal motions only, showing that kinematics augmentation substantially enlarges coverage without significant performance degradation.

相比之下，只靠域随机化——即仅在训练时扰动物体形状和位姿——在我们的强化学习形式下表现较差，因为策略很难探索到远离名义参考的区域。用我们增强数据训练的策略则能获得可靠成功（对比见视频）。诚然，额外的奖励设计可能有帮助，但这与我们的极简设计目标相悖。定量来看，在全部增强数据上训练并评估的成功率为 $79.1\%$，与只在名义动作上评估的 $82.2\%$ 相当，说明运动学增强在显著扩大覆盖的同时，并没有明显降低表现。

	Penetration	Foot Skating	Contact Preservation	Downstream RL Policy
Method	Duration $\downarrow$	Max Depth (cm) $\downarrow$	Duration $\downarrow$	Max Vel. (cm/s) $\downarrow$	Duration $\uparrow$	Success Rate $\uparrow$
Robot-Object Interaction (Retargeting from the OMOMO Dataset)
PHC [10]	0.68 $\pm$ 0.21	5.11 $\pm$ 3.09	0.05 $\pm$ 0.05	1.40 $\pm$ 0.80	0.96 $\pm$ 0.09	71.28% $\pm$ 22.55%
GMR [9]	0.83 $\pm$ 0.14	8.50 $\pm$ 3.94	0.02 $\pm$ 0.01	1.46 $\pm$ 0.45	0.99 $\pm$ 0.04	50.83 % $\pm$ 23.89%
VideoMimic [11]	0.60 $\pm$ 0.27	7.48 $\pm$ 4.95	0.12 $\pm$ 0.07	1.50 $\pm$ 0.70	0.77 $\pm$ 0.25	3.85% $\pm$ 8.41%
OmniRetarget	0.00 $\pm$ 0.01	1.34 $\pm$ 0.34	0	0	0.96 $\pm$ 0.09	82.20% $\pm$ 9.74%
Robot-Terrain Interaction (Retargeting from the In-House MoCap Dataset)
PHC	0.66 $\pm$ 0.36	7.74 $\pm$ 4.53	0.15 $\pm$ 0.04	2.03 $\pm$ 1.83	0.45 $\pm$ 0.28	52.63% $\pm$ 49.93%
GMR	0.91 $\pm$ 0.16	5.72 $\pm$ 3.84	0.04 $\pm$ 0.05	1.75 $\pm$ 3.01	0.67 $\pm$ 0.26	78.94% $\pm$ 40.77%
VideoMimic	0.83 $\pm$ 0.11	5.97 $\pm$ 3.58	0.14 $\pm$ 0.05	1.85 $\pm$ 1.38	0.47 $\pm$ 0.25	51.75% $\pm$ 49.23%
OmniRetarget	0.01 $\pm$ 0.02	1.37 $\pm$ 0.18	0	0	0.72 $\pm$ 0.19	94.73% $\pm$ 22.33%
Robot-Only (Retargeting from the LAFAN1 Dataset)
Unitree [34]	0.09 $\pm$ 0.13	3.22 $\pm$ 2.64	0.06 $\pm$ 0.03	1.46 $\pm$ 0.01	N/A	100%
OmniRetarget	0.00 $\pm$ 0.00	1.07 $\pm$ 0.00	0	0	N/A	100%

TABLE II: Quantitative comparison of kinematic retargeting quality and downstream RL performances.

	穿透	脚滑	接触保持	下游强化学习策略
方法	持续时间 $\downarrow$	最大深度 (cm) $\downarrow$	持续时间 $\downarrow$	最大速度 (cm/s) $\downarrow$	持续时间 $\uparrow$	成功率 $\uparrow$
机器人–物体交互（从 OMOMO 数据集重定向）
PHC [10]	0.68 $\pm$ 0.21	5.11 $\pm$ 3.09	0.05 $\pm$ 0.05	1.40 $\pm$ 0.80	0.96 $\pm$ 0.09	71.28% $\pm$ 22.55%
GMR [9]	0.83 $\pm$ 0.14	8.50 $\pm$ 3.94	0.02 $\pm$ 0.01	1.46 $\pm$ 0.45	0.99 $\pm$ 0.04	50.83 % $\pm$ 23.89%
VideoMimic [11]	0.60 $\pm$ 0.27	7.48 $\pm$ 4.95	0.12 $\pm$ 0.07	1.50 $\pm$ 0.70	0.77 $\pm$ 0.25	3.85% $\pm$ 8.41%
OmniRetarget	0.00 $\pm$ 0.01	1.34 $\pm$ 0.34	0	0	0.96 $\pm$ 0.09	82.20% $\pm$ 9.74%
机器人–地形交互（从内部动作捕捉数据集重定向）
PHC	0.66 $\pm$ 0.36	7.74 $\pm$ 4.53	0.15 $\pm$ 0.04	2.03 $\pm$ 1.83	0.45 $\pm$ 0.28	52.63% $\pm$ 49.93%
GMR	0.91 $\pm$ 0.16	5.72 $\pm$ 3.84	0.04 $\pm$ 0.05	1.75 $\pm$ 3.01	0.67 $\pm$ 0.26	78.94% $\pm$ 40.77%
VideoMimic	0.83 $\pm$ 0.11	5.97 $\pm$ 3.58	0.14 $\pm$ 0.05	1.85 $\pm$ 1.38	0.47 $\pm$ 0.25	51.75% $\pm$ 49.23%
OmniRetarget	0.01 $\pm$ 0.02	1.37 $\pm$ 0.18	0	0	0.72 $\pm$ 0.19	94.73% $\pm$ 22.33%
仅机器人（从 LAFAN1 数据集重定向）
Unitree [34]	0.09 $\pm$ 0.13	3.22 $\pm$ 2.64	0.06 $\pm$ 0.03	1.46 $\pm$ 0.01	N/A	100%
OmniRetarget	0.00 $\pm$ 0.00	1.07 $\pm$ 0.00	0	0	N/A	100%

表 II：运动学重定向质量与下游强化学习表现的定量比较。

V-B Benchmark Against Prior Retargeting Pipelines

V-B 与先前重定向流程的基准比较

Fig. 7: Artifacts resulting from the retargeting baselines.

图 7：重定向基线产生的伪影。

We compare OmniRetarget against several widely-used open-source retargeting baselines[^2]: PHC [10], GMR [9] and VideoMimic [11]. The generated dataset including 2.78 hours of box carrying in OMOMO, 1 hour of in-house MoCap and 4.6 hours of LAFAN1 will be open-sourced.

我们把 OmniRetarget 与几种广泛使用的开源重定向基线比较[^2]：PHC [10]、GMR [9] 和 VideoMimic [11]。生成的数据集包括 OMOMO 中 2.78 小时的搬箱、1 小时内部动作捕捉，以及 4.6 小时 LAFAN1，都将开源。

Kinematics Quality

We evaluate the kinematic quality of retargeted motions on a Unitree G1 with three criteria:

运动学质量

我们在 Unitree G1 上用三项标准评估重定向动作的运动学质量：

1. Penetration: Measured by the time duration (normalized by the trajectory length) and maximum depth of intersections between the robot, objects, and terrain.

1. 穿透：用机器人、物体与地形之间相交的持续时间（按轨迹长度归一化）以及最大深度来衡量。

2. Foot Skating: Quantified by the time duration (normalized by the total desired foot sticking length) and maximum skating velocity of a stance foot.

2. 脚滑：用支撑脚滑动的持续时间（按期望的脚部附着总长度归一化）以及最大滑动速度来衡量。

3. Contact Preservation: Quantified by the time duration (normalized by the desired contact length). For robot-object tasks, we measure hand-object contact. For robot-terrain tasks, we measure contact between the robot’s hands, toes, and heels with the terrain surface.

3. 接触保持：用持续时间（按期望接触长度归一化）来衡量。对于机器人–物体任务，我们测量手–物接触；对于机器人–地形任务，我们测量机器人的手、脚尖和脚跟与地形表面的接触。

As illustrated in Tab. II, OmniRetarget significantly outperforms all baselines across most kinematic metrics. While OmniRetarget occasionally incurs minor penetration due to the linearization of constraints (3b) in the sequential SOCP solver, the violations are minimal and can be efficiently fixed by RL. GMR achieves the highest contact preservation score for robot-object interaction tasks; however, this outcome largely reflects its keypoint-matching objective. In practice, scaling human hand keypoints to the robot’s size often drives the robot’s hands inside the object, leading to substantial penetration errors (Fig. 7b). Overall, all baselines exhibit significant penetration and foot skating (Fig. 7), degrading the downstream RL performance, as discussed next.

如表 II 所示，OmniRetarget 在大多数运动学指标上明显优于全部基线。尽管由于序贯 SOCP 求解器对约束（3b）的线性化，OmniRetarget 偶尔会出现轻微穿透，但违规很小，并且可以被强化学习有效修正。GMR 在机器人–物体交互任务上取得了最高的接触保持分数；不过这一结果在很大程度上反映的是它的关键点匹配目标。实践中，把人手关键点缩放到机器人尺寸，常常会把手送进物体内，从而造成显著的穿透误差（图 7b）。总体来看，所有基线都存在明显的穿透和脚滑（图 7），进而损害下游强化学习表现，如下文所述。

For a direct comparison on pure locomotion, we retarget motions from the LAFAN1 MoCap dataset [2] and benchmark them against the publicly available Unitree LAFAN1 retargeted dataset [34]. This serves as a strong baseline, as it is widely considered a high-quality data source for RL-based locomotion training [33]. Table II shows that OmniRetarget’s motions exhibit fewer physical artifacts, achieving better satisfaction of hard constraints.

为在纯运动上做直接比较，我们重定向 LAFAN1 动作捕捉数据集 [2] 中的动作，并与公开的 Unitree LAFAN1 重定向数据集 [34] 对照。后者是很强的基线，通常被视为基于强化学习的运动训练的高质量数据来源 [33]。表 II 显示，OmniRetarget 的动作物理伪影更少，对硬约束的满足也更好。

Downstream RL Performance

A central observation from prior works [33, 12] is that the quality of retargeted motions strongly influences the performance of downstream RL. To verify this, we select 39 challenging motions for OmniRetarget and baselines, and train RL policies using identical hyperparameters from [33] without manual tuning. We evaluate the policies in simulation, and success is measured by training termination criteria.

下游强化学习表现

先前工作 [33, 12] 的一个核心观察是：重定向动作的质量会强烈影响下游强化学习的表现。为验证这一点，我们为 OmniRetarget 和各基线选出 39 条具有挑战性的动作，并用 [33] 中完全相同的超参数训练强化学习策略，不做手工调节。我们在仿真中评估策略，成功与否按训练终止准则衡量。

Shown in Tab. II, retargeting quality directly impacts RL success rates. OmniRetarget consistently achieves the highest performance across tasks, exceeding baselines by over 10% with lower variance, which indicates more stable learning across different motions. PHC performs better than GMR in object manipulation, likely due to lower penetration with sufficient contact preservation, but worse in terrain interaction, where its contact preservation drops by nearly 50%. Specifically for terrain interaction, we see that contact preservation is directly proportional to the success rate. These results suggest that both contact preservation and penetration reduction are critical for generalizing RL policies across diverse tasks, and OmniRetarget shows strength in both.

如表 II 所示，重定向质量直接影响强化学习成功率。OmniRetarget 在各任务上始终取得最高表现，超过基线 10% 以上，且方差更低，说明在不同动作上的学习更稳定。PHC 在物体操作上好于 GMR，很可能是因为穿透更少、同时接触保持足够；但在地形交互上更差，其接触保持下降了近 50%。具体到地形交互，接触保持与成功率直接成正比。这些结果表明，接触保持和减少穿透对策略在多样任务上的泛化都很关键，而 OmniRetarget 在这两方面都更强。

VideoMimic shows the weakest interaction preservation among all baselines (Fig. 7c), likely due to its collision avoidance soft cost conflicting with the keypoint matching cost. This is compounded by its coarse collision model originally designed for heightmaps, which is ill-suited for precise loco-manipulation. Consequently, while its terrain-interaction results are comparable to PHC, its performance on object manipulation is poor. Although this could be partially attributed to the tuning of its soft penalties, OmniRetarget demonstrates that a hard-constraint formulation avoids such sensitivities altogether.

VideoMimic 在所有基线中的交互保持最弱（图 7c），很可能是因为它的碰撞避免软代价与关键点匹配代价相互冲突。再加上其碰撞模型原本为高度图设计、较为粗糙，并不适合精确的运动–操作。因此，尽管它在地形交互上的结果与 PHC 相当，在物体操作上的表现却很差。这或许可以部分归因其软惩罚的调节，但 OmniRetarget 表明，硬约束形式可以从根本上避免这类敏感性。

VI Conclusion

VI 结论

In this work, we tackled a key data bottleneck caused by a lack of high-quality, interaction-aware retargeting pipeline in humanoid whole-body loco-manipulation. We introduced OmniRetarget, a unified, interaction-preserving data generation engine that leverages an interaction mesh within a single constrained optimization. Our experiments showed that OmniRetarget significantly outperforms prior methods in kinematic quality, producing a diverse set of artifact-free trajectories from single demonstrations. This high-quality data enabled a proprioceptive RL policy, trained with minimal formulation, to achieve long-horizon dynamic skills on a physical humanoid via zero-shot sim-to-real transfer.

本文针对人形全身运动–操作中，缺少高质量、感知交互的重定向流程这一关键数据瓶颈。我们介绍了 OmniRetarget：一个统一的、保持交互关系的数据生成引擎，在同一次约束优化中使用交互网格。实验表明，OmniRetarget 在运动学质量上明显优于先前方法，并能从单条示范产生多样、无伪影的轨迹。这些高质量数据使采用极简形式训练的本体感觉强化学习策略，能够通过零样本仿真到真实迁移，在实体人形机器人上完成长时程动态技能。

Ultimately, OmniRetarget demonstrates a paradigm shift from patching lower-quality reference motions with complex reward engineering to solving the problem at its source with principled data generation. While our current frame-by-frame optimization is mostly effective, future work could explore jointly optimizing the entire trajectory to enhance the framework’s robustness to noisier motion sources, such as video data, or learning autonomous visuomotor policies. By open-sourcing our complete framework and the large-scale dataset of retargeted trajectories, we hope to accelerate progress towards more agile, capable, and versatile humanoid robots.

归根结底，OmniRetarget 展示了一种范式转变：不再用复杂的奖励设计去修补低质量参考动作，而是在源头用有原则的数据生成把问题解开。尽管当前的逐帧优化在大多数情况下已经有效，未来工作可以探索对整条轨迹做联合优化，以增强框架对视频数据等更噪声动作来源的稳健性，或学习自主的视觉运动策略。通过开源完整框架和大规模重定向轨迹数据集，我们希望加速迈向更敏捷、更有能力、也更通用的人形机器人。

References

参考文献

[1] J. Li, J. Wu, and C. K. Liu, “Object motion guided human motion synthesis,” ACM Transactions on Graphics (TOG) , 2023.

[2] F. G. Harvey, M. Yurick, D. Nowrouzezahrai, and C. Pal, “Robust motion in-betweening,” vol. 39, no. 4, 2020.

[3] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,” Science robotics , 2020.

[4] X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne, “Deepmimic: Example-guided deep reinforcement learning of physics-based character skills,” ACM Transactions On Graphics (TOG) , 2018.

[5] M. Xu, Y. Shi, K. Yin, and X. B. Peng, “Parc: Physics-based augmentation with reinforcement learning for character controllers,” in Proceedings of the SIGGRAPH Conference Papers , 2025.

[6] Z. Wu, J. Li, P. Xu, and C. K. Liu, “Human-object interaction from human-level instructions,” arXiv preprint arXiv:2406.17840 , 2024.

[7] Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn, “Humanplus: Humanoid shadowing and imitation from humans,” CoRL , 2024.

[8] T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. M. Kitani, C. Liu, and G. Shi, “Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning,” in CoRL , 2025.

[9] Y. Ze, Z. Chen, J. P. AraÚjo, Z.-a. Cao, X. B. Peng, J. Wu, and C. K. Liu, “Twist: Teleoperated whole-body imitation system,” CoRL , 2025.

[10] Z. Luo, J. Cao, A. W. Winkler, K. Kitani, and W. Xu, “Perpetual humanoid control for real-time simulated avatars,” in ICCV , 2023.

[11] A. Allshire, H. Choi, J. Zhang, D. McAllister, A. Zhang, C. M. Kim, T. Darrell, P. Abbeel, J. Malik, and A. Kanazawa, “Visual imitation enables contextual humanoid control,” CoRL , 2025.

[12] T. Zhang, B. Zheng, R. Nai, Y. Hu, Y.-J. Wang, G. Chen, F. Lin, J. Li, C. Hong, K. Sreenath et al. , “Hub: Learning extreme humanoid balance,” CoRL , 2025.

[13] T. He, J. Gao, W. Xiao, Y. Zhang, Z. Wang, J. Wang, Z. Luo, G. He, N. Sobanbabu, C. Pan, Z. Yi, G. Qu, K. Kitani, J. Hodgins, L. J. Fan, Y. Zhu, C. Liu, and G. Shi, “Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills,” arXiv , 2025.

[14] E. S. L. Ho, T. Komura, and C.-L. Tai, “Spatial relationship preserving character motion adaptation,” ACM Transactions on Graphics , 2010.

[15] L. Yang, H. Suh, T. Zhao, B. P. Graesdal, T. Kelestemur, J. Wang, T. Pang, and R. Tedrake, “Physics-driven data generation for contact-rich manipulation via trajectory optimization,” RSS , 2025.

[16] T. Cheynel, T. Rossi, B. Bellot-Gurlet, D. Rohmer, and M.-P. Cani, “Sparse motion semantics for contact-aware retargeting,” in ACM SIGGRAPH Conference on Motion, Interaction and Games , 2023.

[17] Y. Kim, H. Park, S. Bang, and S.-H. Lee, “Retargeting human-object interaction to virtual avatars,” IEEE transactions on visualization and computer graphics , vol. 22, no. 11, pp. 2405–2412, 2016.

[18] M. Gleicher, “Retargetting motion to new characters,” in Proceedings of the 25th annual conference on Computer graphics and interactive techniques , 1998, pp. 33–42.

[19] K. Aberman, P. Li, D. Lischinski, O. Sorkine-Hornung, D. Cohen-Or, and B. Chen, “Skeleton-aware networks for deep motion retargeting,” ACM Transactions on Graphics (TOG) , vol. 39, no. 4, pp. 62–1, 2020.

[20] R. Villegas, J. Yang, D. Ceylan, and H. Lee, “Neural kinematic networks for unsupervised motion retargetting,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018.

[21] Y. Zhang, D. Gopinath, Y. Ye, J. Hodgins, G. Turk, and J. Won, “Simulation and retargeting of complex multi-character interactions,” in ACM SIGGRAPH 2023 Conference Proceedings , 2023.

[22] S. Nakaoka and T. Komura, “Interaction mesh based motion adaptation for biped humanoid robots,” in Humanoids , 2012.

[23] J. Dao, H. Duan, and A. Fern, “Sim-to-real learning for humanoid box loco-manipulation,” in ICRA . IEEE, 2024.

[24] J. Long, J. Ren, M. Shi, Z. Wang, T. Huang, P. Luo, and J. Pang, “Learning humanoid locomotion with perceptive internal model,” arXiv , 2024.

[25] J. He, C. Zhang, F. Jenelten, R. Grandia, M. Bächer, and M. Hutter, “Attention-based map encoding for learning generalized legged locomotion,” Science Robotics , vol. 10, no. 105, p. eadv3604, 2025.

[26] X. He, R. Dong, Z. Chen, and S. Gupta, “Learning getting-up policies for real-world humanoid robots,” RSS , 2025.

[27] Y. Kuang, H. Geng, A. Elhafsi, T.-D. Do, P. Abbeel, J. Malik, M. Pavone, and Y. Wang, “Skillblender: Towards versatile humanoid whole-body loco-manipulation via skill blending,” arXiv , 2025.

[28] Z. Zhang, C. Chen, H. Xue, J. Wang, S. Liang, Y. Liu, Z. Zhang, H. Wang, and L. Yi, “Unleashing humanoid reaching potential via real-world-ready skill space,” arXiv preprint arXiv:2505.10918 , 2025.

[29] Y. Xue, W. Dong, M. Liu, W. Zhang, and J. Pang, “A unified and general humanoid whole-body controller for versatile locomotion,” RSS , 2025.

[30] C. Zhang, W. Xiao, T. He, and G. Shi, “Wococo: Learning whole-body humanoid control with sequential contacts, 2024,” arXiv , 2024.

[31] Y. Zhang, Y. Yuan, P. Gurunath, T. He, S. Omidshafiei, A.-a. Agha-mohammadi, M. Vazquez-Chanlatte, L. Pedersen, and G. Shi, “Falcon: Learning force-adaptive humanoid loco-manipulation,” arXiv , 2025.

[32] Z. Li, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath, “Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control,” IJRR , 2025.

[33] Q. Liao, T. E. Truong, X. Huang, G. Tevet, K. Sreenath, and C. K. Liu, “Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion,” arXiv e-prints , pp. arXiv–2508, 2025.

[34] U. Robotics and Contributors, “Unitree lafan1 retargeting dataset,” https://huggingface.co/datasets/lvhaidong/LAFAN1_Retargeting_Dataset , 2025.

[35] S. Xu, H. Y. Ling, Y.-X. Wang, and L.-Y. Gui, “Intermimic: Towards universal whole-body control for physics-based human-object interactions,” in Proceedings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 12 266–12 277.

[36] M. Seo, S. Han, K. Sim, S. H. Bang, C. Gonzalez, L. Sentis, and Y. Zhu, “Deep imitation learning for humanoid loco-manipulation through human teleoperation,” in Humanoids , 2023.

[37] Q. Ben, F. Jia, J. Zeng, J. Dong, D. Lin, and J. Pang, “Homie: Humanoid loco-manipulation with isomorphic exoskeleton cockpit,” RSS , 2025.

[38] X. Zhang, M. Chang, P. Kumar, and S. Gupta, “Diffusion meets dagger: Supercharging eye-in-hand imitation learning,” in RSS , 2024.

[39] S. Tian, B. Wulfe, K. Sargent, K. Liu, S. Zakharov, V. C. Guizilini, and J. Wu, “View-invariant policy learning via zero-shot novel view synthesis,” in CoRL , 2025.

[40] L. Y. Chen, C. Xu, K. Dharmarajan, Z. Irshad, R. Cheng, K. Keutzer, M. Tomizuka, Q. Vuong, and K. Goldberg, “Rovi-aug: Robot and viewpoint augmentation for cross-embodiment robot learning,” in Conference on Robot Learning (CoRL) , 2024.

[41] Z. Mandi, H. Bharadhwaj, V. Moens, S. Song, A. Rajeswaran, and V. Kumar, “Cacti: A framework for scalable multi-task multi-scene visual imitation learning,” arXiv preprint arXiv:2212.05711 , 2022.

[42] Z. Chen, S. Kiami, A. Gupta, and V. Kumar, “Genaug: Retargeting behaviors to unseen situations via generative augmentation,” RSS , 2023.

[43] T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, J. Peralta, B. Ichter et al. , “Scaling robot learning with semantically imagined experience,” RSS , 2023.

[44] A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y. Narang, L. Fan, Y. Zhu, and D. Fox, “Mimicgen: A data generation system for scalable robot learning using human demonstrations,” in CoRL , 2023.

[45] Z. Jiang, Y. Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y. Zhu, “Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning,” ICRA , 2025.

[46] C. Garrett, A. Mandlekar, B. Wen, and D. Fox, “Skillmimicgen: Automated demonstration generation for efficient skill learning and deployment,” in Conference on Robot Learning (CoRL) , 2024.

[47] S. Starke, H. Zhang, T. Komura, and J. Saito, “Neural state machine for character-scene interactions,” ACM Transactions on Graphics , vol. 38, no. 6, p. 178, 2019.

[48] K. Zakka, “Mink: Python inverse kinematics based on MuJoCo,” May 2025. [Online]. Available: https://github.com/kevinzakka/mink

[49] H. Si and K. Gärtner, “Meshing piecewise linear complexes by constrained delaunay tetrahedralizations,” in Proceedings of the 14th international meshing roundtable . Springer, 2005, pp. 147–163.

[50] M. Alexa, “Differential coordinates for local mesh morphing and deformation,” The Visual Computer , vol. 19, no. 2, pp. 105–114, 2003.

[51] K. Zhou, J. Huang, J. Snyder, X. Liu, H. Bao, B. Guo, and H.-Y. Shum, “Large mesh deformation using the volumetric graph laplacian,” in ACM SIGGRAPH 2005 Papers . ACM, 2005, pp. 496–503.

[52] R. Tedrake and the Drake Development Team, “Drake: Model-based design and verification for robotics,” 2019.

[53] B. E. Jackson, K. Tracy, and Z. Manchester, “Planning with attitude,” IEEE Robotics and Automation Letters , 2021.

[54] Boston Dynamics, “Atlas Gets a Grip,” YouTube, available: https://www.youtube.com/watch?v=-e1_QhJ1EhQ .

[55] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “SMPL: A skinned multi-person linear model,” ACM Trans. Graphics (Proc. SIGGRAPH Asia) , vol. 34, no. 6, pp. 248:1–248:16, Oct. 2015.

APPENDIX

附录

VI-A Different Sources of Human Motion Data

VI-A 不同来源的人体动作数据

Human motion datasets contain rich pose and shape information, but they differ both in data format and in the physical attributes (e.g., height, body proportions) of the demonstrators. To make them compatible across different sources and suitable for retargeting, we need to convert these inputs into a consistent representation, typically a time series of global 3D keypoint positions $\{p^{\text{source}}_{0:T,i}\}$. This process must account for differences between human demonstrators and the target robot.

人体动作数据集包含丰富的姿态和体型信息，但它们在数据格式，以及示范者的身高、身体比例等物理属性上并不相同。为使它们在不同来源之间兼容、并适合重定向，需要把这些输入转换成一致的表示，通常是一组全局三维关键点位置的时间序列 $\{p^{\text{source}}_{0:T,i}\}$。这一过程必须考虑人体示范者与目标机器人之间的差异。

The datasets used in this work represent two common formats:

本文使用的数据集代表两种常见格式：

• • Parametric Human Models: The OMOMO dataset uses the SMPL format [55], a parametric model representing human body shape and pose-dependent variations using shape ($\beta$) and pose ($q$) parameters.

• • 参数化人体模型：OMOMO 数据集使用 SMPL 格式 [55]，这是一种用体型（$\beta$）和姿态（$q$）参数表示人体形状及姿态相关变化的参数化模型。

• • Skeleton Hierarchy: Both our in-house MoCap data and the LAFAN1 dataset utilize the skeleton hierarchy defined in the BVH format.

• • 骨架层级：我们的内部动作捕捉数据和 LAFAN1 数据集，都使用 BVH 格式定义的骨架层级。

Different retargeting pipelines use different strategies to handle these formats. We detail these preprocessing steps below, denoting the human demonstrator’s pose as $q^{\text{demo}}_{t}$, the SMPL forward model for the $i$-th keypoint as $M_{i}$, the original demonstrator shape as $\beta^{\text{demo}}$, and the demonstrator’s $i$-th keypoint position as $p^{\text{demo}}_{t,i}$.

不同重定向流程用不同策略处理这些格式。我们在下面详述这些预处理步骤，并把人体示范者的姿态记为 $q^{\text{demo}}_{t}$，第 $i$ 个关键点的 SMPL 前向模型记为 $M_{i}$，示范者的原始体型记为 $\beta^{\text{demo}}$，示范者第 $i$ 个关键点位置记为 $p^{\text{demo}}_{t,i}$。

VI-A1 SMPL Data

To handle data from parametric models like SMPL, methods typically follow one of two strategies: fitting the model to the robot’s morphology or directly scaling the human’s keypoints.

VI-A1 SMPL 数据

处理 SMPL 这类参数化模型数据时，方法通常采用两种策略之一：把模型拟合到机器人的形态，或直接缩放人体关键点。

0:  SMPL model $M$, robot urdf with forward kinematics $f$, $\bar{q}^{\text{smpl}}=0_{n_{s}},\bar{q}^{\text{robot}}=0_{n_{x}}$
0:  scaling factors $\alpha,\beta$
1:  $\alpha,\beta\leftarrow 1,0_{10}$
2:  for $\text{iter}=1,\ldots,\text{max\_iter}$ do
3:   $L(\alpha,\beta)=\sum_{i}\left\|(f_{i}(\bar{q}^{\text{robot}})-\alpha\cdot M_{i}(\bar{q}^{\text{smpl}};\beta)\right\|^{2}$
4:   $\alpha\leftarrow\alpha-\eta_{\alpha}\cdot\nabla_{\alpha}L$
5:   $\beta\leftarrow\beta-\eta_{\beta}\cdot\nabla_{\beta}L$
6:  end for

Algorithm 1 Fit SMPL Shape (PHC)

0:  SMPL model $M$, robot urdf with forward kinematics $f$, $\bar{q}^{\text{smpl}}=0_{n_{s}},\bar{q}^{\text{robot}}=0_{n_{x}}$
0:  scaling factors $\alpha,\beta$
1:  $\alpha,\beta\leftarrow 1,0_{10}$
2:  for $\text{iter}=1,\ldots,\text{max\_iter}$ do
3:   $L(\alpha,\beta)=\sum_{i}\left\|(f_{i}(\bar{q}^{\text{robot}})-\alpha\cdot M_{i}(\bar{q}^{\text{smpl}};\beta)\right\|^{2}$
4:   $\alpha\leftarrow\alpha-\eta_{\alpha}\cdot\nabla_{\alpha}L$
5:   $\beta\leftarrow\beta-\eta_{\beta}\cdot\nabla_{\beta}L$
6:  end for

算法 1 拟合 SMPL 体型（PHC）

Model Fitting (PHC, VideoMimic)

This strategy fits a scaled SMPL model to the robot’s morphology. PHC first optimizes for an overall scaling factor $\alpha$, and a set of SMPL shape parameters $\beta$ that best match the robot’s link length in a canonical T-pose, as detailed in Alg. 1. The final source keypoint positions are then generated from this fitted model:

模型拟合（PHC、VideoMimic）

这一策略把缩放后的 SMPL 模型拟合到机器人的形态。PHC 先优化一个总体缩放因子 $\alpha$，以及一组在规范 T 姿态下最匹配机器人连杆长度的 SMPL 体型参数 $\beta$，详见算法 1。最终的源关键点位置再由这一拟合模型生成：

$$p^{\text{source}}_{t,i}=\alpha\cdot M_{i}(q^{\text{demo}}_{t};\beta). \tag{6}$$

VideoMimic adopts a similar philosophy but integrates the scaling directly into its main retargeting optimization, solving for per-link scale factors jointly with the robot’s motion.

VideoMimic 采用相似思路，但把缩放直接并入主重定向优化，与机器人动作一起求解各连杆的缩放因子。

Direct Scaling (GMR & OmniRetarget)

In contrast, GMR and OmniRetarget use a more direct approach. They generate keypoints from the human’s original SMPL parameters $\beta^{\text{demo}}$ and then scale them to the robot’s proportions. Both methods support detailed morphological adaptation via per-bone scaling factors based on corresponding human-robot link lengths. For simplicity in this work, however, we adopt a single global scaling factor $\alpha$, set to the robot-to-human height ratio:

直接缩放（GMR 与 OmniRetarget）

相比之下，GMR 和 OmniRetarget 采用更直接的做法。它们用人体的原始 SMPL 参数 $\beta^{\text{demo}}$ 生成关键点，再缩放到机器人的比例。两种方法都支持按对应的人–机连杆长度做逐骨缩放，以实现细致的形态适配。不过为简单起见，本文采用单一的全局缩放因子 $\alpha$，取为机器人与人的身高比：

$$p^{\text{source}}_{t,i}=\alpha\cdot M_{i}(q^{\text{demo}}_{t};\beta^{\text{demo}}),\alpha=\frac{h_{\text{robot}}}{h_{\text{demo}}}. \tag{7}$$

Method	Optimization Type	Primary Objective	Preprocessing	Data Formats
PHC	Trajectory-wise Optimization	Keypoint Position Matching	Model Fitting	SMPL
GMR	Per-Frame Optimization	Keypoint Position & Orientation Matching	Direct Scaling	SMPL, BVH
VideoMimic	Trajectory-wise Optimization	Pairwise Distance & Orientation Preservation	Model Fitting	SMPL
IMMA	Multi-Stage Trajectory-wise Optimization	Interaction Mesh Deformation + IK	Unknown	Unknown
OmniRetarget	Per-Frame Optimization	Interaction Mesh Deformation	Direct Scaling	SMPL, BVH

TABLE III: Comparison of different retargeting methodologies

方法	优化类型	主要目标	预处理	数据格式
PHC	整段轨迹优化	关键点位置匹配	模型拟合	SMPL
GMR	逐帧优化	关键点位置与朝向匹配	直接缩放	SMPL, BVH
VideoMimic	整段轨迹优化	成对距离与朝向保持	模型拟合	SMPL
IMMA	多阶段整段轨迹优化	交互网格形变 + IK	未知	未知
OmniRetarget	逐帧优化	交互网格形变	直接缩放	SMPL, BVH

表 III：不同重定向方法的比较

VI-A2 Skeleton Hierarchy Data

For formats like BVH, keypoint positions are derived from the skeleton’s forward kinematics $f^{\text{skeleton}}$. This data is then typically scaled to the robot’s size using the height ratio:

VI-A2 骨架层级数据

对于 BVH 这类格式，关键点位置由骨架的前向运动学 $f^{\text{skeleton}}$ 得到。这些数据通常再用身高比缩放到机器人尺寸：

$$p^{\text{source}}_{t,i}=\frac{h_{\text{robot}}}{h_{\text{demo}}}\cdot f_{i}^{\text{skeleton}}(q^{\text{demo}}_{t}). \tag{8}$$

A key distinction among methods is their data compatibility. While GMR and OmniRetarget are designed to process both parametric model data and raw skeleton hierarchies, frameworks like PHC and VideoMimic are primarily designed for SMPL data. Fitting other data formats to the SMPL format is yet another tedious process.

各方法的一个关键区别在于数据兼容性。GMR 和 OmniRetarget 被设计为既能处理参数化模型数据，也能处理原始骨架层级；而 PHC 和 VideoMimic 则主要面向 SMPL 数据。把其他格式再拟合到 SMPL，又是一道繁琐的工序。

VI-B Different Kinematic Retargeting Formulations

VI-B 不同的运动学重定向形式

Once human motion is preprocessed into a series of source keypoint positions $\{p^{\text{source}}_{0:T,i}\}$, different methods formulate the retargeting problem in distinct ways. As summarized in Tab. III, these approaches vary in their optimization strategy and objectives. The following sections detail the mathematical formulation of each baseline method and our proposed approach, OmniRetarget.

一旦人体动作被预处理成一系列源关键点位置 $\{p^{\text{source}}_{0:T,i}\}$，不同方法会以不同方式表述重定向问题。如表 III 所汇总，这些方法在优化策略和目标上各不相同。以下几节分别给出各基线方法以及我们提出的 OmniRetarget 的数学形式。

0:  Robot urdf, source keypoint positions $\{p^{\text{source}}_{0:T,i}\}$
0:  $q_{0:T}$
1:  $q_{0:T}\leftarrow[0_{n_{x}}]_{T}$
2:  for $\text{iter}=1,\ldots,\text{max\_iter}$ do
3:   $\mathcal{L}(q_{0:T})=\sum_{t=0}^{T}\sum_{i}\left\|f_{i}(q_{t})-p^{\text{source}}_{t,i}\right\|^{2}$
4:   $q_{0:T}\leftarrow\text{clamp}(q_{0:T}-\nabla_{q_{0:T}}\mathcal{L}(q_{0:T}),q_{\min},q_{\max})$
5:  end for

Algorithm 2 Retarget Robot Motion (PHC)

0:  Robot urdf, source keypoint positions $\{p^{\text{source}}_{0:T,i}\}$
0:  $q_{0:T}$
1:  $q_{0:T}\leftarrow[0_{n_{x}}]_{T}$
2:  for $\text{iter}=1,\ldots,\text{max\_iter}$ do
3:   $\mathcal{L}(q_{0:T})=\sum_{t=0}^{T}\sum_{i}\left\|f_{i}(q_{t})-p^{\text{source}}_{t,i}\right\|^{2}$
4:   $q_{0:T}\leftarrow\text{clamp}(q_{0:T}-\nabla_{q_{0:T}}\mathcal{L}(q_{0:T}),q_{\min},q_{\max})$
5:  end for

算法 2 重定向机器人动作（PHC）

VI-B1 PHC

PHC formulates retargeting as a large-scale trajectory-wise optimization problem. It applies gradient descent to minimize the error between the source keypoint positions and the robot’s keypoint positions over the entire trajectory, as shown in Alg. 2.

VI-B1 PHC

PHC 把重定向表述为大规模的整段轨迹优化问题。它用梯度下降，在整条轨迹上最小化源关键点位置与机器人关键点位置之间的误差，见算法 2。

VI-B2 GMR

GMR performs retargeting by solving an inverse kinematics (IK) problem at each frame (3). At each timestep, GMR finds the robot configuration $q_{t}$ that matches the source keypoint positions and orientations via the following optimization program:

VI-B2 GMR

GMR 通过在每一帧求解逆运动学（IK）问题来做重定向（3）。在每个时间步，GMR 寻找匹配源关键点位置和朝向的机器人构型 $q_{t}$，优化问题如下：

$$\displaystyle q_{t}^{\star}=\argmin_{q_{t}} \\ \displaystyle\sum_{i}\left\|f_{i}^{p}(q_{t})-p^{\text{source}}_{t,i}\right\|^{2}+\left\|f_{i}^{\theta}(q_{t})-\theta^{\text{source}}_{t,i}\right\|^{2} \\ \displaystyle\text{s.t. } \\ \displaystyle q_{\min}\leq q_{t}\leq q_{\max},$$

where $f_{i}^{p}$ and $f_{i}^{\theta}$ are the robot forward kinematics for the $i$-th keypoint’s position and orientation, respectively. Leveraging the mink [48] library, GMR solves this program in a Sequential Quadratic Programming fashion.

其中 $f_{i}^{p}$ 和 $f_{i}^{\theta}$ 分别是第 $i$ 个关键点位置和朝向的机器人前向运动学。GMR 借助 mink [48] 库，以序贯二次规划的方式求解该问题。

0:  Robot urdf, source keypoint positions $\{p^{\text{source}}_{0:T,i}\}$ and orientations $\{\theta^{\text{source}}_{0:T,i}\}$
0:  $q_{0:T}$
1:  for $t=0,\ldots,T$ do
2:   $q_{t}\leftarrow$ Solve IK (9)
3:  end for

Algorithm 3 Retarget Robot Motion (GMR)

0:  Robot urdf, source keypoint positions $\{p^{\text{source}}_{0:T,i}\}$ and orientations $\{\theta^{\text{source}}_{0:T,i}\}$
0:  $q_{0:T}$
1:  for $t=0,\ldots,T$ do
2:   $q_{t}\leftarrow$ Solve IK (9)
3:  end for

算法 3 重定向机器人动作（GMR）

VI-B3 VideoMimic

VideoMimic jointly optimizes for the robot motion $q_{0:T}$ and SMPL per-link scaling factor $\beta$ over the entire trajectory. The primary objective is to preserve the scaled pairwise distance and orientation between each keypoint pair $(i,j)$:

VI-B3 VideoMimic

VideoMimic 在整条轨迹上联合优化机器人动作 $q_{0:T}$ 和 SMPL 的逐连杆缩放因子 $\beta$。主要目标是保持每对关键点 $(i,j)$ 之间经过缩放的成对距离和朝向：

$$\mathcal{L}_{\text{pairwise}}=\sum_{t,i\in\mathcal{N}(j)}\|\beta_{ij}\cdot(p^{\text{demo}}_{t,i}-p^{\text{demo}}_{t,j})-(f_{i}(q_{t})-f_{j}(q_{t}))\|_{2}^{2}, \tag{10}$$

with soft penalties on foot contact matching $\mathcal{L}_{\text{contact}}$, foot skating $\mathcal{L}_{\text{skating}}$, collision $\mathcal{L}_{\text{collision}}$, joint limits $\mathcal{L}_{\text{joint}}$ and temporal smoothness $\mathcal{L}_{\text{smooth}}$:

并加上对脚部接触匹配 $\mathcal{L}_{\text{contact}}$、脚滑 $\mathcal{L}_{\text{skating}}$、碰撞 $\mathcal{L}_{\text{collision}}$、关节限位 $\mathcal{L}_{\text{joint}}$ 和时间平滑 $\mathcal{L}_{\text{smooth}}$ 的软惩罚：

$$\displaystyle q_{0:T}^{\star},\beta^{\star}= \\ \displaystyle\argmin_{q_{0:T},\beta}\quad\mathcal{L}_{\text{pairwise}}+\lambda_{c}\cdot\mathcal{L}_{\text{contact}}+\lambda_{s}\cdot\mathcal{L}_{\text{skate}}+ \\ \displaystyle\lambda_{cl}\cdot\mathcal{L}_{\text{collision}}+\lambda_{j}\cdot\mathcal{L}_{\text{joint}}+\lambda_{sm}\cdot\mathcal{L}_{\text{smooth}}+\ldots$$

0:  Robot urdf, demonstrator’s original keypoint positions $\{p^{\text{demo}}_{0:T,i}\}$
0:  $q_{0:T},\beta$ $\leftarrow$ Solve (11) for the entire trajectory

Algorithm 4 Retarget Robot Motion (VideoMimic)

0:  Robot urdf, demonstrator’s original keypoint positions $\{p^{\text{demo}}_{0:T,i}\}$
0:  $q_{0:T},\beta$ $\leftarrow$ Solve (11) for the entire trajectory

算法 4 重定向机器人动作（VideoMimic）

VI-B4 IMMA Multi-stage Optimization

IMMA relies on a complex, multi-stage pipeline: first, it optimizes the intermediate robot keypoint positions to warp the interaction mesh from the human to the robot with minimal deformation by solving the following program

VI-B4 IMMA 的多阶段优化

IMMA 依赖复杂的多阶段流程：首先优化中间的机器人关键点位置，以最小形变把交互网格从人翘曲到机器人，求解如下规划

$$\displaystyle p_{t,i}^{\star}=\argmin_{p_{t_{i}}} \\ \displaystyle\sum_{i}\|L(p_{t,i}^{\text{source}})-L(p_{t,i})\|^{2} \\ \displaystyle\text{s.t.} \\ \displaystyle\phi_{j}(q_{t})\geq 0,\forall j \\ \displaystyle\|p_{t,i}-p_{t,j}\|_{2}=l_{ij},\forall\text{bone} \\ \displaystyle p_{t}^{F}=p_{t-1}^{F},\forall\text{stance foot},$$

where $l_{ij}$ is the bone length between the $i$-th and $j$-th joints. Then, it solves a separate IK problem to recover joint angles that best match the intermediate keypoints:

其中 $l_{ij}$ 是第 $i$ 与第 $j$ 个关节之间的骨长。随后再单独求解一个 IK 问题，找回最匹配这些中间关键点的关节角：

$$\displaystyle q_{t}^{\star}=\argmin_{q_{t}}\sum_{i}\|f_{i}(q_{t})-p_{t,i}^{\star}\|_{2}^{2}.$$

In later stages, additional hard constraints on the feet and waist are imposed to prevent foot slipping and ensure dynamic balancing. This sequential and fragmented approach produces dynamically consistent motions but fails to consider crucial kinematic constraints like joint and velocity limits.

在后续阶段，还会对脚和腰施加额外的硬约束，以防止脚滑并保证动态平衡。这种分段、割裂的做法能产生动力学上一致的动作，但没有考虑关节限位和速度限位等关键运动学约束。

0:  Robot urdf, source keypoint positions $\{p^{\text{source}}_{0:T,i}\}$
0:  $q_{0:T}$
1:  for $t=0,\ldots,T$ do
2:   $q_{t}\leftarrow$ Solve interaction mesh optimization (3)
3:  end for

Algorithm 5 Retarget Robot Motion (OmniRetarget)

0:  Robot urdf, source keypoint positions $\{p^{\text{source}}_{0:T,i}\}$
0:  $q_{0:T}$
1:  for $t=0,\ldots,T$ do
2:   $q_{t}\leftarrow$ Solve interaction mesh optimization (3)
3:  end for

算法 5 重定向机器人动作（OmniRetarget）

VI-B5 OmniRetarget

OmniRetarget, as outlined in Alg. 5, operates frame-by-frame by minimizing the Laplacian deformation of the interaction meshes. The core objective (3a) is flexible and can be augmented with task-specific costs, such as the orientation matching term from GMR, providing a unified and extensible framework for motion retargeting.

VI-B5 OmniRetarget

如算法 5 所述，OmniRetarget 逐帧工作，最小化交互网格的拉普拉斯形变。核心目标（3a）很灵活，可以加入任务相关的代价，例如 GMR 中的朝向匹配项，从而为动作重定向提供一个统一、可扩展的框架。

VI-C Data Augmentation Details

VI-C 数据增强细节

VI-C1 Augmented Object Trajectory

To generate a perturbed object trajectory, we introduce a transient offset that decays exponentially over time. Let the original trajectory be denoted by $(p_{obj}(t),\theta_{obj}(t))$. We define an initial positional offset $\Delta p_{obj}$ and rotational offset $\Delta\theta_{obj}$ that are applied at the onset of object motion, $t_{m}$. The augmented trajectory, $(\tilde{p}_{obj}(t),\tilde{\theta}_{obj}(t))$, is then formulated as:

VI-C1 增强后的物体轨迹

为生成受扰动的物体轨迹，我们引入一个随时间指数衰减的瞬时偏移。把原轨迹记为 $(p_{obj}(t),\theta_{obj}(t))$。我们定义在物体运动起始时刻 $t_{m}$ 施加的初始位置偏移 $\Delta p_{obj}$ 和转动偏移 $\Delta\theta_{obj}$。增强后的轨迹 $(\tilde{p}_{obj}(t),\tilde{\theta}_{obj}(t))$ 写为：

$$\displaystyle\tilde{p}{obj}(t) \ \displaystyle=\begin{cases}\Delta p{obj}+p_{obj}(0)&\text{if }t<t_{m}\
\Delta p_{obj},e^{-(t-t_{m})/\tau_{p}}+p_{obj}(t)&\text{if }t\geq t_{m}\end{cases} \ \displaystyle\tilde{\theta}{obj}(t) \ \displaystyle=\begin{cases}\Delta\theta{obj}\oplus\theta_{obj}(0)&\text{if }t<t_{m}\
\Delta\theta_{obj},e^{-(t-t_{m})/\tau_{\theta}}\oplus\theta_{obj}(t)&\text{if }t\geq t_{m}\end{cases} \tag{14a}$$

where $\tau_{p}$ and $\tau_{\theta}$ are time constants governing the rate of decay for the translational and rotational perturbations, respectively. The $\oplus$ operator denotes composition for orientations (e.g., quaternion multiplication).

其中 $\tau_{p}$ 和 $\tau_{\theta}$ 分别是控制平移和转动扰动衰减快慢的时间常数。$\oplus$ 表示朝向的复合运算（例如四元数乘法）。

VI-C2 Interaction Mesh Construction in Object Frame

For robot-object interactions, it is crucial to construct the interaction mesh in the object’s local coordinate frame. This ensures that the Laplacian coordinates, which encode relative spatial relationships, are invariant to the object’s global rotation and translation. As illustrated in Fig. 8, when the object rotates by $180^{\circ}$ (indicated by the black arrow), the Laplacian coordinate of the object in the world frame $L_{W}$ changes from $(0,1)$ to $(0,-1)$, while the Laplacian coordinate calculated in the object frame $L_{O}$ remains constant. Using object-frame coordinates is therefore essential for preserving the intended interaction geometry during object spatial transformation.

VI-C2 在物体坐标系中构建交互网格

对于机器人–物体交互，必须在物体的局部坐标系中构建交互网格。这样才能使编码相对空间关系的拉普拉斯坐标，对物体的全局转动和平移保持不变。如图 8 所示，当物体旋转 $180^{\circ}$（黑箭头所示）时，物体在世界系中的拉普拉斯坐标 $L_{W}$ 从 $(0,1)$ 变为 $(0,-1)$，而在物体坐标系中计算的拉普拉斯坐标 $L_{O}$ 保持不变。因此，在物体空间变换中要保持预期的交互几何，使用物体坐标系下的坐标至关重要。

Fig. 8: The Laplacian coorinate should stay the same when the object rotates 180◦.

图 8：当物体旋转 $180^{\circ}$ 时，拉普拉斯坐标应保持不变。

Fig. 9: The actual target (left) and source (right) interaction meshes used for optimization.

图 9：优化中实际使用的目标（左）和源（右）交互网格。

VI-D Sequential SOCP Details

VI-D 序贯 SOCP 细节

At each time step $t$, we iteratively solve a Second-Order Cone Program (SOCP) for the optimal change in the robot’s configuration $dq$ util convergence for up to 10 iterations. For conciseness, we omit the time index $t$ for variables within the Sequential SOCP loop.

在每个时间步 $t$，我们迭代求解二阶锥规划（SOCP），以得到机器人构型的最优增量 $dq$，直至收敛，最多 10 次迭代。为简洁起见，在序贯 SOCP 循环内的变量上省略时间下标 $t$。

The optimization finds the increment $dq_{n}$ at the $n$-th iteration. The configuration is updated using this increment, starting from the previous time step’s solution ($\bar{q}_{0}=q_{t-1}^{\star}$):

优化在第 $n$ 次迭代中求解增量 $dq_{n}$。构型用该增量更新，并从上一时间步的解出发（$\bar{q}_{0}=q_{t-1}^{\star}$）：

$$\bar{q}_{n+1}=\bar{q}_{n}+dq_{n}^{\star}.$$

The optimal increment $dq_{n}^{\star}$ is the solution to the following SOCP, which is linearized around the current iterate $\bar{q}_{n}$:

最优增量 $dq_{n}^{\star}$ 是如下 SOCP 的解，该规划在当前迭代点 $\bar{q}_{n}$ 附近线性化：

$$\displaystyle dq_{n}^{\star}=\argmin_{dq_{n}}\quad \\ \displaystyle\|L^{\text{source}}-(J_{L}^{n}\cdot dq_{n}+\bar{L}_{n}^{\text{target}})\|^{2} \\ \displaystyle+\|\bar{q}_{n}+dq_{n}-q_{t-1}\|_{Q}^{2} \\ \displaystyle J_{j}^{n}\cdot dq_{n}+\phi_{j}(\bar{q}_{n})\geq 0,\forall j \\ \displaystyle q_{\min}\leq\bar{q}_{n}+dq_{n}\leq q_{\max} \\ \displaystyle v_{\min}\cdot dt\leq\bar{q}_{n}+dq_{n}-q_{t-1}\leq v_{\max}\cdot dt \\ \displaystyle p_{t}^{F}(\bar{q}_{n})+J_{F}^{n}\cdot dq_{n}=p_{t-1}^{F},\forall\text{stance foot} \\ \displaystyle\|dq_{n}\|_{2}\leq\epsilon, \tag{15a}$$

where

其中

• • $L^{\text{source}}=vec(\{L(p_{t,i}^{\text{source}})\})$

• • $L^{\text{target}}(q)=vec(\{L(p_{t,i}^{\text{target}}(q))\})$

• • $\bar{L}_{n}^{\text{target}}=vec(\{L(p_{t,i}^{\text{target}}(\bar{q}_{n}))\})$

• • $J_{L}^{n}=\partial L^{\text{target}}/\partial q|_{q=\bar{q}_{n}}$

• • $J_{j}^{n}=\partial\phi_{j}/\partial q|_{q=\bar{q}_{n}}$

• • $J_{F}^{n}=\partial p_{t}^{F}/\partial q|_{q=\bar{q}_{n}}$.

• • $J_{F}^{n}=\partial p_{t}^{F}/\partial q|_{q=\bar{q}_{n}}$。

The second-order cone constraint (15g) is a trust region constraint with radius $\epsilon$ (we use $\epsilon=0.2$) that keeps the step size small, ensuring the linear approximations remain valid.

二阶锥约束（15g）是半径为 $\epsilon$ 的信任域约束（我们取 $\epsilon=0.2$），用来限制步长，以保证线性近似仍然有效。

VI-E Downstream RL Evaluation Breakdown

VI-E 下游强化学习评测的分解

Shown in Fig. 10, we present histograms from the downstream RL evaluation (Sec. V-B) to illustrate failure patterns and variance across OmniRetarget and baselines. These histograms break down failure rates by each motion for two tasks: robot–object interaction and robot–terrain interaction, highlighting not only overall averages but also how failures distribute across different motions. We do not include augmented motions as baselines do not support augmentation.

如图 10 所示，我们给出下游强化学习评测（第 V-B 节）的直方图，以说明 OmniRetarget 与各基线的失败模式和方差。这些直方图按每条动作分解失败率，覆盖机器人–物体交互和机器人–地形交互两项任务，不仅给出总体平均，也显示失败如何在不同动作上分布。我们没有纳入增强动作，因为基线并不支持增强。

Fig. 10: Histograms from the downstream RL evaluation showing the failure patterns for the baselines in different tasks.

图 10：下游强化学习评测的直方图，展示各基线在不同任务上的失败模式。

In robot–object interaction, the motions are modest while object properties are heavily randomized. Since the motions are not aggressive, most policies can adapt even to low-quality references and achieve at least one success, except VideoMimic, which fails systematically due to poor interaction preservation. This task therefore measures robustness rather than accuracy. We see that GMR shows broader failure spread with lower success rates, likely due to penetration issues that reduce robustness under placement changes. PHC shows improved robustness, while OmniRetarget achieves the most robust performance, with results concentrated in the high-success region.

在机器人–物体交互中，动作本身并不激烈，但物体属性被大幅随机化。由于动作并不猛烈，大多数策略即使面对低质量参考也能适应，并至少成功一次；VideoMimic 除外，它因交互保持很差而系统性失败。因此这一任务衡量的是稳健性而非精度。可以看到，GMR 的失败分布更宽、成功率更低，很可能是因为穿透问题降低了它对摆放变化的稳健性。PHC 的稳健性有所改善，而 OmniRetarget 最为稳健，结果集中在高成功区域。

In contrast, climbing terrains requires much more agile and challenging motions and thus, demands precise reference motions: if the quality is low, the agent fails outright with no successes. Here, PHC and VideoMimic perform the worst, with nearly half the motions failing entirely. GMR delivers somewhat better references but still fails on four motions, while OmniRetarget fails on only one. These results show that OmniRetarget not only provides superior robustness under variation but also higher reference accuracy.

相比之下，攀爬地形需要更加敏捷、更有挑战的动作，因而要求参考动作足够精确：质量一旦偏低，智能体就会彻底失败、一次成功也没有。在这里，PHC 和 VideoMimic 表现最差，近一半动作完全失败。GMR 给出的参考稍好，但仍有四条动作失败，而 OmniRetarget 只失败一条。这些结果表明，OmniRetarget 不仅在变化条件下更稳健，参考精度也更高。

For the one remaining failure, we believe that it is limited by the simple RL formulation we use. For future work, an interesting direction could be to extend the current RL formulation with curriculum learning to support these extremely difficult motions.

对于剩下的那一次失败，我们认为它受限于我们所采用的简单强化学习形式。未来一个有意思的方向，是把当前的强化学习形式扩展为课程学习，以支持这些极其困难的动作。

Footnotes

脚注

[^1]: The motion is acquired from https://actorcore.reallusion.com/3d-motion?asset=parkour-tic-tac-backflip . An IMU capable of measuring angular rates above $15$ rad/s is required for this motion.

[^2]: Baseline performance may depend on their hyperparameters. We initialized from the default settings in their public codes, and further improved to ensure consistent performance across different tasks.