Visual Imitation Enables Contextual Humanoid Control | AGIDaily
AGI Daily
资讯日报词条论文人物新创
返回论文列表
摘要1 引言2 相关工作3 Real-to-Sim 数据获取3.1 预处理3.2 人体与场景联合重建3.3 生成可仿真数据4 策略学习5 结果5.1 重建与数据5.2 真实世界部署6 结论7 局限重建。重定向。传感与策略输入。仿真保真度。数据规模与动作质量。致谢参考文献附录 A Real-to-Sim 细节A.1 符号A.2 人体与场景联合重建的目标函数A.3 生成可仿真数据A.4 评估细节A.5 自我视角渲染附录 B 策略学习B.1 强化学习设置与训练B.2 观察B.3 动作B.4 仿真参数B.4.1 仿真器中的地形处理B.5 奖励形式B.6 终止准则B.7 域随机化B.8 训练数据分布附录 C 机器人部署C.1 真实环境中的分阶段部署
返回论文列表
目录

Visual Imitation Enables Contextual Humanoid Control

收录于 CoRL 2025 Best Student Paper Award被引 142 次
打开论文来源

Visual Imitation Enables Contextual Humanoid Control

视觉模仿使人形机器人实现情境控制

Arthur Allshire∗^*∗, Hongsuk Choi∗^*∗, Junyi Zhang∗^*∗, David McAllister∗^*∗, Anthony Zhang, Chung Min Kim, Trevor Darrell, Pieter Abbeel, Jitendra Malik, Angjoo Kanazawa

UC Berkeley

∗^*∗These authors contributed equally. Correspondence to: allshire@berkeley.edu, hongsuk@berkeley.edu.

Abstract

摘要

How can we teach humanoids to climb staircases and sit on chairs using the surrounding environment context? Arguably, the simplest way is to just show them—casually capture a human motion video and feed it to humanoids. We introduce VideoMimic, a real-to-sim-to-real pipeline that mines everyday videos, jointly reconstructs the humans and the environment, and produces whole-body control policies for humanoid robots that perform the corresponding skills. We demonstrate the results of our pipeline on real humanoid robots, showing robust, repeatable contextual control such as staircase ascents and descents, sitting and standing from chairs and benches, as well as other dynamic whole-body skills—all from a single policy, conditioned on the environment and global root commands. VideoMimic offers a scalable path towards teaching humanoids to operate in diverse real-world environments.

怎样教人形机器人顺着周围环境去爬楼梯、坐椅子?最简单的办法,也许就是直接演示给它看——随手拍一段人体动作视频,再交给人形机器人。我们提出 VideoMimic:一条 real-to-sim-to-real 流水线,它从日常视频中挖掘数据,联合重建人体与环境,并为人形机器人生成能执行相应技能的全身控制策略。我们在真实人形机器人上展示了这条流水线的结果:只需同一条策略,再以环境和全局根指令为条件,就能稳健、可重复地完成情境控制,例如上下楼梯、从椅子和长凳坐下与站起,以及其他动态全身技能。VideoMimic 为教人形机器人在多样的真实环境中工作,提供了一条可扩展的路径。

VideoMimic 把单目视频转为可迁移的人形技能

Figure 1: VideoMimic is a real-to-sim-to-real pipeline that converts monocular videos into transferable humanoid skills, letting robots learn context-aware behaviors (terrain-traversing, stairs-climbing, sitting) in a single policy. Video results are available on our webpage: https://videomimic.net.

图 1:VideoMimic 是一条 real-to-sim-to-real 流水线,把单目视频转化成可迁移的人形技能,使机器人能在同一条策略里学会有情境意识的行为(穿越地形、爬楼梯、坐下)。视频结果见项目主页:https://videomimic.net。

1 Introduction

1 引言

How do we learn to interact with the world around us—like sitting on a chair or climbing a staircase? We watch others perform these actions, try them ourselves, and gradually build up the skill. Over time, we can handle new chairs and staircases, even if we have not seen those exact ones before. If humanoid robots could learn in this way—by observing everyday videos—they could acquire diverse contextual whole-body skills without relying on hand-tuned rewards or motion-capture data for each new behavior and environment. We refer to this ability to execute environment-appropriate actions as contextual control.

我们是怎样学会与周围世界互动的——比如坐到椅子上,或爬上一段楼梯?我们先看别人做,再自己试,逐渐把这项技能练出来。时间长了,即便没见过那把椅子、那段楼梯,也能应付过去。如果人形机器人也能这样学习——通过观察日常视频——它们就可以获得多样的、依赖情境的全身技能,而不必为每一种新行为和每一种新环境去手工调奖励,或重新采集动作捕捉数据。我们把这种按环境采取合适动作的能力,称为情境控制。

We introduce VideoMimic, a real-to-sim-to-real pipeline that turns monocular videos—such as casual smartphone captures—into transferable skills for humanoids. From these videos, we jointly recover the 4D human-scene geometry, retarget the motion to a humanoid, and train an RL policy to track the reference trajectories. We then distill the policy into a single unified policy that observes only proprioception, a local height-map, and the desired root direction. This distilled policy outputs low-level motor actions conditioned on the terrain and body state, allowing it to execute appropriate behaviors—such as stepping, climbing, or sitting—across unseen environments without explicit task labels or skill selection.

我们提出 VideoMimic,一条把单目视频(例如随手用手机拍下的画面)转化成可迁移人形技能的 real-to-sim-to-real 流水线。它从这些视频中联合恢复人体与场景的四维几何,把动作重定向到人形机器人,再用强化学习训练策略去跟踪参考轨迹。随后把该策略蒸馏成一条统一策略,它只观察本体感觉、局部高度图和期望的根方向。这条蒸馏后的策略根据地形和身体状态输出底层电机动作,因而能在未见过的环境里执行迈步、攀爬或坐下等合适行为,无需显式的任务标签,也不必先做技能选择。

We develop a perception module that reconstructs 3D human motion from a monocular RGB video, along with aligned scene point clouds in the world coordinate frame. We convert the point clouds into meshes and align them with gravity to ensure compatibility with physics simulators. The global motion and local poses are retargeted to a humanoid with constraints that ensure physical plausibility, accounting for the embodiment gap. The mesh and retargeted data seed a goal-conditioned DeepMimic [1]-style reinforcement-learning phase in simulation: we warm-start on MoCap data, then train a single policy to track motions from multiple videos in their respective height-mapped environments while randomizing mass, friction, latency, and sensor noise for robustness. Once our tracking policy is trained, we distill it using DAgger [2] to a policy that operates without conditioning on target joint angles. The new policy observes proprioception, an 11 × 11 height-map patch centered on the torso, and the vector to the goal in the robot’s local reference frame. PPO fine-tuning under this reduced observation set yields a generalist controller that, given height-map and root direction at test time, selects and smoothly executes context-appropriate actions such as stepping, climbing, or sitting. In particular, every step of our policy relies only on observations available at real-world deployment, making it immediately runnable on real hardware.

我们开发了一个感知模块,从单目 RGB 视频中重建三维人体动作,以及对齐到世界坐标系的场景点云。点云被转成网格,并与重力对齐,以便接入物理仿真器。全局运动和局部姿态再被重定向到人形机器人,并加上保证物理可行性的约束,以弥补身体形态差异。这些网格和重定向后的数据,会进入仿真中一段以目标为条件、DeepMimic[1] 风格的强化学习:先在动作捕捉数据上热身,再让同一条策略在各自的高度图环境里跟踪来自多段视频的动作,同时对质量、摩擦、延迟和传感器噪声做随机化,以提升稳健性。跟踪策略训好后,我们用 DAgger[2] 把它蒸馏成不再以目标关节角为条件的策略。新策略观察本体感觉、以躯干为中心的 11×1111\times 1111×11 高度图块,以及机器人局部坐标系中指向目标的向量。在这一缩减后的观察集上再用 PPO 微调,就得到一个通才控制器:测试时只要给出高度图和根方向,它就会选择并平滑地执行迈步、攀爬或坐下等与情境相符的动作。尤其是,策略每一步所用的观察在真实部署时都能拿到,因而可以直接跑在真实硬件上。

Our approach bridges 4D video reconstruction and robot skill learning in a single, data-driven loop. Unlike earlier work that recovers only the person or the scene in isolation, we jointly reconstruct both at a physically meaningful scale and represent them as meshes and motion trajectories suitable for physics-based policy learning. We train our approach on 123 monocular RGB videos, which will be released. We validate the approach through deployment on a real Unitree G1 robot, which shows generalized humanoid motor skills in the context of surrounding environments, even on unseen environments. We will release the reconstruction code, policy training framework, and the video dataset to facilitate future research.

我们的方法把四维视频重建和机器人技能学习放进同一条数据驱动的循环里。与先前只单独恢复人或只单独恢复场景的工作不同,我们在具有物理意义的尺度上联合重建二者,并把它们表示成适合基于物理的策略学习的网格和运动轨迹。方法在 123 段单目 RGB 视频上训练,这些视频将会公开。我们在真实的 Unitree G1 机器人上做了验证:即便在未见过的环境中,它也能在周围环境的情境下展现出可泛化的人形运动技能。重建代码、策略训练框架和视频数据集都将发布,以便后续研究使用。

2 Related Work

2 相关工作

Learning Skills on Legged Robots.

腿式机器人上的技能学习。

Method Env. Real-to-Sim Context. Ctrl Real Robot
DeepMimic / SfV [1, 3] ✗ ✗ ✗
Egocentric Loco [4] ✗ ✓ ✓
ASAP [5] ✗ ✗ ✓
Humanoid Loco. [6] ✗ ✗ ✓
H2O / ExBody2 [7, 8] ✗ ✗ ✓
Parkour [9, 10] ✗ ✓ ✓
VideoMimic (Ours) ✓ ✓ ✓

Table 1: Comparison of methods across different features. VideoMimic transfers both human motion and scene geometry from real videos to simulation, learns context-aware control in simulation, and successfully deploys the resulting policy on real-world environments.

方法 环境 Real-to-Sim 情境控制 真实机器人
DeepMimic / SfV [1, 3] ✗ ✗ ✗
Egocentric Loco [4] ✗ ✓ ✓
ASAP [5] ✗ ✗ ✓
Humanoid Loco. [6] ✗ ✗ ✓
H2O / ExBody2 [7, 8] ✗ ✗ ✓
Parkour [9, 10] ✗ ✓ ✓
VideoMimic(本文) ✓ ✓ ✓

表 1:不同方法在各项能力上的对比。VideoMimic 把人体动作和场景几何同时从真实视频迁到仿真中,在仿真里学习有情境意识的控制,并成功把得到的策略部署到真实环境。

Recent progress in legged-robot motor skills follows two complementary streams. Reward-based methods use model-free RL in simulation, shaping behavior with handcrafted objectives that mix task terms (e.g., velocity tracking) and motion-naturalness regularizers; thanks to massive parallel physics engines [11, 12], this paradigm has produced agile locomotion on quadrupeds and humanoids without motion data. However, each new behavior demands tuning of user-defined rewards and environment scripting [13, 14, 15, 4, 6, 9, 16, 17]. Data-driven methods instead imitate reference motion, originally MoCap clips or monocular video, training a simulated character to track them and porting the idea to robots [1, 3, 18, 19, 7, 8]. For example, recent work [20, 21] frames legged locomotion as a next- token prediction task and pre-trains a policy on human data in kinematic space, showing strong performance. While imitation bypasses reward engineering [5], existing works typically assume flat ground or manually designed setups, limiting context-aware whole-body control; even animation systems that model human-scene contact rely on instrumented MoCap stages and thus lack scalability [22, 23]. Our system conditions on visual observations, the local height‑maps, and learns environment‑aware skills such as stair‑climbing and chair‑sitting directly from monocular RGB videos. Joint 4D human–scene reconstruction provides physically consistent reference motions, which RL distills into policies that transfer to a real humanoid (Table 1).

腿式机器人运动技能的近期进展,大致沿着两条互补的路线展开。基于奖励的方法在仿真里做无模型强化学习,用手工设计的目标来塑造行为,其中既有任务项(例如速度跟踪),也有动作自然度的正则;借助大规模并行物理引擎[11, 12],这一范式已经能让四足机器人和人形机器人在没有动作数据的情况下走出敏捷步态。不过,每一种新行为都需要重新调整用户定义的奖励,并编写环境脚本[13, 14, 15, 4, 6, 9, 16, 17]。数据驱动方法则改为模仿参考动作——最初是动作捕捉片段或单目视频——先训练仿真角色去跟踪,再把这一思路迁到机器人上[1, 3, 18, 19, 7, 8]。例如,最近的工作[20, 21]把腿式运动写成下一 token 预测任务,并在运动学空间里用人体数据预训练策略,表现很强。模仿虽然绕开了奖励工程[5],但现有工作通常假定地面平坦,或依赖手工搭好的场景,限制了有情境意识的全身控制;即便是对人体–场景接触建模的动画系统,也往往依赖仪器化的动作捕捉棚,因而难以扩展[22, 23]。我们的系统以视觉观察(局部高度图)为条件,直接从单目 RGB 视频中学习爬楼梯、坐椅子等感知环境的技能。人体与场景的联合四维重建提供了物理一致的参考动作,再经强化学习蒸馏成可迁移到真实人形机器人的策略(表 1)。

Human and Scene Reconstruction from Images and Videos. Early monocular-video methods regress pose and shape of humans [24] in a camera-relative frame with deep networks [24, 25, 26], which suffices for rendering, action recognition, or single-person tracking [27, 28, 29, 30, 31] but leaves the global trajectory—and thus context-aware dynamics—undefined; pioneers like SfV hand-tuned a global scale and even assumed a static camera, limiting generality. Recent methods combine human motion priors with SfM/SLAM to recover metric trajectories [32, 33, 34], yet still model only the person and camera. Advances in general scene parsing [35, 36, 37] have enabled joint human-scene reconstruction that resolves scale via multi-view cues or learned priors [38, 39], but these systems have not been validated on robots. Parallel work injects physics constraints in post-processing or simulation [40, 41, 42, 43, 44], trading scalability for realism. Our pipeline unifies these threads: it simultaneously estimates metric human motion and surrounding geometry from in-the-wild videos—without MoCap, pre-scanned scenes, or reward engineering—and outputs simulator-ready trajectories that respect contacts and collisions, enabling scalable learning of whole-body humanoid skills.

从图像和视频中重建人体与场景。 早期的单目视频方法用深度网络[24, 25, 26]在相机坐标系下回归人体的姿态和体型[24],这对渲染、动作识别或单人跟踪已经够用[27, 28, 29, 30, 31],但全局轨迹——以及因而产生的情境相关动力学——仍是未定义的;像 SfV 这样的先驱工作还要手工设定全局尺度,甚至假定相机静止,限制了通用性。近期方法把人体运动先验与 SfM/SLAM 结合起来,以恢复带度量尺度的轨迹[32, 33, 34],但建模对象仍只有人和相机。通用场景解析的进展[35, 36, 37]使联合的人体–场景重建成为可能,并可借助多视线索或学习到的先验来确定尺度[38, 39],不过这些系统尚未在机器人上得到验证。与此并行的工作在后处理或仿真中注入物理约束[40, 41, 42, 43, 44],往往以可扩展性换取真实感。我们的流水线把这些线索统一起来:它从野外视频中同时估计带度量尺度的人体运动和周围几何——无需动作捕捉、预扫描场景或奖励工程——并输出遵守接触与碰撞约束、可直接进仿真器的轨迹,从而可扩展地学习人形机器人的全身技能。

VideoMimic 的 Real-to-Sim 流程:从手机视频重建人体与场景

Figure 2: VideoMimic Real-to-Sim. A casually captured phone video provides the only input. We first reconstruct per-frame human motion and 2D keypoints, along with a dense scene point cloud. An efficient optimization jointly aligns the motion and point cloud, recovers statistically accurate metric scale using a human height prior, and registers the human trajectory based on human-associated points. The point cloud is then converted to a mesh, aligned with gravity, and the motion is retargeted to a humanoid in the reconstructed scene. This yields world-frame trajectories and simulator-ready meshes that serve as inputs for policy training.

图 2:VideoMimic 的 Real-to-Sim。 唯一输入是一段随手拍摄的手机视频。我们先重建逐帧人体动作和二维关键点,以及稠密的场景点云。随后用一次高效优化,把动作和点云联合对齐,借助身高先验恢复统计上准确的度量尺度,并根据与人体相关的点来配准人体轨迹。点云再被转成网格、与重力对齐,动作则被重定向到重建场景中的人形机器人。由此得到世界坐标系下的轨迹和可直接进仿真的网格,作为策略训练的输入。

3 Real-to-Sim Data Acquisition

3 Real-to-Sim 数据获取

Our real-to-sim pipeline proceeds as summarized in Figure 2. We extract per-frame human poses and a raw scene point cloud from the input video (Sec. 3.1); jointly optimize them to obtain metrically aligned human trajectories and scene geometry (Sec. 3.2); apply gravity alignment and convert the filtered point cloud into a lightweight mesh (Sec. 3.3); and retarget the refined trajectories to the humanoid under joint-limit, contact, and collision constraints (Sec. A.3). The resulting motion–mesh pairs are ready for policy learning in Sec. 4.

我们的 real-to-sim 流水线如图 2 所示。我们从输入视频中提取逐帧人体姿态和原始场景点云(第 3.1 节);再对二者做联合优化,得到度量对齐的人体轨迹和场景几何(第 3.2 节);接着做重力对齐,并把过滤后的点云转成轻量网格(第 3.3 节);最后在关节限位、接触和碰撞约束下,把精炼后的轨迹重定向到人形机器人(第 A.3 节)。得到的动作–网格对即可用于第 4 节的策略学习。

3.1 Preprocessing

3.1 预处理

We preprocess monocular RGB videos with off-the-shelf state-of-the-art human pose estimation and SfM methods. First, people are detected and associated across frames using Grounded SAM2 [45, 46]. For each detected person, we recover per‑frame 3D SMPL [24] parameters with VIMO [47], obtaining per‑frame local pose θt\theta^{t}θt, shape β\betaβ, and SMPL 3D joints J3D t∈RJ×3J_{\mathrm{3D}}^{\,t}\in\mathbb{R}^{J\times 3}J3Dt​∈RJ×3. We detect 2D keypoints J2D tJ_{\mathrm{2D}}^{\,t}J2Dt​, i.e., body joint pixel positions, with ViTPose [48]. Foot contact is regressed by BSTRO [49]. For scene reconstruction, we obtain the world point cloud from either MegaSaM [35] or MonST3R [36], which is parameterized as per-frame depth DtD^{t}Dt, camera pose [Rt∣tt][R^{t}|t^{t}][Rt∣tt], and a shared camera intrinsic matrix KKK. Note that the resulting point cloud is not metrically accurate.

我们用现成的先进人体姿态估计和 SfM 方法,对单目 RGB 视频做预处理。首先用 Grounded SAM2[45, 46] 检测人,并在各帧之间做关联。对每个检测到的人,我们用 VIMO[47] 恢复逐帧的三维 SMPL[24] 参数,得到逐帧局部姿态 θt\theta^{t}θt、体型 β\betaβ,以及 SMPL 三维关节 J3D t∈RJ×3J_{\mathrm{3D}}^{\,t}\in\mathbb{R}^{J\times 3}J3Dt​∈RJ×3。二维关键点 J2D tJ_{\mathrm{2D}}^{\,t}J2Dt​,也就是身体关节的像素位置,由 ViTPose[48] 检测。脚部接触由 BSTRO[49] 回归。场景重建方面,世界点云来自 MegaSaM[35] 或 MonST3R[36],参数化为逐帧深度 DtD^{t}Dt、相机位姿 [Rt∣tt][R^{t}|t^{t}][Rt∣tt],以及共享的相机内参矩阵 KKK。需要注意的是,得到的点云并不具备准确的度量尺度。

To coarsely position the person in the world frame, we follow the initialization strategy of SLAHMR [32], using (i) the camera focal length predicted by SfM and (ii) the ratio between the average 2D limb length from the ViTPose detections J~2D t\tilde{J}_{\mathrm{2D}}^{\,t}J~2Dt​ and the corresponding metric scale 3D limb length in J3D tJ_{\mathrm{3D}}^{\,t}J3Dt​, we estimate a similarity factor per frame that yields a coarse global trajectory (ϕt0,γt0)(\phi^{{t}_{0}},\gamma^{{t}_{0}})(ϕt0​,γt0​). Separately, we also lift J~2D t\tilde{J}_{\mathrm{2D}}^{\,t}J~2Dt​ to 3D by un‑projecting each pixel (u,v)(u,v)(u,v) with its depth Du,vtD_{u,v}^{t}Du,vt​ and intrinsics KKK from SfM: J~3D t=K−1[u,v,1]⊤Du,vt\tilde{J}_{\mathrm{3D}}^{\,t}=K^{-1}[u,v,1]^{\top}D_{u,v}^{t}J~3Dt​=K−1[u,v,1]⊤Du,vt​. The lifted joints are then used to jointly optimize human poses and scene geometry scale, as described in the following section.

为了把人粗略放到世界坐标系中,我们沿用 SLAHMR[32] 的初始化策略:利用(i)SfM 预测的相机焦距,以及(ii)ViTPose 检测 J~2D t\tilde{J}_{\mathrm{2D}}^{\,t}J~2Dt​ 中的平均二维肢体长度,与 J3D tJ_{\mathrm{3D}}^{\,t}J3Dt​ 中对应的度量三维肢体长度之比,为每一帧估计一个相似变换因子,从而得到粗略的全局轨迹 (ϕt0,γt0)(\phi^{{t}_{0}},\gamma^{{t}_{0}})(ϕt0​,γt0​)。另外,我们还用 SfM 给出的深度 Du,vtD_{u,v}^{t}Du,vt​ 和内参 KKK,把每个像素 (u,v)(u,v)(u,v) 反投影,把 J~2D t\tilde{J}_{\mathrm{2D}}^{\,t}J~2Dt​ 抬升到三维:J~3D t=K−1[u,v,1]⊤Du,vt\tilde{J}_{\mathrm{3D}}^{\,t}=K^{-1}[u,v,1]^{\top}D_{u,v}^{t}J~3Dt​=K−1[u,v,1]⊤Du,vt​。这些抬升后的关节随后用于联合优化人体姿态和场景几何尺度,详见下一节。

3.2 Joint Human–Scene Reconstruction

3.2 人体与场景联合重建

Our pipeline jointly optimizes the human trajectory and the scene scale. The variables are the humans’ global translations γ1:T\gamma^{1:T}γ1:T, global orientations ϕ1:T\phi^{1:T}ϕ1:T, local poses θ1:T\theta^{1:T}θ1:T, and the scene point‑cloud scale α\alphaα. Because MegaSam pointclouds are scale‑ambiguous, the metric human height prior in the SMPL body models serves as the metric reference, while the lifted joints J~3D t\tilde{J}_{\mathrm{3D}}^{\,t}J~3Dt​ refine both the global trajectory and the local pose. We therefore solve for α\alphaα simultaneously, reconciling any residual mismatch between the human‑derived scale and the scene geometry.

我们的流水线联合优化人体轨迹和场景尺度。变量包括人体的全局平移 γ1:T\gamma^{1:T}γ1:T、全局朝向 ϕ1:T\phi^{1:T}ϕ1:T、局部姿态 θ1:T\theta^{1:T}θ1:T,以及场景点云尺度 α\alphaα。由于 MegaSam 点云存在尺度模糊,SMPL 人体模型中的度量身高先验充当尺度基准,而抬升后的关节 J~3D t\tilde{J}_{\mathrm{3D}}^{\,t}J~3Dt​ 则同时精炼全局轨迹和局部姿态。因此我们同时求解 α\alphaα,以消解人体导出的尺度与场景几何之间残留的不一致。

Inspired by He et al. [7], we optionally run a scale-adaptation pass that searches for an SMPL shape β⋆\beta^{\star}β⋆ whose height and limb proportions match those of the G1 robot, prior to joint human-scene optimization. The SMPL joints are then extracted from the reshaped mesh. This use of a prefitted G1-scale SMPL β\betaβ effectively rescales the scene geometry to G1 size, improving the feasibility of humanoid motion—e.g., enabling actions like running or climbing over large obstacles—and facilitating reference motion learning. For real-world deployment, we skip this step and operate directly on the original metric-scale scene.

受 He 等人[7]启发,我们可选择在人体–场景联合优化之前,先做一次尺度适配:搜索一个身高和肢体比例与 G1 机器人相符的 SMPL 体型 β⋆\beta^{\star}β⋆。随后从重塑后的网格中提取 SMPL 关节。使用预先拟合到 G1 尺度的 SMPL β\betaβ,相当于把场景几何缩放到 G1 的大小,从而改善人形动作的可行性——例如在大型障碍上奔跑或攀爬——也更便于学习参考动作。真实部署时,我们跳过这一步,直接在原始度量尺度的场景上操作。

The objective combines joint-distance losses in 3D (L3DL_{3\text{D}}L3D​), computed as the L1 distance between J~3D t\tilde{J}_{\mathrm{3D}}^{\,t}J~3Dt​ and J3D tJ_{\mathrm{3D}}^{\,t}J3Dt​, and 2D projection losses (L2DL_{2\text{D}}L2D​), along with a temporal smoothness regularizer (LSmoothL_{\text{Smooth}}LSmooth​) that discourages frame-to-frame jitter:

目标函数由三维关节距离损失(L3DL_{3\text{D}}L3D​,即 J~3D t\tilde{J}_{\mathrm{3D}}^{\,t}J~3Dt​ 与 J3D tJ_{\mathrm{3D}}^{\,t}J3Dt​ 之间的 L1 距离)、二维投影损失(L2DL_{2\text{D}}L2D​),以及抑制帧间抖动的时间平滑正则(LSmoothL_{\text{Smooth}}LSmooth​)组成:

arg⁡min⁡α,γ,ϕ,θ  w3DL3D+w2DL2D+LSmooth.\arg\min_{\alpha,\gamma,\phi,\theta}\;w_{3\text{D}}L_{3\text{D}}+w_{2\text{D}}L_{2\text{D}}+L_{\text{Smooth}}.argα,γ,ϕ,θmin​w3D​L3D​+w2D​L2D​+LSmooth​.

We optimize this objective with a Levenberg–Marquardt solver implemented in JAX [50]. Running on an NVIDIA A100 GPU, the optimizer processes a 300‑frame sequence in approximately 20 ms after compilation. Please refer to Sec. A for more details.

我们用 JAX[50] 实现的 Levenberg–Marquardt 求解器来优化这一目标。在一块 NVIDIA A100 GPU 上,编译完成后,优化器处理一段 300 帧序列大约只需 20 毫秒。更多细节见第 A 节。

3.3 Generating Simulation-Ready Data

3.3 生成可仿真数据

To deploy the monocular reconstruction in a physics engine, we (i) align it with real-world gravity using GeoCalib [51] and (ii) convert the noisy, dense point cloud into a lightweight mesh that imposes meaningful geometric constraints and supports memory-efficient parallel training. We use NKSR [52] for meshification. Please see Sec. A.3 for more details.

要把单目重建放进物理引擎,我们会(i)用 GeoCalib[51] 把它与真实世界的重力对齐,并(ii)把嘈杂、稠密的点云转成轻量网格,既施加有意义的几何约束,也便于节省显存的并行训练。网格化使用 NKSR[52]。更多细节见第 A.3 节。

4 Policy Learning

4 策略学习

VideoMimic 在仿真中的策略训练流程

Figure 3: Policy training in sim. Our pipeline of training RL starts with a dataset of Motion Capture trajectories. We then inject a heightmap observation and track whole-body reference trajectories from our videos in various environments. We proceed to distill a policy conditioned only on the root position of the robot. We then finetune this policy directly with RL using the same reduced observation set. Our pipeline is motivated by three goals: (a) producing motions that are fast and faithful to the original video demonstrations; (b) ensuring observations are available in real-world settings; and (c) training a generalist policy that distills knowledge from all video demonstrations into a single model applicable beyond the training set.

图 3:仿真中的策略训练。 我们的强化学习流水线从一套动作捕捉轨迹数据集开始。随后注入高度图观察,并在多种环境中跟踪来自视频的全身参考轨迹。再蒸馏出一条只以机器人根位置为条件的策略。接着用同一套缩减后的观察,直接用强化学习微调这条策略。流水线服务于三个目标:(a)得到既快又忠实于原始视频示范的动作;(b)确保这些观察在真实环境中也能拿到;(c)训练一条通才策略,把所有视频示范中的知识蒸馏进同一个模型,并能用于训练集之外的情形。

Given the kinematic reference from our clips and scenes, our policy learning pipeline produces a context-conditioned policy that can perform skills from the references when prompted by the appropriate environmental context. Figure 3 gives an overview of our pipeline, detailed below.

有了片段和场景给出的运动学参考,我们的策略学习流水线会得到一条以情境为条件的策略:当周围环境给出相应提示时,它就能执行参考中的技能。图 3 给出了流水线概览,细节如下。

Policy Learning. We use Proximal Policy Optimization [53] implementation from Rudin et al. [54] for training our policy. Our learning takes place in the IsaacGym simulator [12]. Please refer to Sec. B for more details on our formulation and hyperparameters.

策略学习。 我们用 Rudin 等人[54]实现的近端策略优化(Proximal Policy Optimization)[53] 来训练策略。学习在 IsaacGym 仿真器[12]中进行。关于问题形式化与超参数的更多细节,见第 B 节。

Observations. Our policies are conditioned on both proprioceptive and target-related observations. The proprioceptive inputs include a history of the robot’s joint positions (qqq), joint velocities (q˙\dot{q}q˙​), angular velocity (ω\omegaω), projected gravity vector (ggg), and previous actions (at−1a^{t-1}at−1); we use a history length of 5 in practice. In addition, the policy receives local target observations: the target joint angles, target root roll and pitch, and the desired root direction, specified by relative x-y offset and yaw angle between the robot’s current root position and the target root, all expressed in the robot’s local frame. For policies conditioned on heightmaps, we further provide an elevation map around the torso. This is represented as an 11×1111\times 1111×11 grid sampled at 0.1m0.1\mathrm{m}0.1m intervals, which captures local terrain geometry. Finally, the critic receives additional privileged observations, which are detailed in Table 3.

观察。 我们的策略同时以本体感觉和与目标相关的观察为条件。本体感觉输入包括机器人关节位置(qqq)、关节速度(q˙\dot{q}q˙​)、角速度(ω\omegaω)、投影重力向量(ggg)以及上一时刻动作(at−1a^{t-1}at−1)的历史;实践中历史长度为 5。此外,策略还接收局部目标观察:目标关节角、目标根滚转与俯仰,以及期望的根方向——由机器人当前根位置与目标根之间的相对 xxx-yyy 偏移和偏航角给出,全部表示在机器人局部坐标系中。对以高度图为条件的策略,我们还提供躯干周围的高程图,表示为按 0.1m0.1\mathrm{m}0.1m 间隔采样的 11×1111\times 1111×11 网格,用于刻画局部地形几何。最后,critic 还会收到额外的特权观察,详见表 3。

Batched Tracking. Our system utilizes a batched variant of DeepMimic [1] in order to learn to imitate motions using RL. We implement Reference State Initialization [1] in addition to motion load balancing similar to Tessler et al. [55], upweighting motions with a lower success rate.

批量跟踪。 系统采用 DeepMimic[1] 的批量变体,用强化学习来学习模仿动作。我们实现了参考状态初始化(Reference State Initialization)[1],并采用与 Tessler 等人[55]类似的动作负载均衡,提高成功率较低的动作的采样权重。

Rewards. Our RL reward is designed entirely around data-driven tracking terms—specifically, link and joint positions, joint velocities, and foot-contact signals—so that raw demonstrations can be translated into physically executable motions with minimal hand-tuning. We have two objectives: (1) reducing reliance on manually crafted priors that are typically introduced through reward engineering, and (2) ensuring physical feasibility of the resulting motions. These two goals can conflict: because the reference trajectories are purely kinematic data from humans, exact tracking may result in non-physical motion. We therefore introduce an action-rate penalty along with several other penalty criteria designed to discourage exploiting simulator physics. Full formulations and weights are detailed in Sec. B.1. We train our policy over the stages described below.

奖励。 我们的强化学习奖励完全围绕数据驱动的跟踪项来设计——具体包括连杆与关节位置、关节速度,以及脚部接触信号——以便在尽量少做手工调节的情况下,把原始示范转化成物理上可执行的动作。我们有两个目标:(1)减少通常通过奖励工程引入的手工先验;(2)保证所得动作在物理上可行。这两个目标可能冲突:因为参考轨迹只是来自人体的纯运动学数据,精确跟踪有时会得到非物理的动作。因此我们引入动作变化率惩罚,以及其他若干用于抑制钻仿真物理空子的惩罚项。完整公式和权重见第 B.1 节。策略按下面各阶段训练。

Stage 1: MoCap Pre-Training. MoCap pre-training lets a policy learn challenging skills from noisy video reconstruction while keeping hand-crafted priors to a minimum and bridging the human-to-robot embodiment gap. Earlier work tackles this either by sampling start poses with multi-agent RL [3] or by having a privileged simulator imitate the motion [5]. Radosavovic et al. [21] and Singh et al. [56] instead employed a form of kinematic pre‑training on human data. We adopt a simpler yet effective strategy: first pre‑train the policy on MoCap trajectories, then fine‑tune it on our reconstructed video data—both stages use reinforcement learning in a physics simulator. Even the MoCap‑only policy can be deployed directly on the real robot as shown in Fig 9. We used LAFAN motion capture data [57] retargeted to Unitree G1. For the pretrained policies, the conditioning the policy receives is the target joint angles, target root roll/pitch, and desired root direction.

阶段 1:动作捕捉预训练。 动作捕捉预训练让策略能从带噪声的视频重建中学会有挑战性的技能,同时把手工先验压到最低,并弥合人到机器人的身体形态差异。先前工作要么用多智能体强化学习来采样起始姿态[3],要么让带特权的仿真器去模仿动作[5]。Radosavovic 等人[21]和 Singh 等人[56]则改为在人体数据上做一种运动学预训练。我们采用更简单却有效的策略:先在动作捕捉轨迹上预训练策略,再在我们重建的视频数据上微调——两个阶段都在物理仿真器里做强化学习。即便是只用动作捕捉训出的策略,也可以直接部署到真实机器人上,如图 9 所示。我们使用重定向到 Unitree G1 的 LAFAN 动作捕捉数据[57]。对预训练策略而言,它所接收的条件是目标关节角、目标根滚转/俯仰,以及期望的根方向。

Stage 2: Scene-Conditioned Tracking. After MoCap pretraining, we initialize the policy from MPT checkpoints and introduce scene awareness by conditioning on the environment heightmap. The heightmap is integrated via a projection into the MPT policy’s latent space residually with an initial weight of 0. We then randomly sample motions and perform DeepMimic-style tracking across reconstructed terrains. During this stage, the policy continues to receive motion-specific tracking conditioning, including target joint angles, root roll/pitch, and desired root directions.

阶段 2:以场景为条件的跟踪。 动作捕捉预训练之后,我们从 MPT 检查点初始化策略,并通过以环境高度图为条件来引入场景意识。高度图经投影残差式地接入 MPT 策略的潜空间,初始权重为 0。随后随机采样动作,并在重建地形上做 DeepMimic 式跟踪。这一阶段,策略仍接收特定于动作的跟踪条件,包括目标关节角、根滚转/俯仰,以及期望的根方向。

Stage 3: Distillation. Following the stage of batched tracking, we distill via DAgger [2] to a policy that does not observe target joint angles or root roll/pitch observations. We are then able to use the desired root directions observations as conditioning signals to control the robot’s position, which can be fed either from a joystick or potentially a path provided by a high-level controller. In this way, our framework unifies the previously separate approaches of joystick tracking and global reference following. Our distilled policy benefits from the fact that the teacher is also trained with observation randomization, hence it learns actions under some uncertainty, which would not be the case if we started by training with full body observations; this has been shown to be helpful in other contexts with policy learning [58].

阶段 3:蒸馏。 批量跟踪之后,我们用 DAgger[2] 蒸馏出一条不再观察目标关节角或根滚转/俯仰的策略。这样就可以把期望的根方向观察当作控制机器人位置的条件信号,它既可来自手柄,也可来自高层控制器给出的路径。由此,我们的框架把原先分开的手柄跟踪和全局参考跟随统一起来。蒸馏后的策略还受益于教师本身也在观察随机化下训练,因而会在一定不确定性下学习动作;如果一开始就用全身观察来训练,就不会是这样。这一点在其他策略学习情境中也已被证明有帮助[58]。

Stage 4: Under-conditioned RL Finetuning. After distilling our policies to be exclusively conditioned on the root of our trajectories, we perform another round of RL. This is because behaviours which are learned conditioned on target joints may be sub-optimal for policies which are not conditioned on such targets. In practice, we found that this can significantly boost performance as compared to distilled policies. It also makes it possible to add lower-quality reference motions to the reference set since removing targets from the actor in effect makes it a “data-driven” reward signal with an under-constrained actor which is able to follow references appropriate to context.

阶段 4:欠条件下的强化学习微调。 把策略蒸馏成只以轨迹的根为条件之后,我们再做一轮强化学习。这是因为:在以目标关节为条件时学到的行为,对不再以这些目标为条件的策略来说,可能并不是最优的。实践中我们发现,与蒸馏策略相比,这一步能明显提升表现。它也使质量较低的参考动作可以被加入参考集:从 actor 中去掉目标,相当于把跟踪变成一种「数据驱动」的奖励信号,同时让欠约束的 actor 能够跟随与情境相符的参考。

VideoMimic Real-to-Sim 流水线的多样能力

Figure 4: Versatile capabilities of our Real-to-Sim pipeline. VideoMimic enables (i) robust tracking of Internet videos with challenging motion and diverse environments, (ii) simultaneous reconstruction and retargeting of multiple humans, and (iii) ego-view RGB-D rendering for embodied perception—though not used in our current policy, it highlights the framework’s broader applicability across inputs and tasks.

图 4:我们 Real-to-Sim 流水线的多样能力。 VideoMimic 能够:(i)稳健跟踪动作难度高、环境多样的互联网视频;(ii)同时重建并重定向多个人;(iii)为具身感知做自我视角的 RGB-D 渲染——尽管当前策略并未使用它,这仍说明该框架在输入和任务上有更广的适用性。

5 Results

5 结果

We demonstrate that humanoid robots can learn context-aware skills in diverse environments by imitating everyday human videos. We first evaluate the robustness of our reconstruction pipeline against baselines. Next, we demonstrate its versatility, highlighting its potential impact on future research. We then detail our curated video dataset. Finally, we ablate the MPT component and present demonstrations successfully transferred from simulation policies to a physical robot.

我们表明,通过模仿日常人体视频,人形机器人可以在多样环境中学会有情境意识的技能。我们先对照基线评估重建流水线的稳健性,再展示它的通用性,并说明其对后续研究的潜在影响。随后介绍我们整理的视频数据集。最后对 MPT 组件做消融,并给出从仿真策略成功迁到实体机器人上的演示。

5.1 Reconstruction and Data

5.1 重建与数据

Evaluation. We evaluate the robustness of our reconstruction pipeline on a subset of the SLOPER4D dataset [59], assessing both human trajectory and scene geometry reconstruction.

评估。 我们在 SLOPER4D 数据集[59]的一个子集上评估重建流水线的稳健性,同时考察人体轨迹和场景几何的重建。

Methods WA-MPJPE W-MPJPE Chamfer Distance
WHAM* [60] 189.29 1148.49 -
TRAM [47] 149.48 954.90 10.66
Ours 112.13 696.62 0.75

Table 2: Comparison of Reconstruction. *: WHAM does not recover the environment.

方法 WA-MPJPE W-MPJPE Chamfer 距离
WHAM* [60] 189.29 1148.49 -
TRAM [47] 149.48 954.90 10.66
本文 112.13 696.62 0.75

表 2:重建结果对比。*:WHAM 不恢复环境。

We compare our method against baselines following the standard evaluation protocol [60, 47]. As summarized in Table 2, our method consistently achieves the best performance, outperforming prior work in both human trajectory accuracy (WA/W-MPJPE) and scene geometry (Chamfer Distance). Refer to Sec. A.4 for more evaluation details.

我们按标准评估协议[60, 47]与基线比较。如表 2 所示,我们的方法在各项指标上均为最优,人体轨迹精度(WA/W-MPJPE)和场景几何(Chamfer 距离)都超过了先前工作。更多评估细节见第 A.4 节。

策略在真实机器人上完成复杂地形、站立与坐下

Figure 5: The policy performing various skills on the real robot: traversing complex terrain, standing, and sitting. All these skills are in a single policy, which decides what to do based on the context of its heightmap and joystick direction input. Top row: the policy stands from a seated position after sitting down. Second row: the policy walks up a flight of stairs. Third row: the policy walks down a flight of stairs. Bottom row: the policy walks over a kerb and onto a rough terrain. Please find the video results on our webpage.

图 5:策略在真实机器人上完成的多种技能: 穿越复杂地形、站起和坐下。这些技能都在同一条策略里,它根据高度图的情境和手柄方向输入来决定做什么。第一行:策略坐下后再从坐姿站起。第二行:策略走上一段楼梯。第三行:策略走下一段楼梯。最下行:策略跨过路缘,走上粗糙地形。视频结果见项目主页。

Versatility. Figure 4 and our webpage’s Viser [61] visualizations highlight the breadth of our reconstruction pipeline, showcasing (i) robust environment reconstruction from an Internet video involving dynamic human-scene interaction, (ii) multi-human reconstruction and retargeting. Furthermore, the dense point cloud reconstruction enables ego-view RGB-D rendering via simple rasterization. While not used in our current policy, this offers a promising direction for future work—especially given the challenges of rendering naturalistic images in simulation.

通用性。 图 4 以及项目主页上的 Viser[61] 可视化,展现了重建流水线的覆盖面,包括:(i)从一段包含动态人体–场景交互的互联网视频中稳健地重建环境;(ii)多人重建与重定向。此外,稠密点云重建还可以通过简单的光栅化,做自我视角的 RGB-D 渲染。尽管当前策略并未使用它,这仍为后续工作指出了一个有希望的方向——尤其是考虑到在仿真中渲染自然图像本身就很困难。

Video Data. We curated 123 casually recorded smartphone videos of people performing everyday activities in diverse indoor and outdoor settings, including sitting, standing up from furniture, walking up/down stairs (even backwards), and stepping onto blocks. See Sec. B.8 for more details.

视频数据。 我们整理了 123 段用智能手机随手拍摄的视频,记录人们在多样室内外场景中的日常活动,包括坐下、从家具站起、上下楼梯(甚至倒着走),以及踏上台阶。更多细节见第 B.8 节。

动作捕捉预训练(MPT)对成功率的影响

Figure 6: Impact of MoCap pre-training (MPT). Pre-training the policy on motion-capture data facilitates learning on video captures despite noisy references.

图 6:动作捕捉预训练(MPT)的影响。 先在动作捕捉数据上预训练策略,即使参考带有噪声,也有助于在视频采集数据上开展学习。

MPT ablation. We ablate the impact of pretraining on motion capture data. MPT has multiple effects: first, reference motions are noisy and thus harder to learn to track tabula rasa. Second, initial positions of the robot are often not entirely statically stable or may have some interpenetrations with the scene. Hence, MPT can help stabilize learning during the initial phases, whereas a policy from scratch may not even be able to learn how to balance. As shown in Figure 6, removing MPT significantly hinders the policy’s ability to learn effective behaviors.

MPT 消融。 我们消融了在动作捕捉数据上预训练的作用。MPT 有多重效果:其一,参考动作带有噪声,从零开始学跟踪会更难;其二,机器人的初始位置往往并非完全静稳定,或会与场景发生一定程度的相互穿透。因此,MPT 有助于在初期稳定学习,而从零训练的策略甚至可能学不会保持平衡。如图 6 所示,去掉 MPT 会显著妨碍策略学到有效行为。

5.2 Real-world Deployment

5.2 真实世界部署

Setup. We deploy our controller on a 23-DoF Unitree G1 humanoid and run it onboard at 50 Hz. Following [62], we set relatively low joint gains, Kp=75K_{p}=75Kp​=75, to avoid excessively fast or overly stiff behaviour—which helps to avoid excessively violent contact when the robot makes heavy contact with objects such as chairs or stairs. LiDAR odometry and height-maps are computed in real time using Fast-lio2 [63] and probabilistic terrain mapping [64, 17]. The odometry is used to align the local height-map observations into the robot’s frame. We feed joystick targets from a human operator. Including policy running, all operations are run onboard. We found two critical ingredients for successful motion deployment through iterative sim-to-real trials: (i) relaxing the episode-termination tolerances with respect to the reference motion, and (ii) injecting realistic physics perturbations during training. Full details are given in Sec. C.

设置。 我们把控制器部署在 23 自由度的 Unitree G1 人形机器人上,并在机载以 50 Hz 运行。沿用[62],我们把关节增益设得较低,Kp=75K_{p}=75Kp​=75,以免动作过快、过硬——这有助于在机器人与椅子或楼梯等物体发生较重接触时,避免冲击过大。LiDAR 里程计和高度图由 Fast-lio2[63] 与概率地形建图[64, 17]实时计算。里程计用来把局部高度图观察对齐到机器人坐标系。手柄目标由操作员给出。包括策略推理在内,全部运算都在机载完成。通过反复的仿真到真实试验,我们发现成功部署动作有两个关键因素:(i)相对参考动作放宽回合终止容差;(ii)在训练中注入贴近实际的物理扰动。完整细节见第 C 节。

Real-world evaluation. Figure 5 and the accompanying video showcase the policy executing a wide range of whole-body behaviors on the Unitree G1. Without any task-specific tuning, the same network—driven only by proprioception and a noisy LiDAR height-map that provides a full 360∘360^{\circ}360∘ view around the torso—climbs and descends indoor and outdoor staircases, traverses steep earthen slopes and rough vegetation, and reliably sits down on or stands up from chairs and benches. The controller is surprisingly resilient: after unexpected foot slides while descending stairs, it recovers by momentarily hopping on a single leg before regaining nominal gait.

真实世界评估。 图 5 和配套视频展示了该策略在 Unitree G1 上执行的多种全身行为。不必针对单项任务再调参,同一网络只靠本体感觉,以及环绕躯干一圈、带噪声的 LiDAR 高度图所提供的 360∘360^{\circ}360∘ 视野,就能在室内外爬上爬下楼梯,穿过较陡的土坡和杂草,也能可靠地从椅子和长凳上坐下、站起。控制器的韧性出乎意料:下楼梯时如果脚底突然打滑,它会短暂单脚跳一下,再回到正常步态。

To the best of our knowledge, this is the first real-world deployment of a context-aware humanoid policy learned from monocular human videos, jointly demonstrating perceptive locomotion and environment-prompted whole-body skills such as sitting, standing, and climbing stairs. Additional qualitative results are available on the project webpage.

据我们所知,这是首次把从单目人体视频学到的有情境意识的人形策略部署到真实世界,并同时展示有感知的移动,以及由环境提示的全身技能,例如坐下、站起和爬楼梯。更多定性结果见项目主页。

6 Conclusion

6 结论

We introduced VideoMimic, a real-to-sim-to-real pipeline that converts everyday human videos into environment-conditioned control policies for humanoids. The system (i) reconstructs humans and surrounding geometry from monocular clips, (ii) retargets the motion to a kinematically feasible humanoid, and (iii) uses the recovered scene as task terrain for dynamics-aware RL. The result is a single policy that delivers robust, repeatable contextual control—e.g., stair ascents/descents and chair sit-stand—all driven only by the environment geometry and a root direction command. VideoMimic offers a scalable path for teaching humanoids contextual skills directly from videos. We expect future work to extend the system to richer human–environment interactions, multi-modal sensor-based context learning, and multi-agent behavior modeling, among other directions.

我们提出了 VideoMimic,一条把日常人体视频转化成以环境为条件的人形控制策略的 real-to-sim-to-real 流水线。该系统(i)从单目片段中重建人体和周围几何,(ii)把动作重定向到运动学上可行的人形机器人,并(iii)把恢复出的场景当作任务地形,用于考虑动力学的强化学习。最终得到的是一条策略,它能提供稳健、可重复的情境控制——例如上下楼梯,以及从椅子坐下、站起——驱动力只来自环境几何和一条根方向指令。VideoMimic 为直接从视频教给人形机器人情境技能,提供了一条可扩展的路径。我们预期后续工作会把系统扩展到更丰富的人–环境交互、基于多模态传感器的情境学习,以及多智能体行为建模等方向。

7 Limitations

7 局限

Our pipeline delivers encouraging real‑world results, yet several practical weaknesses remain.

这条流水线已经给出了令人鼓舞的真实世界结果,但若干实际弱点仍然存在。

Reconstruction.

Monocular 4D human–scene recovery is still brittle in the wild. Camera pose drift in MegaSaM often yields duplicate “ghost” layers of the same surface. Due to its inability to refine the dynamic points, the dynamic points from the person are mistakenly fused into the static point cloud or inaccurately placed (e.g., feet buried beneath the environment). In particular, we found that MegaSaM performs poorly on images with low texture. Depth filtering and spatio‑temporal subsampling remove many outlier points, but aggressive thresholds leave holes that hinder meshing. NKSR mitigates noise, yet may oversmooth fine geometry (e.g., narrow stair treads); such high-frequency details are crucial for robot control, and we discard videos where these details are missing after reconstruction. Also, during point-to-mesh conversion, spiky artifacts may appear due to stray points.

重建。

野外的单目四维人体–场景恢复仍然脆弱。MegaSaM 中的相机位姿漂移,常常会在同一表面上留下重复的「幽灵」层。由于无法精炼动态点,来自人的动态点会被错误地融合进静态点云,或被放错位置(例如脚陷进地面以下)。我们尤其发现,MegaSaM 在低纹理图像上表现较差。深度过滤和时空子采样去掉了许多离群点,但阈值过严又会留下妨碍网格化的空洞。NKSR 能减轻噪声,却可能把精细几何(例如窄楼梯踏步)过度抹平;这类高频细节对机器人控制很关键,重建后若缺失这些细节,我们会丢弃相应视频。此外,点云转网格时,游离点还可能造成尖刺伪影。

Retargeting.

The kinematic optimizer assumes every reference pose can be made feasible once scaled to the robot. In cluttered scenes, this is not always true, and conflicting costs—strict foot‑contact matching versus collision avoidance—can trap the solver in poor local minima that the RL controller must subsequently “clean up.”

重定向。

运动学优化器假定:一旦缩放到机器人尺度,每个参考姿态都可以被做得可行。在杂乱场景中并不总是如此;相互冲突的代价——严格匹配脚部接触,与避免碰撞——可能把求解器困在较差的局部极小,随后还要靠强化学习控制器再做补救。

Sensing and policy input.

At test time, the controller receives only proprioception and an 11×1111\times 1111×11 LiDAR height‑map. This coarse grid is adequate for terrain and chairs but lacks the resolution for precise contacts, manipulation, or reasoning about overhanging obstacles. Incorporating richer perceptual inputs—such as RGB‑D data or learned occupancy grids—would likely broaden the method’s applicability and improve its semantic understanding of the environment.

传感与策略输入。

测试时,控制器只接收本体感觉和一张 11×1111\times 1111×11 的 LiDAR 高度图。这张粗糙网格对地形和椅子已经够用,但分辨率不足以支持精确接触、操作,或对悬空障碍做推理。引入更丰富的感知输入——例如 RGB-D 数据或学习到的占据栅格——有望拓宽方法的适用范围,并改善它对环境的语义理解。

Simulation fidelity.

We assume the scene can be represented as a single rigid mesh. Scaling to articulated or deformable objects will require more expressive simulators and object‑level reconstruction pipelines—open problems for future work.

仿真保真度。

我们假定场景可以表示成一张刚性网格。若要扩展到关节物体或可变形物体,将需要表达力更强的仿真器,以及面向物体的重建流水线——这仍是留给后续工作的开放问题。

Data scale and motion quality.

The distilled policy is trained on only 123 video clips and occasionally relies on recovery behaviors, leading to jerky motions. Larger, more diverse video corpora and iterative real‑world fine‑tuning should improve smoothness and robustness.

数据规模与动作质量。

蒸馏策略只在 123 段视频上训练,偶尔还要依赖恢复行为,因而动作会显得发僵、发顿。更大规模、更多样的视频语料,再加上在真实世界中的迭代微调,应当能改善平滑性和稳健性。

Moving beyond these limitations—through better dynamic static separation, hole‑resistant meshing, adaptive retargeting costs, richer perception, and larger datasets—is a key direction for future work.

超越这些局限——包括更好的动态与静态分离、能抵抗空洞的网格化、自适应的重定向代价、更丰富的感知,以及更大规模的数据集——是后续工作的一个关键方向。

Acknowledgments

We thank Brent Yi for his guidance with the excellent 3D visualization tool we use, Viser. We are grateful to Ritvik Singh, Jason Liu, Ankur Handa, Ilija Radosavovic, Himanshu Gaurav-Singh, Haven Feng, and Kevin Zakka for helpful advice and discussions during the paper. We thank Lea Müller for helpful discussions at the start of the project. We thank Zhizheng Liu for helpful suggestions on evaluating human and scene reconstruction. We thank Moji Shi and Huayi Wang for advice on using LiDAR heightmap input. We thank Eric Xu, Matthew Liu, Hayeon Jeong, Hyunjoo Lee, Jihoon Choi, Tyler Bonnen, and Yao Tang for their help in capturing and featuring in the video clips used in this project. We acknowledge support from the BAIR Humanoid Intelligence Center. Chung Min Kim is supported by the NSF Research Fellowship Program, Grant DGE 2146752. Pieter Abbeel holds concurrent appointments as a Professor at UC Berkeley and as an Amazon Scholar. This paper describes work performed at UC Berkeley and is not associated with Amazon. This project was funded in part by NSF: CNS-2235013, IARPA DOI/IBC No. 140D0423C0035, ONR MURI awards N00014-21-1-2801 and W911NF-23-1-0277, and Bakar fellows.

致谢

我们感谢 Brent Yi 在我们使用的出色三维可视化工具 Viser 上给予的指导。感谢 Ritvik Singh、Jason Liu、Ankur Handa、Ilija Radosavovic、Himanshu Gaurav-Singh、Haven Feng 和 Kevin Zakka 在论文过程中提供的建议与讨论。感谢 Lea Müller 在项目起步时的有益讨论。感谢 Zhizheng Liu 就人体与场景重建评估提出的建议。感谢 Moji Shi 和 Huayi Wang 就使用 LiDAR 高度图输入给予的指导。感谢 Eric Xu、Matthew Liu、Hayeon Jeong、Hyunjoo Lee、Jihoon Choi、Tyler Bonnen 和 Yao Tang 帮助拍摄并出镜于本项目所用的视频片段。我们感谢 BAIR Humanoid Intelligence Center 的支持。Chung Min Kim 受 NSF Research Fellowship Program 资助,资助号 DGE 2146752。Pieter Abbeel 同时担任加州大学伯克利分校教授与 Amazon Scholar。本文所述工作在加州大学伯克利分校完成,与 Amazon 无关。本项目部分资助来自 NSF: CNS-2235013、IARPA DOI/IBC No. 140D0423C0035、ONR MURI 项目 N00014-21-1-2801 与 W911NF-23-1-0277,以及 Bakar Fellows。

References

参考文献

[1] X. B. Peng, P. Abbeel, S. Levine, and M. van de Panne. DeepMimic. ACM Transactions on Graphics , 37(4):1–14, jul 2018. doi: 10.1145/3197517.3201311 . URL https://doi.org/10.1145%2F3197517.3201311 .

[2] S. Ross, G. J. Gordon, and J. A. Bagnell. No-regret reductions for imitation learning and structured prediction. CoRR , abs/1011.0686, 2010. URL http://arxiv.org/abs/1011.0686 .

[3] X. B. Peng, A. Kanazawa, J. Malik, P. Abbeel, and S. Levine. SFV: reinforcement learning of physical skills from videos. CoRR , abs/1810.03599, 2018. URL http://arxiv.org/abs/1810.03599 .

[4] A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged locomotion in challenging terrains using egocentric vision, 2022. URL https://arxiv.org/abs/2211.07638 .

[5] T. He, J. Gao, W. Xiao, Y. Zhang, Z. Wang, J. Wang, Z. Luo, G. He, N. Sobanbab, C. Pan, Z. Yi, G. Qu, K. Kitani, J. Hodgins, L. J. Fan, Y. Zhu, C. Liu, and G. Shi. Asap: Aligning simulation and real-world physics for learning agile humanoid whole-body skills, 2025. URL https://arxiv.org/abs/2502.01143 .

[6] I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath. Real-world humanoid locomotion with reinforcement learning, 2023. URL https://arxiv.org/abs/2303.03381 .

[7] T. He, Z. Luo, W. Xiao, C. Zhang, K. Kitani, C. Liu, and G. Shi. Learning human-to-humanoid real-time whole-body teleoperation, 2024. URL https://arxiv.org/abs/2403.04436 .

[8] M. Ji, X. Peng, F. Liu, J. Li, G. Yang, X. Cheng, and X. Wang. Exbody2: Advanced expressive humanoid whole-body control, 2025. URL https://arxiv.org/abs/2412.13196 .

[9] D. Hoeller, N. Rudin, D. Sako, and M. Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots, 2023. URL https://arxiv.org/abs/2306.14874 .

[10] Z. Zhuang, S. Yao, and H. Zhao. Humanoid parkour learning, 2024. URL https://arxiv.org/abs/2406.10759 .

[11] J. Hwangbo, J. Lee, and M. Hutter. Per-contact iteration method for solving contact dynamics. IEEE Robotics and Automation Letters , 3(2):895–902, 2018. URL www.raisim.com .

[12] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac gym: High performance gpu-based physics simulation for robot learning. CoRR , abs/2108.10470, 2021. URL https://arxiv.org/abs/2108.10470 .

[13] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter. Learning agile and dynamic motor skills for legged robots. CoRR , abs/1901.08652, 2019. URL http://arxiv.org/abs/1901.08652 .

[14] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning quadrupedal locomotion over challenging terrain. Sci. Robotics , 5(47):5986, 2020. doi: 10.1126/scirobotics.abc5986 . URL https://doi.org/10.1126/scirobotics.abc5986 .

[15] A. Kumar, Z. Fu, D. Pathak, and J. Malik. RMA: rapid motor adaptation for legged robots. CoRR , abs/2107.04034, 2021. URL https://arxiv.org/abs/2107.04034 .

[16] H. Wang, Z. Wang, J. Ren, Q. Ben, T. Huang, W. Zhang, and J. Pang. Beamdojo: Learning agile humanoid locomotion on sparse footholds. In Robotics: Science and Systems (RSS) , 2025.

[17] J. Long, J. Ren, M. Shi, Z. Wang, T. Huang, P. Luo, and J. Pang. Learning humanoid locomotion with perceptive internal model. arXiv preprint arXiv:2411.14386 , 2024.

[18] R. Yu, H. Park, and J. Lee. Human dynamics from monocular video with dynamic camera movements. ACM Transactions on Graphics (TOG) , 40(6):1–14, 2021.

[19] Z. Luo, J. Cao, J. Merel, A. Winkler, J. Huang, K. M. Kitani, and W. Xu. Universal humanoid motion representations for physics-based control. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum?id=OrOd8PxOO2 .

[20] I. Radosavovic, S. Kamat, T. Darrell, and J. Malik. Learning humanoid locomotion over challenging terrain. arXiv preprint arXiv:2410.03654 , 2024a.

[21] I. Radosavovic, B. Zhang, B. Shi, J. Rajasegaran, S. Kamat, T. Darrell, K. Sreenath, and J. Malik. Humanoid locomotion as next token prediction. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024b.

[22] M. Hassan, Y. Guo, T. Wang, M. Black, S. Fidler, and X. B. Peng. Synthesizing physical character-scene interactions. In ACM SIGGRAPH 2023 Conference Proceedings , pages 1–9, 2023.

[23] L. Pan, Z. Yang, Z. Dou, W. Wang, B. Huang, B. Dai, T. Komura, and J. Wang. Tokenhsi: Unified synthesis of physical human-scene interactions through task tokenization, 2025. URL https://arxiv.org/abs/2503.19901 .

[24] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. Smpl: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2 , pages 851–866. 2023.

[25] A. Kanazawa, M. J. Black, D. W. Jacobs, and J. Malik. End-to-end recovery of human shape and pose. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7122–7131, 2018.

[26] M. Kocabas, N. Athanasiou, and M. J. Black. Vibe: Video inference for human body pose and shape estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5253–5263, 2020.

[27] S. Peng, Y. Zhang, Y. Xu, Q. Wang, Q. Shuai, H. Bao, and X. Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9054–9063, 2021.

[28] G. Moon, T. Shiratori, and S. Saito. Expressive whole-body 3d gaussian avatar. In European Conference on Computer Vision , pages 19–35. Springer, 2024.

[29] J. Rajasegaran, G. Pavlakos, A. Kanazawa, and J. Malik. Tracking people by predicting 3d appearance, location and pose. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2740–2749, 2022.

[30] D. C. Luvizon, D. Picard, and H. Tabia. 2d/3d pose estimation and action recognition using multitask deep learning. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5137–5146, 2018.

[31] J. Rajasegaran, G. Pavlakos, A. Kanazawa, C. Feichtenhofer, and J. Malik. On the benefits of 3d pose and tracking for human action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 640–649, 2023.

[32] V. Ye, G. Pavlakos, J. Malik, and A. Kanazawa. Decoupling human and camera motion from videos in the wild. 2023.

[33] Y. Yuan, U. Iqbal, P. Molchanov, K. Kitani, and J. Kautz. Glamr: Global occlusion-aware human mesh recovery with dynamic cameras. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11038–11049, 2022.

[34] M. Kocabas, Y. Yuan, P. Molchanov, Y. Guo, M. J. Black, O. Hilliges, J. Kautz, and U. Iqbal. Pace: Human and camera motion estimation from in-the-wild videos. In 2024 International Conference on 3D Vision (3DV) , pages 397–408. IEEE, 2024.

[35] Z. Li, R. Tucker, F. Cole, Q. Wang, L. Jin, V. Ye, A. Kanazawa, A. Holynski, and N. Snavely. Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos. arXiv preprint arXiv:2412.04463 , 2024.

[36] J. Zhang, C. Herrmann, J. Hur, V. Jampani, T. Darrell, F. Cole, D. Sun, and M.-H. Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arXiv:2410.03825 , 2024.

[37] Q. Wang, Y. Zhang, A. Holynski, A. A. Efros, and A. Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501.12387 , 2025.

[38] L. Müller, H. Choi, A. Zhang, B. Yi, J. Malik, and A. Kanazawa. Reconstructing people, places, and cameras. arXiv preprint arXiv:2412.17806 , 2024.

[39] Z. Liu, J. Lin, W. Wu, and B. Zhou. Joint optimization for 4d human-scene reconstruction in the wild. arXiv preprint arXiv:2501.02158 , 2025.

[40] Y. Yuan, S.-E. Wei, T. Simon, K. Kitani, and J. Saragih. Simpoe: Simulated character control for 3d human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 7159–7169, 2021.

[41] Y. Yuan, V. Makoviychuk, Y. Guo, S. Fidler, X. Peng, and K. Fatahalian. Learning physically simulated tennis skills from broadcast videos. ACM Trans. Graph , 42(4), 2023.

[42] N. Ugrinovic, B. Pan, G. Pavlakos, D. Paschalidou, B. Shen, J. Sanchez-Riera, F. Moreno-Noguer, and L. Guibas. Multiphys: multi-person physics-aware 3d motion estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2331–2340, 2024.

[43] S. Zhang, Y. Zhang, F. Bogo, M. Pollefeys, and S. Tang. Learning motion priors for 4d human body capture in 3d scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11343–11353, 2021.

[44] J. Li, S. Bian, C. Xu, G. Liu, G. Yu, and C. Lu. D &d: Learning human dynamics from dynamic camera. In European Conference on Computer Vision , pages 479–496. Springer, 2022.

[45] N. Ravi, V. Gabeur, Y.-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V. Alwala, N. Carion, C.-Y. Wu, R. Girshick, P. Dollár, and C. Feichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714 , 2024. URL https://arxiv.org/abs/2408.00714 .

[46] T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y. Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024.

[47] Y. Wang, Z. Wang, L. Liu, and K. Daniilidis. Tram: Global trajectory and motion of 3d humans from in-the-wild videos. arXiv preprint arXiv:2403.17346 , 2024.

[48] Y. Xu, J. Zhang, Q. Zhang, and D. Tao. Vitpose: Simple vision transformer baselines for human pose estimation. Advances in neural information processing systems , 35:38571–38584, 2022.

[49] C.-H. P. Huang, H. Yi, M. Höschle, M. Safroshkin, T. Alexiadis, S. Polikovsky, D. Scharstein, and M. J. Black. Capturing and inferring dense full-body human-scene contact. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) , pages 13274–13285, June 2022.

[50] B. Yi, V. Ye, M. Zheng, Y. Li, L. Müller, G. Pavlakos, Y. Ma, J. Malik, and A. Kanazawa. Estimating body and hand motion in an ego-sensed world. arXiv preprint arXiv:2410.03665 , 2024.

[51] A. Veicht, P.-E. Sarlin, P. Lindenberger, and M. Pollefeys. Geocalib: Learning single-image calibration with geometric optimization. In European Conference on Computer Vision , pages 1–20. Springer, 2024.

[52] J. Huang, Z. Gojcic, M. Atzmon, O. Litany, S. Fidler, and F. Williams. Neural kernel surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4369–4379, 2023.

[53] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. CoRR , abs/1707.06347, 2017. URL http://arxiv.org/abs/1707.06347 .

[54] N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In A. Faust, D. Hsu, and G. Neumann, editors, Proceedings of the 5th Conference on Robot Learning , volume 164 of Proceedings of Machine Learning Research , pages 91–100. PMLR, 08–11 Nov 2022. URL https://proceedings.mlr.press/v164/rudin22a.html .

[55] C. Tessler, Y. Guo, O. Nabati, G. Chechik, and X. B. Peng. Maskedmimic: Unified physics-based character control through masked motion inpainting. ACM Transactions on Graphics (TOG) , 2024.

[56] H. G. Singh, A. Loquercio, C. Sferrazza, J. Wu, H. Qi, P. Abbeel, and J. Malik. Hand-object interaction pretraining from videos, 2024. URL https://arxiv.org/abs/2409.08273 .

[57] F. G. Harvey, M. Yurick, D. Nowrouzezahrai, and C. Pal. Robust motion in-betweening. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) , 39(4), 2020.

[58] T. G. W. Lum, M. Matak, V. Makoviychuk, A. Handa, A. Allshire, T. Hermans, N. D. Ratliff, and K. V. Wyk. Dextrah-g: Pixels-to-action dexterous arm-hand grasping with geometric fabrics, 2024. URL https://arxiv.org/abs/2407.02274 .

[59] Y. Dai, Y. Lin, X. Lin, C. Wen, L. Xu, H. Yi, S. Shen, Y. Ma, and C. Wang. Sloper4d: A scene-aware dataset for global 4d human pose estimation in urban environments. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 682–692, 2023.

[60] S. Shin, J. Kim, E. Halilaj, and M. J. Black. WHAM: Reconstructing world-grounded humans with accurate 3D motion. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) , 2024.

[61] B. Yi, C. M. Kim, J. Kerr, G. Wu, R. Feng, A. Zhang, J. Kulhanek, H. Choi, Y. Ma, M. Tancik, et al. Viser: Imperative, web-based 3d visualization in python. arXiv preprint arXiv:2507.22885 , 2025.

[62] K. Zakka, B. Tabanpour, Q. Liao, M. Haiderbhai, S. Holt, J. Y. Luo, A. Allshire, E. Frey, K. Sreenath, L. A. Kahrs, C. Sferrazza, Y. Tassa, and P. Abbeel. Mujoco playground, 2025. URL https://arxiv.org/abs/2502.08844 .

[63] W. Xu, Y. Cai, D. He, J. Lin, and F. Zhang. Fast-lio2: Fast direct lidar-inertial odometry, 2021. URL https://arxiv.org/abs/2107.06829 .

[64] P. Fankhauser, M. Bloesch, and M. Hutter. Probabilistic terrain mapping for mobile robots with uncertain localization. IEEE Robotics and Automation Letters (RA-L) , 3(4):3019–3026, 2018. doi: 10.1109/LRA.2018.2849506 .

[65] J. Lu, T. Huang, P. Li, Z. Dou, C. Lin, Z. Cui, Z. Dong, S.-K. Yeung, W. Wang, and Y. Liu. Align3r: Aligned monocular depth estimation for dynamic videos. arXiv preprint arXiv:2412.03079 , 2024.

[66] T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. Kitani, C. Liu, and G. Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning, 2024. URL https://arxiv.org/abs/2406.08858 .

[67] T. Cheynel, T. Rossi, B. Bellot-Gurlet, D. Rohmer, and M.-P. Cani. Sparse motion semantics for contact-aware retargeting. In ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) , 2023.

[68] C. M. Kim, B. Yi, H. Choi, Y. Ma, K. Goldberg, and A. Kanazawa. Pyroki: A modular toolkit for robot kinematic optimization, 2025. URL https://arxiv.org/abs/2505.03728 .

[69] Y. Li, Y. Lin, J. Cui, T. Liu, W. Liang, Y. Zhu, and S. Huang. Clone: Closed-loop whole-body humanoid teleoperation for long-horizon tasks, 2025. URL https://arxiv.org/abs/2506.08931 .

[70] D. Makoviichuk and V. Makoviychuk. rl-games: A high-performance framework for reinforcement learning. https://github.com/Denys88/rl_games , May 2021.

Appendix A Real-to-Sim Details

附录 A Real-to-Sim 细节

Our goal is to endow a humanoid with whole-body skills—walking, climbing, sitting—that account for the surrounding geometry after watching a collection of monocular RGB videos of humans performing such skills. We assume (i) a monocular RGB video that clearly captures both the person and the scene; (ii) a static environment during the clip so human motion and terrain can be treated as rigid at training time; and (iii) known robot kinematics and joint limits, but no multi-view rig or motion-capture setup, depth sensors, or pre-scanned meshes. From the video, we jointly reconstruct metric-scale 4D human trajectories and dense scene geometry, align them to gravity, meshify scene point clouds, and retarget the kinematics to the robot while enforcing contacts and collisions. The resulting motion-and-mesh pairs serve as simulator-ready training clips.

我们的目标是:在观看一批人体完成行走、攀爬、坐下等技能的单目 RGB 视频之后,让人形机器人获得能够顾及周围几何的全身技能。我们假定:(i)一段单目 RGB 视频能够清楚地同时拍到人和场景;(ii)片段期间环境静止,因而训练时可以把人体运动和地形都当作刚体来处理;(iii)机器人的运动学和关节限位已知,但不使用多相机架、动作捕捉装置、深度传感器或预先扫描的网格。我们从视频中联合重建度量尺度的四维人体轨迹和稠密场景几何,将它们与重力对齐,把场景点云网格化,并在施加接触与碰撞约束的同时把运动学重定向到机器人。得到的动作与网格对,即可作为仿真就绪的训练片段。

A.1 Notations

A.1 符号

Setup. Our method takes a monocular video sequence as input. We denote each video frame as ItI^{t}It, with resolution H×WH\times WH×W. Given this input, along with initial estimates of camera parameters and human poses, our method jointly reconstructs global human motion trajectories and dense environmental geometry in a metric-scale 3D world.

设置。 方法以一段单目视频序列为输入。我们将每一视频帧记为 ItI^{t}It,分辨率为 H×WH\times WH×W。在该输入以及相机参数和人体姿态的初始估计之上,方法在度量尺度的三维世界中联合重建全局人体运动轨迹和稠密环境几何。

Human. We represent reconstructed humans from a video using the SMPL model [24]. SMPL is a differentiable function mapping pose parameters θ∈SO(3)J\theta\in\mathrm{SO}(3)^{J}θ∈SO(3)J and shape parameters β∈RB\beta\in\mathbb{R}^{B}β∈RB to a mesh with JJJ joints. The mesh is positioned in the world coordinate system by global orientation ϕ∈SO(3)\phi\in\mathrm{SO}(3)ϕ∈SO(3) and translation γ∈R3\gamma\in\mathbb{R}^{3}γ∈R3. Thus, at frame ttt, a human is defined by:

人体。 我们用 SMPL 模型[24]来表示从视频中重建出的人体。SMPL 是一个可微函数,把姿态参数 θ∈SO(3)J\theta\in\mathrm{SO}(3)^{J}θ∈SO(3)J 和体型参数 β∈RB\beta\in\mathbb{R}^{B}β∈RB 映射成带有 JJJ 个关节的网格。网格通过全局朝向 ϕ∈SO(3)\phi\in\mathrm{SO}(3)ϕ∈SO(3) 和平移 γ∈R3\gamma\in\mathbb{R}^{3}γ∈R3 放到世界坐标系中。因此,在第 ttt 帧,一个人由下式定义:

Pt=ϕt,θt,βt,γt.(1)\textbf{P}^{t}={\phi^{t},\theta^{t},\beta^{t},\gamma^{t}}. \tag{1}Pt=ϕt,θt,βt,γt.(1)

Camera. We assume a perspective camera model with intrinsics K∈R3×3K\in\mathbb{R}^{3\times 3}K∈R3×3 and extrinsics defined by rotation R∈SO(3)R\in\mathrm{SO}(3)R∈SO(3) and translation t∈R3t\in\mathbb{R}^{3}t∈R3. A 3D point x3D∈R3x^{\text{3D}}\in\mathbb{R}^{3}x3D∈R3 is first transformed into the camera frame and then projected onto the image plane as:

相机。 我们采用透视相机模型,内参为 K∈R3×3K\in\mathbb{R}^{3\times 3}K∈R3×3,外参由旋转 R∈SO(3)R\in\mathrm{SO}(3)R∈SO(3) 和平移 t∈R3t\in\mathbb{R}^{3}t∈R3 给出。三维点 x3D∈R3x^{\text{3D}}\in\mathbb{R}^{3}x3D∈R3 先变换到相机坐标系,再投影到像平面:

$$x_{\text{2D}}=\Pi\left(K\begin{bmatrix}R&t\end{bmatrix}\begin{bmatrix}x_{\text{3D}}\
1\end{bmatrix}\right), \tag{2}$$

where Π:R3→R2\Pi:\mathbb{R}^{3}\to\mathbb{R}^{2}Π:R3→R2 denotes the perspective projection, defined as Π([uvw])=(uw,vw)\Pi\left(\begin{bmatrix}u\\ v\\ w\end{bmatrix}\right)=\left(\frac{u}{w},\frac{v}{w}\right)Π​​uvw​​​=(wu​,wv​).

其中 Π:R3→R2\Pi:\mathbb{R}^{3}\to\mathbb{R}^{2}Π:R3→R2 表示透视投影,定义为 Π([uvw])=(uw,vw)\Pi\left(\begin{bmatrix}u\\ v\\ w\end{bmatrix}\right)=\left(\frac{u}{w},\frac{v}{w}\right)Π​​uvw​​​=(wu​,wv​)。

Scene. The scene is represented using dense per-frame depth and camera output by MegaSaM [35] or MonST3R [36]. Specifically, we use MegaSaM for scenes with richer textures where the correspondence-based Bundle-Adjustment is more reliable. For textureless scenes, we adopt MonST3R and its depth-conditioned variant [65] for better reconstruction results, although MegaSam generalizes better across a wider distribution of videos. To resolve scale ambiguity of the camera translation and scene geometry, we introduce a scaling parameter α\alphaα. Given per-pixel depth Di,jD_{i,j}Di,j​, the corresponding world coordinate Si,j\mathcal{S}{i,j}Si,j for pixel (i,j)(i,j)(i,j) is obtained by unprojecting the points:

场景。 场景用 MegaSaM[35] 或 MonST3R[36] 输出的稠密逐帧深度和相机来表示。具体而言,纹理较丰富、基于对应的束调整更可靠的场景,我们使用 MegaSaM;对缺少纹理的场景,则采用 MonST3R 及其以深度为条件的变体[65],以获得更好的重建,尽管 MegaSam 在更广的视频分布上泛化更好。为消除相机平移和场景几何的尺度模糊,我们引入尺度参数 α\alphaα。给定逐像素深度 Di,jD_{i,j}Di,j​,像素 (i,j)(i,j)(i,j) 对应的世界坐标 Si,j\mathcal{S}{i,j}Si,j 由反投影得到:

Si,j=α(R⊤[K−1Di,j[i,j,1]⊤]−R⊤t).(3)\mathcal{S}{i,j}=\alpha(R^{\top}[K^{-1}D_{i,j}[i,j,1]^{\top}]-R^{\top}t). \tag{3}Si,j=α(R⊤[K−1Di,j​[i,j,1]⊤]−R⊤t).(3)

优化前后的人体轨迹与场景点云

Figure 7: Before and after optimization. We visualize human trajectories and scene point clouds before and after optimization, showing each from two different viewpoints.

图 7:优化前与优化后。 我们可视化优化前后的人体轨迹和场景点云,并各从两个不同视点展示。

A.2 Objectives for Joint Human–Scene Reconstruction

A.2 人体与场景联合重建的目标函数

As shown in Figure 7(a), the coarse initialization produces inaccurate human trajectories and an incorrect scene scale. Our optimization procedure refines these estimates by jointly optimizing the human’s global translations (γ1:T\gamma^{1:T}γ1:T), global orientations (ϕ1:T\phi^{1:T}ϕ1:T), local poses (θ1:T\theta^{1:T}θ1:T), and the scene point cloud scale (α\alphaα). The optimization aligns the human and world as shown in Figure 7(b), and aids in generating simulation-ready data.

如图 7(a) 所示,粗初始化给出的人体轨迹并不准确,场景尺度也不正确。我们的优化过程通过联合优化人体的全局平移(γ1:T\gamma^{1:T}γ1:T)、全局朝向(ϕ1:T\phi^{1:T}ϕ1:T)、局部姿态(θ1:T\theta^{1:T}θ1:T)以及场景点云尺度(α\alphaα)来精炼这些估计。优化将人体与世界对齐,如图 7(b) 所示,并有助于生成可仿真数据。

The objective combines joint‑distance losses in 3D and through 2D projection with a temporal‑smoothness regularizer, which discourages frame‑to‑frame jitter in both the root trajectory and the articulated pose:

目标函数把三维关节距离损失、经二维投影的损失,以及时间平滑正则结合在一起,后者用于抑制根轨迹和关节姿态在帧与帧之间的抖动:

arg⁡min⁡α,γ,ϕ,θ  w3DL3D+w2DL2D+LSmooth.\arg\min_{\alpha,\gamma,\phi,\theta}\;w_{3\text{D}}L_{3\text{D}}+w_{2\text{D}}L_{2\text{D}}+L_{\text{Smooth}}.argα,γ,ϕ,θmin​w3D​L3D​+w2D​L2D​+LSmooth​.

$$\displaystyle L_{3\text{D}} \ \displaystyle=\sum_{t}\bigl\lVert J_{\mathrm{3D}}^{,t}-\tilde{J}{\mathrm{3D}}^{,t}\bigr\rVert{1}, \ \displaystyle L_{2\text{D}} \ \displaystyle=\sum_{t}\bigl\lVert\tilde{J}{\mathrm{2D}}^{,t}-\Pi!\Bigl(K!\begin{bmatrix}R^{t}&t^{t}\end{bmatrix}\begin{bmatrix}J{\mathrm{3D}}^{,t}\
\mathbf{1}\end{bmatrix}\Bigr)\bigr\rVert_{1}, \ \displaystyle L_{\text{Smooth}} \ \displaystyle=\lambda_{\gamma}\sum_{t}\bigl\lVert\gamma^{t}-\gamma^{t-1}\bigr\rVert_{2}^{2}+\lambda_{\theta}\sum_{t}\bigl\lVert\theta^{t}-\theta^{t-1}\bigr\rVert_{2}^{2}, \tag{4}$$

where Π:R3→R2\Pi:\mathbb{R}^{3}\to\mathbb{R}^{2}Π:R3→R2 denotes the perspective projection, defined as Π([uvw])=(uw,vw)\Pi\left(\begin{bmatrix}u\\ v\\ w\end{bmatrix}\right)=\left(\frac{u}{w},\frac{v}{w}\right)Π​​uvw​​​=(wu​,wv​). We optimize this objective with a Levenberg–Marquardt solver implemented in JAX [50]. Running on an NVIDIA A100 GPU, the optimizer processes a 300‑frame sequence in approximately 20 ms after compilation.

其中 Π:R3→R2\Pi:\mathbb{R}^{3}\to\mathbb{R}^{2}Π:R3→R2 表示透视投影,定义为 Π([uvw])=(uw,vw)\Pi\left(\begin{bmatrix}u\\ v\\ w\end{bmatrix}\right)=\left(\frac{u}{w},\frac{v}{w}\right)Π​​uvw​​​=(wu​,wv​)。我们用 JAX[50] 实现的 Levenberg–Marquardt 求解器来优化这一目标。在一块 NVIDIA A100 GPU 上,编译完成后,优化器处理一段 300 帧序列大约只需 20 毫秒。

A.3 Generating Simulation Ready Data

A.3 生成可仿真数据

Gravity alignment. We first estimate the gravity direction in the initial camera frame and define the transformation that converts the 3D reconstruction from world coordinates to a gravity-aligned coordinate system compatible with physics engines. GeoCalib [51] provides the roll–pitch angles for that frame; the composite rotation Rgravity,world ⁣= ⁣Ry↑ ⁣→zRx(π)Rz(ρ)Rcam,worldR_{\text{gravity,world}}\!=\!R_{y\uparrow\!\to z}R_{x}(\pi)R_{z}(\rho)R_{\text{cam,world}}Rgravity,world​=Ry↑→z​Rx​(π)Rz​(ρ)Rcam,world​ re-orients the reconstruction so that +z+z+z points up, ensuring consistency with gravity in physics engines. The transformation is applied to both human keypoints and static scene geometry.

重力对齐。 我们首先估计初始相机坐标系中的重力方向,并定义把三维重建从世界坐标转换到与物理引擎兼容的重力对齐坐标系的变换。GeoCalib[51] 为该帧提供滚转–俯仰角;复合旋转 Rgravity,world ⁣= ⁣Ry↑ ⁣→zRx(π)Rz(ρ)Rcam,worldR_{\text{gravity,world}}\!=\!R_{y\uparrow\!\to z}R_{x}(\pi)R_{z}(\rho)R_{\text{cam,world}}Rgravity,world​=Ry↑→z​Rx​(π)Rz​(ρ)Rcam,world​ 将重建重新定向,使 +z+z+z 朝上,从而与物理引擎中的重力保持一致。该变换同时作用于人体关键点和静态场景几何。

Pointcloud filtering. For each world pointcloud we discard noisy background points and human pointclouds with thresholding on depth gradient, rotate by Rgravity,worldR_{\text{gravity,world}}Rgravity,world​, scale, and crop to a ±2 m\pm 2\mathrm{\,m}±2m box around the SMPL joints per frame. A 0.1 m0.1\mathrm{\,m}0.1m voxel grid then keeps at most 20 samples per occupied cell, shrinking the pointcloud to about 5% of its original size without losing surface detail.

点云过滤。 对每个世界点云,我们根据深度梯度阈值丢弃嘈杂的背景点和人体点云,再乘以 Rgravity,worldR_{\text{gravity,world}}Rgravity,world​ 做旋转,缩放,并裁到每帧 SMPL 关节周围 ±2 m\pm 2\mathrm{\,m}±2m 的盒子内。随后用 0.1 m0.1\mathrm{\,m}0.1m 的体素网格,每个被占用的格子最多保留 20 个样本,从而在不丢失表面细节的情况下,把点云缩小到原来的约 5%。

Meshification. We first surface it using NKSR [52] to obtain a coarse mesh. Top-down ray casting fills large holes via inverse-distance interpolation inside the convex hull; merging the infilled points with the originals and re-running NKSR produces the final mesh. The entire pipeline processes a 300-frame sequence in roughly 60 s (∼ ⁣5\sim\!5∼5 s for gravity alignment and ∼ ⁣55\sim\!55∼55 s for filtering and meshing).

网格化。 我们先用 NKSR[52] 做表面重建,得到一张粗糙网格。自上而下的射线投射再在凸包内用反距离插值填补大空洞;把填充点与原始点合并,重新运行 NKSR,即得到最终网格。整条流水线处理一段 300 帧序列大约需要 60 秒(重力对齐约 5 秒,过滤与网格化约 55 秒)。

Humanoid Motion Retargeting. Given the human trajectories and environment meshes as input, our objective is to transform these motions to the robot’s embodiment. We approach the retargeting task as an optimization problem similar to previous work [7, 66], but instead use a Levenberg-Marquardt solver to handle the highly nonlinear landscape of the human-in-scene problem setting.

人形动作重定向。 在以人体轨迹和环境网格为输入的前提下,我们的目标是把这些动作变换到机器人的身体形态上。我们把重定向当作一个优化问题,做法与先前工作[7, 66]相近,但改用 Levenberg-Marquardt 求解器,以应对「人在场景中」这一高度非线性的问题设定。

We solve for the following variables: the G1 joint angles q1:Tq^{1:T}q1:T, its root poses (ϕR1:T,γR1:T)(\phi_{\text{R}}^{1:T},\gamma_{\text{R}}^{1:T})(ϕR1:T​,γR1:T​), and the set of per-link scale factors sss between the two embodiments. Note that sss is constant across timesteps, and for all distance-related costs we penalize xxx, yyy, and zzz components independently (i.e., no norm).

我们求解如下变量:G1 的关节角 q1:Tq^{1:T}q1:T、其根位姿 (ϕR1:T,γR1:T)(\phi_{\text{R}}^{1:T},\gamma_{\text{R}}^{1:T})(ϕR1:T​,γR1:T​),以及两套身体形态之间各连杆的尺度因子集合 sss。注意 sss 在时间上是恒定的;对所有与距离有关的代价,我们分别惩罚 xxx、yyy、zzz 分量(即不取范数)。

Inspired by Cheynel et al. [67], we implement a kinematic tree-based motion transfer cost Lmotiont=Lpositiont+LangletL_{\text{motion}}^{t}=L_{\text{position}}^{t}+L_{\text{angle}}^{t}Lmotiont​=Lpositiont​+Langlet​:

受 Cheynel 等人[67]启发,我们实现了基于运动学树的动作迁移代价 Lmotiont=Lpositiont+LangletL_{\text{motion}}^{t}=L_{\text{position}}^{t}+L_{\text{angle}}^{t}Lmotiont​=Lpositiont​+Langlet​:

Lpositiont=∑i≠jmij∥ΔijSMPL(t)−s⋅ΔijG1(t)∥22,(7)L_{\text{position}}^{t}=\sum_{i\neq j}m_{ij}\left\|\Delta_{ij}^{\text{SMPL}}(t)-s\cdot\Delta_{ij}^{\text{G1}}(t)\right\|_{2}^{2}, \tag{7}Lpositiont​=i=j∑​mij​​ΔijSMPL​(t)−s⋅ΔijG1​(t)​22​,(7)

Langlet=∑i≠jmij(1−⟨ΔijSMPL(t), ΔijG1(t)⟩),(8)L_{\text{angle}}^{t}=\sum_{i\neq j}m_{ij}\left(1-\left\langle\Delta_{ij}^{\text{SMPL}}(t),\ \Delta_{ij}^{\text{G1}}(t)\right\rangle\right), \tag{8}Langlet​=i=j∑​mij​(1−⟨ΔijSMPL​(t), ΔijG1​(t)⟩),(8)

where Δij\Delta_{ij}Δij​ is the position difference between joints iii and jjj. mijm_{ij}mij​ denotes if the joints are immediate neighbors in the robot’s kinematic chain — 1 if true, 0 if not. These two costs are weighed equally. These terms are regularized to be close to 1.01.01.0 and stay non-negative.

其中 Δij\Delta_{ij}Δij​ 是关节 iii 与 jjj 的位置差。mijm_{ij}mij​ 表示这两个关节在机器人运动学链中是否为直接邻居——是则为 1,否则为 0。这两项代价等权。这些项被正则到接近 1.01.01.0,并保持非负。

We also ground the robot with the environment through a set of contact costs (e.g., foot contact point matching, foot skating penalty) and collision costs (self- and world-collision avoidance). The robot is approximated as a set of capsules, and the world as a heightmap. The learning-based feet contact estimation [49] is used to get contact signals.

我们还通过一组接触代价(例如脚部接触点匹配、脚滑惩罚)和碰撞代价(自身碰撞与世界碰撞规避),把机器人与环境接地。机器人被近似为一组胶囊体,世界则表示为高度图。脚部接触信号来自基于学习的脚部接触估计[49]。

The skating cost is defined as:

滑步代价定义为:

Lskatingt=∑foot∈{L,R}∥pfoott−pfoott−1∥+∥panklet−panklet−1∥,(9)L_{\text{skating}}^{t}=\sum_{\text{foot}\in\{\text{L},\text{R}\}}\left\|p_{\text{foot}}^{t}-p_{\text{foot}}^{t-1}\right\|+\left\|p_{\text{ankle}}^{t}-p_{\text{ankle}}^{t-1}\right\|, \tag{9}Lskatingt​=foot∈{L,R}∑​​pfoott​−pfoott−1​​+​panklet​−panklet−1​​,(9)

where pfoott=Troot,world(ϕRt,γRt)Tfoot,root(qt)p_{\text{foot}}^{t}=\mathbf{T}_{\text{root},\text{world}}(\phi_{R}^{t},\gamma_{R}^{t})\mathbf{T}_{\text{foot},\text{root}}(q^{t})pfoott​=Troot,world​(ϕRt​,γRt​)Tfoot,root​(qt) is the position of the foot link in the world at time ttt.

其中 pfoott=Troot,world(ϕRt,γRt)Tfoot,root(qt)p_{\text{foot}}^{t}=\mathbf{T}_{\text{root},\text{world}}(\phi_{R}^{t},\gamma_{R}^{t})\mathbf{T}_{\text{foot},\text{root}}(q^{t})pfoott​=Troot,world​(ϕRt​,γRt​)Tfoot,root​(qt) 是 ttt 时刻脚部连杆在世界中的位置。

We also include common robot costs (e.g., joint limits, temporal smoothness of root pose and robot joints) and a small regularization cost on the G1 robot’s knee yaw joint for stable leg poses. Processing a 300-frame clip takes around 10 seconds on a single A100, using the PyRoki library [68].

我们还包括常见的机器人代价(例如关节限位,以及根位姿与机器人关节的时间平滑),并对 G1 机器人的膝关节偏航加一项很小的正则,以保持腿部姿态稳定。在单块 A100 上,用 PyRoki 库[68]处理一段 300 帧片段大约需要 10 秒。

A.4 Evaluation Details

A.4 评估细节

For quantitative evaluation of our real-to-sim pipeline, we conduct experiments on the SLOPER4D dataset [59]. Following established protocols [39, 60, 47], we assess human trajectory reconstruction using World-frame Mean Per Joint Position Error (W-MPJPE) and World-frame Aligned MPJPE (WA-MPJPE), and evaluate scene geometry reconstruction using the Chamfer Distance. Specifically, for human trajectory evaluation, we first slice each sequence into 100-frame segments. W-MPJPE is then computed by aligning only the first two frames to the ground truth, emphasizing global consistency, while WA-MPJPE aligns the entire segment to evaluate local trajectory accuracy. For geometry evaluation, we compute the Chamfer Distance (in meters) between the predicted pointcloud and LiDAR points within the RGB camera’s field of view. For benchmarking, we use a subset of SLOPER‑4D that contains only those sequences in which SAM2 tracking—human detection plus cross‑frame association—succeeds. This yields two sequences each of running, walking, and stair ascent/descent. Results for WHAM and TRAM are reproduced with their official code.

为了对 real-to-sim 流水线做定量评估,我们在 SLOPER4D 数据集[59]上做实验。按照已有协议[39, 60, 47],人体轨迹重建用世界坐标系下的平均每关节位置误差(W-MPJPE)和世界坐标系对齐后的 MPJPE(WA-MPJPE)来衡量,场景几何重建则用 Chamfer 距离来衡量。具体而言,评估人体轨迹时,我们先把每条序列切成 100 帧的片段。W-MPJPE 只把前两帧与真值对齐,以强调全局一致性;WA-MPJPE 则对齐整个片段,以评估局部轨迹精度。几何评估方面,我们计算预测点云与 RGB 相机视野内 LiDAR 点之间的 Chamfer 距离(以米计)。做基准比较时,我们使用 SLOPER‑4D 的一个子集,其中只保留 SAM2 跟踪——即人体检测加跨帧关联——获得成功的序列。这样得到跑步、行走和上下楼梯各两条序列。WHAM 和 TRAM 的结果用其官方代码复现。

VideoMimic 重建场景的自我视角 RGBD 渲染

Figure 8: Ego-view RGBD-rendering. While our policy does not use RGB conditioning for now, we demonstrate the potential of our reconstruction for ego-view rendering. We rasterize the metrically‑scaled point cloud into color and depth images by projecting every 3D point onto the image plane of a virtual camera pitched to match the G1’s head‑mounted sensor. This first‑person rendering gives the robot realistic observations of its surroundings, unlocking future work on active vision and semantic understanding, and can be injected directly into the simulator during RL training to couple perception and control.

图 8:自我视角的 RGBD 渲染。 尽管当前策略暂不以 RGB 为条件,我们仍展示了该重建用于自我视角渲染的潜力。我们把度量缩放后的点云光栅化成彩色图和深度图,方法是把每个三维点投影到一台虚拟相机的像平面上,其俯仰与 G1 头部传感器一致。这种第一人称渲染为机器人提供了周围环境的真实观察,为后续的主动视觉和语义理解打开了空间,也可以在强化学习训练期间直接注入仿真器,从而把感知与控制耦合起来。

A.5 Ego-view Rendering

A.5 自我视角渲染

We leverage our metrically‑scaled 4D reconstruction to render ego‑view RGB‑D frames by projecting every 3D point into a virtual camera at the Unitree G1 head sensor. This first‑person perspective is not consumed by the policy in the present work, yet it demonstrates the pipeline’s future potential. Our reconstructions can supply realistic visual observations to future perception‑conditioned policies, enabling active vision and semantic scene understanding directly from the robot’s viewpoint. A key limitation, however, is that monocular capture leaves many surfaces unobserved; when rasterized, these occluded regions manifest as holes or gaps in the rendered images. Bridging these gaps is an exciting direction for follow‑up work: modern data-driven novel‑view‑synthesis models can hallucinate plausible geometry and appearance for invisible surfaces, promising photorealistic egocentric renderings that would further narrow the sim‑to‑real perceptual gap for visuomotor policies.

我们利用度量缩放后的四维重建,把每个三维点投影到位于 Unitree G1 头部传感器处的虚拟相机中,从而渲染出自我视角的 RGB-D 帧。这一第一人称视角在本文中并未被策略使用,但展示了流水线面向未来的潜力。我们的重建可以为后续以感知为条件的策略提供真实的视觉观察,使机器人能直接从自身视角做主动视觉和场景语义理解。一个关键局限在于:单目采集会留下许多未被观察到的表面;光栅化时,这些被遮挡的区域会在渲染图像中表现为空洞或缺口。填补这些缺口是后续工作的一个有吸引力的方向:现代数据驱动的新视角合成模型,可以为不可见表面补全出合理的几何和外观,有望得到逼真的第一人称渲染,从而进一步缩小视觉运动策略的仿真到真实感知差距。

Appendix B Policy Learning

附录 B 策略学习

B.1 RL Setup and Training

B.1 强化学习设置与训练

We use the PPO algorithm [53]. We use a discount factor of γ=0.99\gamma=0.99γ=0.99, and a GAE parameter of λ=0.95\lambda=0.95λ=0.95. We use an adaptive learning rate with a desired KL of 0.020.020.02. For adaptive learning rate, we use 1.21.21.2 as opposed to the usual 1.51.51.5 learning rate change, as we find that this leads to faster training. When doing RL from scratch, we set the learning rate (which is modified from this value by adaptive KL) at the start to 1e−31e-31e−3 however when finetuning we find it is important to start from a very low learning rate of 2e−52e-52e−5 to prevent early updates with a high learning rate from destroying the existing checkpoint when the distribution of data is biased early during an episode when all environments start. We use a max gradient norm of 1.0. 5 learning epochs are used per rollout, and our rollout length is 24. Our entropy coefficient is set to 0.00250.00250.0025. We use 2 x NVIDIA 4090 GPUs for training, each with 4096 environments (so an effective total of 8192 environments). We use MLPs with 4 layers of [1024, 512, 256, 128] dimensions for different layers.

我们使用 PPO 算法[53]。折扣因子为 γ=0.99\gamma=0.99γ=0.99,GAE 参数为 λ=0.95\lambda=0.95λ=0.95。学习率采用自适应方案,目标 KL 为 0.020.020.02。自适应学习率的变化倍数我们取 1.21.21.2,而非常用的 1.51.51.5,因为我们发现这样训练更快。从零做强化学习时,学习率(随后仍会按自适应 KL 修改)起始设为 1e−31e-31e−3;但在微调时,我们发现必须从非常低的学习率 2e−52e-52e−5 起步,以免在回合早期、所有环境同时起步、数据分布尚有偏时,用过高学习率做出的更新破坏已有检查点。梯度范数上限为 1.0。每次 rollout 做 5 个学习 epoch,rollout 长度为 24。熵系数为 0.00250.00250.0025。训练使用 2 块 NVIDIA 4090 GPU,每块跑 4096 个环境(有效总计 8192 个环境)。我们使用 4 层、各层维度为 [1024, 512, 256, 128] 的 MLP。

B.2 Observations

B.2 观察

Table 3 details the actor and critic observations. We note that, unlike works in graphics [1, 3], the actor gets all unprivileged observations that are all available on the real robot (see Sec. 5). Furthermore, the target root reference during train time is fed into the global frame to ensure it is aligned with the terrain. Thus, unlike [5, 8], we are able to train policies using a global root reference, allowing our policies to correctly imitate references in the world frame over long horizons, which is critical for correctly imitating motions tied to a terrain.

表 3 给出了 actor 与 critic 的观察。需要指出的是,与图形学中的工作[1, 3]不同,actor 得到的都是非特权观察,并且在真实机器人上全部可以拿到(见第 5 节)。此外,训练时的目标根参考被送入全局坐标系,以确保它与地形对齐。因此,与[5, 8]不同,我们能够用全局根参考来训练策略,使策略在长时域上正确模仿世界坐标系中的参考,这对正确模仿与地形绑定的动作至关重要。

Fast-lio2 [63] and probabilistic terrain mapping [64, 17] are used to derive heightmaps and a live odometry signal in real. This can be used to follow 2D commands in the global frame as we did in testing (see Figure 9) and as is done in a concurrent work [69] for the purpose of teleoperation. However, we note that we do not do this in the final version of the paper to avoid any hysteresis in the position commands, and instead multiply the joystick value by a fixed amount to obtain a position which is interpreted by the policy as an offset to the target position in the local frame.

在真实环境中,Fast-lio2[63] 与概率地形建图[64, 17]用于得到高度图和实时里程计信号。这可以用来在全局坐标系中跟随二维指令,正如我们在测试中所做的(见图 9),以及一项并发工作[69]出于遥操作目的所做的那样。不过需要说明:为了避免位置指令中的滞后,论文最终版本并不采用这种方式,而是把手柄数值乘以一个固定量,得到一个被策略解释为局部坐标系中目标位置偏移的量。

Input Dim. Actor (MPT) Actor (tracking) Actor (distill/ft.) Critic
Base angular velocity (last 5) 5×3=155\times 3=155×3=15D ✓ ✓ ✓ ✓
Base linear velocity (last 5) 5×3=155\times 3=155×3=15D ×\times× ×\times× ×\times× ✓
Projected gravity (last 5) 5×3=155\times 3=155×3=15D ✓ ✓ ✓ ✓
DoF positions (last 5) 5×ndof5\times n_{\mathrm{dof}}5×ndof​ ✓ ✓ ✓ ✓
DoF velocities (last 5) 5×ndof5\times n_{\mathrm{dof}}5×ndof​ ✓ ✓ ✓ ✓
Actions (last 5) 5×nact5\times n_{\mathrm{act}}5×nact​ ✓ ✓ ✓ ✓
Local frame pos. to target (last 5) 5×2=105\times 2=105×2=10D ✓ ✓ ✓ ✓
Local frame yaw to target (last 5) 5×1=55\times 1=55×1=5D ✓ ✓ ✓ ✓
Target joint angles ndofn_{\mathrm{dof}}ndof​ ✓ ✓ ×\times× ✓
Target root roll 1D ✓ ✓ ×\times× ✓
Target root pitch 1D ✓ ✓ ×\times× ✓
Terrain height (height-map) 121D ×\times× ✓ ✓ ✓
Root height 1D ×\times× ×\times× ×\times× ✓
Link heights nlinksn_{\mathrm{links}}nlinks​ ×\times× ×\times× ×\times× ✓
Root orientation (quat) 4D ×\times× ×\times× ×\times× ✓
Root position 3D ×\times× ×\times× ×\times× ✓
Joint positions ndofn_{\mathrm{dof}}ndof​ ×\times× ×\times× ×\times× ✓
Link positions 3 nlinks3\,n_{\mathrm{links}}3nlinks​ ×\times× ×\times× ×\times× ✓
Link velocities 3 nlinks3\,n_{\mathrm{links}}3nlinks​ ×\times× ×\times× ×\times× ✓
Feet contact flags nfeetn_{\mathrm{feet}}nfeet​ ×\times× ×\times× ×\times× ✓
Feet target contact flags nfeetn_{\mathrm{feet}}nfeet​ ×\times× ×\times× ×\times× ✓

Table 3: Observations for each network: Motion Capture Pretraining (MPT) actor, motion-tracking actor, distillation/finetuning actor, and critic. Here ndof=23n_{\mathrm{dof}}=23ndof​=23 for the Unitree G1.

输入 维数 Actor(MPT) Actor(跟踪) Actor(蒸馏/微调) Critic
基座角速度(最近 5 步) 5×3=155\times 3=155×3=15D ✓ ✓ ✓ ✓
基座线速度(最近 5 步) 5×3=155\times 3=155×3=15D ×\times× ×\times× ×\times× ✓
投影重力(最近 5 步) 5×3=155\times 3=155×3=15D ✓ ✓ ✓ ✓
关节位置(最近 5 步) 5×ndof5\times n_{\mathrm{dof}}5×ndof​ ✓ ✓ ✓ ✓
关节速度(最近 5 步) 5×ndof5\times n_{\mathrm{dof}}5×ndof​ ✓ ✓ ✓ ✓
动作(最近 5 步) 5×nact5\times n_{\mathrm{act}}5×nact​ ✓ ✓ ✓ ✓
局部坐标系到目标的位置(最近 5 步) 5×2=105\times 2=105×2=10D ✓ ✓ ✓ ✓
局部坐标系到目标的偏航(最近 5 步) 5×1=55\times 1=55×1=5D ✓ ✓ ✓ ✓
目标关节角 ndofn_{\mathrm{dof}}ndof​ ✓ ✓ ×\times× ✓
目标根滚转 1D ✓ ✓ ×\times× ✓
目标根俯仰 1D ✓ ✓ ×\times× ✓
地形高度(高度图) 121D ×\times× ✓ ✓ ✓
根高度 1D ×\times× ×\times× ×\times× ✓
连杆高度 nlinksn_{\mathrm{links}}nlinks​ ×\times× ×\times× ×\times× ✓
根朝向(四元数) 4D ×\times× ×\times× ×\times× ✓
根位置 3D ×\times× ×\times× ×\times× ✓
关节位置 ndofn_{\mathrm{dof}}ndof​ ×\times× ×\times× ×\times× ✓
连杆位置 3 nlinks3\,n_{\mathrm{links}}3nlinks​ ×\times× ×\times× ×\times× ✓
连杆速度 3 nlinks3\,n_{\mathrm{links}}3nlinks​ ×\times× ×\times× ×\times× ✓
脚部接触标志 nfeetn_{\mathrm{feet}}nfeet​ ×\times× ×\times× ×\times× ✓
脚部目标接触标志 nfeetn_{\mathrm{feet}}nfeet​ ×\times× ×\times× ×\times× ✓

表 3:各网络的观察:动作捕捉预训练(MPT)actor、动作跟踪 actor、蒸馏/微调 actor,以及 critic。此处对 Unitree G1 有 ndof=23n_{\mathrm{dof}}=23ndof​=23。

B.3 Actions

B.3 动作

We use unbounded actions without activations on the actor output following [54], however following [70] we clip the actions to our chosen range prevent the policy from learning bang-bang style control earlier in training and add a ”bounds loss” to the actor with the following formulation to enforce this limit in the actor output:

我们沿用[54],对 actor 输出不使用激活函数,动作为无界;同时又沿用[70],把动作裁剪到选定范围,以免策略在训练早期学会 Bang-Bang 式控制,并向 actor 加入如下形式的「边界损失」,以在 actor 输出上执行这一限制:

μ~j=clip⁡(μj, −ϵ, ϵ),Lbounds=1D∑j=1D∣μ~j−μj∣,(10)\displaystyle\tilde{\mu}_{j} \\ \displaystyle=\operatorname{clip}\bigl(\mu_{j},\,-\epsilon,\,\epsilon\bigr), \\ \displaystyle\mathcal{L}_{\mathrm{bounds}} \\ \displaystyle=\frac{1}{D}\sum_{j=1}^{D}\bigl|\tilde{\mu}_{j}-\mu_{j}\bigr|, \tag{10}μ~​j​=clip(μj​,−ϵ,ϵ),Lbounds​=D1​j=1∑D​​μ~​j​−μj​​,(10)

where the μj\mu_{j}μj​ are the policy actions per-dimension. We use a coefficient of 0.00050.00050.0005 for this loss. For the G1 humanoid, we limit the action magnitude to 8.0. During experiments, our initial action standard deviation is set to 0.8.

其中 μj\mu_{j}μj​ 是策略在各维上的动作。该项损失的系数为 0.00050.00050.0005。对 G1 人形机器人,我们把动作幅度限制为 8.0。实验中,初始动作标准差设为 0.8。

B.4 Simulation Parameters

B.4 仿真参数

We run Isaac Gym [12] at 200Hz with control decimation 4, giving an effective policy Δt\Delta tΔt of 0.02s0.02s0.02s.

我们以 200Hz 运行 Isaac Gym[12],控制降采样为 4,因而策略的有效 Δt\Delta tΔt 为 0.02s0.02s0.02s。

We set the maximum depenetration velocity especially low – to 0.1m/sec – to prevent large velocities being applied in the occasional case of interpenetration with terrain during the reference motion.

我们把最大去穿透速度设得特别低——为 0.1m/sec——以免在参考动作偶尔与地形发生相互穿透时,被施加过大的速度。

B.4.1 Dealing with Terrains in the Simulator

Due to an undocumented bug in IsaacGym, the simulator registers collisions between robots in different virtual and theoretically isolated “environments” but which occupy the same world-frame spatial volume. This leads to a dramatic increase in memory usage and a significant drop in simulator throughput. To mitigate this, when training on a single clip or a few clips, we duplicate the task terrain and spatially distribute robot spawns to reduce inter-robot collisions.

B.4.1 仿真器中的地形处理

由于 IsaacGym 中一个未写入文档的缺陷,仿真器会在不同虚拟、理论上相互隔离、却占用相同世界坐标空间的「环境」之间,把机器人之间的碰撞也登记下来。这会急剧增加显存占用,并显著降低仿真吞吐。为缓解这一问题,在只训练一条或少数几条片段时,我们复制任务地形,并把机器人的出生点在空间上分散开,以减少机器人之间的碰撞。

B.5 Reward Formulation

B.5 奖励形式

Our concern above all is to have a system that can take in data and produce new physical motions; hence, our reward is made up mostly of data-driven terms --- tracking link and joint positions and velocities, as well as matching feet contact, with minimal re-weighting. In reward design, We have two objectives: firstly, to minimize the strength of manually-designed priors injected into RL tracking via reward engineering.[^1] Secondly, to produce physically feasible motions. Note that the two objectives are in tension as since the kinematic data comes from humans, perfect tracking would result in nonphysical motions. We also have several penalty criteria, namely action rate penalties, and two penalties to discourage exploiting simulator physics. Table 4 details the reward terms used for policy learning.

[^1]: The more such human priors that are added, the less general our motion tracking system will be. This is the inverse of classical sim-to-real RL pipelines, which largely rely on a practitioner’s “reward hacking” to get desired behaviours.

我们首先关心的,是建立一个吃进数据、就能产生新的物理动作的系统;因此,奖励主要由数据驱动项构成——跟踪连杆和关节的位置与速度,以及匹配脚部接触——并尽量少做重新加权。在奖励设计上,我们有两个目标:第一,尽量削弱通过奖励工程注入强化学习跟踪的手工先验。[^1] 第二,得到物理上可行的动作。需要指出,这两个目标彼此冲突:运动学数据来自人体,完美跟踪会得到非物理的动作。我们还有若干惩罚项,即动作变化率惩罚,以及两项用于抑制钻仿真物理空子的惩罚。表 4 给出了策略学习所用的奖励项。

[^1]:加入的这类人体先验越多,动作跟踪系统的通用性就越弱。这与经典的仿真到真实强化学习流水线正好相反,后者在很大程度上依赖实践者的「reward hacking」来得到想要的行为。

Reward Formula Weight kkk Justification
Penalties / Regularization
Action‐Rate Penalty ∥at−at−1∥2\lVert\mathbf{a}_{t}-\mathbf{a}_{t-1}\rVert^{2}∥at​−at−1​∥2 −8.0-8.0−8.0 – Can’t move too quickly on real robot
Ankle Action Penalty ∥at∥2\lVert\mathbf{a}_{t}\rVert^{2}∥at​∥2 −4.0-4.0−4.0 – Ankles should stay near neutral
DOF-Pos Limits Penalty ∑[overflow w.r.t.  0.98 range]\displaystyle\sum[\text{overflow w.r.t.\;0.98\,range}]∑[overflow w.r.t.0.98range] −50.0-50.0−50.0 – Penalize joints outside 98 % of their range
Collision Penalty ∑1 ⁣[∥fc∥>0.1]\displaystyle\sum 1\!\bigl[\lVert\mathbf{f}_{c}\rVert>0.1\bigr]∑1[∥fc​∥>0.1] −1.0-1.0−1.0 – Discourage collisions on penalized links
Contact–No–Velocity Pen. ∥vfeet∥(c>1)\lVert\mathbf{v}_{\text{feet}}\rVert_{(c>1)}∥vfeet​∥(c>1)​ −100-100−100 – Penalize stationary foot contact
Tracking Rewards
Joint-Pos Tracking exp⁡ ⁣(−k ∥q−q∗∥2)\exp\!\bigl(-k\,\lVert\mathbf{q}-\mathbf{q}^{\ast}\rVert^{2}\bigr)exp(−k∥q−q∗∥2) 120120120 2.02.02.0 Follow reference joint angles
Joint-Vel Tracking exp⁡ ⁣(−k ∥q˙−q˙∗∥2)\exp\!\bigl(-k\,\lVert\dot{\mathbf{q}}-\dot{\mathbf{q}}^{\ast}\rVert^{2}\bigr)exp(−k∥q˙​−q˙​∗∥2) 242424 0.010.010.01 Follow reference joint velocities
Root Ori Tracking exp⁡ ⁣(−k θroot)\exp\!\bigl(-k\,\theta_{\text{root}}\bigr)exp(−kθroot​) 151515 3.03.03.0 Keep base orientation on track
Torso Pos Tracking exp⁡ ⁣(−k ∥ptorso−p∗∥2)\exp\!\bigl(-k\,\lVert\mathbf{p}_{\text{torso}}-\mathbf{p}^{\ast}\rVert^{2}\bigr)exp(−k∥ptorso​−p∗∥2) 151515 50.050.050.0 Precise torso translation
Torso Ori Tracking exp⁡ ⁣(−k θtorso)\exp\!\bigl(-k\,\theta_{\text{torso}}\bigr)exp(−kθtorso​) 151515 3.03.03.0 Precise torso attitude
Link Pos Tracking exp⁡ ⁣(−k ∥plinks−p∗∥2)\exp\!\bigl(-k\,\lVert\mathbf{p}_{\text{links}}-\mathbf{p}^{\ast}\rVert^{2}\bigr)exp(−k∥plinks​−p∗∥2) 303030 5.05.05.0 Track 13 key link positions
Link Vel Tracking exp⁡ ⁣(−k ∥p˙links−p˙∗∥2)\exp\!\bigl(-k\,\lVert\dot{\mathbf{p}}_{\text{links}}-\dot{\mathbf{p}}^{\ast}\rVert^{2}\bigr)exp(−k∥p˙​links​−p˙​∗∥2) 555 0.10.10.1 Track link velocities
Feet Contact Match ∑\mathbbm1 ⁣[ct(i)=ct∗(i)]\sum\mathbbm{1}\!\bigl[c_{t}^{(i)}=c_{t}^{\ast(i)}\bigr]∑\mathbbm1[ct(i)​=ct∗(i)​] 111 – Match contacts from reference motion (BSTRO)
Feet Air-Time Bonus ∑i(tair(i)−k) \mathbbm1first_contact(i)\displaystyle\sum_{i}(t^{(i)}_{\text{air}}-k)\,\mathbbm{1}_{\text{first\_contact}}^{(i)}i∑​(tair(i)​−k)\mathbbm1first_contact(i)​ 200020002000 0.250.250.25 Reward long, ballistic steps
Reset Condition Weight kkk
Termination Penalty episode end (failure) −500-500−500 – Harsh penalty on failure events
Alive Reward added every step 300300300 – Incentive to stay alive

Table 4: Reward terms, corresponding weights, and scaling factors kkk. These rewards remain the same across training phases. However, we anneal the action rate and ankle action penalty from 0.2 and 0.0, respectively, to the given values in the table.

奖励 公式 权重 kkk 说明
惩罚 / 正则
动作变化率惩罚 ∥at−at−1∥2\lVert\mathbf{a}_{t}-\mathbf{a}_{t-1}\rVert^{2}∥at​−at−1​∥2 −8.0-8.0−8.0 – 真实机器人上不能动得太快
踝关节动作惩罚 ∥at∥2\lVert\mathbf{a}_{t}\rVert^{2}∥at​∥2 −4.0-4.0−4.0 – 踝关节应保持在中立附近
关节位置限位惩罚 ∑[overflow w.r.t.  0.98 range]\displaystyle\sum[\text{overflow w.r.t.\;0.98\,range}]∑[overflow w.r.t.0.98range] −50.0-50.0−50.0 – 惩罚超出自身范围 98% 的关节
碰撞惩罚 ∑1 ⁣[∥fc∥>0.1]\displaystyle\sum 1\!\bigl[\lVert\mathbf{f}_{c}\rVert>0.1\bigr]∑1[∥fc​∥>0.1] −1.0-1.0−1.0 – 抑制受罚连杆上的碰撞
接触–无速度惩罚 ∥vfeet∥(c>1)\lVert\mathbf{v}_{\text{feet}}\rVert_{(c>1)}∥vfeet​∥(c>1)​ −100-100−100 – 惩罚接触期间的脚部速度
跟踪奖励
关节位置跟踪 exp⁡ ⁣(−k ∥q−q∗∥2)\exp\!\bigl(-k\,\lVert\mathbf{q}-\mathbf{q}^{\ast}\rVert^{2}\bigr)exp(−k∥q−q∗∥2) 120120120 2.02.02.0 跟随参考关节角
关节速度跟踪 exp⁡ ⁣(−k ∥q˙−q˙∗∥2)\exp\!\bigl(-k\,\lVert\dot{\mathbf{q}}-\dot{\mathbf{q}}^{\ast}\rVert^{2}\bigr)exp(−k∥q˙​−q˙​∗∥2) 242424 0.010.010.01 跟随参考关节速度
根朝向跟踪 exp⁡ ⁣(−k θroot)\exp\!\bigl(-k\,\theta_{\text{root}}\bigr)exp(−kθroot​) 151515 3.03.03.0 使基座朝向保持在轨迹上
躯干位置跟踪 exp⁡ ⁣(−k ∥ptorso−p∗∥2)\exp\!\bigl(-k\,\lVert\mathbf{p}_{\text{torso}}-\mathbf{p}^{\ast}\rVert^{2}\bigr)exp(−k∥ptorso​−p∗∥2) 151515 50.050.050.0 精确的躯干平移
躯干朝向跟踪 exp⁡ ⁣(−k θtorso)\exp\!\bigl(-k\,\theta_{\text{torso}}\bigr)exp(−kθtorso​) 151515 3.03.03.0 精确的躯干姿态
连杆位置跟踪 exp⁡ ⁣(−k ∥plinks−p∗∥2)\exp\!\bigl(-k\,\lVert\mathbf{p}_{\text{links}}-\mathbf{p}^{\ast}\rVert^{2}\bigr)exp(−k∥plinks​−p∗∥2) 303030 5.05.05.0 跟踪 13 个关键连杆位置
连杆速度跟踪 exp⁡ ⁣(−k ∥p˙links−p˙∗∥2)\exp\!\bigl(-k\,\lVert\dot{\mathbf{p}}_{\text{links}}-\dot{\mathbf{p}}^{\ast}\rVert^{2}\bigr)exp(−k∥p˙​links​−p˙​∗∥2) 555 0.10.10.1 跟踪连杆速度
脚部接触匹配 ∑\mathbbm1 ⁣[ct(i)=ct∗(i)]\sum\mathbbm{1}\!\bigl[c_{t}^{(i)}=c_{t}^{\ast(i)}\bigr]∑\mathbbm1[ct(i)​=ct∗(i)​] 111 – 匹配参考动作中的接触(BSTRO)
脚部腾空时间奖励 ∑i(tair(i)−k) \mathbbm1first_contact(i)\displaystyle\sum_{i}(t^{(i)}_{\text{air}}-k)\,\mathbbm{1}_{\text{first\_contact}}^{(i)}i∑​(tair(i)​−k)\mathbbm1first_contact(i)​ 200020002000 0.250.250.25 奖励长而有腾空的一步
重置 条件 权重 kkk
终止惩罚 回合结束(失败) −500-500−500 – 对失败事件施以严厉惩罚
存活奖励 每一步都加 300300300 – 激励保持存活

表 4:奖励项、对应权重以及缩放因子 kkk。这些奖励在各训练阶段保持不变。不过,动作变化率和踝关节动作惩罚会分别从 0.2 和 0.0 退火到表中给出的数值。

B.6 Termination Criteria

B.6 终止准则

Our termination criteria depend on the maximum error of the tracked link joints in the robot. The episode terminates if the Cartesian error exceeds the termination threshold during any step in training. During MPT, we set this to 0.3. When doing the tracking stage over terrains, we set this to 0.5 (higher is better on sometimes noisy references from vision). During RL finetuning, we set this to 1.2. The reason for the high threshold in finetuning is that during RL, we care a lot about ensuring both recovery behaviours and diversity which are not seen during normal DeepMimic-style training on a dataset of the size we are using (and indeed, with too strict a tolerance recovery behaviours will not be seen at all). Hence, a loose tolerance on termination while still using the same tracking reward helps to maintain the essence of the motions in the data while still achieving strong recovery behaviours.

我们的终止准则取决于机器人上被跟踪连杆关节的最大误差。训练中只要某一步的笛卡尔误差超过终止阈值,回合就会终止。MPT 阶段将阈值设为 0.3。在地形上做跟踪时设为 0.5(面对有时来自视觉、带有噪声的参考,阈值更高更合适)。强化学习微调时设为 1.2。微调阶段阈值较高,是因为在强化学习中,我们非常看重恢复行为和多样性,而它们在我们所用规模的数据集上、按常规 DeepMimic 式训练时并不会出现(事实上,容差过严时,恢复行为根本不会被看到)。因此,在仍使用同一套跟踪奖励的同时放宽终止容差,有助于既保持数据中动作的要旨,又能获得较强的恢复行为。

B.7 Domain Randomization

B.7 域随机化

We add various domain randomizations in the simulation to simulate the impacts of various unmodeled physical effects in the simulator, which leads to more robust policies. These are detailed in Table 5.

我们在仿真中加入多种域随机化,以模拟仿真器中各类未建模物理效应的影响,从而使策略更加稳健。详情见表 5。

Category Parameter Value / Range Comment
Dynamics randomization
DoF friction μDoF\mu_{\text{DoF}}μDoF​ U[0, 0.02]\mathcal{U}[0,\,0.02]U[0,0.02] Different per environment.
Random pushes (xy) Δvpush,xy\Delta v_{\text{push},xy}Δvpush,xy​ U[−0.25, 0.25]\mathcal{U}[-0.25,\,0.25]U[−0.25,0.25] m/s Additive to the robot’s reference state or current velocity.
Random pushes (z) Δvpush,z\Delta v_{\text{push},z}Δvpush,z​ U[−0.10, 0.10]\mathcal{U}[-0.10,\,0.10]U[−0.10,0.10] m/s Additive to the robot’s reference state or current velocity.
Random push interval TpushT_{\text{push}}Tpush​ 101010 s Interval between random pushes.
Observation offset noise (additive bias, re-sampled from gaussian per episode with given standard deviation)
Gravity bias Δg\Delta gΔg 0.010.010.01g Fixed bias added to gravity vector for an episode in policy input.
DoF-position bias Δq\Delta qΔq 0.0050.0050.005rad Fixed bias added to each joint position in policy input.
Observation white noise (re-sampled every step from Gaussian with divergent standard deviation)
Relative odom (x,yx,yx,y) to target σxy\sigma_{xy}σxy​ 0.010.010.01 m Resampled every step.
Relative odom yaw to target σψ\sigma_{\psi}σψ​ 0.010.010.01 rad
DoF positions σq\sigma_{q}σq​ 0.010.010.01 rad
DoF velocities σq˙\sigma_{\dot{q}}σq˙​​ 1.51.51.5 rad/s
Linear base velocity σv\sigma_{v}σv​ 0.10.10.1 m/s
Angular base velocity σω\sigma_{\omega}σω​ 0.20.20.2 rad/s
Gravity (per-axis) σg\sigma_{g}σg​ 0.050.050.05 g
Odom update-rate randomization
Update frequency (steps) nodomn_{\text{odom}}nodom​ U{2,6}\mathcal{U}\{2,6\}U{2,6} Odometry observation is held constant for nnn env-steps to mimic drop-outs.
Height-map sensing noise
White noise (cells) σh\sigma_{h}σh​ 0.020.020.02 m Gaussian sampled per-cell and per-step.
Offset noise (cells) Δh\Delta_{h}Δh​ 0.020.020.02 m Height-bias (applied to all cells, Gaussian sampled per episode).
Roll / Pitch noise σϕ,σθ\sigma_{\phi},\sigma_{\theta}σϕ​,σθ​ 0.040.040.04 rad Projection frame tilt, Gaussian per-env.
Yaw noise σψ\sigma_{\psi}σψ​ 0.080.080.08 rad Map frame rotation, Gaussian per-env.
Sensor delay dmax⁡d_{\max}dmax​ U{0,3}\mathcal{U}\{0,3\}U{0,3} Frames of latency before height-map update.
Update frequency nmapn_{\text{map}}nmap​ U{1,5}\mathcal{U}\{1,5\}U{1,5} Heightmap refresh period (steps).
Bad-distance probability pbadp_{\text{bad}}pbad​ 0.010.010.01 Chance a cell is replaced by a random value.

Table 5: Domain-randomization and noise settings used during training. “Offset” terms are fixed per episode; “white” terms are resampled each simulation step. Update-frequency variables simulate low-rate sensors by holding observations constant for a random number of steps.

类别 参数 取值 / 范围 说明
动力学随机化
关节摩擦 μDoF\mu_{\text{DoF}}μDoF​ U[0, 0.02]\mathcal{U}[0,\,0.02]U[0,0.02] 每个环境各自不同。
随机推挤(xy) Δvpush,xy\Delta v_{\text{push},xy}Δvpush,xy​ U[−0.25, 0.25]\mathcal{U}[-0.25,\,0.25]U[−0.25,0.25] m/s 叠加到机器人的参考状态或当前速度上。
随机推挤(z) Δvpush,z\Delta v_{\text{push},z}Δvpush,z​ U[−0.10, 0.10]\mathcal{U}[-0.10,\,0.10]U[−0.10,0.10] m/s 叠加到机器人的参考状态或当前速度上。
随机推挤间隔 TpushT_{\text{push}}Tpush​ 101010 s 两次随机推挤之间的间隔。
观察 偏移 噪声(加性偏置,每个回合按给定标准差从高斯分布重新采样)
重力偏置 Δg\Delta gΔg 0.010.010.01g 在一个回合内,作为固定偏置加到策略输入中的重力向量上。
关节位置偏置 Δq\Delta qΔq 0.0050.0050.005rad 在一个回合内,作为固定偏置加到策略输入中的各关节位置上。
观察 白 噪声(每一步按发散的标准差从高斯分布重新采样)
相对里程计(x,yx,yx,y)到目标 σxy\sigma_{xy}σxy​ 0.010.010.01 m 每一步重新采样。
相对里程计偏航到目标 σψ\sigma_{\psi}σψ​ 0.010.010.01 rad
关节位置 σq\sigma_{q}σq​ 0.010.010.01 rad
关节速度 σq˙\sigma_{\dot{q}}σq˙​​ 1.51.51.5 rad/s
基座线速度 σv\sigma_{v}σv​ 0.10.10.1 m/s
基座角速度 σω\sigma_{\omega}σω​ 0.20.20.2 rad/s
重力(按轴) σg\sigma_{g}σg​ 0.050.050.05 g
里程计更新率随机化
更新频率(步数) nodomn_{\text{odom}}nodom​ U{2,6}\mathcal{U}\{2,6\}U{2,6} 里程计观察连续 nnn 个环境步保持不变,以模拟丢包。
高度图传感噪声
白噪声(格子) σh\sigma_{h}σh​ 0.020.020.02 m 按格子、按步从高斯分布采样。
偏移噪声(格子) Δh\Delta_{h}Δh​ 0.020.020.02 m 高度偏置(作用于所有格子,每个回合从高斯分布采样)。
滚转 / 俯仰噪声 σϕ,σθ\sigma_{\phi},\sigma_{\theta}σϕ​,σθ​ 0.040.040.04 rad 投影坐标系倾斜,按环境从高斯分布采样。
偏航噪声 σψ\sigma_{\psi}σψ​ 0.080.080.08 rad 地图坐标系旋转,按环境从高斯分布采样。
传感器延迟 dmax⁡d_{\max}dmax​ U{0,3}\mathcal{U}\{0,3\}U{0,3} 高度图更新前的延迟帧数。
更新频率 nmapn_{\text{map}}nmap​ U{1,5}\mathcal{U}\{1,5\}U{1,5} 高度图刷新周期(步数)。
坏距离概率 pbadp_{\text{bad}}pbad​ 0.010.010.01 某个格子被替换为随机值的概率。

表 5:训练中使用的域随机化与噪声设置。 「偏移」项在每个回合内固定;「白」项在每个仿真步重新采样。更新频率变量通过把观察随机保持若干步不变,来模拟低速率传感器。

B.8 Training Data Distribution

B.8 训练数据分布

After the MPT stage, when training our pipelines on our own data, we train on 123 clips of human motion data collected from our pipeline. We also include during the training process the 10 clips of flat terrain walking data from LaFan  [57] that we used during MPT. Since we sample to be class balanced per-clip, this means we train  90% on our own data and 10% on LaFan data. The motion distribution of our 123 collected clips is reported in Table 6.

MPT 阶段之后,在用我们自己的数据训练流水线时,我们使用由该流水线采集的 123 段人体动作片段。训练过程中还会纳入 MPT 阶段用过的、来自 LaFan[57] 的 10 段平坦地形行走数据。由于我们按类别、按片段做均衡采样,这意味着训练中约 90% 是我们自己的数据,10% 是 LaFan 数据。这 123 段自采片段的动作分布见表 6。

Category Keyword examples # Sequences
Climbing up stairs “climb up”, “walk up stairs (backwards)”, “step‑up platform / block” 39
Climbing down stairs “climb / walk down stairs”, “down stairs backwards / sideways” 36
Both up & down stairs “up then down”, “mixed up / down”, “two people opposite directions” 13
Standing (only) “standing up” 7
Sitting (only) “sit down”, “sitting on sofa / chair / bench” 12
Both sitting & standing “sit then stand” 5
Terrain traversing “stepping stones / chairs”, “side‑walk”, “step on board / block” 11
Total 123

Table 6: Categorization and distribution of our 123 self-collected training clips.

类别 关键词示例 片段数
上楼梯 “climb up”, “walk up stairs (backwards)”, “step‑up platform / block” 39
下楼梯 “climb / walk down stairs”, “down stairs backwards / sideways” 36
上、下楼梯都有 “up then down”, “mixed up / down”, “two people opposite directions” 13
仅站起 “standing up” 7
仅坐下 “sit down”, “sitting on sofa / chair / bench” 12
坐下与站起都有 “sit then stand” 5
穿越地形 “stepping stones / chairs”, “side‑walk”, “step on board / block” 11
合计 123

表 6:我们自行采集的 123 段训练片段的类别与分布。

Appendix C Robot Deployment

附录 C 机器人部署

We implement deployment code in C++ using ROS and the Unitree SDK 2 to enable fast running on the onboard Jetson Orin NX at 50Hz. We use gains of Kp=75Kp=75Kp=75 and Kd=2Kd=2Kd=2 for all joints except the ankle, where we use Kp=20Kp=20Kp=20 and Kd=0.1Kd=0.1Kd=0.1 and Kd=0.2Kd=0.2Kd=0.2 for roll and pitch, respectively, following [62].

我们用 C++,结合 ROS 和 Unitree SDK 2 实现部署代码,以便在机载 Jetson Orin NX 上以 50Hz 快速运行。除踝关节外,所有关节使用 Kp=75Kp=75Kp=75、Kd=2Kd=2Kd=2 的增益;踝关节则沿用[62],滚动使用 Kp=20Kp=20Kp=20、Kd=0.1Kd=0.1Kd=0.1,俯仰使用 Kp=20Kp=20Kp=20、Kd=0.2Kd=0.2Kd=0.2。

C.1 Stages of Deployment in Real

C.1 真实环境中的分阶段部署

多阶段策略在真实场景中的评估

Figure 9: Evaluation of policies at multiple stages. Due to the multi-stage nature of our pipeline, we evaluate the performance on real at multiple stages before going out into the real world. Left: trying our MoCap Pre-Trained policy. Right: Trying the first generalist in a lab environment tracking a pre-defined root trajectory.

图 9:多阶段策略评估。 由于流水线具有多阶段性质,我们在进入真实世界之前,先在多个阶段评估真实环境中的表现。左:试验动作捕捉预训练策略。右:在实验室环境中试验第一条通才策略,使其跟踪一条预先给定的根轨迹。

We employed a progressive evaluation approach to deploying our policies, to gradually debug capabilities in the real world. The first stage was being able to track motion capture data from MPT policies. We then distilled motion capture policies trained on a large dataset to test various approaches to distillation; this is how we found that root position conditioning works better than root velocity. Finally, we started to deploy heightmap conditioned distilled policies from our full pipeline. Figure 9 shows two examples of this.

我们采用循序渐进的评估方式来部署策略,以便逐步在真实世界中调试各项能力。第一阶段是能够用 MPT 策略跟踪动作捕捉数据。随后,我们把在大规模数据集上训练的动作捕捉策略加以蒸馏,以试验各种蒸馏做法;正是由此发现,以根位置为条件优于以根速度为条件。最后,我们开始部署来自完整流水线、以高度图为条件的蒸馏策略。图 9 给出了两个例子。