Learning Unified Force and Position Control for Legged Loco-Manipulation

Peiyuan Zhi Affiliation:  State Key Laboratory of General Artificial Intelligence,Beijing Institute for General Artificial Intelligence (BIGAI)    Peiyang Li Affiliation:  State Key Laboratory of General Artificial Intelligence,Beijing Institute for General Artificial Intelligence (BIGAI) Affiliation:  Beijing University of Posts and Telecommunications https://unified-force.github.io/    Jianqin Yin Affiliation:  Beijing University of Posts and Telecommunications https://unified-force.github.io/    Baoxiong Jia Affiliation:  State Key Laboratory of General Artificial Intelligence,Beijing Institute for General Artificial Intelligence (BIGAI)    Siyuan Huang Affiliation:  State Key Laboratory of General Artificial Intelligence,Beijing Institute for General Artificial Intelligence (BIGAI)

Abstract

Robotic loco-manipulation tasks often involve contact-rich interactions with the environment, requiring the joint modeling of contact force and robot position. However, recent visuomotor policies often focus solely on learning position or force control, overlooking their co-learning. In this work, we propose the first unified policy for legged robots that jointly models force and position control learned without reliance on force sensors. By simulating diverse combinations of position and force commands alongside external disturbance forces, we use rl to learn a policy that estimates forces from historical robot states and compensates for them through position and velocity adjustments. This policy enables a wide range of manipulation behaviors under varying force and position inputs, including position tracking, force application, force tracking, and compliant interactions. Furthermore, we demonstrate that the learned policy enhances trajectory-based imitation learning pipelines by incorporating essential contact information through its force estimation module, achieving approximately $\sim$39.5% higher success rates across four challenging contact-rich manipulation tasks compared to position-control policies. Extensive experiments on both a quadrupedal manipulator and a humanoid robot validate the versatility and robustness of the proposed policy across diverse scenarios.

机器人 loco-manipulation 任务常与环境发生富接触交互，需要同时建模接触力与机器人位姿。然而，近期视觉运动策略往往只学位置控制或只学力控制，忽视二者的联合学习。本文提出面向腿式机器人的首个统一策略，在无需力传感器的条件下联合建模力控制与位置控制。我们在仿真中组合多样的位置与力指令以及外部扰动，用强化学习训练策略：它从历史状态估计外力，并通过位置与速度补偿加以抵消。该策略可在不同力/位置输入下支持位置跟踪、施力、力跟踪与柔顺交互等多种操作行为。进一步地，我们表明所学策略可通过力估计模块把关键接触信息注入基于轨迹的模仿学习流水线，在四项富接触操作任务上相对纯位置控制策略成功率约高 $\sim$39.5%。在四足机械臂与人形机器人上的大量实验验证了该策略在多样场景中的通用性与鲁棒性。

Figure 1: We present a unified force-position policy for legged robots that enables diverse loco-manipulation behaviors, including position tracking, force application, and compliant interactions (top). When used for imitation learning data collection, the policy’s learned internal force estimator provides force-aware demonstrations, improving model performance in contact-rich tasks without relying on force sensors (middle). Results on both quadruped and humanoid robots demonstrate the policy’s versatility and robustness (bottom).

图 1：我们提出面向腿式机器人的统一力–位置策略，可实现位置跟踪、施力与柔顺交互等多种 loco-manipulation 行为（上）。将该策略用于模仿学习数据采集时，其内部力估计器可提供力感知示教，从而在无需力传感器的条件下提升富接触任务表现（中）。在四足与人形机器人上的结果展示了策略的通用性与鲁棒性（下）。

Keywords: Unified Force and Position Control, Force-aware Imitation Learning

关键词： 统一力与位置控制，力感知模仿学习

1 Introduction

Legged robots have recently advanced in locomotion and manipulation  [1, 2, 3, 4], enabling them to traverse complex terrains (e.g., stairs) and extend their workspace through adaptive body posture, revitalizing interest in loco-manipulation [5, 6, 7, 8]. However, controlling legged manipulators is challenging due to their complex kinematic structures. This difficulty is further exacerbated in contact-rich manipulation tasks, where accurate modeling of contact forces is essential for desired control behaviors (e.g., compliance), yet is hindered by the absence of force-sensing hardware. These challenges underscore the need for robust, adaptable policies to support effective robot-environment and human-robot interactions.

腿式机器人近期在运动与操作上进展显著 [1, 2, 3, 4]，既能穿越楼梯等复杂地形，又能通过调整身体姿态扩大工作空间，重新点燃了对 loco-manipulation 的兴趣 [5, 6, 7, 8]。但腿式操作臂运动学结构复杂，控制难度大；在富接触操作中，精确建模接触力对柔顺等期望行为至关重要，却又常受制于缺少力传感硬件。这些挑战凸显了需要鲁棒、可适配的策略，以支撑有效的机器人–环境与人–机交互。

To tackle the control challenge of legged manipulators, rl (rl) algorithms have emerged as effective alternatives to traditional control methods, offering robust and generalizable policies trained through domain randomization [6, 7, 8, 9, 2, 10]. These policies integrate locomotion and manipulation in complex tasks but primarily depend on precise position control, limiting their applicability in contact-rich scenarios. This reliance has also driven the rise of position-based robot imitation learning [11, 12, 13, 14, 15], with large datasets [16, 13, 17, 18] focused solely on robot trajectories, omitting crucial contact information due to the lack of force sensing. As shown in Section 4.2, such trajectory-only data is insufficient for training effective policies, even for basic contact-rich tasks (e.g., wiping a blackboard). This underscores the limitation of position control and emphasizes the necessity of integrating force sensing and modeling into learning-based policies for more effective task execution.

为应对腿式操作臂的控制难题，强化学习（RL）已成为传统控制之外的有效选择，可借助域随机化训练出鲁棒且可泛化的策略 [6, 7, 8, 9, 2, 10]。这些策略能在复杂任务中融合运动与操作，但主要依赖精确位置控制，限制了在富接触场景的适用性。这一依赖也推动了基于位置的机器人模仿学习 [11, 12, 13, 14, 15]，以及大量只含机器人轨迹、因缺力传感而省略接触信息的数据集 [16, 13, 17, 18]。如第 4.2 节所示，仅靠轨迹数据甚至难以训出擦黑板这类基础富接触任务的有效策略。这凸显了位置控制的局限，也强调了把力传感与力建模纳入学习型策略的必要性。

In light of the aforementioned challenges and observations, we propose the first unified policy for legged robots that seamlessly integrates force and position control without the need for force sensors. Unlike previous methods [9] that handle force and position control independently, we train a single control policy using rl in Isaac Gym [19] by simulating diverse combinations of position and force commands alongside external disturbance forces. The policy leverages a force estimator to predict external forces based on the robot’s historical states and offsets to target positions, enabling adaptive adjustments to the robot’s position and velocity. The learned policy supports versatile manipulation behaviors, including position tracking, force application, force tracking, and compliant responses to varied force and position inputs. We also verify the generalizability of this learning framework to different robot embodiments through an extensive spectrum of 7 experiments on both the Unitree B2-Z1 quadrupedal manipulator platform and the Unitree G1 humanoid robot.

基于上述挑战与观察，我们提出面向腿式机器人、无需力传感器即可无缝融合力控制与位置控制的首个统一策略。不同于将力与位置控制分开处理的先前方法 [9]，我们在 Isaac Gym [19] 中用强化学习，通过仿真多样的位置/力指令组合与外部扰动，训练单一控制策略。策略借助力估计器，根据机器人历史状态与相对目标位置的偏移预测外力，从而自适应调整位置与速度。所学策略支持位置跟踪、施力、力跟踪以及对不同力/位置输入的柔顺响应等多种操作行为。我们还在 Unitree B2-Z1 四足操作平台与 Unitree G1 人形机器人上共做 7 组实验，验证该学习框架对不同本体的可泛化性。

Additionally, we highlight the capabilities of the learned policy in facilitating imitation learning with contact force information. Specifically, we develop a force-aware data collection pipeline that utilizes our learned policy as the base teleoperation policy, simultaneously passing position and force commands to the robot while collecting contact-rich manipulation data via the embedded contact force estimator. We validate the effectiveness of this data by integrating the estimated force into a position-based imitation learning policy, leading to a significant improvement ($\sim$39.5%) in success rates over the vanilla position-based methods across three challenging contact-rich tasks. These experimental results underscore the potential of our learned policy as a general framework for curating contact-rich robot interaction data, particularly in the absence of explicit force sensors.

此外，我们强调所学策略在为模仿学习提供接触力信息方面的能力。具体地，我们以所学策略作为底层遥操作策略，构建力感知数据采集流水线：同时向机器人下发位置与力指令，并通过内嵌接触力估计器采集富接触操作数据。将估计力接入基于位置的模仿学习策略后，在三项困难的富接触任务上相对纯位置方法成功率显著提升（$\sim$39.5%）。这些结果说明，在缺少显式力传感器时，所学策略可作为整理富接触交互数据的通用框架。

Overall, our contributions can be summarized as follows:

总体而言，本文贡献可概括如下：

1. We propose the first model for learning unified force and position control in legged loco-manipulation, enabling diverse control behaviors such as position tracking, force control, and compliance with a single policy.

1. 我们提出首个面向腿式 loco-manipulation 的统一力–位置控制学习模型，单一策略即可覆盖位置跟踪、力控制与柔顺等多种控制行为。

2. Through 7 experiments on a quadrupedal manipulator and a humanoid robot, we demonstrate the effectiveness and robustness of our learned policy across diverse and challenging task scenarios.

2. 在四足操作臂与人形机器人上的 7 组实验表明，所学策略在多样且困难的任务场景中有效且鲁棒。

3. We develop a force-aware robot imitation learning data collection pipeline using our learned force estimator, improving position-based imitation learning baselines by $\sim$39.5% on three challenging contact-rich manipulation tasks, highlighting our policy’s promise as a general and efficient framework for contact-rich task demonstration curation.

3. 我们基于所学力估计器构建力感知模仿学习数据采集流水线，使基于位置的模仿学习基线在三项富接触操作任务上提升约 $\sim$39.5%，凸显该策略作为富接触示教整理通用高效框架的潜力。

2 Related Works

Whole-body Control

wbc (wbc) has been widely adopted to enhance robotic capabilities in mobile manipulation, particularly within classical control frameworks [20, 21, 5]. More recently, rl with parallel simulators [19, 22] has become the mainstream approach for addressing complex control challenges in legged robots. Several learning-based methods [6, 23, 24, 7, 25] have improved the robustness of wbc, while others have extended its application to force-intensive tasks [26, 27, 28, 29, 30]. For instance, [27] coordinates joint movements to apply sufficient force during pushing, and ALMA[29] combines wbc with force control to achieve precise end-effector actuation. [30] integrates Cartesian impedance control into a QP formulation, enabling compliant loco-manipulation through a double mass-damper-spring model. These works collectively highlight wbc’s effectiveness in unifying force and position control.

全身控制

全身控制（WBC）已被广泛用于提升移动操作能力，尤其在经典控制框架中 [20, 21, 5]。近来，结合并行仿真器的强化学习 [19, 22] 成为应对腿式机器人复杂控制挑战的主流路径。若干学习方法 [6, 23, 24, 7, 25] 提升了 WBC 的鲁棒性，另一些则将其拓展到大力相关任务 [26, 27, 28, 29, 30]。例如，[27] 协调关节运动以在推物时施加足够力；ALMA [29] 将 WBC 与力控制结合以实现精确末端执行；[30] 把笛卡尔阻抗控制嵌入二次规划，通过双质量–阻尼–弹簧模型实现柔顺 loco-manipulation。这些工作共同表明 WBC 在统一力与位置控制上的有效性。

Hybrid Force and Position Control

In contact-rich manipulation tasks, relying solely on end-effector trajectory control is often insufficient due to the inherent coupling between force and position. Early work [31, 32, 33, 34], including the introduction of impedance control [34], laid the foundation for hybrid force-position strategies. Recent studies [1, 35, 36, 37, 9] have advanced compliance control, with some leveraging force sensors and others estimating force indirectly via internal signals or reinforcement learning. Inspired by these trends, our work eliminates the need for force sensors by using reinforcement learning to train a quadruped robot to simultaneously control force and position. This enables flexible switching between force following, impedance control, and hybrid modes through different command configurations.

力–位置混合控制

在富接触操作中，因力与位置固有耦合，仅靠末端轨迹控制往往不够。早期工作 [31, 32, 33, 34]（含阻抗控制 [34] 的提出）奠定了力–位置混合策略的基础。近期研究 [1, 35, 36, 37, 9] 推进了柔顺控制，有的依赖力传感器，有的通过内部信号或强化学习间接估计力。受此启发，本文取消对力传感器的依赖，用强化学习训练四足机器人同时控制力与位置，并通过不同指令配置在力跟随、阻抗控制与混合模式之间灵活切换。

Imitation Learning for Mobile Manipulation

Imitation learning [38, 39, 40, 41, 42] has recently become a prominent approach for training robots to perform various tasks. bc (bc) [43, 44] is a straightforward method of imitation learning that learns policies by supervising observation-action pairs from expert demonstrations. Studies leveraging image data and proprioceptive sensing [45, 46, 47, 8] to generate robot control commands have shown remarkable success in mobile manipulation tasks. Furthermore, recent research [48, 49] has begun incorporating tactile sensing to enhance the sensory capabilities of robots. Similarly, our work utilizes force inputs without relying on force sensors, demonstrating that force information is critical in enabling robots to complete challenging tasks effectively.

移动操作中的模仿学习

模仿学习 [38, 39, 40, 41, 42] 近来成为训练机器人执行多样任务的重要途径。行为克隆（BC） [43, 44] 是一种直接的模仿学习，通过对专家示教中的观测–动作对做监督来学策略。利用图像与本体感觉 [45, 46, 47, 8] 生成控制指令的研究在移动操作上取得显著成功。此外，近期工作 [48, 49] 开始引入触觉以增强感知。与之类似，本文在不依赖力传感器的条件下使用力输入，表明力信息对完成困难任务至关重要。

3 Method

3.1 A Unified Formulation for Force and Position Control

We begin by introducing the general problem formulation of our approach. As shown in the upper part in Fig. 2(c), given the position command relative to the robot body frame and force command, ${\mathbf{x}}^{\text{cmd}}$ and ${\bm{F}}^{\text{cmd}}$, our goal is to learn a rl policy that ensures the robot’s behavior adheres to these commands under net force ${\bm{F}}$. To achieve this goal, we adopt the impedance control formulation:

我们先给出方法的一般问题表述。如 图 2(c) 上半部分所示，给定相对机器人机体坐标系的位置指令与力指令 ${\mathbf{x}}^{\text{cmd}}$、$\bm{F}^{\text{cmd}}$，目标是学习一个强化学习策略，使机器人在净力 ${\bm{F}}$ 作用下仍服从这些指令。为此我们采用阻抗控制形式：

$${\bm{F}}=K({\mathbf{x}}-{\mathbf{x}}^{\text{des}})+D(\dot{{\mathbf{x}}}-\dot{{\mathbf{x}}}^{\text{des}})+M(\ddot{{\mathbf{x}}}-\ddot{{\mathbf{x}}}^{\text{des}}), \tag{1}$$

where ${\mathbf{x}}$ denotes the actual position of the robot. ${\mathbf{x}}^{\text{des}}$, $\dot{{\mathbf{x}}}^{\text{des}}$, and $\ddot{{\mathbf{x}}}^{\text{des}}$ denotes the desired goal position, velocity, and acceleration of the robot. The parameters $K$, $D$, and $M$ correspond to the stiffness and damping coefficients, and equivalent mass (inertia), respectively.

其中 ${\mathbf{x}}$ 为机器人实际位置；${\mathbf{x}}^{\text{des}}$、$\dot{{\mathbf{x}}}^{\text{des}}$、$\ddot{{\mathbf{x}}}^{\text{des}}$ 分别为期望位置、速度与加速度；$K$、$D$、$M$ 分别为刚度、阻尼与等效质量（惯量）参数。

End-effector Modeling

As the end-effector typically moves slowly during manipulation tasks, we can make the following simplification over Eq. 1: ${\bm{F}}=K({\mathbf{x}}-{\mathbf{x}}^{\text{des}})$. The net force ${\bm{F}}$ primarily consists of three components: the active force ${\bm{F}}^{\text{cmd}}$, the passive reaction force ${\bm{F}}^{\text{react}}$ which arises from applying ${\bm{F}}^{\text{cmd}}$ to the environment, and additional external disturbances ${\bm{F}}^{\text{ext}}$. Therefore, the desired target position ${\mathbf{x}}^{\text{target}}$ of the end-effector is given by:

末端执行器建模

操作任务中末端通常运动较慢，可将式 1 简化为 ${\bm{F}}=K({\mathbf{x}}-{\mathbf{x}}^{\text{des}})$。净力 ${\bm{F}}$ 主要由三部分构成：主动力 ${\bm{F}}^{\text{cmd}}$、对环境施加 ${\bm{F}}^{\text{cmd}}$ 产生的被动反力 ${\bm{F}}^{\text{react}}$，以及额外外扰 ${\bm{F}}^{\text{ext}}$。因此末端期望目标位置 ${\mathbf{x}}^{\text{target}}$ 为：

$${\mathbf{x}}^{\text{target}}={\mathbf{x}}^{\text{cmd}}+\frac{{\bm{F}}^{\text{ext}}+({\bm{F}}^{\text{cmd}}-{\bm{F}}^{\text{react}})}{K}, \tag{2}$$

where the environment reaction force prevents the end-effector from reaching the commanded position ${\mathbf{x}}^{\text{cmd}}$. Under the formulation of Eq. 2, several manipulation behaviors can be derived by appropriately specifying the position command ${\mathbf{x}}^{\text{cmd}}$ and force command ${\bm{F}}^{\text{cmd}}$, including Position Control, Force Control, Impedance Control and Hybrid Position and Force Control. We present a detailed formulation of these control behaviors in Appendix A. In complex scenarios involving simultaneous position commands, force commands, and external disturbances, the system adheres to Eq. 2, integrating these basic control modes.

其中环境反力阻止末端到达指令位置 ${\mathbf{x}}^{\text{cmd}}$。在式 2 框架下，通过恰当指定位置指令 ${\mathbf{x}}^{\text{cmd}}$ 与力指令 ${\bm{F}}^{\text{cmd}}$，可导出多种操作行为，包括位置控制、力控制、阻抗控制与力–位置混合控制。附录 A 给出这些控制行为的详细表述。在位置指令、力指令与外扰同时存在的复杂场景中，系统仍服从式 2，并融合上述基本控制模式。

Figure 2: Method Overview. (a) Architecture of the unified position-force policy trained via reinforcement learning to track position and force commands under external disturbances. (b) Force-aware imitation learning enabled by demonstrations collected using our learned policy, without requiring force sensors. (c) Illustration of position and velocity compensation for force interactions modeled at both the end-effector and the robot base. (d) Visualization of sampled force commands and disturbances used to simulate diverse contact scenarios during policy training.

图 2：方法概览。(a) 经强化学习训练、在外扰下跟踪位置与力指令的统一力–位置策略结构。(b) 用所学策略采集示教、无需力传感器即可实现的力感知模仿学习。(c) 在末端与机体底座处建模力交互时的位置与速度补偿示意。(d) 训练时用于仿真多样接触场景的力指令与扰动采样可视化。

Multi-contact Modeling

For other robot body parts beyond the end-effector, the formulation in Eq. 2 can be extended accordingly. Taking the robot base as an example, we typically care not about its joint state, but rather its velocity or global position. In such cases, Eq. 2 can be simplified by assuming the robot is controlled through base velocity commands. Specifically, given the velocity and force commands ${\mathbf{v}}_{\text{base}}^{\text{cmd}}=\dot{{\mathbf{x}}}_{\text{base}}^{\text{cmd}}$ and ${\bm{F}}^{\text{cmd}}_{\text{base}}$, along with external disturbances ${\bm{F}}^{\text{ext}}_{\text{base}}$, we can derive from Eq. 1:

多接触建模

对末端之外的机体部位，式 2 可相应扩展。以机体底座为例，我们通常关心的不是关节状态，而是速度或全局位置。此时可假设机器人通过底座速度指令控制，从而简化式 2。具体地，给定速度与力指令 ${\mathbf{v}}_{\text{base}}^{\text{cmd}}=\dot{{\mathbf{x}}}_{\text{base}}^{\text{cmd}}$、${\bm{F}}^{\text{cmd}}_{\text{base}}$ 以及外扰 ${\bm{F}}^{\text{ext}}_{\text{base}}$，由式 1 可得：

$${\bm{F}}_{\text{base}}=D(\dot{{\mathbf{x}}}_{\text{base}}-\dot{{\mathbf{x}}}_{\text{base}}^{\text{des}})=D({\mathbf{v}}_{\text{base}}-{\mathbf{v}}_{\text{base}}^{\text{des}}), \tag{3}$$

where ${\bm{F}}_{\text{base}}={\bm{F}}_{\text{base}}^{\text{ext}}+({\bm{F}}_{\text{base}}^{\text{cmd}}-{\bm{F}}_{\text{base}}^{\text{react}})$ is the net force and transform Eq. 2 into:

其中 ${\bm{F}}_{\text{base}}={\bm{F}}_{\text{base}}^{\text{ext}}+({\bm{F}}_{\text{base}}^{\text{cmd}}-{\bm{F}}_{\text{base}}^{\text{react}})$ 为净力，并将式 2 变换为：

$$\displaystyle{\mathbf{v}}_{\text{base}}^{\text{target}} \\ \displaystyle={\mathbf{v}}^{\text{cmd}}_{\text{base}}+\frac{{\bm{F}}_{\text{base}}^{\text{ext}}+({\bm{F}}_{\text{base}}^{\text{cmd}}-{\bm{F}}_{\text{base}}^{\text{react}})}{D}.$$

After this transformation, we can implement similar basic control modes using derivations from Eq. 4. Furthermore, this formulation can be extended to scenarios where external disturbances and force commands on body parts are transformed to end-effectors by converting the net force on the robot base ${\bm{F}}_{\text{base}}$ into an external force to the end-effector ${\bm{F}}_{\text{base}2{\text{ee}}}$, and vice versa. However, due to the learning complexity of such methods, this work focuses on treating the end-effector and robot base independently, leaving the integrated derivation as important future work.

变换后，可用式 4 的推导实现类似的基本控制模式。该表述还可扩展到将机体部位上的外扰与力指令变换到末端：把底座净力 ${\bm{F}}_{\text{base}}$ 转为作用在末端的外力 ${\bm{F}}_{\text{base}2{\text{ee}}}$，反之亦然。但这类方法学习复杂度高，本文暂将末端与底座独立处理，把一体化推导留作重要未来工作。

As a summary, we build our policy learning with reward provided following Eqs. 2 and 4 which models the behavior of the end-effector and the base of the legged robot considering both active and passive forces. We provide the detailed training settings and model in Section 3.2.

综上，我们按式 2 与式 4 提供奖励来构建策略学习，建模腿式机器人末端与底座在主动力与被动力下的行为。详细训练设置与模型见第 3.2 节。

3.2 Learning a Unified Force-Position Control Policy

We detail the learning of the proposed unified force-position control policy by first defining the space of observations, commands, and actions. Specifically, we define the robot’s observation ${\mathbf{o}}_{t}$ with the robot’s base orientation ${\mathbf{g}}^{\text{base}}_{t}$, angular velocity $\omega^{\text{base}}_{t}$, joint position ${\mathbf{q}}_{t}$, joint velocities $\dot{{\mathbf{q}}}_{t}$, previous action ${\mathbf{a}}_{t-1}$, command ${\mathbf{c}}^{\text{cmd}}_{t}$, and the feet clock timings $\theta^{\text{feet}}_{t}$:

我们先定义观测、指令与动作空间，再详述统一力–位置控制策略的学习。具体地，机器人观测 ${\mathbf{o}}_{t}$ 包含底座朝向 ${\mathbf{g}}^{\text{base}}_{t}$、角速度 $\omega^{\text{base}}_{t}$、关节位置 ${\mathbf{q}}_{t}$、关节速度 $\dot{{\mathbf{q}}}_{t}$、上一动作 ${\mathbf{a}}_{t-1}$、指令 ${\mathbf{c}}^{\text{cmd}}_{t}$ 以及足端相位时钟 $\theta^{\text{feet}}_{t}$：

$${\mathbf{o}}_{t}=\left[{\mathbf{g}}^{\text{base}}_{t},{\mathbf{\omega}}^{\text{base}}_{t},{\mathbf{q}}_{t},\dot{{\mathbf{q}}}_{t},{\mathbf{a}}_{t-1},{\mathbf{c}}^{\text{cmd}}_{t},{\mathbf{\theta}}^{\text{feet}}_{t}\right] \tag{5}$$

where the input command ${\mathbf{c}}^{\text{cmd}}_{t}=[{\mathbf{v}}_{\text{base}}^{\text{cmd}},{\mathbf{x}}_{\text{ee}}^{\text{cmd}},{\bm{F}}_{\text{ee}}^{\text{cmd}},{\bm{F}}_{\text{base}}^{\text{cmd}}]$ covers the base velocity, end-effector position, end-effector force, and base force commands. For quadrupedal robots, we consider all four command types. For humanoid robots, only the locomotion command ${\mathbf{v}}^{\text{cmd}}_{\text{base}}$ and base force command ${\bm{F}}^{\text{cmd}}_{\text{base}}$ are considered as there is no gripper available on the robot for manipulation tasks. The action ${\mathbf{a}}_{t}$ specifies joint position targets for a PD controller, calculated as ${\mathbf{q}}^{\text{target}}_{t}=\sigma_{a}{\mathbf{a}}_{t}+{\mathbf{q}}^{\text{default}}$, where $\sigma_{a}$ scales the policy output and ${\mathbf{q}}^{\text{default}}$ represents a standard pose.

其中输入指令 ${\mathbf{c}}^{\text{cmd}}_{t}=[{\mathbf{v}}_{\text{base}}^{\text{cmd}},{\mathbf{x}}_{\text{ee}}^{\text{cmd}},{\bm{F}}_{\text{ee}}^{\text{cmd}},{\bm{F}}_{\text{base}}^{\text{cmd}}]$ 覆盖底座速度、末端位置、末端力与底座力指令。对四足机器人我们使用全部四类指令；对人形机器人，因无可用于操作的夹爪，仅考虑运动指令 ${\mathbf{v}}^{\text{cmd}}_{\text{base}}$ 与底座力指令 ${\bm{F}}^{\text{cmd}}_{\text{base}}$。动作 ${\mathbf{a}}_{t}$ 指定 PD 控制器的关节位置目标，计算为 ${\mathbf{q}}^{\text{target}}_{t}=\sigma_{a}{\mathbf{a}}_{t}+{\mathbf{q}}^{\text{default}}$，其中 $\sigma_{a}$ 缩放策略输出，${\mathbf{q}}^{\text{default}}$ 为标准姿态。

Policy Design

We provide an overview of our policy model in Fig. 2(a). Our policy model comprises three modules: the observation encoder, the state estimator, and the actor. The encoder processes the observation history ${\mathbf{o}}_{[t-H,\dots,t-1,t]}$ $(H=32)$ and outputs a latent feature, which is then sent to the state estimator and the actor. The state estimator then predicts the robot’s state, including the external force ${\bm{F}}={\bm{F}}^{\text{ext}}+{\bm{F}}^{\text{react}}$, the end-effector position, and the base velocity. This estimated force could then be translated to command signals in certain desired control behaviors.

策略设计

图 2(a) 给出策略模型概览。模型含三个模块：观测编码器、状态估计器与 actor。编码器处理观测历史 ${\mathbf{o}}_{[t-H,\dots,t-1,t]}$（$H=32$）并输出潜变量特征，再送入状态估计器与 actor。状态估计器预测机器人状态，包括外力 ${\bm{F}}={\bm{F}}^{\text{ext}}+{\bm{F}}^{\text{react}}$、末端位置与底座速度。估计力可在特定期望控制行为中进一步转为指令信号。

Force Simulation

To simulate diverse scenarios for learning the unified force-position policy, we randomly sample position, velocity, and force commands, along with external net forces as required in Eqs. 2 and 4. The ranges of input commands and external forces are detailed in Section C.1. Notably, the reaction force ${\bm{F}}^{\text{react}}$ is not modeled explicitly but is incorporated into the net external force ${\bm{F}}={\bm{F}}^{\text{ext}}+{\bm{F}}^{\text{react}}$. The sampling range of ${\mathbf{x}}^{\text{cmd}}$ slightly exceeds the arm’s original workspace without whole-body movement, while ensuring that the resulting ${\mathbf{x}}^{\text{target}}_{\text{ee}}$ remains within the operational limits when whole-body motion is allowed. During training, as illustrated in Fig. 2(d), sampled forces are linearly ramped up to target values, held constant for a fixed interval, and then reduced back to zero according to a pre-defined schedule. After a brief zero-force period, new forces are and the cycle repeats. This sampling strategy exposes the policy to a variety of control conditions, echoing the different desired control behavior discussed in Section 3.1 and enabling a single policy to adapt to varying control task demands.

力仿真

为学习统一力–位置策略，我们按式 2 与式 4 的需求随机采样位置、速度、力指令以及外部净力。指令与外力范围见第 C.1 节。注意反力 ${\bm{F}}^{\text{react}}$ 不单独显式建模，而是并入净外力 ${\bm{F}}={\bm{F}}^{\text{ext}}+{\bm{F}}^{\text{react}}$。${\mathbf{x}}^{\text{cmd}}$ 的采样范围略超出手臂在无全身运动时的原始工作空间，但保证在允许全身运动时所得 ${\mathbf{x}}^{\text{target}}_{\text{ee}}$ 仍在可达范围内。训练中如 图 2(d) 所示，采样力按预定义日程线性升至目标、保持固定时长再降回零；短暂零力后重新采样并循环。该策略使策略接触多种控制条件，呼应第 3.1 节讨论的不同期望控制行为，从而让单一策略适配多变任务需求。

Policy Learning

We adopt a two-stage training procedure: first focusing on whole-body reaching and locomotion, then introducing random force commands and external disturbances. This staged approach empirically yields more stable training than a single-stage setup, as further analyzed in Appendix C. Policy learning is supervised by rewarding accurate tracking of the target end-effector position ${\mathbf{x}}_{\text{ee}}^{\text{target}}$ and base velocity ${\mathbf{v}}_{\text{base}}^{\text{target}}$ under varying input and disturbance combinations. Additionally, an MSE loss is used to improve the accuracy of the state estimator for both robot state and external force. Full reward specifications are provided in Table A.1.

策略学习

我们采用两阶段训练：先聚焦全身到达与运动，再引入随机力指令与外扰。经验上，该课程比单阶段设置训练更稳，附录 C 有进一步分析。策略学习通过奖励在多变输入与扰动组合下准确跟踪目标末端位置 ${\mathbf{x}}_{\text{ee}}^{\text{target}}$ 与底座速度 ${\mathbf{v}}_{\text{base}}^{\text{target}}$ 来监督；另用 MSE 损失提升状态估计器对机器人状态与外力的精度。完整奖励见 表 A.1。

3.3 Force-aware Imitation Learning

Recognizing the importance of force information in real-world manipulation tasks and its absence in most existing datasets, we leverage our learned force-position policy to collect force-aware data for imitation learning. Concretely, we teleoperate the robot to record joint states, base states, control commands, estimated end-effector contact forces, and RGB images from cameras mounted on both the end-effector and the robot base. This data is used to train a diffusion-based force-aware imitation learning policy that takes as input the robot states, estimated forces, and image observations, and predicts both force and end-effector position commands as inputs to our low-level force-position policy. Unlike prior works relying solely on visual inputs, our force estimator supplements the policy with contact information, enabling more accurate object interaction and force application. We validate the impact of the collected data and the effectiveness of our approach in Section 4.2. Details of the teleoperation pipeline and training procedures are provided in Appendix B and Appendix D.

认识到力信息在真实操作中的重要性及其在多数现有数据集中的缺失，我们用所学力–位置策略采集力感知数据以供模仿学习。具体地，遥操作机器人记录关节状态、底座状态、控制指令、估计的末端接触力，以及末端与底座相机的 RGB 图像。数据用于训练基于扩散的力感知模仿学习策略：输入为机器人状态、估计力与图像观测，输出力与末端位置指令，再交给底层力–位置策略。与仅依赖视觉的先前工作不同，力估计器为策略补充接触信息，使物体交互与施力更准确。第 4.2 节验证了采集数据的影响与方法有效性；遥操作流水线与训练细节见附录 B 与附录 D。

4 Experiment

4.1 Force and Position Command Tracking

Figure 3: Force and position control evaluation. (a)–(c) Evaluation of force and position control tracking errors in simulation environments. (d) Real-world evaluation of force control, shaded areas indicate variance measured across 5 different end-effector positions.

图 3：力与位置控制评估。(a)–(c) 仿真中力与位置控制跟踪误差评估。(d) 真实世界力控制评估，阴影表示在 5 个不同末端位置上测得的方差。

Position Tracking

To evaluate performance in simulation, we conduct 6000-step rollouts with randomly generated end-effector trajectories with position commands only, covering the entire training workspace. We report the average position tracking and estimation errors across these trials. As shown in Fig. 3(b), the end-effector tracking error remains mostly within 0.1m when in the absence of external forces and force commands. Slightly higher errors are observed along the Y-axis, likely due to fewer available degrees of freedom in that direction, which limits precision. We also assess the accuracy of the state estimator by comparing the estimated end-effector positions with ground-truth simulation values. Across all axes, the estimation error remains within 0.05m, as shown in Fig. 3(a).

位置跟踪

为评估仿真表现，我们仅用位置指令，对随机生成的末端轨迹做 6000 步 rollout，覆盖整个训练工作空间，并报告平均位置跟踪与估计误差。如图 3(b) 所示，在无外力与无力指令时，末端跟踪误差大多保持在 0.1 m 以内。Y 轴误差略高，可能因该方向可用自由度更少、精度受限。我们还将估计末端位置与仿真真值比较以评估状态估计器；各轴估计误差均在 0.05 m 内，见图 3(a)。

Force Control

We evaluate the ability of our proposed policy to estimate and act with forces under two settings. First, we assess position tracking performance when the policy receives force commands that match the applied external forces, serving as an indirect evaluation for evaluating unified force-position control. As shown in Fig. 3(c), compared to the experiment without external forces, tracking error increases slightly compared to the no-force setting but remains mostly within 0.1m, demonstrating effective force-aware behavior. Second, we conduct a direct force control evaluation on real robots by applying force commands ranging from 0 N to 60 N and measuring end-effector forces using a dynamometer. Measurements at five different end-effector positions yield average errors within 10 N, as shown in Fig. 3(d). Force estimation across six discrete levels shows errors between 5–10 N. Due to hardware limitations, evaluations along the Y- and Z-axes are capped at 40 N. Despite minor sim-to-real discrepancies, particularly along the Y-axis, the estimator remains sufficiently accurate for the targeted manipulation tasks. More analyses are provided in Appendix E.

力控制

我们在两种设置下评估策略估计并用力的能力。首先，在力指令与所施外力一致时评估位置跟踪，作为统一力–位置控制的间接检验。如图 3(c)，相对无外力实验，跟踪误差略升但仍大多在 0.1 m 内，表明力感知行为有效。其次，在真机上做直接力控制评估：力指令从 0 N 到 60 N，用测力计测末端力；五个末端位置的平均误差在 10 N 内，见图 3(d)。六个离散力级上的力估计误差约 5–10 N。受硬件限制，Y、Z 轴评估上限为 40 N。尽管存在轻微仿真到现实差距（尤其 Y 轴），估计器对目标操作任务仍足够准确。更多分析见附录 E。

4.2 Force-aware Imitation Learning

Task Settings

We evaluate our method on four real-world tasks that require hybrid force-position control and force sensing: wipe-blackboard, open-cabinet, close-cabinet, and open-drawer-occlusion. In the wipe-blackboard task, the robot must maintain continuous contact with the surface while moving laterally to erase ink marks. We collect $50$ trajectories and trained the force-aware diffusion policy for $30k$ steps. In open-cabinet and close-cabinet, the robot interacts with a push-to-open cabinet, while in open-drawer-occlusion, it opens a drawer that gradually becomes occluded in visual observations (see the setup in Fig. A.8). For each of the three open/close tasks, we collect $30$ episodes per task and train the policy for $20k$ steps. As a baseline, we deploy the trained low-level policy but exclude the force estimator and force command signals during teleoperation data collection. Each task is performed 50 times with each trial constrained to a maximum of $1000$ steps ($\sim$20 seconds) for successful completion. For completeness, we provide additional details about the task definitions and experimental settings in Appendix D.

任务设置

我们在四项需混合力–位置控制与力感知的真机任务上评估：擦黑板、开柜、关柜、遮挡开抽屉。擦黑板要求机器人在横向擦拭墨迹时保持与表面持续接触；采集 $50$ 条轨迹，力感知扩散策略训练 $30k$ 步。开/关柜任务中机器人与按压开启式柜门交互；遮挡开抽屉中抽屉在视觉上逐渐被遮挡（设置见图 A.8）。三项开合任务各采集 $30$ 条 episode，策略训练 $20k$ 步。作为基线，我们部署已训底层策略，但在遥操作采集时去掉力估计器与力指令信号。每项任务各执行 50 次，单次成功上限 $1000$ 步（约 20 秒）。任务定义与实验设置详见附录 D。

Figure 4: Force-aware imitation learning. (a) Time-series outputs of position and force commands to the trained force-aware imitation policy in the wipe-blackboard task. cmd denotes the output of the imitation learning policy, while pred indicates the external force estimated by the low-level policy. (b) A visualization of the data collection process. (c) The performance comparison between our policy and a baseline vision-only policy over 50 trials across four tasks.

图 4：力感知模仿学习。(a) 擦黑板任务中，已训力感知模仿策略输出的位置与力指令时间序列；cmd 为模仿学习策略输出，pred 为底层策略估计的外力。(b) 数据采集过程可视化。(c) 我们的策略与仅视觉基线在四项任务、各 50 次试验上的表现对比。

Results and Analyses

In Fig. 4 , we compare our method to the baseline in the four real-world tasks. Our approach achieves $\sim$39.5% higher success rates than the baseline. In wipe-blackboard, the position-only policy fails to maintain stable contact, often resulting in insufficient wiping or excessive force that risks surface damage. In contrast, our force-aware policy ensures consistent contact pressure, while the low-level policy improves compliance and reduces mechanical stress. For open- and close-cabinet, the primary challenge lies in the push-to-open mechanism’s narrow 3mm stroke, which is difficult to detect using vision alone. Our force estimator accurately senses the required contact force, enabling reliable activation. In open-drawer-occlusion, the baseline policy, relying solely on visual cues, suffers a sharp success rate drop to 0.3 due to unobservable contact. Our method leverages force sensing to detect contact under occlusion, increasing the success rate to 0.76 and underscoring the importance of force estimation in vision-compromised scenarios. We provide all quantitative comparisons and additional details in Appendix D.

结果与分析

图 4 比较了我们的方法与基线在四项真机任务上的表现。相对基线，我们的方法成功率约高 $\sim$39.5%。擦黑板时，纯位置策略难以维持稳定接触，常出现擦不净或用力过大损伤表面；力感知策略可保持稳定接触压强，底层策略也提升柔顺、减轻机械应力。开/关柜的主要难点在于按压开启机构仅约 3 mm 行程，单靠视觉难察觉；力估计器可准确感知所需接触力，从而可靠触发。遮挡开抽屉中，仅视觉基线因接触不可观测成功率骤降至 0.3；我们的方法借力感知在遮挡下检测接触，成功率升至 0.76，凸显视觉受损场景下力估计的重要性。全部定量对比与细节见附录 D。

4.3 Basic Manipulation Policies

Force Control

Force control directly applies a commanded force by moving the end-effector in the force direction until the applied and commanded forces match. Our unified strategy requires no additional training; following  Eq. A.7, where the sum of the estimated external force and the force command determines displacement compensation until equilibrium is achieved. As demonstrated in  Fig. 5(a) and the supplementary video, when a $2.5kg$ dumbbell is attached to the end-effector, the robot achieves balance with a $25N$ upward force command. Without this command, however, the end-effector drops due to the dumbbell’s gravity.

力控制

力控制沿力方向移动末端，直到施加力与指令力匹配。我们的统一策略无需额外训练；按式 A.7，估计外力与力指令之和决定位移补偿直至平衡。如图 5(a) 与补充视频，当末端挂上 $2.5\,\mathrm{kg}$ 哑铃并给出 $25\,\mathrm{N}$ 向上力指令时，机器人可保持平衡；若无力指令，末端会因重力下坠。

Figure 5: Diverse skills facilitated by our policy. (a) Force control: The robot counteracts gravity to support a payload when given a $25N$ force command. (d) Base force tracking: The robot responds compliantly to pushes on its base, enabling intuitive human guidance. (c) Force tracking: The robot tracks a zero-force command by minimizing external force interactions. (d) Impedance control: The robot adjusts its whole-body posture to counteract and comply with external disturbances.

图 5：策略促成的多样技能。(a) 力控制：在 $25\,\mathrm{N}$ 力指令下抵消重力以支撑负载。(d) 底座力跟踪：对底座推挤做柔顺响应，便于直观人工引导。(c) 力跟踪：跟踪零力指令以最小化外力交互。(d) 阻抗控制：调整全身姿态以对抗并顺应外扰。

Force tracking

Force tracking is a special case of force control where the end-effector tracks a zero-force command. When external forces are applied, the end-effector moves in the force direction. Once the force is removed, it stays in the displaced position instead of returning to the original target. This behavior is implemented using our unified policy, following  Eq. A.9. As shown in  Fig. 5(c) and the supplementary video, this capability is demonstrated by setting the force command to zero. In this case, the end-effector follows the external force and remains in the displaced position after the force is removed, achieving effective force tracking.

力跟踪

力跟踪是力控制的特例，末端跟踪零力指令。外力作用时末端沿力方向移动；力撤销后停在偏移位置而非回到原目标。该行为用统一策略按式 A.9 实现。如图 5(c) 与补充视频，将力指令设为零即可演示：末端跟随外力，力消失后留在偏移位，实现有效力跟踪。

Impedance Control

A key application of force control is impedance control, where the end-effector tracks a target position while responding compliantly to external forces, following the dynamics of a spring-mass-damper system. We implement impedance control using the unified policy, following  Eq. A.8. As shown in  Fig. 5(d) and the supplementary video, we demonstrate this capability in human-robot tug-of-war and arm-wrestling scenarios. In these tasks, the further the end-effector deviates from the target position, the greater the resistive force exerted by the robot, showcasing impedance behavior.

阻抗控制

力控制的关键应用是阻抗控制：末端跟踪目标位置的同时对外力柔顺响应，遵循弹簧–质量–阻尼动力学。我们用统一策略按式 A.8 实现。如图 5(d) 与补充视频，在人机拔河与掰腕场景中，末端偏离目标越远，机器人施加的阻力越大，呈现阻抗行为。

4.4 Cross Embodiment Performance

To validate the cross-embodiment capability of our unified policy,we tested it on the Unitree G1 humanoid robot and Unitree B2-Z1 quadrupedal manipulator. For locomotion, unlike manipulation tasks where force compensation applies directly to the end-effector, we adjust the robot base velocity to compensate for external forces, following  Eq. 4. As shown in the third row of  Fig. 1, when the compensated velocity equals and opposes the velocity command, the humanoid robot halts and leans its body to maintain balance. Similarly, as shown in  Fig. 5(b), the quadrupedal robot begins walking forward when kicked, even with zero force and velocity commands.

为验证统一策略的跨本体能力，我们在 Unitree G1 人形机器人与 Unitree B2-Z1 四足操作臂上测试。对运动而言，不同于操作中力补偿直接作用在末端，我们按式 4 通过调整底座速度补偿外力。如图 1 第三行，当补偿速度与速度指令等大反向时，人形机器人停下并倾斜身体以保持平衡。类似地，如图 5(b)，即使力与速度指令为零，四足机器人被踢后仍会开始向前行走。

5 Conclusion

We propose a unified force-position control policy for legged robots, enabling contact-rich loco-manipulation tasks without explicit force sensors. Using reinforcement learning, our policy estimates external forces from historical states and compensates for them through position and velocity adjustments. This approach supports diverse behaviors like position tracking, force application, and compliance. Additionally, integrating force estimation into imitation learning improves task success in contact-rich environments. Experiments on quadrupedal and humanoid robots validate the policy’s adaptability and robustness in real-world scenarios.

我们提出面向腿式机器人的统一力–位置控制策略，使富接触 loco-manipulation 无需显式力传感器即可完成。通过强化学习，策略从历史状态估计外力，并以位置与速度调整加以补偿，从而支持位置跟踪、施力与柔顺等多样行为。将力估计融入模仿学习还可提升富接触环境中的任务成功率。四足与人形机器人上的实验验证了策略在真实场景中的适应性与鲁棒性。

6 Limitations and Future Work

First, while the policy successfully estimates external forces without direct force sensing, its accuracy tends to degrade in high-frequency interactions and at the edges of the robot’s workspace. Future work could focus on improving force estimation in these corner cases. One possible direction is to incorporate velocity and acceleration terms from Eq. 2 to enhance force estimation, allowing the model to better capture dynamic interactions.

首先，尽管策略能在无直接力传感下成功估计外力，但在高频交互与工作空间边缘时精度往往下降。未来工作可改进这些边角情形的力估计；一个可能方向是引入式 2 中的速度与加速度项，以更好刻画动态交互。

Second, while our policy generalizes well from simulation to real-world deployment, discrepancies remain due to the sim-to-real gap, particularly in force accuracy along different coordinate axes. These differences likely stem from mismatches in actuator dynamics and contact modeling between simulation and real hardware. Future work could explore techniques such as domain randomization and real-to-sim corrections to improve robustness across varying real-world conditions.

其次，虽策略从仿真到真机部署泛化良好，但仍存在仿真到现实差距，尤其不同坐标轴上的力精度。差异可能来自仿真与真机在执行器动力学与接触建模上的不匹配。未来可探索域随机化与 real-to-sim 校正等技术，提升在多变真实条件下的鲁棒性。

Additionally, our current framework primarily focuses on estimating force at a single interaction point. Future work could explore multi-point force estimation and whole-body force interaction tasks. For example, in scenarios such as a quadrupedal robot opening a heavy door, the robot could use its body to brace against the door while simultaneously using its manipulator to press down on the handle. Developing policies that coordinate multiple contact forces across different body parts could enable more complex and effective real-world interactions.

此外，当前框架主要估计单交互点上的力。未来可探索多点力估计与全身力交互任务。例如四足机器人开沉重门时，可用身体撑住门，同时用操作臂下压门把手。发展协调身体多部位接触力的策略，有望实现更复杂有效的真实交互。

References

Hwangbo et al. [2019] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter. Learning agile and dynamic motor skills for legged robots. Science Robotics , 4(26):eaau5872, 2019.

Zhuang et al. [2024] Z. Zhuang, S. Yao, and H. Zhao. Humanoid parkour learning. arXiv preprint arXiv:2406.10759 , 2024.

He et al. [2024] T. He, C. Zhang, W. Xiao, G. He, C. Liu, and G. Shi. Agile but safe: Learning collision-free high-speed legged locomotion. arXiv preprint arXiv:2401.17583 , 2024.

Hoeller et al. [2024] D. Hoeller, N. Rudin, D. Sako, and M. Hutter. Anymal parkour: Learning agile navigation for quadrupedal robots. Science Robotics , 9(88):eadi7566, 2024.

Sleiman et al. [2023] J.-P. Sleiman, F. Farshidian, and M. Hutter. Versatile multicontact planning and control for legged loco-manipulation. Science Robotics , 8(81):eadg5014, 2023.

Fu et al. [2023] Z. Fu, X. Cheng, and D. Pathak. Deep whole-body control: learning a unified policy for manipulation and locomotion. In Conference on Robot Learning , pages 138–149. PMLR, 2023.

Liu et al. [2024] M. Liu, Z. Chen, X. Cheng, Y. Ji, R. Qiu, R. Yang, and X. Wang. Visual whole-body control for legged loco-manipulation. The 8th Conference on Robot Learning , 2024.

Qiu et al. [2024] R.-Z. Qiu, Y. Song, X. Peng, S. A. Suryadevara, G. Yang, M. Liu, M. Ji, C. Jia, R. Yang, X. Zou, et al. Wildlma: Long horizon loco-manipulation in the wild. arXiv preprint arXiv:2411.15131 , 2024.

Portela et al. [2024] T. Portela, G. B. Margolis, Y. Ji, and P. Agrawal. Learning force control for legged manipulation. arXiv preprint arXiv:2405.01402 , 2024.

Zhuang et al. [2023] Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao. Robot parkour learning. arXiv preprint arXiv:2309.05665 , 2023.

Shridhar et al. [2023] M. Shridhar, L. Manuelli, and D. Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning (CoRL) , 2023.

Chi et al. [2023] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research , page 02783649241273668, 2023.

Brohan et al. [2023] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818 , 2023.

Team et al. [2024] O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213 , 2024.

Black et al. [2024] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164 , 2024.

Walke et al. [2023] H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V. Myers, M. J. Kim, M. Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL) , 2023.

Khazatsky et al. [2024] A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945 , 2024.

O’Neill et al. [2024] A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 6892–6903. IEEE, 2024.

Makoviychuk et al. [2021] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470 , 2021.

Sleiman et al. [2021] J.-P. Sleiman, F. Farshidian, M. V. Minniti, and M. Hutter. A unified mpc framework for whole-body dynamic locomotion and manipulation. IEEE Robotics and Automation Letters , 6(3):4688–4695, 2021.

Polverini et al. [2020] M. P. Polverini, A. Laurenzi, E. M. Hoffman, F. Ruscelli, and N. G. Tsagarakis. Multi-contact heavy object pushing with a centaur-type humanoid robot: Planning and control for a real demonstrator. IEEE Robotics and Automation Letters , 5(2):859–866, 2020.

Rudin et al. [2022] N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning , pages 91–100. PMLR, 2022.

Pan et al. [2024] G. Pan, Q. Ben, Z. Yuan, G. Jiang, Y. Ji, J. Pang, H. Liu, and H. Xu. Roboduet: A framework affording mobile-manipulation and cross-embodiment. arXiv preprint arXiv:2403.17367 , 2024.

Ma et al. [2022] Y. Ma, F. Farshidian, T. Miki, J. Lee, and M. Hutter. Combining learning-based locomotion policy with model-based manipulation for legged mobile manipulators. IEEE Robotics and Automation Letters , 7(2):2377–2384, 2022.

Wang et al. [2024] J. Wang, J. Rajabov, C. Xu, Y. Zheng, and H. Wang. Quadwbg: Generalizable quadrupedal whole-body grasping. arXiv preprint arXiv:2411.06782 , 2024.

Murphy et al. [2012] M. P. Murphy, B. Stephens, Y. Abe, and A. A. Rizzi. High degree-of-freedom dynamic manipulation. In Unmanned Systems Technology XIV , volume 8387, pages 339–348. SPIE, 2012.

Murooka et al. [2015] M. Murooka, S. Nozawa, Y. Kakiuchi, K. Okada, and M. Inaba. Whole-body pushing manipulation with contact posture planning of large and heavy object for humanoid robot. In 2015 IEEE International Conference on Robotics and Automation (ICRA) , pages 5682–5689. IEEE, 2015.

Rehman et al. [2016] B. U. Rehman, M. Focchi, J. Lee, H. Dallali, D. G. Caldwell, and C. Semini. Towards a multi-legged mobile manipulator. In 2016 IEEE International Conference on Robotics and Automation (ICRA) , pages 3618–3624. IEEE, 2016.

Bellicoso et al. [2019] C. D. Bellicoso, K. Krämer, M. Stäuble, D. Sako, F. Jenelten, M. Bjelonic, and M. Hutter. Alma-articulated locomotion and manipulation for a torque-controllable robot. In 2019 International conference on robotics and automation (ICRA) , pages 8477–8483. IEEE, 2019.

Risiglione et al. [2022] M. Risiglione, V. Barasuol, D. G. Caldwell, and C. Semini. A whole-body controller based on a simplified template for rendering impedances in quadruped manipulators. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 9620–9627. IEEE, 2022.

Raibert and Craig [1981] M. H. Raibert and J. J. Craig. Hybrid position/force control of manipulators. 1981.

Mason [1981] M. T. Mason. Compliance and force control for computer controlled manipulators. IEEE Transactions on Systems, Man, and Cybernetics , 11(6):418–432, 1981.

Yoshikawa [1987] T. Yoshikawa. Dynamic hybrid position/force control of robot manipulators–description of hand constraints and calculation of joint driving force. IEEE Journal on Robotics and Automation , 3(5):386–392, 1987.

Hogan [1984] N. Hogan. Impedance control: An approach to manipulation. In 1984 American control conference , pages 304–313. IEEE, 1984.

Zhang et al. [2021] X. Zhang, L. Sun, Z. Kuang, and M. Tomizuka. Learning variable impedance control via inverse reinforcement learning for force-related tasks. IEEE Robotics and Automation Letters , 6(2):2225–2232, 2021.

Hou et al. [2024] Y. Hou, Z. Liu, C. Chi, E. Cousineau, N. Kuppuswamy, S. Feng, B. Burchfiel, and S. Song. Adaptive compliance policy: Learning approximate compliance for diffusion guided control. arXiv preprint arXiv:2410.09309 , 2024.

de Wolde et al. [2024] J. de Wolde, L. Knoedler, G. Garofalo, and J. Alonso-Mora. Current-based impedance control for interacting with mobile manipulators. arXiv preprint arXiv:2403.13079 , 2024.

Bousmalis et al. [2023] K. Bousmalis, G. Vezzani, D. Rao, C. M. Devin, A. X. Lee, M. B. Villalonga, T. Davchev, Y. Zhou, A. Gupta, A. Raju, et al. Robocat: A self-improving generalist agent for robotic manipulation. Transactions on Machine Learning Research , 2023.

Mees et al. [2024] O. Mees, D. Ghosh, K. Pertsch, K. Black, H. R. Walke, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, et al. Octo: An open-source generalist robot policy. In First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024 , 2024.

Vuong et al. [2023] Q. Vuong, S. Levine, H. R. Walke, K. Pertsch, A. Singh, R. Doshi, C. Xu, J. Luo, L. Tan, D. Shah, et al. Open x-embodiment: Robotic learning datasets and rt-x models. In Towards Generalist Robots: Learning Paradigms for Scalable Skill Acquisition@ CoRL2023 , 2023.

Yang et al. [2023] J. Yang, D. Sadigh, and C. Finn. Polybot: Training one policy across robots while embracing variability. arXiv preprint arXiv:2307.03719 , 2023.

Fang et al. [2024] H.-S. Fang, H. Fang, Z. Tang, J. Liu, C. Wang, J. Wang, H. Zhu, and C. Lu. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 653–660. IEEE, 2024.

Pomerleau [1988] D. A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems , 1, 1988.

Bojarski et al. [2016] M. Bojarski, D. D. Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, X. Zhang, J. Zhao, and K. Zieba. End to end learning for self-driving cars, 2016. URL https://arxiv.org/abs/1604.07316 .

Fu et al. [2024] Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. In Conference on Robot Learning (CoRL) , 2024.

Ha et al. [2024] H. Ha, Y. Gao, Z. Fu, J. Tan, and S. Song. Umi on legs: Making manipulation policies mobile with manipulation-centric whole-body controllers. arXiv preprint arXiv:2407.10353 , 2024.

He et al. [2024] Z. He, K. Lei, Y. Ze, K. Sreenath, Z. Li, and H. Xu. Learning visual quadrupedal loco-manipulation from demonstrations. arXiv preprint arXiv:2403.20328 , 2024.

Lin et al. [2024] T. Lin, Y. Zhang, Q. Li, H. Qi, B. Yi, S. Levine, and J. Malik. Learning visuotactile skills with two multifingered hands. arXiv preprint arXiv:2404.16823 , 2024.

Yang et al. [2023] W. Yang, A. Angleraud, R. S. Pieters, J. Pajarinen, and J.-K. Kämäräinen. Seq2seq imitation learning for tactile feedback-based manipulation. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 5829–5836. IEEE, 2023.

Schulman et al. [2017] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017.

Appendix A Problem Formulation

Our policy enables a wide range of manipulation behaviors under varying force and position inputs, including position control, force control, impedance control, and hybrid position and force control. Under the formulation of Eq. 2, we can easily derive the following basic manipulation policies:

我们的策略可在多变力/位置输入下实现位置控制、力控制、阻抗控制与力–位置混合控制等多种操作行为。在式 2 框架下，可方便导出如下基本操作策略：

• a) Position Control: When there is no external disturbance or active force command applied, Eq. 2 becomes $${\mathbf{x}}^{\text{target}}={\mathbf{x}}^{\text{cmd}},$$ (A.6) where the target end-effector position ${\mathbf{x}}^{\text{target}}$ should reach the commanded position ${\mathbf{x}}^{\text{cmd}}$.

• a) 位置控制：无外扰且无主动力指令时，式 2 变为 $${\mathbf{x}}^{\text{target}}={\mathbf{x}}^{\text{cmd}},$$ (A.6)，即目标末端位置 ${\mathbf{x}}^{\text{target}}$ 应到达指令位置 ${\mathbf{x}}^{\text{cmd}}$。

• b) Force Control: When in contact with the environment and applying a force ${\bm{F}}^{\text{cmd}}$ without external disturbances, the desired goal position of the end-effector is defined following Eq. 2 as: $\displaystyle{\mathbf{x}}^{\text{target}}={\mathbf{x}}^{\text{cmd}}+\frac{({\bm{F}}^{\text{cmd}}-{\bm{F}}^{\text{react}})}{K}.$ (A.7) During the system execution, the reaction force ${\bm{F}}^{\text{react}}$ gradually increases to match the force command ${\bm{F}}^{\text{cmd}}$, leaving the final target pose ${\mathbf{x}}^{\text{target}}_{\text{final}}={\mathbf{x}}^{\text{cmd}}$.

• b) 力控制：与环境接触并施加力 ${\bm{F}}^{\text{cmd}}$、无外扰时，末端期望目标按式 2 为：$\displaystyle{\mathbf{x}}^{\text{target}}={\mathbf{x}}^{\text{cmd}}+\frac{({\bm{F}}^{\text{cmd}}-{\bm{F}}^{\text{react}})}{K}.$ (A.7)执行过程中反力 ${\bm{F}}^{\text{react}}$ 逐渐增大到匹配力指令 ${\bm{F}}^{\text{cmd}}$，最终目标位姿 ${\mathbf{x}}^{\text{target}}_{\text{final}}={\mathbf{x}}^{\text{cmd}}$。

• c) Impedance Control: When the end-effector is subjected to an external disturbance force but does not apply force to the environment, Eq. 2 simplify to: $${\mathbf{x}}^{\text{target}}={\mathbf{x}}^{\text{cmd}}+\frac{{\bm{F}}^{\text{ext}}}{K},$$ (A.8) where the end-effector, when subjected to external disturbances, adjusts its position to exhibit compliance in response to the external force ${\bm{F}}^{\text{ext}}$. Notably, we can also implement force tracking and gravity compensation using Eq. A.8 by dynamically adjusting the position command ${\mathbf{x}}^{\text{cmd}}$ following: $$\Delta{\mathbf{x}}^{\text{cmd}}=\frac{{\bm{F}}^{\text{ext}}}{K},$$ (A.9) where ${\bm{F}}^{\text{ext}}$ can be the gravity term or an external force.

• c) 阻抗控制：末端受外扰但不对环境施力时，式 2 简化为：$${\mathbf{x}}^{\text{target}}={\mathbf{x}}^{\text{cmd}}+\frac{{\bm{F}}^{\text{ext}}}{K},$$ (A.8)，末端在外扰下调整位置以对外力 ${\bm{F}}^{\text{ext}}$ 表现柔顺。此外，也可按 $$\Delta{\mathbf{x}}^{\text{cmd}}=\frac{{\bm{F}}^{\text{ext}}}{K},$$ (A.9) 动态调整位置指令，用式 A.8 实现力跟踪与重力补偿，其中 ${\bm{F}}^{\text{ext}}$ 可以是重力项或外力。

• d) Hybrid Position and Force Control: As defined in [31], hybrid position and force control refers to controlling the end-effector’s movement using ${\mathbf{x}}^{\text{cmd}}$ while applying a force command ${\bm{F}}^{\text{cmd}}={\bm{F}}^{\text{cmd}}_{\perp}$ perpendicular to the tangential direction of the movement. In this scenario, the system follows Eq. A.6 along the tangential direction without force command and satisfies Eq. A.7 in the perpendicular direction where the force command is active.

• d) 力–位置混合控制：按 [31] 的定义，指末端运动由 ${\mathbf{x}}^{\text{cmd}}$ 控制，同时沿运动切向的垂直方向施加力指令 ${\bm{F}}^{\text{cmd}}={\bm{F}}^{\text{cmd}}_{\perp}$。此时系统在切向无力指令、服从式 A.6，在力指令生效的法向服从式 A.7。

Appendix B Hardware Settings and Teleoperation System

(a) B2-Z1 robot hardware. A Unitree B2 robot with a Z1 arm is teleoperated via a wireless controller, with two RealSense cameras for visual input.

(a) B2-Z1 机器人硬件。 Unitree B2 搭载 Z1 机械臂，经无线手柄遥操作，两台 RealSense 相机提供视觉输入。

Robot System Setup

The humanoid robot system ( Fig. 2) is the 29-DOF Unitree G1 robot. The quadruped robot system ( Fig. 6(a)) consists of a 12-DOF Unitree B2 robot and a 6-DOF Unitree Z1 robot arm, both powered by the battery of B2. We customize two RealSense cameras mounted on the arm and head of the quadruped robot. Our whole-body controller and diffusion policy inference are executed on a dedicated desktop with an RTX 3090 GPU, interfaced with the B2-Z1 robot via an internet connection.

机器人系统设置

人形机器人系统（图 2）为 29 自由度 Unitree G1。四足系统（图 6(a)）由 12 自由度 Unitree B2 与 6 自由度 Unitree Z1 臂组成，均由 B2 电池供电。我们在四足臂与头部定制安装两台 RealSense 相机。全身控制器与扩散策略推理在配备 RTX 3090 的专用台式机上运行，经网络与 B2-Z1 通信。

Teleoperation System

As shown in  Fig. 6(b), for manipulation tasks that require only position control, we utilize the MuJoCo AR application, which allows flexible teleoperation of the robot using an iPhone. However, as illustrated in  Fig. 6(a), this approach is no longer suitable for manipulation tasks involving hybrid force and position control. To address this, we developed a dedicated teleoperation system based on the B2 robot’s built-in wireless controller. In this system, two joysticks are used to control the robot’s base movement, while eight buttons below are mapped to control the end-effector’s position and the opening/closing of the gripper. Additionally, by holding the “L1” button, the system switches from issuing position commands to force commands for the end-effector.

遥操作系统

如图 6(b)，对仅需位置控制的操作任务，我们使用 MuJoCo AR 应用，可用 iPhone 灵活遥操作。但如图 6(a)，该方法不再适用于力–位置混合控制任务。为此我们基于 B2 自带无线手柄开发专用遥操作系统：双摇杆控制底座运动，下方八个按键映射末端位置与夹爪开合；按住 “L1” 则从位置指令切换为末端力指令。

Appendix C Details on Policy Learning with Reinforcement Learning

We utilize ppo (ppo) [50] to train the actor policy and implement the state estimator with a mlp (mlp) network for state prediction outputs. We train our rl policy in Isaac Gym [19] with $4096$ parallel environments.

我们用近端策略优化（PPO） [50] 训练 actor，并用多层感知机（MLP）实现状态估计器的状态预测输出。强化学习策略在 Isaac Gym [19] 中以 $4096$ 个并行环境训练。

C.1 Input Commands and Disturbance Forces

We sample input commands and disturbance forces within the ranges below during training:

训练时在如下范围内采样输入指令与扰动：

1. End-effector position command in spherical coordinates within the body frame ${\mathbf{x}}^{\text{cmd}}_{\text{ee}}=(r^{\text{cmd}},\theta^{\text{cmd}},\phi^{\text{cmd}})$, where $r^{\text{cmd}}\in[0.35,0.85\,\text{m}]$, $\theta^{\text{cmd}}\in[-0.4\pi,0.4\pi\,\text{rad}]$, $\phi^{\text{cmd}}\in[-0.6\pi,0.6\pi\,\text{rad}]$.

1. 机体系球坐标下的末端位置指令 ${\mathbf{x}}^{\text{cmd}}_{\text{ee}}=(r^{\text{cmd}},\theta^{\text{cmd}},\phi^{\text{cmd}})$，其中 $r^{\text{cmd}}\in[0.35,0.85\,\text{m}]$，$\theta^{\text{cmd}}\in[-0.4\pi,0.4\pi\,\text{rad}]$，$\phi^{\text{cmd}}\in[-0.6\pi,0.6\pi\,\text{rad}]$。

2. End-effector force command in cartesian coordinates within the body frame ${\bm{F}}^{\text{cmd}}_{\text{ee}}\in\mathbb{R}^{3}$: $[-60N,60N]$.

2. 机体系笛卡尔坐标下的末端力指令 ${\bm{F}}^{\text{cmd}}_{\text{ee}}\in\mathbb{R}^{3}$：$[-60N,60N]$。

3. Base velocity command ${\mathbf{v}}^{\text{cmd}}_{\text{base}}=(v^{\text{cmd}}_{x},v^{\text{cmd}}_{y},\omega^{\text{cmd}}_{z})$, where $v_{x}\in[-0.8,0.8\,\text{m/s}]$, $v_{y}\in[-0.6,0.6\,\text{m/s}]$, $\omega_{z}\in[-0.8,0.8\,\text{rad/s}]$.

3. 底座速度指令 ${\mathbf{v}}^{\text{cmd}}_{\text{base}}=(v^{\text{cmd}}_{x},v^{\text{cmd}}_{y},\omega^{\text{cmd}}_{z})$，其中 $v_{x}\in[-0.8,0.8\,\text{m/s}]$，$v_{y}\in[-0.6,0.6\,\text{m/s}]$，$\omega_{z}\in[-0.8,0.8\,\text{rad/s}]$。

4. Base force command within the body frame ${\bm{F}}^{\text{cmd}}_{\text{base}}\in\mathbb{R}^{3}$: $[-60N,60N]$.

4. 机体系底座力指令 ${\bm{F}}^{\text{cmd}}_{\text{base}}\in\mathbb{R}^{3}$：$[-60N,60N]$。

5. External net force from the environment at the end-effector ${\bm{F}}_{\text{ee}}\in\mathbb{R}^{3}$: $[-60N,60N]$ and at robot base ${\bm{F}}_{\text{base}}\in\mathbb{R}^{3}$: $[-60N,60N]$.

5. 环境在末端的外部净力 ${\bm{F}}_{\text{ee}}\in\mathbb{R}^{3}$：$[-60N,60N]$，以及在底座的 ${\bm{F}}_{\text{base}}\in\mathbb{R}^{3}$：$[-60N,60N]$。

C.2 Reward and Domain Radomization

Table A.1 provides a detailed overview of the reward structure employed in this study, and  Table A.2 outlines the adopted domain randomization scheme.

表 A.1 给出本研究所用奖励结构概览，表 A.2 列出采用的域随机化方案。

Term	Equation	Weight
end-effector Unified Position and Force Control
gripper position	$\exp\{-|{\mathbf{x}}_{\text{ee}}-({\mathbf{x}}^{\text{cmd}}+({\bm{F}}^{\text{ext}}+{\bm{F}}^{\text{cmd}}-{\bm{F}}^{\text{react}})/{B}|/0.5\}$	$2.0$
Base Unified Position and Force Control ($\mathbf{r}_{v}^{b}$)
base velocity	$\exp\{-{|{\mathbf{v}}_{\text{base}}-({\mathbf{v}}^{\text{cmd}}_{\text{base}}+F_{\text{base}}/D)|/{0.25}}\}$	$2.0$
Safety and Smoothness
collision penalty	$\mathds{1}_{\text{collision}}$	$-5.0$
joint limit	$\mathds{1}_{q>0.8*q^{max}||q<0.8*q^{min}}$	$-10.0$
torques	$|{\mathbf{\tau}}|^{2}$	$-5\text{\times}{10}^{-6}$
joint velocities	$|\dot{q}|^{2}$	$-8\text{\times}{10}^{-4}$
joint acceleration	$|\ddot{q}|^{2}$	$-2\text{\times}{10}^{-7}$
action rate	$|a_{t-1}-a_{t}|$	$-0.02$
torque limit	$\mathds{1}_{\mathbf{\tau}>0.9*|\mathbf{\tau}^{max}|}$	$-0.005$
Gait
contact number	$\sum_{\text{foot}}\mathds{1}_{\mathbf{\tau}_{contact}>5.}*stance\_mask$	$2.0$
reference motion	$|q-q^{ref}|^{2}$	$1.0$

Table A.1: Reward terms for learning the whole-body policy.

项	公式	权重
末端统一位置与力控制
夹爪位置	$\exp\{-|{\mathbf{x}}_{\text{ee}}-({\mathbf{x}}^{\text{cmd}}+({\bm{F}}^{\text{ext}}+{\bm{F}}^{\text{cmd}}-{\bm{F}}^{\text{react}})/{B}|/0.5\}$	$2.0$
底座统一位置与力控制 ($\mathbf{r}_{v}^{b}$)
底座速度	$\exp\{-{|{\mathbf{v}}_{\text{base}}-({\mathbf{v}}^{\text{cmd}}_{\text{base}}+F_{\text{base}}/D)|/{0.25}}\}$	$2.0$
安全与平滑
碰撞惩罚	$\mathds{1}_{\text{collision}}$	$-5.0$
关节限位	$\mathds{1}_{q>0.8*q^{max}||q<0.8*q^{min}}$	$-10.0$
力矩	$|{\mathbf{\tau}}|^{2}$	$-5\text{\times}{10}^{-6}$
关节速度	$|\dot{q}|^{2}$	$-8\text{\times}{10}^{-4}$
关节加速度	$|\ddot{q}|^{2}$	$-2\text{\times}{10}^{-7}$
动作变化率	$|a_{t-1}-a_{t}|$	$-0.02$
力矩限位	$\mathds{1}_{\mathbf{\tau}>0.9*|\mathbf{\tau}^{max}|}$	$-0.005$
步态
接触数量	$\sum_{\text{foot}}\mathds{1}_{\mathbf{\tau}_{contact}>5.}*stance\_mask$	$2.0$
参考运动	$|q-q^{ref}|^{2}$	$1.0$

表 A.1：学习全身策略的奖励项。

Term	Unit	Range
Friction	-	$[0.3,\,2.0]$
Body Mass	$kg$	$[0.0,\,15.0]$
base com (x,y,z axis)	m	$[-0.15,\,0.15]$
Motor Strength	%	$[85,\,115]$
Gripper Payload	$kg$	$[0.0,\,0.5]$
Push robot	$m/s$	$[0.0,\,0.8]$, interval = 8s

Table A.2: Domain radomization for learning the whole-body policy.

项	单位	范围
摩擦	-	$[0.3,\,2.0]$
身体质量	$kg$	$[0.0,\,15.0]$
底座质心 (x,y,z)	m	$[-0.15,\,0.15]$
电机强度	%	$[85,\,115]$
夹爪负载	$kg$	$[0.0,\,0.5]$
推机器人	$m/s$	$[0.0,\,0.8]$，间隔 = 8s

表 A.2：学习全身策略的域随机化。

C.3 Additional Analyses on Policy Learning

Figure A.7: Training reward curve (view with zoom-in).

图 A.7：训练奖励曲线（可放大查看）。

Introducing external disturbances early makes training challenging, as initial policies struggle with body balance and end-effector stability. To address this, we used a two-stage curriculum: first, training whole-body reaching and locomotion, then adding random force commands and disturbances. The training reward curve (Fig. A.7) shows an initial drop in stage two, with locomotion recovering and whole-body reaching stabilizing, despite a slight reaching reward decrease due to increased sampling diversity of force commands and disturbances.

过早引入外扰会使训练变难，因初期策略尚难维持身体平衡与末端稳定。为此我们采用两阶段课程：先训全身到达与运动，再加入随机力指令与扰动。训练奖励曲线（图 A.7）显示第二阶段初期有所下降，随后运动能力恢复、全身到达趋于稳定；因力指令与扰动采样更丰富，到达奖励略有下降。

Appendix D Details on Force-aware Imitation Learning Policy

Task	wipe-blackboard	open-cabinet	close-cabinet	open-drawer-occlusion
w/o Force	0.22	0.36	0.30	0.30
w/ Force	0.58	0.70	0.72	0.76

Table A.3: Imitation learning results (50 trials per task)

任务	wipe-blackboard	open-cabinet	close-cabinet	open-drawer-occlusion
无 Force	0.22	0.36	0.30	0.30
有 Force	0.58	0.70	0.72	0.76

表 A.3：模仿学习结果（每任务 50 次试验）

Figure A.8: Open drawer with occlusion, the gripper becomes occluded during manipulation.

图 A.8：遮挡开抽屉，操作过程中夹爪被遮挡。

Task Settings

We select four tasks that require a combination of hybrid force and position control and force sensing capabilities in the real world.

任务设置

我们选取四项在真实世界中需结合力–位置混合控制与力感知能力的任务。

• • For the wipe-blackboard task, the robot must press against the blackboard while moving laterally to effectively remove ink marks. If the robot loses contact or fails to apply sufficient force during movement, the ink will not be wiped away. A purely position-controlled approach struggles with maintaining consistent contact force, often resulting in intermittent contact or excessive force application. In contrast, our force estimator enables the robot to sustain stable contact pressure, ensuring effective wiping while minimizing the risk of excessive force that could damage the robot.

• • 擦黑板任务中，机器人须边贴紧黑板边横向移动以有效擦除墨迹。若运动中失去接触或施力不足，墨迹擦不掉。纯位置控制难以维持稳定接触力，常出现间歇接触或用力过大；力估计器则使机器人保持稳定接触压强，在有效擦拭的同时降低损伤风险。

• • For the two tasks of open- and close-cabinet with a push-to-open cabinet, the robot must apply sufficient force to press the door, triggering the built-in mechanism that causes it to spring open or close. Unlike the previous task, this scenario requires the robot to exert enough force to overcome the mechanism’s resistance, despite the minimal displacement during activation.

• • 对按压开启式柜门的开/关柜两项任务，机器人须施加足够力按压门板，触发内置机构使其弹开或关闭。与前一任务不同，激活时位移极小，但仍需克服机构阻力。

• • For the task of open-drawer-occlusion with a rebound mechanism, the robot must apply sufficient force to press on the drawer front under visual occlusion, triggering the built-in mechanism that makes the drawer spring open, as shown in Fig. A.8.

• • 对带回弹机构的遮挡开抽屉任务，机器人须在视觉遮挡下对抽屉正面施加足够力，触发内置机构使抽屉弹开，如图 A.8。

Evaluation

We provide the quantitative comparison between our method and the baseline in Table A.3. And specifically:

评估

表 A.3 给出我们的方法与基线的定量对比，具体如下：

• • For the task of wipe-blackboard, success is defined as the robot erasing $90\%$ of the ink marks. Failure occurs if the robot exceeds the time limit without achieving this goal.

• • 擦黑板：成功定义为擦除 $90\%$ 墨迹；超时未达成则失败。

• • For the two tasks of open- and close-cabinet with a rebound mechanism, success is defined as the robot pressing the cabinet door to open (or close) the cabinet and then releasing the surface. Failure occurs if the robot is unable to activate the mechanism or does not release the cabinet door after pressing it.

• • 带回弹机构的开/关柜：成功定义为按下柜门使之打开（或关闭）后松开表面；未能触发机构或按下后未松开关门则失败。

• • For the task of open-drawer-occlusion with a rebound mechanism, success is defined as the robot pressing the drawer front under visual occlusion and then releasing the surface. Failure occurs if the robot is unable to activate the mechanism or does not release the drawer front after pressing it.

• • 带回弹机构的遮挡开抽屉：成功定义为在视觉遮挡下按下抽屉正面后松开；未能触发机构或按下后未松开则失败。

Figure A.9: Real-world force control evaluation.

图 A.9：真实世界力控制评估。

Appendix E Assessing Force Estimation Accuracy Along X and Y Axes

We evaluate the force estimator by randomly selecting five positions and applying forces ranging from -60N to 60N along the X-axis (as in Fig. 4 of the main paper). We additionally evaluate on the Y- and Z-axis within 40N (due to hardware constraints of Unitree-Z1) in Fig. A.9. While sim-to-real discrepancies introduce inaccurate estimations, especially along Y-axis, we argue that the current estimator suffices for the coarse-grained manipulation tasks discussed in this study. Reducing this sim-to-real gap for finer control will be one focus Additionalof our future work.

我们随机选取五个位置，沿 X 轴施加 -60 N 到 60 N 的力以评估力估计器（同正文图 4）。另在硬件限制（Unitree-Z1）下于 Y、Z 轴 40 N 范围内评估，见图 A.9。仿真到现实差距会带来不准确估计（尤其 Y 轴），但我们认为当前估计器足以支撑本文讨论的粗粒度操作任务。缩小该差距以实现更精细控制将是未来工作之一。