AGI Daily
资讯日报词条论文人物新创
返回论文列表

暂无章节

返回论文列表
目录

NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception

收录于 RSS 2026 Outstanding Systems Paper in Memory of Seth Teller Award;Robotics: Science and Systems XXII;DOI 10.15607/rss.2026.xxii.159被引 6 次
打开论文来源

Robotics: Science and Systems 2026
Sydney, Australia, July 13–17, 2026

Robotics: Science and Systems 2026
澳大利亚悉尼,2026年7月13–17日

NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception

NeuralActuator:面向机器人动力学与外力感知的神经执行器建模

Zhiyang Dou1^{1}1, John U. Onyemelukwe1∗^{1*}1∗, Hangxing Zhang1∗^{1*}1∗, Heng Zhang1^{1}1, Minghao Guo1^{1}1, Yunsheng Tian1^{1}1, Michal Piotr Lipiec1^{1}1, Joshua Jacob1^{1}1, Chao Liu1^{1}1, Peter Yichen Chen1^{1}1, Yuri Ivanov2,†^{2,\dagger}2,† and Wojciech Matusik1^{1}1

1^{1}1MIT  2^{2}2Amazon Robotics

∗^{*}∗Research Assistant at MIT CDFG, equal contribution.
†^{\dagger}†The work of this author does not relate to their position at Amazon.

Abstract

摘要

Differentiable simulators have advanced policy learning and model-based control across diverse robotic tasks. To date, actuator dynamics remain underexplored and are a major source of sim-to-real error, especially on low-cost platforms where the linear current–torque model τ=KtI\tau = K_t Iτ=Kt​I breaks down under commanded-target tracking due to friction, hysteresis, backlash, and thermal effects. Beyond forward dynamics, accurate actuator models also support force perception, which is crucial for jointly modeling force and position control in manipulation tasks. We present NeuralActuator, a neural actuator model that jointly predicts (i) torques to capture the full nonlinear and time-varying current–torque relationship on low-cost servos, (ii) external contact forces as well as force detection gates for sensorless force perception, (iii) motor conditions indicating their operating regime. We introduce a twin-arm teleoperation system that collects motor states alongside ground-truth forces, contributing a dataset named Neural Actuation Dataset (NAD). The torque head is trained through differentiable simulation from pose trajectories without torque labels; the force and motor-condition heads use direct supervision from their respective ground truth. A Transformer-based architecture captures temporal dependencies while enabling efficient real-time inference. We validate NeuralActuator across three platforms—a 5-DoF OpenManipulator-X, a 6-DoF SO-101 from LeRobot, and a 7-DoF Franka Emika Panda with on-board joint torque sensors—spanning three actuator families and the 500–500–500–30k+ cost spectrum, and show that it enables accurate dynamics modeling, sensorless force estimation, motor condition estimation, and improved behavior-cloning control when used as a pretrained module. Project Page: https://frank-zy-dou.github.io/projects/NeuralActuator/index.html

可微分仿真已经推动了多样机器人任务中的策略学习与基于模型的控制。直到现在,执行器动力学仍探讨不足,并且是仿真到真机误差的主要来源,尤其是在低成本平台上:由于摩擦、磁滞、齿隙和热效应,线性电流–力矩模型 τ=KtI\tau = K_t Iτ=Kt​I 在指令目标跟踪下会失效。在前向动力学之外,精确的执行器模型也支撑力感知,这对在操作任务中联合建模力控制与位置控制至关重要。我们提出 NeuralActuator,一个神经执行器模型,它同时预测(i)力矩,以刻画低成本舵机上完整的非线性、时变电流–力矩关系;(ii)外部接触力以及用于无传感器力感知的力检测门控;(iii)指示运行状态的电机工况。我们引入一套双臂遥操作系统,在采集电机状态的同时记录力的真值,并贡献名为 Neural Actuation Dataset(NAD)的数据集。力矩头通过可微分仿真、由位姿轨迹训练,不使用力矩标签;力和电机工况头则用来自各自真值的直接监督。基于 Transformer 的架构既能抓住时间依赖,又能做高效的实时推理。我们在三个平台上验证 NeuralActuator——5 自由度 OpenManipulator-X、来自 LeRobot 的 6 自由度 SO-101,以及带板载关节力矩传感器的 7 自由度 Franka Emika Panda——覆盖三类执行器家族和 500 美元至 3 万美元以上的价位区间,并表明它能够做准确的动力学建模、无传感器力估计、电机工况估计,以及在作为预训练模块时改进行为克隆控制。项目主页:https://frank-zy-dou.github.io/projects/NeuralActuator/index.html

Fig. 1: (a) Push–force gauge validation at three end-effector heights. We show NeuralActuator pushing a force gauge under low, middle, and high postures (top to bottom). Left: real-robot execution; middle: corresponding simulated rollout; right: force profiles. The force Mean Absolute Error (MAE) is 0.037 N, 0.015 N, and 0.028 N, respectively. The solid curve shows the predicted external force, and the dashed curve shows the force-gauge ground truth. (b) Estimated forces for downstream tasks. NeuralActuator provides estimated forces that facilitate controller training in Sim-to-Real for object lifting and holding (top) and pick-and-place (bottom) tasks.

Fig. 1: (a) Push–force gauge validation at three end-effector heights. We show NeuralActuator pushing a force gauge under low, middle, and high postures (top to bottom). Left: real-robot execution; middle: corresponding simulated rollout; right: force profiles. The force Mean Absolute Error (MAE) is 0.037 N, 0.015 N, and 0.028 N, respectively. The solid curve shows the predicted external force, and the dashed curve shows the force-gauge ground truth. (b) Estimated forces for downstream tasks. NeuralActuator provides estimated forces that facilitate controller training in Sim-to-Real for object lifting and holding (top) and pick-and-place (bottom) tasks.

图 1:(a)在三种末端高度下用推压力计做验证。我们展示 NeuralActuator 在低、中、高姿态下(自上而下)推压测力计。左:真机执行;中:对应的仿真滚动;右:力曲线。力的平均绝对误差(MAE)分别为 0.037 N、0.015 N 和 0.028 N。实线表示预测的外力,虚线表示测力计真值。(b)下游任务中的估计力。NeuralActuator 提供的估计力,便于在仿真到真机中为物体抬举保持(上)和拾取放置(下)训练控制器。

I. INTRODUCTION

I. 引言

Differentiable simulators [23, 35, 14, 21, 2, 44] enable efficient gradient-based optimization for robot control, with substantial advances in modeling rigid-body dynamics and recent extensions to soft bodies and fluids. However, a crucial and often underexplored component remains: the actuators themselves. For decades, control frameworks have assumed joint torque scales linearly with current, e.g., τ=KtI\tau = K_t Iτ=Kt​I. While this could hold for high-end industrial motors, it degrades substantially on cost-effective, servo-driven platforms [5, 39, 34, 40]. These platforms employ mechanically commutated actuators (e.g., coreless brushed DC servo modules) that prioritize hardware simplicity but introduce significant non-idealities: gearbox friction, hysteresis, backlash, current saturation, and thermal drift distort the actual joint torque; see visualization in Sec. IV-J. These unmeasured, time-varying effects—with inaccessible internal signals—resist analytical modeling, resulting in poor torque tracking, unstable impedance behavior, and degraded sim-to-real transfer.

可微分仿真 [23, 35, 14, 21, 2, 44] 为机器人控制提供了高效的基于梯度的优化,刚体动力学建模已有显著进展,近来也延伸到软体和流体。然而,一个关键且常常探讨不足的部件仍然是:执行器本身。几十年来,控制框架都假定关节力矩与电流成线性关系,例如 τ=KtI\tau = K_t Iτ=Kt​I。这对高端工业电机或许还能成立,但在讲究成本、由舵机驱动的平台上会明显退化 [5, 39, 34, 40]。这类平台采用机械换向执行器(例如空心杯有刷直流舵机模块),硬件更简单,却引入可观的非理想因素:减速箱摩擦、磁滞、齿隙、电流饱和和热漂移会扭曲真实关节力矩;可视化见第 IV-J 节。这些未测量、随时间变化的效应——内部信号又无法读取——难以用解析模型刻画,从而导致力矩跟踪差、阻抗行为不稳,以及仿真到真机迁移变差。

Beyond accurate modeling, a reliable actuator model unlocks force perception without dedicated sensors. Recent studies demonstrate torque-based sensing for e-skin–like contact perception [25], proprioceptive estimation in legged robots [15], and force control in manipulation [29, 18, 10, 51]. Yet these approaches rely on high-end actuators with reliable torque feedback; on low-cost platforms, the corrupted current-to-torque mapping undermines such force inference. Since the medium of interaction between a manipulator and its environment is ultimately force, accurate contact-force prediction makes many tasks tractable without precise geometric modeling of arbitrary or deformable objects.

除了精确建模之外,可靠的执行器模型还能在没有专用传感器时打开力感知。近期研究展示了用于类电子皮肤接触感知的基于力矩的传感 [25]、腿式机器人中的本体感觉估计 [15],以及操作中的力控制 [29, 18, 10, 51]。不过这些方法依赖带有可靠力矩反馈的高端执行器;在低成本平台上,被破坏的电流到力矩映射会削弱这类力推断。既然操作臂与环境交互的媒介归根结底是力,准确预测接触力就可以在不必对任意或可变形物体做精细几何建模的情况下,让许多任务变得可处理。

We introduce NeuralActuator, a data-driven actuation modeling method for low-cost robot platforms that captures actuator behavior and its coupling with robot dynamics. NeuralActuator integrates naturally with a differentiable simulator, enabling end-to-end training via gradient-based supervision from real-robot rollouts. NeuralActuator learns a history-dependent mapping from commanded poses, motor currents, and exogenous operating conditions (e.g., supply voltage, temperature) to joint torques, external forces, and motor health indicators. Prior work that trains actuator models typically relies on torque labels derived from torque sensors [41, 24], but such supervision is unreliable on low-cost actuators without dedicated torque sensing. Instead, we supervise using pose trajectories: forward integration through our differentiable dynamics maps predicted torques to future configurations, making pose an effective surrogate signal. To effectively capture nonlinear, time-varying actuator effects, NeuralActuator adopts a Transformer architecture that processes state-and-command sequences through multi-head self-attention, capturing long-horizon temporal dependencies and inter-joint coupling. It outputs joint-specific torque predictions, external forces, and motor condition estimates in a multi-task manner.

我们提出 NeuralActuator,一种面向低成本机器人平台的数据驱动执行器建模方法,用于刻画执行器行为及其与机器人动力学的耦合。NeuralActuator 可以自然地接入可微分仿真器,从而借助来自真机滚动的基于梯度的监督,做端到端训练。它学习一个依赖历史的映射:由指令位姿、电机电流以及外在运行条件(例如供电电压、温度),到关节力矩、外力和电机健康指示。以往训练执行器模型的工作通常依赖由力矩传感器得到的力矩标签 [41, 24],但在没有专用力矩传感的低成本执行器上,这类监督并不可靠。我们改为用位姿轨迹来监督:通过可微分动力学做前向积分,把预测力矩映射到未来构型,使位姿成为一种有效的替代信号。为了有效抓住非线性、时变的执行器效应,NeuralActuator 采用 Transformer 架构,用多头自注意力处理状态与指令序列,刻画长时程时间依赖和关节间耦合。它以多任务方式输出各关节的力矩预测、外力以及电机工况估计。

To train and benchmark NeuralActuator, we introduce a real-robot dataset capturing actuation, motion, and force signals. We build a twin-arm leader–follower teleoperation system where an operator kinesthetically drives a leader arm to generate diverse trajectories, while the follower executes joint-space commands under closed-loop control. For each trajectory, we log (i) robot states (joint positions, velocities, commanded targets), (ii) actuator telemetry (motor currents, temperature, supply voltage), and (iii) external forces from force sensors. The dataset spans free-space motions and contact-rich manipulation (pushing, pulling, lifting, known-weight loads), covering actuator effects induced by friction, load changes, and intermittent contacts. We release it as the Neural Actuation Dataset (NAD), providing ground-truth data for model training and benchmarking.

为了训练和评测 NeuralActuator,我们引入一个捕捉驱动、运动和力信号的真机数据集。我们搭建了一套双臂主从遥操作系统:操作者通过动觉驱动主臂以生成多样轨迹,从臂则在闭环控制下执行关节空间指令。对每条轨迹,我们记录(i)机器人状态(关节位置、速度、指令目标),(ii)执行器遥测(电机电流、温度、供电电压),以及(iii)来自力传感器的外力。数据集覆盖自由空间运动和富接触操作(推、拉、抬、已知重量载荷),涵盖由摩擦、载荷变化和间歇接触引起的执行器效应。我们将其发布为 Neural Actuation Dataset(NAD),为模型训练和评测提供带真值的数据。

We conduct several experiments across three platforms—a cost-effective, servo-driven 5-DoF OpenManipulator-X (four revolute joints and a 1-DoF gripper), a 6-DoF SO-101 low-cost arm built on the LeRobot stack [5], and a 7-DoF industrial arm, i.e., Franka Emika Panda robot, with on-board joint torque sensors (Sec. IV-I)—spanning three actuator families and the 500–500–500–30k+ cost spectrum, to evaluate modeling fidelity, online adaptation, throughput, and sim-to-real transfer. On held-out rollouts, NeuralActuator achieves low rollout error (Sec. IV-A), accurate external force estimation (Sec. IV-B) as well as motor condition estimation (Sec. IV-D). The simulator attains high runtime speed and throughput with a modest memory footprint, making it practical for both training and on-robot deployment (Sec. IV-E). We also showcase rapid online adaptation from only few new trajectories (Appendix I). For sim-to-real control, NeuralActuator enables efficient imitation learning for object manipulation. We show that learning actuator behavior together with external force perception improves task success rates (Sec. IV-F). Finally, we demonstrate that combining differentiable simulation with multiple rendering modalities enables visual supervision for NeuralActuator training (Sec. IV-G). We summarize our contributions below:

我们在三个平台上做了多项实验——一台讲求成本、由舵机驱动的 5 自由度 OpenManipulator-X(四个转动关节和 1 自由度夹爪),一台基于 LeRobot 技术栈构建的 6 自由度低成本臂 SO-101 [5],以及一台带板载关节力矩传感器的 7 自由度工业臂,即 Franka Emika Panda(第 IV-I 节)——覆盖三类执行器家族和 500 美元至 3 万美元以上的价位区间,以评估建模保真度、在线适配、吞吐量和仿真到真机迁移。在留出的滚动上,NeuralActuator 达到较低的滚动误差(第 IV-A 节)、准确的外力估计(第 IV-B 节)以及电机工况估计(第 IV-D 节)。仿真器具有较高的运行速度和吞吐量,内存占用适中,因而对训练和机上部署都实用(第 IV-E 节)。我们也展示了仅用少量新轨迹即可做快速在线适配(附录 I)。对仿真到真机控制,NeuralActuator 使物体操作上的模仿学习更为高效。我们表明,把执行器行为与外力感知一起学习,可以提高任务成功率(第 IV-F 节)。最后,我们证明把可微分仿真与多种渲染模态结合,可以为 NeuralActuator 的训练提供视觉监督(第 IV-G 节)。贡献概括如下:

https://emanual.robotis.com/docs/en/platform/openmanipulator_x/overview/

• A data-driven, differentiable actuator model named NeuralActuator that captures actuator behavior and its coupling with robot dynamics via neural torque prediction, with additional outputs for external force prediction and contact detection—enabling force perception without dedicated sensors—and a torque-supervision-free training scheme that leverages differentiable physics to avoid unreliable current–torque labels on low-cost hardware.

• 一个名为 NeuralActuator 的数据驱动、可微分执行器模型:通过神经力矩预测刻画执行器行为及其与机器人动力学的耦合,并额外输出外力预测和接触检测——从而在没有专用传感器时实现力感知——以及一套无需力矩监督的训练方案,借助可微分物理,避开低成本硬件上不可靠的电流–力矩标签。

• A twin-arm teleoperation system for efficient data collection on low-cost arms, time-synchronizing robot proprioception, commanded targets, and actuator-side telemetry.

• 一套用于在低成本臂上高效采集数据的双臂遥操作系统,将机器人本体感觉、指令目标和执行器侧遥测在时间上对齐。

• Neural Actuation Dataset (NAD): a real-robot dataset with time-aligned robot states, actuator telemetry, and end-effector force measurements from interactions measured by external force sensors, enabling training and benchmarking for low-cost actuator modeling.

• Neural Actuation Dataset(NAD):一个真机数据集,包含时间对齐的机器人状态、执行器遥测,以及由外部力传感器测得的末端交互力,用于低成本执行器建模的训练与评测。

• Comprehensive experiments demonstrate that NeuralActuator enables accurate actuator dynamics modeling and sensorless force as well as motor-condition estimation, and improves performance on downstream manipulation tasks, while remaining computationally efficient.

• 全面的实验表明,NeuralActuator 能够做准确的执行器动力学建模以及无传感器的力和电机工况估计,并改进下游操作任务的表现,同时保持计算高效。

II. RELATED WORK

II. 相关工作

A. Learning Neural-Augmented Simulation

A. 学习神经增强仿真

To close the sim-to-real gap, prior works augment analytical simulators with learned residual models: the approach in [1] augments analytical simulators with a stochastic RNN that captures residual dynamics to match the simulation with reality. In [26], an RNN is trained to learn the mismatch in joint positions and velocities between simulated and real robot trajectories for refining the simulator output. TossingBot [50] further combines an analytic ballistic prior with a neural residual learned from visual features. NeuralSim [19] embeds neural modules in differentiable simulators to capture unmodeled dynamics and jointly optimizes them with physical parameters on real data; at the component level, Serifi et al. [42] uses a Transformer to predict residual state corrections for actuators and rigid bodies from simulated states and constraint forces. Recently, [48] presents NeRD, which learns robot-specific dynamics models for predicting future states for articulated rigid bodies under contact constraints. [24] trains a network on the physical system via self-supervision and uses it in the simulation loop to model each joint of ANYmal; the network maps state histories and position targets to joint torques for simulation. [41] introduces a system identification method that combines a physics model with a neural network to learn residual dynamics, enabling domain-consistent dynamics modeling. In their setting, joint torques can be accurately estimated from motor-current measurements. However, these works [24, 41] rely on ground-truth torque labels from dedicated sensors for supervision, which are often unavailable on low-cost platforms. In contrast, NeuralActuator is torque-label-free: it is supervised through differentiable simulation using pose trajectories, without assuming reliable current–torque calibration. Concurrently, [13] learns residual corrective torques via RL from tracking errors; NeuralActuator instead directly predicts forward torques (raw current readings are too noisy on low-cost platforms; see Fig. 9) and additionally estimates external contact forces, trained end-to-end via differentiable simulation with pose-only supervision.

为了缩小仿真到真机的差距,已有工作用学到的残差模型来增强解析仿真器:[1] 中的方法用随机 RNN 增强解析仿真器,以刻画残差动力学,使仿真与现实对齐。[26] 训练一个 RNN 来学习仿真与真机轨迹在关节位置和速度上的失配,从而改进仿真器输出。TossingBot [50] 进一步把解析的弹道先验与从视觉特征学到的神经残差结合起来。NeuralSim [19] 把神经模块嵌进可微分仿真器,以刻画未建模动力学,并在真实数据上与物理参数联合优化;在部件层面,Serifi 等人 [42] 用 Transformer 根据仿真状态和约束力,预测执行器与刚体的残差状态修正。近来,[48] 提出 NeRD,学习机器人特定的动力学模型,用于在接触约束下预测铰接刚体的未来状态。[24] 通过自监督在物理系统上训练网络,并将其放进仿真循环,为 ANYmal 的每个关节建模;该网络把状态历史和位置目标映射为用于仿真的关节力矩。[41] 提出一种系统辨识方法,把物理模型与神经网络结合起来学习残差动力学,从而实现与域一致的动力学建模。在他们的设定中,关节力矩可以由电机电流测量准确地估计。然而,这些工作 [24, 41] 依赖来自专用传感器的力矩真值做监督,而这在低成本平台上往往得不到。相比之下,NeuralActuator 不需要力矩标签:它通过可微分仿真、用位姿轨迹来监督,并不假定电流–力矩标定可靠。与此同时,[13] 通过强化学习,从跟踪误差学习残差修正力矩;NeuralActuator 则直接预测前向力矩(在低成本平台上,原始电流读数噪声过大;见图 9),并额外估计外部接触力,在仅有位姿监督的可微分仿真中端到端训练。

a) Learning for Actuator: Existing works mostly target electronically commutated drives (e.g., PMSMs and BLDCs), learning electromagnetic torque from electrical inputs [49, 32, 46, 43]. These approaches assume high-performance, electronically commutated servos. Previous works also cover behaviors of hydraulics [27] and twisted-string actuators [30]. Unlike these efforts, NeuralActuator targets low-cost coreless DC servo actuators, where friction, backlash, saturation, and thermal drift are more pronounced and can corrupt the effective current–torque relationship.

a)面向执行器的学习:已有工作大多针对电子换向驱动(例如永磁同步电机和无刷直流电机),从电输入学习电磁力矩 [49, 32, 46, 43]。这些方法假定的是高性能、电子换向的伺服。以往工作也覆盖液压 [27] 和扭弦执行器 [30] 的行为。与这些努力不同,NeuralActuator 面向低成本空心杯直流舵机执行器,其中摩擦、齿隙、饱和和热漂移更为显著,并可能破坏有效的电流–力矩关系。

B. Force Estimation and Proprioceptive Sensing

B. 力估计与本体感觉传感

a) Torque- and effort-based contact inference: A long-standing line of work reconstructs external joint torques from model discrepancies and uses them for collision detection and safe reaction [11, 12, 16]. Building on reliable effort feedback, recent systems further elevate this idea to richer “intrinsic” contact perception (e.g., e-skin-like touch without tactile skins) [25], whole-robot touch sensing without dedicated tactile sensors [15], and learning-based controllers that couple force and motion in manipulation and loco-manipulation [29, 51]. These approaches highlight the growing role of actuation as an implicit sensing channel, but they often rely on high-quality torque measurements or well-calibrated current-to-torque mappings.

a)基于力矩和努力的接触推断:一条长期的研究路线从模型偏差重构外部关节力矩,并用于碰撞检测与安全反应 [11, 12, 16]。在可靠的努力反馈之上,近期系统进一步把这一想法提升为更丰富的“内在”接触感知(例如,没有触觉皮肤的类电子皮肤触摸)[25]、无需专用触觉传感器的整机触摸传感 [15],以及在操作和运动操作中把力与运动耦合起来的基于学习的控制器 [29, 51]。这些方法凸显了驱动作为隐式传感通道的日益重要的角色,但它们往往依赖高质量的力矩测量,或经过良好标定的电流到力矩映射。

b) External force estimation on robot arms: For manipulators, complementary literature estimates external forces and contacts by explicitly modeling the robot dynamics and attributing residual terms to interaction wrenches. Representative examples include virtual force sensing and contact localization along the robot structure [36, 37], as well as observer and filtering formulations that improve robustness under friction and model uncertainty [22, 33, 17, 31]. Recently, learning has also been used to directly map joint-level proprioception and measured joint torques from sensors to end-effector wrenches in static or quasi-static regimes [38]. Chen et al. [8] leverage differentiable simulation to infer external object properties solely from proprioceptive signals. Despite this progress, most proprioceptive force-estimation methods assume accurate torque feedback, while the setting of low-cost, servo-driven arms remains largely unexplored. A summary can be found in Tab. A15 in Appendix.

b)机械臂上的外力估计:对操作臂而言,互补的文献通过显式建模机器人动力学、并把残差项归到交互力旋量,来估计外力和接触。代表性例子包括沿机器人结构的虚拟力传感与接触定位 [36, 37],以及在摩擦和模型不确定下提高鲁棒性的观测器与滤波表述 [22, 33, 17, 31]。近来,学习也被用来在静态或准静态工况下,把关节级本体感觉和传感器测得的关节力矩直接映射到末端力旋量 [38]。Chen 等人 [8] 借助可微分仿真,仅从本体感觉信号推断外部物体属性。尽管有这些进展,大多数基于本体感觉的力估计方法仍假定力矩反馈是准确的,而低成本、舵机驱动手臂的设定在很大程度上仍未被探讨。摘要可见附录中的表 A15。

III. METHOD

III. 方法

A. Problem Formulation

A. 问题表述

Consider a robotic manipulator with nnn joints, where each joint is driven by a servo motor. The relationship between commanded pose, current, and output torque exhibits dependencies on multiple factors:

考虑一台有 nnn 个关节的机器人操作臂,每个关节由一台伺服电机驱动。指令位姿、电流和输出力矩之间的关系依赖于多种因素:

τjreal(t)=fτ ⁣(qjcmd(t), ij(t), qj(t), q˙j(t), Tj(t), Ht)(1)\tau_j^{\mathrm{real}}(t)=f_\tau\!\left(q_j^{\mathrm{cmd}}(t),\,i_j(t),\,q_j(t),\,\dot{q}_j(t),\,T_j(t),\,\mathcal{H}_t\right)\tag{1}τjreal​(t)=fτ​(qjcmd​(t),ij​(t),qj​(t),q˙​j​(t),Tj​(t),Ht​)(1)

where qjcmd(t)q_j^{\mathrm{cmd}}(t)qjcmd​(t) is the commanded (goal) position, qj(t)q_j(t)qj​(t) and q˙j(t)\dot{q}_j(t)q˙​j​(t) are joint position and velocity, Tj(t)T_j(t)Tj​(t) represents thermal state, and Ht={(qjcmd(τ),ij(τ),qj(τ),q˙j(τ))}τ=t−wt\mathcal{H}_t=\{(q_j^{\mathrm{cmd}}(\tau),i_j(\tau),q_j(\tau),\dot{q}_j(\tau))\}_{\tau=t-w}^{t}Ht​={(qjcmd​(τ),ij​(τ),qj​(τ),q˙​j​(τ))}τ=t−wt​ denotes the historical window of states over time horizon www.

其中 qjcmd(t)q_j^{\mathrm{cmd}}(t)qjcmd​(t) 是指令(目标)位置,qj(t)q_j(t)qj​(t) 和 q˙j(t)\dot{q}_j(t)q˙​j​(t) 是关节位置与速度,Tj(t)T_j(t)Tj​(t) 表示热状态,而 Ht={(qjcmd(τ),ij(τ),qj(τ),q˙j(τ))}τ=t−wt\mathcal{H}_t=\{(q_j^{\mathrm{cmd}}(\tau),i_j(\tau),q_j(\tau),\dot{q}_j(\tau))\}_{\tau=t-w}^{t}Ht​={(qjcmd​(τ),ij​(τ),qj​(τ),q˙​j​(τ))}τ=t−wt​ 表示时间范围 www 上的状态历史窗口。

Beyond torque prediction, we estimate external contact forces and motor conditions from the same input features:

在力矩预测之外,我们从同一组输入特征估计外部接触力和电机工况:

fext(t)=fforce ⁣(qcmd(t), i(t), q(t), q˙(t), T(t), Ht)∈R3(2)\mathbf{f}^{\mathrm{ext}}(t)=f_{\mathrm{force}}\!\left(q^{\mathrm{cmd}}(t),\,i(t),\,q(t),\,\dot{q}(t),\,T(t),\,\mathcal{H}_t\right)\in\mathbb{R}^{3}\tag{2}fext(t)=fforce​(qcmd(t),i(t),q(t),q˙​(t),T(t),Ht​)∈R3(2)

c(t)=fcond ⁣(qcmd(t), i(t), q(t), q˙(t), T(t), Ht)∈[0,1](3)c(t)=f_{\mathrm{cond}}\!\left(q^{\mathrm{cmd}}(t),\,i(t),\,q(t),\,\dot{q}(t),\,T(t),\,\mathcal{H}_t\right)\in[0,1]\tag{3}c(t)=fcond​(qcmd(t),i(t),q(t),q˙​(t),T(t),Ht​)∈[0,1](3)

where fext(t)\mathbf{f}^{\mathrm{ext}}(t)fext(t) is the predicted end-effector contact force and c(t)c(t)c(t) is a motor health score (probability of normal operation, with c=1c=1c=1 for healthy and c=0c=0c=0 for degraded motors). A physical interpretation of these outputs in terms of the rigid-body equations of motion is given in Appendix B.

其中 fext(t)\mathbf{f}^{\mathrm{ext}}(t)fext(t) 是预测的末端接触力,c(t)c(t)c(t) 是电机健康分数(正常运行的概率,c=1c=1c=1 表示健康,c=0c=0c=0 表示退化电机)。这些输出在刚体运动方程中的物理解释见附录 B。

Fig. 2: Neural Actuation Dataset (NAD) data collection. (a) Overview of the twin-arm leader–follower system with an external camera. (b) An oprator kinesthetically drives the leader to generate diverse motions, while the follower mirrors the motion and records synchronized actuator currents, robot states, and end-effector forces when interacting with a weight.

Fig. 2: Neural Actuation Dataset (NAD) data collection. (a) Overview of the twin-arm leader–follower system with an external camera. (b) An oprator kinesthetically drives the leader to generate diverse motions, while the follower mirrors the motion and records synchronized actuator currents, robot states, and end-effector forces when interacting with a weight.

图 2:Neural Actuation Dataset(NAD)的数据采集。(a)带外部相机的双臂主从系统概览。(b)操作者通过动觉驱动主臂以生成多样运动,从臂则镜像该运动,并在与重物交互时记录同步的执行器电流、机器人状态和末端力。

B. Neural Actuation Dataset (NAD)

B. Neural Actuation Dataset(NAD)

To learn a model for Actuation Modeling, we first collect paired motor-state and force data using a twin-arm teleoperation system built on two identical OpenManipulator-X robots [40]. The system consists of a leader arm kinesthetically operated by a human demonstrator and a follower arm that mirrors the leader’s motion while interacting with the environment (Fig. 2).

为了学习执行器建模的模型,我们首先用一套搭建在两台相同 OpenManipulator-X 机器人上的双臂遥操作系统,采集成对的电机状态与力数据 [40]。该系统包括由人类示范者通过动觉操作的主臂,以及在与环境交互时镜像主臂运动的从臂(图 2)。

Note that in this system, the commanded target joint positions sent to the follower’s actuators are set to the leader’s instantaneous joint states. Both arms are equipped with Dynamixel XM430-W350 servos, which provide synchronized actuator-level measurements, including:

需要说明的是,在这套系统中,送往从臂执行器的指令目标关节位置被设为主臂的瞬时关节状态。两臂都配备 Dynamixel XM430-W350 舵机,可提供同步的执行器级测量,包括:

• Joint positions qj(t)q_j(t)qj​(t) and velocities q˙j(t)\dot{q}_j(t)q˙​j​(t)

• 关节位置 qj(t)q_j(t)qj​(t) 和速度 q˙j(t)\dot{q}_j(t)q˙​j​(t)

• Motor currents ij(t)i_j(t)ij​(t), PWM signals, and bus voltages

• 电机电流 ij(t)i_j(t)ij​(t)、PWM 信号和总线电压

• Coil temperatures Tj(t)T_j(t)Tj​(t)

• 线圈温度 Tj(t)T_j(t)Tj​(t)

For a subset of interaction trajectories, the follower arm is further instrumented with a wrist-mounted 3-axis force sensor, which provides ground-truth end-effector contact force measurements fgt(t)∈R3\mathbf{f}^{\mathrm{gt}}(t)\in\mathbb{R}^{3}fgt(t)∈R3 during execution.

对一部分交互轨迹,从臂还加装了腕部三轴力传感器,在执行过程中提供末端接触力的真值测量 fgt(t)∈R3\mathbf{f}^{\mathrm{gt}}(t)\in\mathbb{R}^{3}fgt(t)∈R3。

Fig. 3: The pipeline of NeuralActuator. At each time step $t$, NeuralActuator takes the commanded pose $q^{\mathrm{cmd}}(t)$, motor current $i(t)$, actuator-side telemetry, and tracking feedback $e(t)$, together with a history buffer $\mathcal{H}_t$ that summarizes recent robot states (e.g., joint poses and velocities). A Transformer encoder ($\times N$ blocks) maps these inputs to three outputs: (i) the directly predicted joint torque $\tau$, (ii) a predicted external force $\hat{f}$, and (iii) a motor-condition signal $c$ indicating the operating regime (e.g., normal vs. anomalous). The predicted $\tau$ and $\hat{f}$ are applied in a differentiable simulator (Differentiable Simulation) to advance the system to time step $t+1$ and update the history buffer. Gradients through Diff Sim enable end-to-end training by supervising simulated state transitions against real-robot rollouts. Right: Transformer block details with input projection $W_{\mathrm{in}}$, positional encoding, multi-head attention, a gated skip connection, SiLU feed-forward network, pooling, and lightweight MLP heads for $\tau$, $\hat{f}$, and $c$.

Fig. 3: The pipeline of NeuralActuator. At each time step ttt, NeuralActuator takes the commanded pose qcmd(t)q^{\mathrm{cmd}}(t)qcmd(t), motor current i(t)i(t)i(t), actuator-side telemetry, and tracking feedback e(t)e(t)e(t), together with a history buffer Ht\mathcal{H}_tHt​ that summarizes recent robot states (e.g., joint poses and velocities). A Transformer encoder (×N\times N×N blocks) maps these inputs to three outputs: (i) the directly predicted joint torque τ\tauτ, (ii) a predicted external force f^\hat{f}f^​, and (iii) a motor-condition signal ccc indicating the operating regime (e.g., normal vs. anomalous). The predicted τ\tauτ and f^\hat{f}f^​ are applied in a differentiable simulator (Differentiable Simulation) to advance the system to time step t+1t+1t+1 and update the history buffer. Gradients through Diff Sim enable end-to-end training by supervising simulated state transitions against real-robot rollouts. Right: Transformer block details with input projection WinW_{\mathrm{in}}Win​, positional encoding, multi-head attention, a gated skip connection, SiLU feed-forward network, pooling, and lightweight MLP heads for τ\tauτ, f^\hat{f}f^​, and ccc.

图 3:NeuralActuator 的流程。在每个时间步 ttt,NeuralActuator 接收指令位姿 qcmd(t)q^{\mathrm{cmd}}(t)qcmd(t)、电机电流 i(t)i(t)i(t)、执行器侧遥测和跟踪反馈 e(t)e(t)e(t),以及概括近期机器人状态(例如关节位姿和速度)的历史缓冲 Ht\mathcal{H}_tHt​。Transformer 编码器(×N\times N×N 个块)把这些输入映射为三个输出:(i)直接预测的关节力矩 τ\tauτ;(ii)预测外力 f^\hat{f}f^​;(iii)指示运行状态(例如正常与异常)的电机工况信号 ccc。预测的 τ\tauτ 和 f^\hat{f}f^​ 被送入可微分仿真器(Differentiable Simulation),把系统推进到时间步 t+1t+1t+1 并更新历史缓冲。经过 Diff Sim 的梯度使端到端训练成为可能:用仿真的状态转移去对真机滚动。右:Transformer 块的细节,包括输入投影 WinW_{\mathrm{in}}Win​、位置编码、多头注意力、带门控的跳跃连接、SiLU 前馈网络、池化,以及用于 τ\tauτ、f^\hat{f}f^​ 和 ccc 的轻量 MLP 头。

All data streams are temporally synchronized and logged per trajectory. For each trajectory of length TTT, we record time-aligned tuples containing: (i) robot states (e.g., joint positions, velocities, and commanded targets), (ii) motor currents and actuator-side telemetry, and (iii) end-effector external forces, which provides supervision for training a neural actuation model. To elicit diverse actuation and interaction regimes, we collect three subsets:

所有数据流都按时间同步,并按轨迹记录。对每条长度为 TTT 的轨迹,我们记录时间对齐的元组,其中包括:(i)机器人状态(例如关节位置、速度和指令目标),(ii)电机电流与执行器侧遥测,以及(iii)末端外力,为训练神经执行器模型提供监督。为引出多样的驱动与交互工况,我们采集三个子集:

Free motion (no external force). This subset characterizes intrinsic actuator dynamics and provides force-free baselines. We record a diverse set of trajectories (119,933 frames, avg. 20.49 s), including: (a) Clockwise and counterclockwise circular end-effector motions; (b) individual joint sweeps covering approximately 90% of feasible ranges; and (c) whole-arm primitives such as lean and extend. These trajectories contain no intentional external contact.

自由运动(无外力)。这一子集刻画执行器的固有动力学,并提供无力的基线。我们记录了一组多样轨迹(119,933 帧,平均 20.49 秒),包括:(a)顺时针和逆时针的末端圆周运动;(b)覆盖约 90% 可行范围的单关节扫描;(c)如倾斜和伸展这类整臂原语。这些轨迹不包含有意的外部接触。

Force-labeled interactions. This subset provides explicit force supervision via two modalities. (a) Known-weight loads. The robot manipulates objects of known mass (m∈{200,300,400,500}m\in\{200,300,400,500\}m∈{200,300,400,500} g). During loaded/grasped intervals, the static payload yields a base-frame gravity vector fgravgt=mgbase∈R3\mathbf{f}^{\mathrm{gt}}_{\mathrm{grav}}=m\mathbf{g}_{\mathrm{base}}\in\mathbb{R}^{3}fgravgt​=mgbase​∈R3 (with gbase=[0,0,−g]⊤\mathbf{g}_{\mathrm{base}}=[0,0,-g]^{\top}gbase​=[0,0,−g]⊤), which we use as the force-magnitude and vertical-component label during those intervals only; dynamic terms and gripping friction are not labeled here. Tasks include go up and stay still (persistent load) and pick and place (intermittent load during grasping), totaling 88,611 frames with an average duration of 21.63 s. (b) Axis-aligned sensor interactions. Operators apply controlled push and pull forces along the force sensor’s principal axes (±X\pm X±X, ±Y\pm Y±Y, ±Z\pm Z±Z). For each force direction, we also record matched no-interaction trajectories that follow the same motion patterns but are executed without physical contact. The measured wrench is transformed into the robot base coordinate to capture full 3D force components arising from both dominant-axis interactions and kinematic coupling (74,020 frames, avg. 10.50 s).

带力标注的交互。这一子集通过两种方式提供显式的力监督。(a)已知重量载荷。机器人操作已知质量的物体(m∈{200,300,400,500}m\in\{200,300,400,500\}m∈{200,300,400,500} g)。在加载/抓取区间内,静态载荷在基座坐标系中产生重力向量 fgravgt=mgbase∈R3\mathbf{f}^{\mathrm{gt}}_{\mathrm{grav}}=m\mathbf{g}_{\mathrm{base}}\in\mathbb{R}^{3}fgravgt​=mgbase​∈R3(其中 gbase=[0,0,−g]⊤\mathbf{g}_{\mathrm{base}}=[0,0,-g]^{\top}gbase​=[0,0,−g]⊤),我们仅在这些区间把它用作力大小和垂直分量的标签;动态项和抓取摩擦在此不标注。任务包括升起并保持静止(持续载荷)以及拾取放置(抓取期间的间歇载荷),共 88,611 帧,平均时长 21.63 秒。(b)沿轴的传感器交互。操作者沿力传感器的主轴(±X\pm X±X、±Y\pm Y±Y、±Z\pm Z±Z)施加受控的推力和拉力。对每个力方向,我们还记录匹配的无交互轨迹,运动模式相同,但不发生物理接触。测得的力旋量被变换到机器人基座坐标系,以捕捉主轴交互和运动学耦合带来的完整三维力分量(74,020 帧,平均 10.50 秒)。

Fig. 4: Data verification in NAD. We time-synchronize the real-robot video (left) with the logged trajectory and verify alignment in a GUI (right), where the red arrow visualizes the estimated external force.

Fig. 4: Data verification in NAD. We time-synchronize the real-robot video (left) with the logged trajectory and verify alignment in a GUI (right), where the red arrow visualizes the estimated external force.

图 4:NAD 中的数据核验。我们把真机视频(左)与记录的轨迹在时间上对齐,并在图形界面中核验对准(右),其中红色箭头可视化估计的外力。

We further collect additional data for the cross-platform evaluation in Sec. IV-I, using both SO-101 logged via the LeRobot stack [5] and a 7-DoF Franka Emika Panda industrial arm, whose on-board torque sensors provide ground-truth joint torques for direct validation.

我们还为第 IV-I 节的跨平台评估采集了额外数据,既包括经 LeRobot 技术栈记录的 SO-101 [5],也包括 7 自由度 Franka Emika Panda 工业臂,其板载力矩传感器为直接验证提供关节力矩真值。

Manipulation under degraded motor conditions. In this task, we mechanically constrain Joint 3 using rubber bands, reducing its effective range and torque capacity; see the inset. Under this asymmetric constraint, we collect pick and place trajectories with and without a 200 g payload (49,898 frames, avg. 21.20 s), enabling disentanglement of contact-induced and motor-induced force deviations.

退化电机工况下的操作。在这一任务中,我们用橡皮筋从机械上约束关节 3,缩小其有效行程和力矩能力;见插图。在这种不对称约束下,我们采集带或不带 200 g 载荷的拾取放置轨迹(49,898 帧,平均 21.20 秒),以便把接触引起的力偏差和电机引起的力偏差分开。

TABLE I: Dataset composition summary. See Appendix A for a detailed task-level breakdown.

Component Description Duration (min)
Free motion No external force ∼\sim∼34.15
Force labeled Known weights or force ∼\sim∼46.24
Motor condition Degraded actuation ∼\sim∼14.13
Total ∼\sim∼94.52

表 I:数据集组成摘要。任务级的详细拆分见附录 A。

组成 说明 时长(分钟)
自由运动 无外力 ∼\sim∼34.15
带力标注 已知重量或力 ∼\sim∼46.24
电机工况 退化驱动 ∼\sim∼14.13
合计 ∼\sim∼94.52

Our dataset is split into training, validation, and test sets with a ratio of 8:1:1, which supports (i) quantitative evaluation against known ground-truth forces, (ii) qualitative analysis of force prediction during contact-rich manipulation, and (iii) robust modeling of contact-free actuation dynamics for improved generalization. All the data in NAD are calibrated and human-verified as shown in Fig. 4. See more details in Appendix A.

我们的数据集按 8:1:1 划分为训练、验证和测试集,以支持(i)相对已知力真值的定量评估,(ii)富接触操作中力预测的定性分析,以及(iii)对无接触驱动动力学的稳健建模,从而改进泛化。NAD 中的全部数据都经过标定和人工核验,如图 4 所示。更多细节见附录 A。

C. NeuralActuator

To capture temporal dependencies and nonlinear actuator dynamics, we employ a Transformer encoder architecture that processes historical state sequences and generates torque corrections. Given a sequence length LLL and hidden dimension ddd, our model consists of:

为了刻画时间依赖和非线性执行器动力学,我们采用 Transformer 编码器架构,处理历史状态序列并生成力矩修正。给定序列长度 LLL 和隐维度 ddd,模型组成如下:

In NeuralActuator, the input sequence X∈RL×nf\mathbf{X}\in\mathbb{R}^{L\times n_f}X∈RL×nf​ stacks per-timestep feature vectors xt∈Rnf\mathbf{x}_t\in\mathbb{R}^{n_f}xt​∈Rnf​ over a history window of L=H+1L=H+1L=H+1 timesteps. Each xt\mathbf{x}_txt​ aggregates three families of signals—(i) commanded targets (qcmdq^{\mathrm{cmd}}qcmd, acmda^{\mathrm{cmd}}acmd), (ii) proprioception (qqq, q˙\dot{q}q˙​, aaa), and (iii) actuator telemetry (iii, VVV, TTT)—together with tracking-error features (e=qcmd−qe=q^{\mathrm{cmd}}-qe=qcmd−q, ea=acmd−ae_a=a^{\mathrm{cmd}}-aea​=acmd−a). All features are normalized before being fed to the network. Note that for grippers, we derive and validate the mapping from Motor 5 rotation to gripper aperture for simulation and training; see Appendix G.

在 NeuralActuator 中,输入序列 X∈RL×nf\mathbf{X}\in\mathbb{R}^{L\times n_f}X∈RL×nf​ 把逐步的特征向量 xt∈Rnf\mathbf{x}_t\in\mathbb{R}^{n_f}xt​∈Rnf​ 在长度为 L=H+1L=H+1L=H+1 的历史窗口上堆叠起来。每个 xt\mathbf{x}_txt​ 聚合三类信号——(i)指令目标(qcmdq^{\mathrm{cmd}}qcmd,acmda^{\mathrm{cmd}}acmd),(ii)本体感觉(qqq,q˙\dot{q}q˙​,aaa),以及(iii)执行器遥测(iii,VVV,TTT)——并加上跟踪误差特征(e=qcmd−qe=q^{\mathrm{cmd}}-qe=qcmd−q,ea=acmd−ae_a=a^{\mathrm{cmd}}-aea​=acmd−a)。所有特征在送入网络前都做归一化。需要说明的是,对夹爪,我们推导并验证了从电机 5 的转角到夹爪开度的映射,用于仿真和训练;见附录 G。

We project the input sequence to the hidden dimension via Henc(0)=XWin\mathbf{H}_{\mathrm{enc}}^{(0)}=\mathbf{X}\mathbf{W}_{\mathrm{in}}Henc(0)​=XWin​, where Win∈Rnf×d\mathbf{W}_{\mathrm{in}}\in\mathbb{R}^{n_f\times d}Win​∈Rnf​×d is the input projection matrix.

我们通过 Henc(0)=XWin\mathbf{H}_{\mathrm{enc}}^{(0)}=\mathbf{X}\mathbf{W}_{\mathrm{in}}Henc(0)​=XWin​ 把输入序列投影到隐维度,其中 Win∈Rnf×d\mathbf{W}_{\mathrm{in}}\in\mathbb{R}^{n_f\times d}Win​∈Rnf​×d 是输入投影矩阵。

We stack NencN_{\mathrm{enc}}Nenc​ Transformer encoder layers to extract contextual representations from the input sequence:

我们堆叠 NencN_{\mathrm{enc}}Nenc​ 层 Transformer 编码器,以从输入序列中提取上下文表示:

Zenc(l)=LayerNorm ⁣(Henc(l−1)+γa(l)⋅MHA ⁣(Henc(l−1)))(4)\mathbf{Z}_{\mathrm{enc}}^{(l)}=\mathrm{LayerNorm}\!\left(\mathbf{H}_{\mathrm{enc}}^{(l-1)}+\gamma_a^{(l)}\cdot\mathrm{MHA}\!\left(\mathbf{H}_{\mathrm{enc}}^{(l-1)}\right)\right)\tag{4}Zenc(l)​=LayerNorm(Henc(l−1)​+γa(l)​⋅MHA(Henc(l−1)​))(4)

Henc(l)=LayerNorm ⁣(Zenc(l)+γf(l)⋅FFN ⁣(Zenc(l)))(5)\mathbf{H}_{\mathrm{enc}}^{(l)}=\mathrm{LayerNorm}\!\left(\mathbf{Z}_{\mathrm{enc}}^{(l)}+\gamma_f^{(l)}\cdot\mathrm{FFN}\!\left(\mathbf{Z}_{\mathrm{enc}}^{(l)}\right)\right)\tag{5}Henc(l)​=LayerNorm(Zenc(l)​+γf(l)​⋅FFN(Zenc(l)​))(5)

where MHA denotes multi-head self-attention, FFN is a feed-forward network with SiLU activation, and γa(l),γf(l)∈R\gamma_a^{(l)},\gamma_f^{(l)}\in\mathbb{R}γa(l)​,γf(l)​∈R are learned scalar gates for the attention and FFN skip connections. After the encoder layers, we aggregate the sequence representation via temporal pooling, hpool=Pool(Henc(Nenc))∈Rd\mathbf{h}_{\mathrm{pool}}=\mathrm{Pool}(\mathbf{H}_{\mathrm{enc}}^{(N_{\mathrm{enc}})})\in\mathbb{R}^{d}hpool​=Pool(Henc(Nenc​)​)∈Rd, which is passed to the output heads. The network produces three types of outputs from the pooled representation hpool\mathbf{h}_{\mathrm{pool}}hpool​:

其中 MHA 表示多头自注意力,FFN 是带 SiLU 激活的前馈网络,γa(l),γf(l)∈R\gamma_a^{(l)},\gamma_f^{(l)}\in\mathbb{R}γa(l)​,γf(l)​∈R 是注意力与 FFN 跳跃连接上学习到的标量门。编码器层之后,我们用时间池化聚合序列表示,hpool=Pool(Henc(Nenc))∈Rd\mathbf{h}_{\mathrm{pool}}=\mathrm{Pool}(\mathbf{H}_{\mathrm{enc}}^{(N_{\mathrm{enc}})})\in\mathbb{R}^{d}hpool​=Pool(Henc(Nenc​)​)∈Rd,再送入输出头。网络由池化后的表示 hpool\mathbf{h}_{\mathrm{pool}}hpool​ 产生三类输出:

1. Torque Prediction. NeuralActuator achieves the torque prediction via

1. 力矩预测。 NeuralActuator 通过下式得到力矩预测

τpred(t)=gθ(Ht),(6)\tau^{\mathrm{pred}}(t)=g_{\theta}(\mathcal{H}_t),\tag{6}τpred(t)=gθ​(Ht​),(6)

where τpred(t)∈Rn\tau^{\mathrm{pred}}(t)\in\mathbb{R}^{n}τpred(t)∈Rn denotes the predicted joint torques and Ht={xt−w,…,xt}\mathcal{H}_t=\{\mathbf{x}_{t-w},\ldots,\mathbf{x}_t\}Ht​={xt−w​,…,xt​} is a history window of proprioception, commands, and actuator telemetry. This design is motivated by the fact that the effective current–torque relationship on low-cost servo-driven arms can be highly nonlinear and time-varying during closed-loop tracking; See visualization in Sec. IV-J. Direct torque prediction lets the network learn the full nonlinearity without relying on a fixed linear prior.

其中 τpred(t)∈Rn\tau^{\mathrm{pred}}(t)\in\mathbb{R}^{n}τpred(t)∈Rn 表示预测的关节力矩,Ht={xt−w,…,xt}\mathcal{H}_t=\{\mathbf{x}_{t-w},\ldots,\mathbf{x}_t\}Ht​={xt−w​,…,xt​} 是本体感觉、指令和执行器遥测的历史窗口。这一设计的动机是:在低成本舵机驱动手臂上,有效的电流–力矩关系在闭环跟踪中可以高度非线性且随时间变化;可视化见第 IV-J 节。直接预测力矩让网络学习完整的非线性,而不依赖固定的线性先验。

A shared MLP predicts joint torques for all joints: τpred=MLPτ(hpool)∈Rn\tau^{\mathrm{pred}}=\mathrm{MLP}_{\tau}(\mathbf{h}_{\mathrm{pool}})\in\mathbb{R}^{n}τpred=MLPτ​(hpool​)∈Rn, where MLPτ\mathrm{MLP}_{\tau}MLPτ​ consists of two linear layers with SiLU activation.

一个共享的 MLP 为所有关节预测力矩:τpred=MLPτ(hpool)∈Rn\tau^{\mathrm{pred}}=\mathrm{MLP}_{\tau}(\mathbf{h}_{\mathrm{pool}})\in\mathbb{R}^{n}τpred=MLPτ​(hpool​)∈Rn,其中 MLPτ\mathrm{MLP}_{\tau}MLPτ​ 由两层带 SiLU 激活的线性层组成。

2. External Force Prediction. We predict end-effector contact forces using a two-stage approach that decouples force magnitude estimation from contact detection. First, a shared MLP predicts the raw 3D contact force: f^raw=MLPforce(hpool)∈R3\hat{\mathbf{f}}_{\mathrm{raw}}=\mathrm{MLP}^{\mathrm{force}}(\mathbf{h}_{\mathrm{pool}})\in\mathbb{R}^{3}f^raw​=MLPforce(hpool​)∈R3, which estimates force components from motor states alone, without dedicated force sensors. To suppress spurious force predictions during non-contact phases, we introduce a contact gate g∈[0,1]g\in[0,1]g∈[0,1] predicted by a separate head: g=σ(MLPgate(hpool))g=\sigma(\mathrm{MLP}^{\mathrm{gate}}(\mathbf{h}_{\mathrm{pool}}))g=σ(MLPgate(hpool​)), where σ(⋅)\sigma(\cdot)σ(⋅) denotes the sigmoid function. The gate represents the probability of physical contact: g≈1g\approx 1g≈1 indicates contact, while g≈0g\approx 0g≈0 indicates free motion. The final force output is computed as: f^=g⋅f^raw\hat{\mathbf{f}}=g\cdot\hat{\mathbf{f}}_{\mathrm{raw}}f^=g⋅f^raw​. During training, the gate supervision is derived from ground-truth force measurements ggt=1[∥fgt∥2>ϵ]g_{\mathrm{gt}}=\mathbf{1}[\|\mathbf{f}_{\mathrm{gt}}\|_2>\epsilon]ggt​=1[∥fgt​∥2​>ϵ], where ϵ=0.01 N\epsilon=0.01\,\mathrm{N}ϵ=0.01N serves as a noise threshold.

2. 外力预测。 我们用两阶段方法预测末端接触力,把力大小估计与接触检测分开。首先,一个共享 MLP 预测原始三维接触力:f^raw=MLPforce(hpool)∈R3\hat{\mathbf{f}}_{\mathrm{raw}}=\mathrm{MLP}^{\mathrm{force}}(\mathbf{h}_{\mathrm{pool}})\in\mathbb{R}^{3}f^raw​=MLPforce(hpool​)∈R3,它仅从电机状态估计力分量,不使用专用力传感器。为了在无接触阶段抑制虚假的力预测,我们引入由单独一头预测的接触门控 g∈[0,1]g\in[0,1]g∈[0,1]:g=σ(MLPgate(hpool))g=\sigma(\mathrm{MLP}^{\mathrm{gate}}(\mathbf{h}_{\mathrm{pool}}))g=σ(MLPgate(hpool​)),其中 σ(⋅)\sigma(\cdot)σ(⋅) 表示 sigmoid 函数。该门表示发生物理接触的概率:g≈1g\approx 1g≈1 表示接触,g≈0g\approx 0g≈0 表示自由运动。最终力输出计算为:f^=g⋅f^raw\hat{\mathbf{f}}=g\cdot\hat{\mathbf{f}}_{\mathrm{raw}}f^=g⋅f^raw​。训练时,门控的监督来自力真值测量 ggt=1[∥fgt∥2>ϵ]g_{\mathrm{gt}}=\mathbf{1}[\|\mathbf{f}_{\mathrm{gt}}\|_2>\epsilon]ggt​=1[∥fgt​∥2​>ϵ],其中 ϵ=0.01 N\epsilon=0.01\,\mathrm{N}ϵ=0.01N 作为噪声阈值。

3. Motor Condition Estimation. A separate head predicts a motor condition score c∈[0,1]c\in[0,1]c∈[0,1]: c=σ(MLPcond(hpool))c=\sigma(\mathrm{MLP}^{\mathrm{cond}}(\mathbf{h}_{\mathrm{pool}}))c=σ(MLPcond(hpool​)), where c=1c=1c=1 indicates normal motor operation and c=0c=0c=0 indicates degraded or damaged motor state. This enables condition monitoring by detecting anomalous patterns from current-torque discrepancies and thermal signatures.

3. 电机工况估计。 另一头预测电机工况分数 c∈[0,1]c\in[0,1]c∈[0,1]:c=σ(MLPcond(hpool))c=\sigma(\mathrm{MLP}^{\mathrm{cond}}(\mathbf{h}_{\mathrm{pool}}))c=σ(MLPcond(hpool​)),其中 c=1c=1c=1 表示电机正常运行,c=0c=0c=0 表示退化或损坏的电机状态。这可以通过检测电流–力矩偏差和热特征中的异常模式,来做工况监测。

Differentiable Simulation Integration. NeuralActuator is integrated with differentiable physics simulators. We embed our model within the simulation loop to enable gradient-based optimization through the entire system dynamics.

可微分仿真集成。 NeuralActuator 与可微分物理仿真器集成。我们把模型嵌进仿真循环,使基于梯度的优化能够穿过整个系统动力学。

Forward Dynamics. At each simulation timestep ttt, the simulator computes: (1) Predicted torque: τtpred=gθ(Ht)\tau^{\mathrm{pred}}_t=g_{\theta}(\mathcal{H}_t)τtpred​=gθ​(Ht​) and (2) State evolution: st+1=DiffSim(st,τtpred,f^text)s_{t+1}=\mathrm{DiffSim}(s_t,\tau^{\mathrm{pred}}_t,\hat{\mathbf{f}}^{\mathrm{ext}}_t)st+1​=DiffSim(st​,τtpred​,f^text​), where st=[qt,q˙t]s_t=[q_t,\dot{q}_t]st​=[qt​,q˙​t​] represents the joint state and qtcmdq^{\mathrm{cmd}}_tqtcmd​ is the commanded position.

前向动力学。 在每个仿真时间步 ttt,仿真器计算:(1)预测力矩:τtpred=gθ(Ht)\tau^{\mathrm{pred}}_t=g_{\theta}(\mathcal{H}_t)τtpred​=gθ​(Ht​);(2)状态演化:st+1=DiffSim(st,τtpred,f^text)s_{t+1}=\mathrm{DiffSim}(s_t,\tau^{\mathrm{pred}}_t,\hat{\mathbf{f}}^{\mathrm{ext}}_t)st+1​=DiffSim(st​,τtpred​,f^text​),其中 st=[qt,q˙t]s_t=[q_t,\dot{q}_t]st​=[qt​,q˙​t​] 表示关节状态,qtcmdq^{\mathrm{cmd}}_tqtcmd​ 是指令位置。

Gradient Computation. Through automatic differentiation, we compute gradients of a trajectory loss L\mathcal{L}L with respect to network parameters:

梯度计算。 通过自动微分,我们计算轨迹损失 L\mathcal{L}L 相对网络参数的梯度:

∂L∂θ=∑t=1T∂L∂st∂st∂τtpred∂τtpred∂θ.(7)\frac{\partial\mathcal{L}}{\partial\theta}=\sum_{t=1}^{T}\frac{\partial\mathcal{L}}{\partial s_t}\frac{\partial s_t}{\partial\tau^{\mathrm{pred}}_t}\frac{\partial\tau^{\mathrm{pred}}_t}{\partial\theta}.\tag{7}∂θ∂L​=t=1∑T​∂st​∂L​∂τtpred​∂st​​∂θ∂τtpred​​.(7)

This enables end-to-end learning where the network parameters are optimized to minimize discrepancies between simulated and real trajectories.

这使得端到端学习成为可能:网络参数被优化,以缩小仿真轨迹与真实轨迹之间的偏差。

Model Training. Since ground-truth torque values cannot be directly measured on low-cost platforms, the torque head is supervised through pose trajectories: forward integration maps predicted torques to future configurations, making pose measurements effective surrogate targets. The force, gate, and motor-condition heads use direct supervision from their respective ground truth. We minimize discrepancies between simulated and real trajectories through five loss components.

模型训练。由于在低成本平台上无法直接测量力矩真值,力矩头通过位姿轨迹来监督:前向积分把预测力矩映射到未来构型,使位姿测量成为有效的替代目标。力、门控和电机工况头则用来自各自真值的直接监督。我们通过五个损失项来缩小仿真轨迹与真实轨迹之间的偏差。

Joint Angle Loss: To handle angular wraparound, we define wrapπ(x)=((x+π) mod 2π)−π\mathrm{wrap}_{\pi}(x)=((x+\pi)\bmod 2\pi)-\piwrapπ​(x)=((x+π)mod2π)−π and compute Ljoint=1∣T∣∑t∥wrapπ(ϕtsim−ϕtreal)∥1\mathcal{L}_{\mathrm{joint}}=\frac{1}{|\mathcal{T}|}\sum_{t}\|\mathrm{wrap}_{\pi}(\phi^{\mathrm{sim}}_{t}-\phi^{\mathrm{real}}_{t})\|_1Ljoint​=∣T∣1​∑t​∥wrapπ​(ϕtsim​−ϕtreal​)∥1​. Gripper Loss: For the prismatic gripper, Lgrip=1∣T∣∑t∣qgrip,tsim−qgrip,treal∣\mathcal{L}_{\mathrm{grip}}=\frac{1}{|\mathcal{T}|}\sum_{t}|q^{\mathrm{sim}}_{\mathrm{grip},t}-q^{\mathrm{real}}_{\mathrm{grip},t}|Lgrip​=∣T∣1​∑t​∣qgrip,tsim​−qgrip,treal​∣. Force Loss: When ground-truth forces are available, Lforce=1∣T∣∑t∥ftpred−ftgt∥1\mathcal{L}_{\mathrm{force}}=\frac{1}{|\mathcal{T}|}\sum_{t}\|\mathbf{f}^{\mathrm{pred}}_{t}-\mathbf{f}^{\mathrm{gt}}_{t}\|_1Lforce​=∣T∣1​∑t​∥ftpred​−ftgt​∥1​. Gate Loss: The contact gate is supervised with binary cross-entropy against ggt=1[∥fgt∥2>ϵ]g_{\mathrm{gt}}=\mathbf{1}[\|\mathbf{f}_{\mathrm{gt}}\|_2>\epsilon]ggt​=1[∥fgt​∥2​>ϵ]. Condition Loss: Motor condition is supervised with binary cross-entropy: Lcond=−1∣T∣∑t[stlog⁡ct+(1−st)log⁡(1−ct)]\mathcal{L}_{\mathrm{cond}}=-\frac{1}{|\mathcal{T}|}\sum_{t}[s_t\log c_t+(1-s_t)\log(1-c_t)]Lcond​=−∣T∣1​∑t​[st​logct​+(1−st​)log(1−ct​)], where st=1s_t=1st​=1 for normal motors and st=0s_t=0st​=0 for degraded (restricted) motors. The total loss is: L=λjointLjoint+λgripLgrip+λforceLforce+λgateLgate+λcondLcond\mathcal{L}=\lambda_{\mathrm{joint}}\mathcal{L}_{\mathrm{joint}}+\lambda_{\mathrm{grip}}\mathcal{L}_{\mathrm{grip}}+\lambda_{\mathrm{force}}\mathcal{L}_{\mathrm{force}}+\lambda_{\mathrm{gate}}\mathcal{L}_{\mathrm{gate}}+\lambda_{\mathrm{cond}}\mathcal{L}_{\mathrm{cond}}L=λjoint​Ljoint​+λgrip​Lgrip​+λforce​Lforce​+λgate​Lgate​+λcond​Lcond​, where T\mathcal{T}T denotes timesteps with available ground-truth. We apply focal weighting for force samples to address contact/free-motion imbalance, and train with Adam [28] for 10000 epochs. Unless otherwise specified, experiments are conducted on a standard workstation with an Intel Core Ultra 7 265K processor (20 cores, 5.0 GHz), 32 GB of memory, and an NVIDIA GeForce RTX 5080 GPU with 16 GB of VRAM; the no-load benchmark in Appendix C uses an RTX 4090, and the hand-eye calibration measurements in Appendix D use an A6000. Network training and inference were implemented in JAX JIT [4] with GPU acceleration, and differentiable simulation for rigid body dynamics was based on MuJoCo MJX [44]. More details are in our Appendix.

关节角损失: 为处理角度环绕,我们定义 wrapπ(x)=((x+π) mod 2π)−π\mathrm{wrap}_{\pi}(x)=((x+\pi)\bmod 2\pi)-\piwrapπ​(x)=((x+π)mod2π)−π,并计算 Ljoint=1∣T∣∑t∥wrapπ(ϕtsim−ϕtreal)∥1\mathcal{L}_{\mathrm{joint}}=\frac{1}{|\mathcal{T}|}\sum_{t}\|\mathrm{wrap}_{\pi}(\phi^{\mathrm{sim}}_{t}-\phi^{\mathrm{real}}_{t})\|_1Ljoint​=∣T∣1​∑t​∥wrapπ​(ϕtsim​−ϕtreal​)∥1​。夹爪损失: 对移动夹爪,Lgrip=1∣T∣∑t∣qgrip,tsim−qgrip,treal∣\mathcal{L}_{\mathrm{grip}}=\frac{1}{|\mathcal{T}|}\sum_{t}|q^{\mathrm{sim}}_{\mathrm{grip},t}-q^{\mathrm{real}}_{\mathrm{grip},t}|Lgrip​=∣T∣1​∑t​∣qgrip,tsim​−qgrip,treal​∣。力损失: 当力的真值可用时,Lforce=1∣T∣∑t∥ftpred−ftgt∥1\mathcal{L}_{\mathrm{force}}=\frac{1}{|\mathcal{T}|}\sum_{t}\|\mathbf{f}^{\mathrm{pred}}_{t}-\mathbf{f}^{\mathrm{gt}}_{t}\|_1Lforce​=∣T∣1​∑t​∥ftpred​−ftgt​∥1​。门控损失: 接触门控用相对 ggt=1[∥fgt∥2>ϵ]g_{\mathrm{gt}}=\mathbf{1}[\|\mathbf{f}_{\mathrm{gt}}\|_2>\epsilon]ggt​=1[∥fgt​∥2​>ϵ] 的二元交叉熵来监督。工况损失: 电机工况用二元交叉熵监督:Lcond=−1∣T∣∑t[stlog⁡ct+(1−st)log⁡(1−ct)]\mathcal{L}_{\mathrm{cond}}=-\frac{1}{|\mathcal{T}|}\sum_{t}[s_t\log c_t+(1-s_t)\log(1-c_t)]Lcond​=−∣T∣1​∑t​[st​logct​+(1−st​)log(1−ct​)],其中对正常电机 st=1s_t=1st​=1,对退化(受限)电机 st=0s_t=0st​=0。总损失为:L=λjointLjoint+λgripLgrip+λforceLforce+λgateLgate+λcondLcond\mathcal{L}=\lambda_{\mathrm{joint}}\mathcal{L}_{\mathrm{joint}}+\lambda_{\mathrm{grip}}\mathcal{L}_{\mathrm{grip}}+\lambda_{\mathrm{force}}\mathcal{L}_{\mathrm{force}}+\lambda_{\mathrm{gate}}\mathcal{L}_{\mathrm{gate}}+\lambda_{\mathrm{cond}}\mathcal{L}_{\mathrm{cond}}L=λjoint​Ljoint​+λgrip​Lgrip​+λforce​Lforce​+λgate​Lgate​+λcond​Lcond​,其中 T\mathcal{T}T 表示具有可用真值的时间步。我们对力样本使用焦点加权,以应对接触/自由运动不平衡,并用 Adam [28] 训练 10000 个周期。除非另有说明,实验在一台标准工作站上进行,配置为 Intel Core Ultra 7 265K 处理器(20 核,5.0 GHz)、32 GB 内存,以及 16 GB 显存的 NVIDIA GeForce RTX 5080 GPU;附录 C 中的空载基准使用 RTX 4090,附录 D 中的手眼标定测量使用 A6000。网络训练与推理在带 GPU 加速的 JAX JIT [4] 中实现,刚体动力学的可微分仿真基于 MuJoCo MJX [44]。更多细节见我们的附录。

IV. EXPERIMENTS

IV. 实验

A. Rollout Accuracy

A. 滚动精度

Fig. 5: Robot arm simulation with external force estimation on the test set. (a) NeuralActuator estimates end-effector external forces along six directions ($\pm X$, $\pm Y$, $\pm Z$); red arrows visualize the estimated force vectors in simulation (paired with corresponding real-robot executions). (b) Force estimation during force-aware manipulation: lifting and holding 200 g and 300 g payloads (top) and pick-and-place with 200 g and 300 g payloads (bottom). Please refer to our video for more results.

Fig. 5: Robot arm simulation with external force estimation on the test set. (a) NeuralActuator estimates end-effector external forces along six directions (±X\pm X±X, ±Y\pm Y±Y, ±Z\pm Z±Z); red arrows visualize the estimated force vectors in simulation (paired with corresponding real-robot executions). (b) Force estimation during force-aware manipulation: lifting and holding 200 g and 300 g payloads (top) and pick-and-place with 200 g and 300 g payloads (bottom). Please refer to our video for more results.

图 5:测试集上带外力估计的机械臂仿真。(a)NeuralActuator 沿六个方向(±X\pm X±X、±Y\pm Y±Y、±Z\pm Z±Z)估计末端外力;红色箭头在仿真中可视化估计的力向量(并与对应的真机执行配对)。(b)力感知操作中的力估计:抬举并保持 200 g 和 300 g 载荷(上),以及带 200 g 和 300 g 载荷的拾取放置(下)。更多结果请见我们的视频。

We evaluate the fidelity of our differentiable simulator by comparing predicted trajectories against ground truth on a held-out test dataset. For each joint jjj at horizon TTT, we compute the mean absolute error (MAE) across NNN test trajectories: MAEj(T)=1N∑n=1N1T∑t=1T∣qj,t(n)−q^j,t(n)∣\mathrm{MAE}_j(T)=\frac{1}{N}\sum_{n=1}^{N}\frac{1}{T}\sum_{t=1}^{T}|q^{(n)}_{j,t}-\hat{q}^{(n)}_{j,t}|MAEj​(T)=N1​∑n=1N​T1​∑t=1T​∣qj,t(n)​−q^​j,t(n)​∣, where qj,t(n)q^{(n)}_{j,t}qj,t(n)​ and q^j,t(n)\hat{q}^{(n)}_{j,t}q^​j,t(n)​ denote the ground truth and predicted positions, respectively. Tab. III presents the simulation accuracy across different time horizons. Our model achieves relatively high short-term accuracy for revolute joints and prismatic joints. The error growth remains bounded even at longer horizons—a critical property for model-predictive control. We believe this horizon is sufficient for most online planning applications, where replanning typically occurs at higher frequencies. The consistent error growth pattern across joints validates our physics-in-the-loop learning approach, which captures actuator-specific, nonlinear effects while remaining compatible with differentiable rigid-body dynamics.

我们通过在留出的测试集上把预测轨迹与真值比较,来评估可微分仿真器的保真度。对每个关节 jjj 在时域 TTT 上,我们计算 NNN 条测试轨迹的平均绝对误差(MAE):MAEj(T)=1N∑n=1N1T∑t=1T∣qj,t(n)−q^j,t(n)∣\mathrm{MAE}_j(T)=\frac{1}{N}\sum_{n=1}^{N}\frac{1}{T}\sum_{t=1}^{T}|q^{(n)}_{j,t}-\hat{q}^{(n)}_{j,t}|MAEj​(T)=N1​∑n=1N​T1​∑t=1T​∣qj,t(n)​−q^​j,t(n)​∣,其中 qj,t(n)q^{(n)}_{j,t}qj,t(n)​ 和 q^j,t(n)\hat{q}^{(n)}_{j,t}q^​j,t(n)​ 分别表示真值和预测位置。表 III 给出不同时间范围上的仿真精度。我们的模型在转动关节和移动关节上都达到相对较高的短期精度。即使在更长时域上,误差增长仍然有界——这对模型预测控制是一项关键性质。我们认为这一时域对大多数在线规划应用已经足够,因为重规划通常以更高频率发生。各关节上一致的误差增长模式,验证了我们这种把物理放进回路的学习方法:它能抓住执行器特有的非线性效应,同时仍与可微分刚体动力学兼容。

TABLE III: Simulation accuracy on the test set. J1–J4: joint-angle MAE (deg). Grip: gripper-aperture MAE (mm). @600 steps correspond to 10 seconds.

@100 steps @300 steps @600 steps
Task J1 J2 J3 J4 Grip J1 J2 J3 J4 Grip J1 J2 J3 J4 Grip
backward_forward 2.4 4.3 4.4 4.2 0 2.3 5.9 3.9 4.2 0 3.1 4.8 4.9 4.5 0
circular_ccw 1.7 5.2 2.2 2.6 0 1.5 3.1 2.0 3.4 0 2.5 2.3 2.1 2.8 0
circular_cw 3.6 3.2 1.4 2.2 0 3.1 1.9 1.2 2.6 0 2.6 1.7 2.0 2.8 0
go_up_stay_still 2.5 1.8 2.3 1.2 0 3.1 2.1 3.1 2.5 0 3.1 1.5 3.5 2.7 0
joint_sweep_1 2.3 2.5 1.8 1.4 0 1.9 1.0 2.6 1.8 0 2.9 1.9 2.0 1.6 0
joint_sweep_2 2.1 4.3 4.6 1.2 0 1.8 6.7 8.4 1.1 0 2.4 4.6 5.2 1.7 0
joint_sweep_3 3.7 4.5 3.1 1.9 0 3.4 2.6 2.8 2.8 0 3.1 2.6 2.5 3.4 0
joint_sweep_4 2.6 4.8 2.5 3.4 0 2.9 3.3 2.4 3.2 0 2.4 2.9 2.2 4.6 0
joint_sweep_5 1.7 3.0 1.7 3.1 0.4 2.6 3.9 2.3 2.7 0.5 5.5 3.3 5.1 3.0 0.7
pick_place_empty 2.6 2.5 2.2 3.5 1.1 1.9 3.6 2.7 3.2 1.1 3.1 2.7 2.0 3.9 1.0
Average 2.5 3.6 2.6 2.5 0.2 2.5 3.4 3.1 2.8 0.2 3.1 2.8 3.2 3.1 0.2

表 III:测试集上的仿真精度。J1–J4:关节角 MAE(度)。Grip:夹爪开度 MAE(毫米)。@600 步对应 10 秒。

@100 步 @300 步 @600 步
任务 J1 J2 J3 J4 Grip J1 J2 J3 J4 Grip J1 J2 J3 J4 Grip
backward_forward 2.4 4.3 4.4 4.2 0 2.3 5.9 3.9 4.2 0 3.1 4.8 4.9 4.5 0
circular_ccw 1.7 5.2 2.2 2.6 0 1.5 3.1 2.0 3.4 0 2.5 2.3 2.1 2.8 0
circular_cw 3.6 3.2 1.4 2.2 0 3.1 1.9 1.2 2.6 0 2.6 1.7 2.0 2.8 0
go_up_stay_still 2.5 1.8 2.3 1.2 0 3.1 2.1 3.1 2.5 0 3.1 1.5 3.5 2.7 0
joint_sweep_1 2.3 2.5 1.8 1.4 0 1.9 1.0 2.6 1.8 0 2.9 1.9 2.0 1.6 0
joint_sweep_2 2.1 4.3 4.6 1.2 0 1.8 6.7 8.4 1.1 0 2.4 4.6 5.2 1.7 0
joint_sweep_3 3.7 4.5 3.1 1.9 0 3.4 2.6 2.8 2.8 0 3.1 2.6 2.5 3.4 0
joint_sweep_4 2.6 4.8 2.5 3.4 0 2.9 3.3 2.4 3.2 0 2.4 2.9 2.2 4.6 0
joint_sweep_5 1.7 3.0 1.7 3.1 0.4 2.6 3.9 2.3 2.7 0.5 5.5 3.3 5.1 3.0 0.7
pick_place_empty 2.6 2.5 2.2 3.5 1.1 1.9 3.6 2.7 3.2 1.1 3.1 2.7 2.0 3.9 1.0
平均 2.5 3.6 2.6 2.5 0.2 2.5 3.4 3.1 2.8 0.2 3.1 2.8 3.2 3.1 0.2

Sweeping the rollout horizon H∈{64,128,256,320,500}H\in\{64,128,256,320,500\}H∈{64,128,256,320,500} (Tab. II), simulator-side torque gradients remain O(10−2)O(10^{-2})O(10−2) with no collapse and the parameter-gradient direction stays nearly fixed (cosine alignment ≥0.96\ge 0.96≥0.96 relative to H=128H=128H=128); ∥∇θL∥\|\nabla_{\theta}\mathcal{L}\|∥∇θ​L∥ plateaus around H=256H=256H=256–320320320, motivating the curriculum used in training.

扫过滚动时域 H∈{64,128,256,320,500}H\in\{64,128,256,320,500\}H∈{64,128,256,320,500}(表 II),仿真器侧的力矩梯度保持在 O(10−2)O(10^{-2})O(10−2) 且没有崩溃,参数梯度方向几乎固定(相对 H=128H=128H=128 的余弦对齐 ≥0.96\ge 0.96≥0.96);∥∇θL∥\|\nabla_{\theta}\mathcal{L}\|∥∇θ​L∥ 在 H=256H=256H=256–320320320 附近进入平台,这为训练中使用的课程式安排提供了依据。

TABLE II: Gradient behavior across rollout horizons HHH.

HHH torque-grad ∣∇θL∣|\nabla_{\theta}L|∣∇θ​L∣ cos⁡\coscos
64 2.73e−22.73\mathrm{e}{-2}2.73e−2 8.77 0.96
128 2.59e−22.59\mathrm{e}{-2}2.59e−2 17.33 1.00
256 1.72e−21.72\mathrm{e}{-2}1.72e−2 22.35 0.99
320 1.45e−21.45\mathrm{e}{-2}1.45e−2 22.91 0.99
500 9.37e−39.37\mathrm{e}{-3}9.37e−3 20.29 0.98

表 II:不同滚动时域 HHH 下的梯度行为。

HHH torque-grad ∣∇θL∣|\nabla_{\theta}L|∣∇θ​L∣ cos⁡\coscos
64 2.73e−22.73\mathrm{e}{-2}2.73e−2 8.77 0.96
128 2.59e−22.59\mathrm{e}{-2}2.59e−2 17.33 1.00
256 1.72e−21.72\mathrm{e}{-2}1.72e−2 22.35 0.99
320 1.45e−21.45\mathrm{e}{-2}1.45e−2 22.91 0.99
500 9.37e−39.37\mathrm{e}{-3}9.37e−3 20.29 0.98

B. Force Estimation Accuracy

B. 力估计精度

We evaluate force prediction on (i) an external force sensing benchmark using a calibrated 6-DoF force-torque sensor, and (ii) a known-weight benchmark where ground-truth forces are defined by gravitational loads (200 g–500 g). See visualization in Fig. 5.

我们在以下两项上评估力预测:(i)使用经过标定的 6 自由度力–力矩传感器的外力传感基准;(ii)由重力载荷(200 g–500 g)定义力真值的已知重量基准。可视化见图 5。

TABLE IV: Simulation and force prediction accuracy on the force-sensor test set. J1–J4: joint-angle MAE (deg). Grip: gripper-aperture MAE (mm). F: force MAE (N).

@100 steps @300 steps @500 steps
Task J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F
With External Force Contact
force_X+ 0.9 2.3 0.9 1.6 1.0 0.36 1.1 1.8 2.4 1.3 1.0 0.46 0.9 1.9 2.7 1.7 1.0 0.50
force_X− 1.4 1.1 0.8 1.1 1.0 0.39 1.6 2.3 1.3 2.3 1.0 0.42 2.0 2.9 1.5 3.0 1.0 0.44
force_Y+ 2.3 5.7 2.8 0.6 0.1 0.36 2.1 4.5 2.6 0.9 0.2 0.38 2.7 4.8 2.1 0.9 0.3 0.44
force_Y− 1.4 1.6 3.1 0.5 0.1 0.38 1.5 3.0 3.1 1.3 0.1 0.36 1.4 3.6 3.1 1.5 0.2 0.38
force_Z+ 3.3 6.2 3.3 0.8 0.1 0.39 2.3 5.6 2.5 1.9 0.2 0.43 2.5 4.4 2.2 2.3 0.3 0.46
force_Z− 1.0 5.9 0.6 0.6 1.0 0.36 2.9 6.6 2.8 2.5 1.0 0.64 3.8 6.4 3.5 2.0 1.0 0.57
Reference (No External Force)
force_X+ 0.6 2.0 1.5 1.2 1.0 0.01 0.8 1.8 1.0 1.1 1.0 0.02 1.0 1.6 1.1 1.1 1.0 0.01
force_X− 2.0 2.5 0.5 1.1 1.0 0.00 1.4 2.3 0.5 1.0 1.0 0.00 1.5 2.2 0.8 1.3 1.0 0.00
force_Y+ 2.0 1.0 2.0 0.6 0.1 0.00 2.1 3.3 2.1 1.1 0.2 0.00 1.9 3.7 1.7 1.1 0.4 0.00
force_Y− 1.0 1.8 1.2 1.3 0.1 0.00 1.6 4.5 2.0 2.2 0.1 0.00 1.2 4.0 2.4 1.6 0.3 0.00
force_Z+ 1.2 1.7 2.1 0.5 0.1 0.00 1.6 3.8 2.2 1.2 0.2 0.00 1.5 2.9 2.0 1.3 0.3 0.00
force_Z− 0.6 2.3 0.9 0.9 1.0 0.02 0.7 1.6 1.1 1.1 1.0 0.01 0.9 1.3 1.0 1.1 1.0 0.01
Avg 1.48 2.84 1.64 0.90 0.55 0.19 1.64 3.43 1.97 1.49 0.58 0.23 1.78 3.31 2.01 1.58 0.65 0.23

表 IV:力传感器测试集上的仿真与力预测精度。J1–J4:关节角 MAE(度)。Grip:夹爪开度 MAE(毫米)。F:力 MAE(牛)。

@100 steps @300 steps @500 steps
任务 J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F
存在外力接触
force_X+ 0.9 2.3 0.9 1.6 1.0 0.36 1.1 1.8 2.4 1.3 1.0 0.46 0.9 1.9 2.7 1.7 1.0 0.50
force_X− 1.4 1.1 0.8 1.1 1.0 0.39 1.6 2.3 1.3 2.3 1.0 0.42 2.0 2.9 1.5 3.0 1.0 0.44
force_Y+ 2.3 5.7 2.8 0.6 0.1 0.36 2.1 4.5 2.6 0.9 0.2 0.38 2.7 4.8 2.1 0.9 0.3 0.44
force_Y− 1.4 1.6 3.1 0.5 0.1 0.38 1.5 3.0 3.1 1.3 0.1 0.36 1.4 3.6 3.1 1.5 0.2 0.38
force_Z+ 3.3 6.2 3.3 0.8 0.1 0.39 2.3 5.6 2.5 1.9 0.2 0.43 2.5 4.4 2.2 2.3 0.3 0.46
force_Z− 1.0 5.9 0.6 0.6 1.0 0.36 2.9 6.6 2.8 2.5 1.0 0.64 3.8 6.4 3.5 2.0 1.0 0.57
参照(无外力)
force_X+ 0.6 2.0 1.5 1.2 1.0 0.01 0.8 1.8 1.0 1.1 1.0 0.02 1.0 1.6 1.1 1.1 1.0 0.01
force_X− 2.0 2.5 0.5 1.1 1.0 0.00 1.4 2.3 0.5 1.0 1.0 0.00 1.5 2.2 0.8 1.3 1.0 0.00
force_Y+ 2.0 1.0 2.0 0.6 0.1 0.00 2.1 3.3 2.1 1.1 0.2 0.00 1.9 3.7 1.7 1.1 0.4 0.00
force_Y− 1.0 1.8 1.2 1.3 0.1 0.00 1.6 4.5 2.0 2.2 0.1 0.00 1.2 4.0 2.4 1.6 0.3 0.00
force_Z+ 1.2 1.7 2.1 0.5 0.1 0.00 1.6 3.8 2.2 1.2 0.2 0.00 1.5 2.9 2.0 1.3 0.3 0.00
force_Z− 0.6 2.3 0.9 0.9 1.0 0.02 0.7 1.6 1.1 1.1 1.0 0.01 0.9 1.3 1.0 1.1 1.0 0.01
Avg 1.48 2.84 1.64 0.90 0.55 0.19 1.64 3.43 1.97 1.49 0.58 0.23 1.78 3.31 2.01 1.58 0.65 0.23

External Force Sensing. Tab. IV reports 3D external force prediction across six directions (±X\pm X±X, ±Y\pm Y±Y, ±Z\pm Z±Z), with forces up to 9 N applied at the end-effector. On contact trajectories, the force MAE is 0.36–0.39 N at 100 steps and 0.38–0.57 N at 500 steps (4–6% relative error). On reference trajectories without contact, NeuralActuator predicts near-zero forces (0.00–0.02 N), reliably distinguishing contact from no-contact states. We found payloads at or below ∼\sim∼50 g (∼\sim∼0.5 N) sit close to the noise floor of this low-cost platform. Reliable detection in that regime is left to future work.

外力传感。 表 IV 报告了沿六个方向(±X\pm X±X、±Y\pm Y±Y、±Z\pm Z±Z)的三维外力预测,施加在末端的力最大到 9 N。在接触轨迹上,100 步时力 MAE 为 0.36–0.39 N,500 步时为 0.38–0.57 N(相对误差 4–6%)。在无接触的参照轨迹上,NeuralActuator 预测接近零的力(0.00–0.02 N),能够可靠地区分接触与无接触状态。我们发现,约 50 g 或更轻的载荷(约 0.5 N)接近这一低成本平台的噪声底。该区间内的可靠检测留待后续工作。

TABLE V: Simulation and force prediction accuracy on the weight-based test set. J1–J4: joint-angle MAE (deg). Grip: gripper-aperture MAE (mm). F: force MAE (N).

@100 steps @300 steps @600 steps
Task Weight J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F
go up and stay 200g 1.3 5.0 2.6 2.4 0.1 0.12 3.4 3.9 4.2 2.3 0.1 0.10 3.1 3.3 4.3 2.9 0.1 0.16
300g 2.6 5.3 5.7 3.7 0.1 0.20 1.8 3.4 5.7 5.4 0.0 0.17 3.5 2.5 3.4 4.6 0.0 0.20
400g 2.2 4.1 1.4 5.0 0.1 0.07 2.9 2.1 2.1 6.1 0.1 0.12 2.0 1.4 3.0 7.1 0.2 0.11
pick and place 200g 0.7 3.1 4.3 3.4 1.1 0.24 2.7 3.9 4.0 4.0 1.1 0.08 2.9 5.1 2.9 3.0 1.0 0.11
300g 0.5 4.5 3.7 0.9 0.3 0.00 1.2 5.1 4.8 1.5 0.2 0.00 2.7 7.8 4.9 3.2 0.3 0.09
400g 0.8 2.1 2.8 2.9 1.1 0.19 2.8 1.4 3.4 3.6 1.1 0.06 2.4 2.0 3.8 3.3 1.0 0.03
500g 2.7 1.8 1.7 3.3 1.1 0.05 4.8 2.8 1.7 2.3 1.1 0.02 4.2 6.3 2.3 2.3 0.9 0.04
Avg 1.54 3.70 3.17 3.09 0.56 0.12 2.80 3.23 3.70 3.60 0.53 0.08 2.97 4.06 3.51 3.77 0.50 0.11

表 V:基于重量的测试集上的仿真与力预测精度。J1–J4:关节角 MAE(度)。Grip:夹爪开度 MAE(毫米)。F:力 MAE(牛)。

@100 steps @300 steps @600 steps
任务 重量 J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F J1 J2 J3 J4 Grip F
go up and stay 200g 1.3 5.0 2.6 2.4 0.1 0.12 3.4 3.9 4.2 2.3 0.1 0.10 3.1 3.3 4.3 2.9 0.1 0.16
300g 2.6 5.3 5.7 3.7 0.1 0.20 1.8 3.4 5.7 5.4 0.0 0.17 3.5 2.5 3.4 4.6 0.0 0.20
400g 2.2 4.1 1.4 5.0 0.1 0.07 2.9 2.1 2.1 6.1 0.1 0.12 2.0 1.4 3.0 7.1 0.2 0.11
pick and place 200g 0.7 3.1 4.3 3.4 1.1 0.24 2.7 3.9 4.0 4.0 1.1 0.08 2.9 5.1 2.9 3.0 1.0 0.11
300g 0.5 4.5 3.7 0.9 0.3 0.00 1.2 5.1 4.8 1.5 0.2 0.00 2.7 7.8 4.9 3.2 0.3 0.09
400g 0.8 2.1 2.8 2.9 1.1 0.19 2.8 1.4 3.4 3.6 1.1 0.06 2.4 2.0 3.8 3.3 1.0 0.03
500g 2.7 1.8 1.7 3.3 1.1 0.05 4.8 2.8 1.7 2.3 1.1 0.02 4.2 6.3 2.3 2.3 0.9 0.04
Avg 1.54 3.70 3.17 3.09 0.56 0.12 2.80 3.23 3.70 3.60 0.53 0.08 2.97 4.06 3.51 3.77 0.50 0.11

Weight Benchmark. Tab. V summarizes force prediction accuracy across weight loads and prediction horizons. NeuralActuator achieves an average force MAE of 0.08–0.12 N across tasks and horizons, with single-task errors below 0.24 N. The go-up-and-stay task exhibits relatively larger per-row errors (up to 0.20 N) under sustained static loading.

重量基准。表 V 汇总了不同重量载荷和预测时域上的力预测精度。NeuralActuator 在各任务和时域上的平均力 MAE 为 0.08–0.12 N,单项任务误差低于 0.24 N。升起并保持任务在持续静载下表现出相对更大的分行误差(最高 0.20 N)。

Novel Contact Geometries. Because the force head operates at the joint-torque level, it is less tied to explicit object geometry than vision-based force inference. To probe this, we evaluate the pretrained model on two unseen objects (inset) with shapes and surface frictions absent from training (a 261 g and a 226 g weight, measured during the still-holding phase). Predicted forces are 2.80 N (GT: 2.56 N) and 2.40 N (GT: 2.21 N), indicating some generalization to unseen contact geometries for this restricted setting.

Unseen contact objects used to test force inference.

新的接触几何。由于力头在关节力矩层面运作,它不像基于视觉的力推断那样紧密绑定显式物体几何。为了检验这一点,我们在两个训练中未见过的物体(插图)上评估预训练模型,其形状和表面摩擦在训练中都未出现(在保持静止阶段测得的重量分别为 261 g 和 226 g)。预测力为 2.80 N(真值:2.56 N)和 2.40 N(真值:2.21 N),表明在这一受限设定下,对未见过的接触几何仍有一定泛化。

C. Comparison with Baselines

C. 与基线的比较

To place NeuralActuator against principled model-based alternatives, we compare it with three classical sensorless force estimators on the weight benchmark: (i) ID-Linear [36], the textbook inverse-dynamics residual τ~ext=Kti+b−τID(q,q˙,q¨)\tilde{\tau}_{\mathrm{ext}}=K_t i+b-\tau_{\mathrm{ID}}(q,\dot{q},\ddot{q})τ~ext​=Kt​i+b−τID​(q,q˙​,q¨​) with per-joint KtK_tKt​ calibrated on free motion; (ii) ID-Friction [31], which augments (i) with Stribeck friction and backlash compensation; and (iii) the Generalized Momentum Observer (GMO) [11], which avoids q¨\ddot{q}q¨​ via momentum integration with auto-tuned gain. All three baselines convert motor current to torque through the linear current–torque map and map the residual joint torque to end-effector force through the Jacobian pseudoinverse; importantly, they consume ground-truth robot states at every step, whereas NeuralActuator predicts forces from its own simulated rollout. Two observations stand out. First, fitting KtK_tKt​ per joint on the OpenManipulator-X almost exposes the limits of the linear assumption: two joints yield negative slopes before clamping, consistent with the high gear ratio gearbox masking the underlying current–torque relation. Second, even after Stribeck and backlash compensation, the best classical method (GMO) trails NeuralActuator by 5.5×5.5\times5.5× in mean force MAE (Tab. VI) 0.66 N vs. 0.12 N. In the no-contact regime, classical methods produce 33–59% false-positive contact rates whereas NeuralActuator remains near zero. Adding Stribeck friction over the linear baseline provides only modest improvement (1.23 vs. 1.41 N), confirming parametric models cannot capture the nonlinearities of the geared servos used on this low-cost platform.

为了把 NeuralActuator 放到有原则的基于模型的替代方法旁边比较,我们在重量基准上把它与三种经典的无传感器力估计器对照:(i)ID-Linear [36],教科书式的逆动力学残差 τ~ext=Kti+b−τID(q,q˙,q¨)\tilde{\tau}_{\mathrm{ext}}=K_t i+b-\tau_{\mathrm{ID}}(q,\dot{q},\ddot{q})τ~ext​=Kt​i+b−τID​(q,q˙​,q¨​),其中各关节的 KtK_tKt​ 在自由运动上标定;(ii)ID-Friction [31],在(i)上增加 Stribeck 摩擦和齿隙补偿;(iii)广义动量观测器(GMO)[11],通过动量积分和自动调节增益来避开 q¨\ddot{q}q¨​。这三种基线都通过线性电流–力矩映射把电机电流换成力矩,再用雅可比伪逆把残差关节力矩映射到末端力;重要的是,它们在每一步都使用真实机器人状态,而 NeuralActuator 则从自己的仿真滚动中预测力。有两点观察很突出。第一,在 OpenManipulator-X 上为每个关节拟合 KtK_tKt​,几乎暴露了线性假设的限度:有两个关节在钳位前就出现负斜率,这与高减速比齿轮箱掩盖底层电流–力矩关系相一致。第二,即使加上 Stribeck 和齿隙补偿,最好的经典方法(GMO)在平均力 MAE 上仍落后 NeuralActuator 5.5×5.5\times5.5×(表 VI)0.66 N 对 0.12 N。在无接触工况下,经典方法产生 33–59% 的假阳性接触率,而 NeuralActuator 接近零。在线性基线之上加入 Stribeck 摩擦只有有限改进(1.23 N 对 1.41 N),说明参数模型无法抓住这一低成本平台上带减速器舵机的非线性。

TABLE VI: Force MAE (N) against model-based sensorless baselines on the weight benchmark (lower is better). Baselines consume ground-truth states at every step; NeuralActuator uses simulated rollout states.

Method Go Up & Stay Pick & Place Avg
200g 300g 400g 200g 300g 400g 500g
ID-Linear [36] 1.37 1.81 2.30 0.72 0.95 1.22 1.47 1.41
ID-Friction [31] 1.06 1.59 2.15 0.62 0.82 1.10 1.31 1.23
GMO [11] 0.58 0.66 1.23 0.33 0.47 0.63 0.75 0.66
NeuralActuator 0.12 0.20 0.07 0.24 0.00 0.19 0.05 0.12

表 VI:重量基准上相对基于模型的无传感器基线的力 MAE(牛,越低越好)。基线在每一步使用真实状态;NeuralActuator 使用仿真滚动状态。

方法 升起并保持 拾取放置 平均
200g 300g 400g 200g 300g 400g 500g
ID-Linear [36] 1.37 1.81 2.30 0.72 0.95 1.22 1.47 1.41
ID-Friction [31] 1.06 1.59 2.15 0.62 0.82 1.10 1.31 1.23
GMO [11] 0.58 0.66 1.23 0.33 0.47 0.63 0.75 0.66
NeuralActuator 0.12 0.20 0.07 0.24 0.00 0.19 0.05 0.12

D. Motor Condition Estimation

D. 电机工况估计

Fig. 6: Motor Condition Estimation. (a) Rubber bands wrapped around Joint 3 to simulate increased resistance from motor degradation or wear. (b) Despite the same position trajectories (top), the abnormal motor requires a higher current (middle), yielding a detectable difference (bottom).

Fig. 6: Motor Condition Estimation. (a) Rubber bands wrapped around Joint 3 to simulate increased resistance from motor degradation or wear. (b) Despite the same position trajectories (top), the abnormal motor requires a higher current (middle), yielding a detectable difference (bottom).

图 6:电机工况估计。(a)用橡皮筋缠绕关节 3,以模拟电机退化或磨损带来的阻力增大。(b)尽管位置轨迹相同(上),异常电机需要更高的电流(中),从而产生可检测的差异(下)。

Next, we simulate motor degradation by wrapping rubber bands around joints to introduce additional mechanical resistance (Fig. 6 (a)). When commanding identical positions, the perturbed motor draws higher current to overcome the resistance while maintaining similar trajectories (Fig. 6 (b)). This consistent current deviation enables sensorless fault detection without additional sensors. In Tab. VII we compare NeuralActuator with baselines for motor condition detection on a pick-and-place task with a 200 g object. NeuralActuator achieves the best performance with 91.0% accuracy with 96.2% recall and 0.95 AUC-ROC, demonstrating potential for predictive maintenance and fault detection on the platforms. Baseline details (Threshold (Thres.), SVM, Random Forest (RF)) are in Appendix F. The rubber band acts as a controlled proxy for mechanical degradation: it imposes a position-dependent resistance that raises the current required for the same trajectory, qualitatively matching the signature of increased bearing friction or gearbox wear. It does not capture intermittent electrical faults or coil-side failures, which we leave to future work.

接下来,我们用橡皮筋缠绕关节来模拟电机退化,以引入额外的机械阻力(图 6(a))。在指令相同位置时,被扰动的电机为克服阻力会汲取更高电流,同时保持相似的轨迹(图 6(b))。这种一致的电流偏差使无传感器故障检测成为可能,而不需要额外传感器。在表 VII 中,我们在带 200 g 物体的拾取放置任务上,把 NeuralActuator 与电机工况检测基线做比较。NeuralActuator 达到最好的表现:准确率 91.0%,召回率 96.2%,AUC-ROC 为 0.95,显示了在这些平台上做预测性维护和故障检测的潜力。基线细节(阈值(Thres.)、SVM、随机森林(RF))见附录 F。橡皮筋作为机械退化的受控代理:它施加随位置变化的阻力,提高完成同一轨迹所需的电流,在性质上与轴承摩擦增大或减速箱磨损的特征相符。它并不能捕捉间歇性电气故障或线圈侧失效,这些我们留待后续工作。

TABLE VII: Motor condition detection.

Metric Thres. SVM RF Ours
Accuracy 58.6% 59.9% 67.1% 91.0%
Precision 0.0% 52.6% 62.3% 84.5%
Recall 0.0% 31.7% 52.4% 96.2%
AUC-ROC 0.45 0.62 0.72 0.95

表 VII:电机工况检测。

指标 Thres. SVM RF 本文
准确率 58.6% 59.9% 67.1% 91.0%
精确率 0.0% 52.6% 62.3% 84.5%
召回率 0.0% 31.7% 52.4% 96.2%
AUC-ROC 0.45 0.62 0.72 0.95

E. Running time performance

E. 运行时间表现

Tab. VIII shows that NeuralActuator is efficient enough for simulation and real-time control: JAX JIT compilation yields sub-millisecond GPU inference latency and throughput well above the 60 Hz control rate. Batch processing further improves throughput, while the total parameter footprint remains small, enabling deployment on embedded platforms.

表 VIII 表明,NeuralActuator 的效率足以用于仿真和实时控制:JAX JIT 编译带来亚毫秒级的 GPU 推理延迟,吞吐量远高于 60 Hz 的控制频率。批处理进一步提高吞吐量,而总参数量仍然很小,从而可以部署到嵌入式平台。

TABLE VIII: Model parameters and runtime performance.

Model Sizes Inference
Metric Value Unit Metric Value Unit
Parameters 1.42M — Mean time 0.25 ms
FLOPs (fwd) 5.46M — P95 time 0.31 ms
Parameter memory 5.43 MB Thru. (b=1) 4,019 Hz
Thru. (b=32) 10,992 Hz

表 VIII:模型参数与运行时间表现。

模型规模 推理
指标 数值 单位 指标 数值 单位
参数量 1.42M — 平均时间 0.25 ms
FLOPs(前向) 5.46M — P95 时间 0.31 ms
参数内存 5.43 MB 吞吐(b=1) 4,019 Hz
吞吐(b=32) 10,992 Hz

F. Sim-to-Real for Real Robot Control — Imitation Learning

F. 面向真机控制的仿真到真机——模仿学习

To demonstrate the utility of NeuralActuator as a simulation component, we train a behavior cloning (BC) controller on top of a frozen neural actuator layer. Following [29], we collect expert demonstrations via teleoperation and train a policy network to predict commanded joint positions (see details in Appendix H). We train the policy using demonstrations with 100 g, 200 g, 300 g, and 500 g payloads, and evaluate on 400 g for object lift-and-hold and 500 g for pick-and-place. Force-aware policy input. In our setting, the BC policy is conditioned not only on proprioception and target joint positions, but also on the external force signal predicted by the frozen NeuralActuator module, f^ext\hat{\mathbf{f}}_{\mathrm{ext}}f^ext​. This augments position-only control with force feedback, which matters under varying payloads and intermittent contacts. The NeuralActuator module is frozen throughout policy training and the NAD data itself is collected through teleoperation rather than a learned controller, so the actuator mapping is identified from the hardware’s input–output behavior and is not coupled to any specific control policy. Position-only baseline. The baseline trains the same policy without NeuralActuator and without any external force input. Tab. IX compares success rates with and without the neural actuator layer, showing that accurate actuator modeling together with force feedback consistently improves sim-to-real transfer for manipulation tasks (Fig. 1).

为了展示 NeuralActuator 作为仿真组件的用处,我们在冻结的神经执行器层之上训练一个行为克隆(BC)控制器。沿用 [29],我们通过遥操作采集专家示范,并训练策略网络来预测指令关节位置(细节见附录 H)。我们用带 100 g、200 g、300 g 和 500 g 载荷的示范训练策略,并在 400 g 上评估物体抬举保持,在 500 g 上评估拾取放置。力感知的策略输入。在我们的设定中,BC 策略不仅以本体感觉和目标关节位置为条件,还以冻结的 NeuralActuator 模块所预测的外力信号 f^ext\hat{\mathbf{f}}_{\mathrm{ext}}f^ext​ 为条件。这为仅有位置的控制补上了力反馈,在载荷变化和间歇接触下很重要。在整个策略训练中 NeuralActuator 模块保持冻结,且 NAD 数据本身是通过遥操作而非学习到的控制器采集的,因此执行器映射是从硬件的输入–输出行为中辨识出来的,并不耦合到任何特定控制策略。仅位置基线。该基线训练同样的策略,但不使用 NeuralActuator,也不使用任何外力输入。表 IX 比较了有无神经执行器层时的成功率,表明准确的执行器建模加上力反馈,能够一致地改进操作任务的仿真到真机迁移(图 1)。

TABLE IX: Behavior cloning success rates with NeuralActuator. Results are averaged over 40 trials.

Task w/o NeuralActuator w/ NeuralActuator
Pick-and-Place 80% 92.5%
Go Up-and-Stay 85% 95%

表 IX:使用 NeuralActuator 时的行为克隆成功率。结果对 40 次试验取平均。

任务 不使用 NeuralActuator 使用 NeuralActuator
拾取放置 80% 92.5%
升起并保持 85% 95%

G. Visual Supervision

G. 视觉监督

Fig. 7: NeuralActuator for hand-eye calibration. Combining NeuralActuator with a differentiable rendering pipeline for end-to-end hand-eye calibration.

Fig. 7: NeuralActuator for hand-eye calibration. Combining NeuralActuator with a differentiable rendering pipeline for end-to-end hand-eye calibration.

图 7:用于手眼标定的 NeuralActuator。把 NeuralActuator 与可微分渲染流程结合起来,做端到端手眼标定。

We demonstrate the application of our differentiable simulation framework with differentiable rendering for fully differentiable hand-eye calibration. We initialize TcbT_{cb}Tcb​ (camera-from-base) with a geometric 3-point initialization that aligns depth-lifted image keypoints with the corresponding forward-kinematics keypoints, and jointly refine the camera extrinsics and per-frame joint configurations through differentiable silhouette-based optimization, following the EasyHeC [7] formulation; camera intrinsics are estimated using VGGT [45] and held fixed. The rendered robot silhouette is matched to a segmentation mask produced by SAM3 [6] with “robot arm” as the text prompt, and consistency is evaluated by mask IoU, reaching an average IoU of 0.8515. Please check out more details in Appendix D.

我们展示了把可微分仿真框架与可微分渲染结合起来,用于完全可微分的手眼标定。我们用几何三点初始化来初始化 TcbT_{cb}Tcb​(相机相对基座),使由深度抬升得到的图像关键点与对应的前向运动学关键点对齐,并按照 EasyHeC [7] 的表述,通过基于可微分轮廓的优化,联合精化相机外参和逐帧关节构型;相机内参用 VGGT [45] 估计并保持固定。渲染出的机器人轮廓与 SAM3 [6] 在文本提示“robot arm”下给出的分割掩膜对齐,一致性用掩膜 IoU 评估,平均 IoU 达到 0.8515。更多细节见附录 D。

H. Ablation on Network Architecture

H. 网络架构消融

Because there is no baseline specifically designed for this task, we evaluate the impact of different neural architectures for the actuator model, comparing three architectures with matched parameter counts (∼\sim∼1.4M parameters) against our proposed approach. Let xt∈RF\mathbf{x}_t\in\mathbb{R}^{F}xt​∈RF denote the feature vector containing joint states, motor currents, and goals. We consider: (i) MLP: A feedforward network with LayerNorm predicting torques from flattened history τt=fθ(xt−L:t)\tau_t=f_{\theta}(\mathbf{x}_{t-L:t})τt​=fθ​(xt−L:t​); (ii) GRU [9]: A recurrent model ht=GRUθ(xt,ht−1)\mathbf{h}_t=\mathrm{GRU}_{\theta}(\mathbf{x}_t,\mathbf{h}_{t-1})ht​=GRUθ​(xt​,ht−1​) with learnable initial states; (iii) LSTM [20]: Long short-term memory cells (ht,ct)=LSTMθ(xt,ht−1,ct−1)(\mathbf{h}_t,\mathbf{c}_t)=\mathrm{LSTM}_{\theta}(\mathbf{x}_t,\mathbf{h}_{t-1},\mathbf{c}_{t-1})(ht​,ct​)=LSTMθ​(xt​,ht−1​,ct−1​) for long-range dependencies. As shown in Tab. X, our Transformer-based approach achieves high dynamics modeling as well as force prediction accuracy compared with other design choices.

由于没有专门为这一任务设计的基线,我们评估不同神经架构对执行器模型的影响,在参数量匹配(约 1.4M 参数)的前提下,把三种架构与我们提出的方法比较。设 xt∈RF\mathbf{x}_t\in\mathbb{R}^{F}xt​∈RF 表示包含关节状态、电机电流和目标的特征向量。我们考虑:(i)MLP:带 LayerNorm 的前馈网络,从展平的历史预测力矩 τt=fθ(xt−L:t)\tau_t=f_{\theta}(\mathbf{x}_{t-L:t})τt​=fθ​(xt−L:t​);(ii)GRU [9]:循环模型 ht=GRUθ(xt,ht−1)\mathbf{h}_t=\mathrm{GRU}_{\theta}(\mathbf{x}_t,\mathbf{h}_{t-1})ht​=GRUθ​(xt​,ht−1​),带可学习的初始状态;(iii)LSTM [20]:长短期记忆单元 (ht,ct)=LSTMθ(xt,ht−1,ct−1)(\mathbf{h}_t,\mathbf{c}_t)=\mathrm{LSTM}_{\theta}(\mathbf{x}_t,\mathbf{h}_{t-1},\mathbf{c}_{t-1})(ht​,ct​)=LSTMθ​(xt​,ht−1​,ct−1​),用于长程依赖。如表 X 所示,相对其他设计选择,我们基于 Transformer 的方法在动力学建模和力预测精度上都达到了较高水平。

TABLE X: Comparison of neural architectures for actuation modeling. We report rollout and force prediction accuracy at a 500-step prediction horizon. Bold marks the best result per column; ties are bolded jointly.

Model J1 J2 J3 J4 Grip F
MLP 4.55 5.81 3.48 2.16 0.91 0.47
GRU 1.83 2.08 1.68 1.70 0.65 0.49
LSTM 2.91 7.66 3.21 3.08 0.71 0.41
Ours 1.78 3.31 2.01 1.58 0.65 0.23

表 X:用于执行器建模的神经架构比较。我们报告 500 步预测时域上的滚动与力预测精度。每列最好结果加粗;并列则一并加粗。

模型 J1 J2 J3 J4 Grip F
MLP 4.55 5.81 3.48 2.16 0.91 0.47
GRU 1.83 2.08 1.68 1.70 0.65 0.49
LSTM 2.91 7.66 3.21 3.08 0.71 0.41
本文 1.78 3.31 2.01 1.58 0.65 0.23

Fig. 8: Cross-platform validation. We train NeuralActuator from scratch on a 7-DoF Franka Emika Panda robot with on-board joint torque sensors and a 6-DoF SO-101 low-cost arm (STS3215 servos), spanning three actuator families and the $500–$30k+ cost spectrum.

Fig. 8: Cross-platform validation. We train NeuralActuator from scratch on a 7-DoF Franka Emika Panda robot with on-board joint torque sensors and a 6-DoF SO-101 low-cost arm (STS3215 servos), spanning three actuator families and the 500–500–500–30k+ cost spectrum.

图 8:跨平台验证。我们从零开始,在带板载关节力矩传感器的 7 自由度 Franka Emika Panda 机器人和 6 自由度低成本臂 SO-101(STS3215 舵机)上训练 NeuralActuator,覆盖三类执行器家族以及 500 美元至 3 万美元以上的价位区间。

I. More Platforms

I. 更多平台

7-DoF Franka Emika Panda with joint torque sensors. This platform exposes per-joint torque measurements, enabling a direct predicted-vs-measured torque validation rather than the pose-only validation used on low-cost servos. For force estimation on Franka, the predicted joint torque τ^\hat{\tau}τ^ is fed back as an input feature to the force head. Across 200–600 g lift-and-hold trajectories, the model attains ≤3.7∘\le 3.7^{\circ}≤3.7∘ per-joint MAE at all settings, 0.26–0.42 N force MAE, and 0.50–0.98 Nm torque MAE against the on-board sensors (ground-truth torque range 0.1–30 Nm), giving the first sensor-grounded confirmation that NeuralActuator’s torque predictions are physically meaningful. Per-joint breakdown at both 100- and 500-step rollouts is reported in Appendix I, Tab. A18.

带关节力矩传感器的 7 自由度 Franka Emika Panda。 这一平台提供各关节的力矩测量,从而可以做直接的预测对测量力矩验证,而不是低成本舵机上所用的仅位姿验证。在 Franka 上做力估计时,预测关节力矩 τ^\hat{\tau}τ^ 被作为力头的输入特征送回。在 200–600 g 的抬举保持轨迹上,模型在所有设定下都达到不超过 3.7∘3.7^{\circ}3.7∘ 的单关节 MAE、0.26–0.42 N 的力 MAE,以及相对板载传感器 0.50–0.98 N·m 的力矩 MAE(力矩真值范围 0.1–30 N·m),这是首次有传感器依据的确认:NeuralActuator 的力矩预测在物理上是有意义的。100 步和 500 步滚动上的分关节细目见附录 I 的表 A18。

6-DoF low-cost arm (SO-101). SO-101 uses STS3215 serial-bus servos with heterogeneous gearing (1.147–1.345), validating transfer beyond the Dynamixel family used in NAD. We collected 40,426 frames (649 s at 62.3 Hz) across six task–weight combinations following the NAD protocol. At a 500-step rollout, the model reaches 0.47–0.73 N force MAE for 300–500 g payloads. Joint MAE is larger than on the original platform (up to ∼\sim∼9.8∘^{\circ}∘ on Joint 3; per-joint breakdown in Appendix I, Tab. A17), attributable to the heterogeneous gearing and the smaller per-platform dataset; the few-trajectory online adaptation routine of Appendix I mitigates both.

6 自由度低成本臂(SO-101)。 SO-101 使用带异构减速比(1.147–1.345)的 STS3215 串行总线舵机,用于验证从 NAD 所用 Dynamixel 家族之外的迁移。我们按照 NAD 规程,在六种任务–重量组合上采集了 40,426 帧(以 62.3 Hz 计为 649 秒)。在 500 步滚动上,模型对 300–500 g 载荷达到 0.47–0.73 N 的力 MAE。关节 MAE 比原平台更大(关节 3 上最高约 9.8∘9.8^{\circ}9.8∘;分关节细目见附录 I 的表 A17),原因可归于异构减速比以及每个平台上更小的数据集;附录 I 中的少轨迹在线适配流程对二者都有缓解。

J. Learned Torque Values

J. 学到的力矩值

Fig. 9: Relationship between motor current and predicted joint torque. The effective gain and offset vary across joints and phases, with evident saturation and noise, indicating a distinctly nonlinear current-to-torque mapping. The averaged joint-angle MAE and gripper-position MAE are 1.26° and 1 mm, respectively (at 500 rollout steps). The y-axis values are model-predicted torques (Nm); the OpenManipulator-X has no on-board torque sensor, so this visualization illustrates the learned nonlinear current–torque mapping rather than a torque-sensor validation. A direct predicted-vs-measured comparison on a 7-DoF arm with joint torque sensors is in Sec. IV-I.

Fig. 9: Relationship between motor current and predicted joint torque. The effective gain and offset vary across joints and phases, with evident saturation and noise, indicating a distinctly nonlinear current-to-torque mapping. The averaged joint-angle MAE and gripper-position MAE are 1.26° and 1 mm, respectively (at 500 rollout steps). The y-axis values are model-predicted torques (Nm); the OpenManipulator-X has no on-board torque sensor, so this visualization illustrates the learned nonlinear current–torque mapping rather than a torque-sensor validation. A direct predicted-vs-measured comparison on a 7-DoF arm with joint torque sensors is in Sec. IV-I.

图 9:电机电流与预测关节力矩之间的关系。有效增益和偏置随关节和阶段变化,并可见饱和与噪声,表明电流到力矩的映射明显非线性。平均关节角 MAE 和夹爪位置 MAE 分别为 1.26° 和 1 mm(在 500 步滚动上)。纵轴数值是模型预测的力矩(N·m);OpenManipulator-X 没有板载力矩传感器,因此这一可视化说明的是学到的非线性电流–力矩映射,而不是力矩传感器验证。在带关节力矩传感器的 7 自由度臂上,直接的预测对测量比较见第 IV-I 节。

We visualize the learned joint torques during real robot arm control. Fig. 9 compares the model-predicted torques with the measured motor currents over time for four joints. No torque sensor is available on this platform, so the comparison here is a qualitative illustration of the nonlinear current–torque relation learned by the model rather than a sensor-based validation; we report a direct predicted-vs-measured comparison on a 7-DoF arm with joint torque sensors in Sec. IV-I. Although the signals are roughly correlated, the effective current–torque gain and offset vary across joints and motion phases, exhibiting saturation, hysteresis, and noise. These patterns are consistent with low-cost actuator effects such as static friction, thermal drift, and gearbox backlash, under which simplified traditional models are often unstable or unidentifiable. In contrast, our neural torque prediction design allows the network to learn the full nonlinear and history-dependent mapping from telemetry to torque for more accurate actuation modeling.

我们可视化真机手臂控制过程中学到的关节力矩。图 9 把四个关节上模型预测的力矩与测得的电机电流随时间做比较。该平台没有力矩传感器,因此这里的比较是对模型所学非线性电流–力矩关系的定性说明,而不是基于传感器的验证;我们在第 IV-I 节报告了在带关节力矩传感器的 7 自由度臂上直接的预测对测量比较。尽管信号大致相关,有效的电流–力矩增益和偏置随关节和运动阶段变化,并表现出饱和、磁滞和噪声。这些模式与低成本执行器效应相一致,例如静摩擦、热漂移和减速箱齿隙,在这些效应下,简化的传统模型往往不稳定或难以辨识。相比之下,我们的神经力矩预测设计让网络学习从遥测到力矩的完整非线性、依赖历史的映射,从而更准确地做执行器建模。

V. CONCLUSION

V. 结论

We presented NeuralActuator, a differentiable neural actuator model for low-cost robots that supports direct torque prediction and force perception without dedicated sensors. Instead of assuming a fixed linear current–torque relation, NeuralActuator learns a history-dependent mapping from telemetry and commands to torque, while jointly estimating external forces and contact. We provide a twin-arm teleoperation pipeline for collecting force-labeled data, a multi-task Transformer that predicts torques, forces, and contact states, and a torque-sensor-free training procedure through differentiable simulation. Experiments across three platforms spanning three actuator families and the 500–500–500–30k+ cost spectrum show improved motion prediction, reliable force estimation on weight benchmarks, motor condition detection, and behavior cloning with NeuralActuator. Limitations and Future Work. Like all data-driven dynamics models, NeuralActuator accumulates error over long-horizon open-loop rollouts, eventually degrading tracking accuracy; the few-trajectory online adaptation routine in Appendix I mitigates this drift. Future work could explore multi-point force prediction and more scalable ways to learn force estimation from visual cues.

我们提出了 NeuralActuator,一个面向低成本机器人的可微分神经执行器模型,支持直接的力矩预测,以及在没有专用传感器时的力感知。NeuralActuator 并不假定固定的线性电流–力矩关系,而是学习从遥测和指令到力矩的、依赖历史的映射,同时联合估计外力和接触。我们提供了用于采集带力标注数据的双臂遥操作流程、一个预测力矩、力和接触状态的多任务 Transformer,以及一套通过可微分仿真、无需力矩传感器的训练程序。在覆盖三类执行器家族以及 500 美元至 3 万美元以上价位区间的三个平台上的实验表明,运动预测有所改进,重量基准上的力估计可靠,能够做电机工况检测,并且可以用 NeuralActuator 做行为克隆。限度与后续工作。与所有数据驱动的动力学模型一样,NeuralActuator 会在长时程开环滚动中累积误差,最终使跟踪精度下降;附录 I 中的少轨迹在线适配流程可以缓解这种漂移。后续工作可以探讨多点力预测,以及更可扩展地从视觉线索学习力估计的途径。

VI. ACKNOWLEDGEMENTS

VI. 致谢

This work has been done under the provisions of MIT-Amazon Science Hub. The authors would like to extend their sincere appreciation to Yuxiang Ma and Yuxin Song for their generous assistance with some of the hardware experiments. We are also deeply grateful to the anonymous reviewers for their thoughtful and constructive comments, which have improved the quality of the work described in this paper.

本工作在 MIT-Amazon Science Hub 的安排下完成。作者衷心感谢 Yuxiang Ma 和 Yuxin Song 在部分硬件实验上的慷慨帮助。我们也深深感谢匿名审稿人审慎而建设性的意见,这些意见改进了本文所述工作的质量。

REFERENCES

参考文献

[1] Anurag Ajay, Jiajun Wu, Nima Fazeli, Maria Bauza, Leslie P Kaelbling, Joshua B Tenenbaum, and Alberto Rodriguez. Augmenting physical simulators with stochastic neural networks: Case study of planar pushing and bouncing. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3066–3073. IEEE, 2018.

[2] Genesis Authors. Genesis: A universal and generative physics engine for robotics and beyond. URL https://github. com/Genesis-Embodied-AI/Genesis, 2024.

[3] Seyed Ali Baradaran Birjandi, Johannes Kühn, and Sami Haddadin. Observer-extended direct method for collision monitoring in robot manipulators using proprioception and imu sensing. IEEE Robotics and Automation Letters, 5(2):954–961, 2020.

[4] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/jax-ml/jax.

[5] Remi Cadene, Simon Alibert, Alexander Soare, Quentin Gallouedec, Adil Zouitine, Steven Palma, Pepijn Kooijmans, Michel Aractingi, Mustafa Shukor, Dana Aubakirova, Martino Russi, Francesco Capuano, Caroline Pascal, Jade Choghari, Jess Moss, and Thomas Wolf. Lerobot: State-of-the-art machine learning for real-world robotics in pytorch. https://github.com/huggingface/ lerobot, 2024.

[6] Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman Rädle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zhou, Liliane Momeni, Rishi Hazra, Shuangrui Ding, Sagar Vaze, Francois Porcher, Feng Li, Siyuan Li, Aishwarya Kamath, Ho Kei Cheng, Piotr Doll´ar, Nikhila Ravi, Kate Saenko, Pengchuan Zhang, and Christoph Feichtenhofer. Sam 3: Segment anything with concepts, 2025. URL https://arxiv.org/abs/2511.16719.

[7] Linghao Chen, Yuzhe Qin, Xiaowei Zhou, and Hao Su. Easyhec: Accurate and automatic hand-eye calibration via differentiable rendering and space exploration. IEEE Robotics and Automation Letters, 8(11): 7234–7241, November 2023. ISSN 2377-3774. doi: 10. 1109/lra.2023.3315551. URL http://dx.doi.org/10.1109/ LRA.2023.3315551.

[8] Peter Yichen Chen, Chao Liu, Pingchuan Ma, John Eastman, Daniela Rus, Dylan Randle, Yuri Ivanov, and Wojciech Matusik. Learning object properties using robot proprioception via differentiable robot-object interaction. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 5997–6004. IEEE, 2025.

[9] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.

[10] Jeremy A Collins, Cody Houff, Patrick Grady, and Charles C Kemp. Visual contact pressure estimation for grippers in the wild. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10947–10954. IEEE, 2023.

[11] Alessandro De Luca and Raffaella Mattone. Sensorless robot collision detection and hybrid force/motion control. In Proceedings of the 2005 IEEE international conference on robotics and automation, pages 999–1004. IEEE, 2005.

[12] Alessandro De Luca, Alin Albu-Schaffer, Sami Haddadin, and Gerd Hirzinger. Collision detection and safe reaction with the dlr-iii lightweight manipulator arm. In 2006 IEEE/RSJ international conference on intelligent robots and systems, pages 1623–1630. IEEE, 2006.

[13] Nolan Fey, Gabriel B Margolis, Martin Peticco, and Pulkit Agrawal. Bridging the sim-to-real gap for athletic loco-manipulation. arXiv preprint arXiv:2502.10894, 2025.

[14] C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax - a differentiable physics engine for large scale rigid body simulation, 2021. URL http://github.com/google/brax.

[15] Wanjia Fu, Hongyu Li, Ivy X He, Stefanie Tellex, and Srinath Sridhar. Unitac: Whole-robot touch sensing without tactile sensors. arXiv preprint arXiv:2507.07980, 2025.

[16] Sami Haddadin, Alin Albu-Schaffer, Alessandro De Luca, and Gerd Hirzinger. Collision detection and reaction: A contribution to safe physical human-robot interaction. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3356–3363. IEEE, 2008.

[17] Linyan Han, Jianliang Mao, Pengfei Cao, Yahui Gan, and Shihua Li. Toward sensorless interaction force estimation for industrial robots using high-order finite-time observers. IEEE Transactions on Industrial Electronics, 69(7):7275–7284, 2021.

[18] Ryo Hanai, Yukiyasu Domae, Ixchel G Ramirez-Alpizar, Bruno Leme, and Tetsuya Ogata. Force map: Learning to predict contact force distribution from vision. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3129–3136. IEEE, 2023.

[19] Eric Heiden, David Millard, Erwin Coumans, Yizhou Sheng, and Gaurav S Sukhatme. Neuralsim: Augmenting differentiable simulators with neural networks. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9474–9481. IEEE, 2021.

[20] Sepp Hochreiter and Jürgen Schmidhuber. Long shortterm memory. Neural computation, 9(8):1735–1780, 1997.

[21] Taylor A Howell, Simon Le Cleac’h, Jan Brüdigam, Qianzhong Chen, Jiankai Sun, J Zico Kolter, Mac Schwager, and Zachary Manchester. Dojo: A differentiable physics engine for robotics. arXiv preprint arXiv:2203.00806, 2022.

[22] Jin Hu and Rong Xiong. Contact force estimation for robot manipulator using semiparametric model and disturbance kalman filter. IEEE Transactions on Industrial Electronics, 65(4):3365–3375, 2017.

[23] Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelley, and Fr´edo Durand. Difftaichi: Differentiable programming for physical simulation. ICLR, 2020.

[24] Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter. Learning agile and dynamic motor skills for legged robots. Science Robotics, 4(26):eaau5872, 2019.

[25] Maged Iskandar, Alin Albu-Schäffer, and Alexander Dietrich. Intrinsic sense of touch for intuitive physical human-robot interaction. Science Robotics, 9(93): eadn4008, 2024.

[26] Yifeng Jiang, Jiazheng Sun, and C Karen Liu. Dataaugmented contact model for rigid body simulation. In Learning for dynamics and control conference, pages 378–390. PMLR, 2022.

[27] Sung-Woo Kim, Buyoun Cho, Seunghoon Shin, Jun-Ho Oh, Jemin Hwangbo, and Hae-Won Park. Force control of a hydraulic actuator with a neural network inverse model. IEEE Robotics and Automation Letters, 6(2): 2814–2821, 2021.

[28] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

[29] Masato Kobayashi, Thanpimon Buamanee, Yuki Uranishi, and Haruo Takemura. Ilbit: Imitation learning for robot using position and torque information based on bilateral control with transformer. IEEJ Journal of Industry Applications, 14(2):161–168, 2025.

[30] Hyeokjun Kwon, Sung-Woo Kim, and Hyun-Min Joe. Learning-based force control of twisted string actuators using a neural network-based inverse model. IEEE Robotics and Automation Letters, 2024.

[31] Magnus Linderoth, Andreas Stolt, Anders Robertsson, and Rolf Johansson. Robotic force estimation using motor torques and modeling of low velocity friction disturbances. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3550–3556. IEEE, 2013.

[32] Hongda Liu, Wentie Niu, and Yonghao Guo. Direct torque control for pmsm based on the rbfnn surrogate model of electromagnetic torque and stator flux linkage. Control Engineering Practice, 148:105943, 2024. doi: 10.1016/j.conengprac.2024.105943.

[33] Sichao Liu, Lihui Wang, and Xi Vincent Wang. Sensorless force estimation for industrial robots using disturbance observer and neural learning of friction approximation. Robotics and Computer-Integrated Manufacturing, 71:102168, 2021.

[34] Tribotix Pty Ltd. Pincherx 150 5dof robot arm. https://tribotix.com/product/pincherx-150-robot-arm/, 2020. Accessed: 2025-08-12.

[35] Miles Macklin. Warp: A high-performance python framework for gpu simulation and graphics. https://github.com/ nvidia/warp, March 2022. NVIDIA GPU Technology Conference (GTC).

[36] Emanuele Magrini, Fabrizio Flacco, and Alessandro De Luca. Estimation of contact forces using a virtual force sensor. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2126–2133. IEEE, 2014.

[37] Lucas Manuelli and Russ Tedrake. Localizing external contact using proprioceptive sensors: The contact particle filter. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5062–5069. IEEE, 2016.

[38] Jonas Osburg, Ivo Kuhlemann, Jannis Hagenah, and Floris Ernst. Using deep neural networks to improve contact wrench estimation of serial robotic manipulators in static tasks. Frontiers in Robotics and AI, 9:892916, 2022.

[39] Interbotix (Trossen Robotics). Phantomx reactor robot arm (ax-12a). https://www.interbotix.com/p/ phantomx-ax-12-reactor-robot-arm.aspx, 2025. Accessed: 2025-08-12.

[40] ROBOTIS Co., Ltd. Openmanipulator-x: Overview (rmx52-tnm). https://emanual.robotis.com/docs/en/platform/ openmanipulator x/overview/, 2017. Accessed: 2025-07- 12.

[41] Laura Schwendeman, Andrew SaLoutos, Elijah Stanger- Jones, and Sangbae Kim. Improving domain transfer of robot dynamics models with geometric system identification and learned friction compensation. In 2023 IEEE- RAS 22nd International Conference on Humanoid Robots (Humanoids), pages 1–8. IEEE, 2023.

[42] Agon Serifi, Espen Knoop, Christian Schumacher, Naveen Kumar, Markus Gross, and Moritz Bächer. Transformer-based neural augmentation of robot simulation representations. IEEE Robotics and Automation Letters, 8(6):3748–3755, 2023.

[43] Mikko Tahkola, Janne Keränen, Denis Sedov, Mehrnaz Farzam Far, and Juha Kortelainen. Surrogate modeling of electrical machine torque using artificial neural networks. IEEE Access, 8:220027–220045, 2020. doi: 10.1109/ACCESS.2020.3042834.

[44] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012. doi: 10.1109/IROS.2012.6386109.

[45] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025.

[46] Songyi Wang and Xinjian Wang. A pinn-based nonlinear pmsm electromagnetic model using differential inductance theory. Applied Sciences, 15(13):7162, 2025. doi: 10.3390/app15137162.

[47] Yufei Wang, David Held, and Zackory Erickson. Visual haptic reasoning: Estimating contact forces by observing deformable object interactions. IEEE Robotics and Automation Letters, 7(4):11426–11433, 2022.

[48] Jie Xu, Eric Heiden, Iretiayo Akinola, Dieter Fox, Miles Macklin, and Yashraj Narang. Neural robot dynamics. arXiv preprint arXiv:2508.15755, 2025.

[49] Yu-Bai Yan, Jia-Ning Liang, Tian-Fu Sun, Jian-Ping Geng, Gang Xie, and Dong-Jia Pan. Torque estimation and control of pmsm based on deep learning. In 2019 22nd International Conference on Electrical Machines and Systems (ICEMS). IEEE, 2019. doi: 10.1109/ICEMS. 2019.8921886.

[50] Andy Zeng, Shuran Song, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. Tossingbot: Learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics, 36(4):1307–1319, 2020.

[51] Peiyuan Zhi, Peiyang Li, Jianqin Yin, Baoxiong Jia, and Siyuan Huang. Learning a unified policy for position and force control in legged loco-manipulation. In Conference on Robot Learning, pages 652–669. PMLR, 2025.

NeuralActuator: Neural Actuation Modeling for Robot Dynamics and External Force Perception | AGIDaily