GPT-as-Policy \pageProject page

GPT-as-Policy
项目主页

Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies

系统探索 GPT-6 Astra 作为具身策略的能力

Galbot Team

Abstract

摘要

GPT-6 Astra exhibits a remarkable ability to generate numerical robot actions, extending its role beyond high-level planning. To assess Astra’s capabilities as general-purpose embodied policies, we conduct comprehensive evaluations across six domains, examining direct control, cooperation with learned policies, and feedback-driven adaptation. In gripper manipulation, Astra can correct task targets and prepare contact conditions for subsequent policy execution; hybrid control with $\pi_{0.5}$ achieves 48% success on the evaluated RoboDojo subset. In dexterous manipulation, hybrid control achieves 50% success in ten experience-guided DexJoCo trials, while direct in-hand control struggles to coordinate finger contacts. In mobile manipulation, hybrid control reaches 38.7% success on the evaluated RoboCasa365. In navigation, Astra leads our local comparisons, reaching 92% success on RxR instruction following and 82% on HM3D object search, although search incurs substantial detours. In locomotion, dense motion-reference generation remains unreliable: none of five sequential attempts on a single obstacle course reaches the goal, despite improvements in stability and forward progress. In humanoid loco-manipulation, Astra exceeds baseline methods on 13 of 30 HumanoidBench tasks with pretrained whole-body controllers. These findings reveal a gap between useful task decisions and reliable physical control. Inference latency further constrains practical control: across 50 RoboDojo instances per condition, policy-assisted and direct control consume 624.8 million and 1.132 billion tokens. A 30-second locomotion run requires 250 model calls averaging 39.86 seconds each, with physics paused during inference.

GPT-6 Astra 展现出生成数值化机器人动作的显著能力，使其角色超出高层规划。为评估 Astra 作为通用具身策略的能力，我们在六个领域开展综合评测，考察直接控制、与已学策略的协作，以及反馈驱动的适应。在夹爪操控中，Astra 能修正任务目标，并为后续策略执行准备接触条件；与 $\pi_{0.5}$ 的混合控制在所评测的 RoboDojo 子集上达到 48% 成功率。在灵巧操控中，混合控制在十次经验引导的 DexJoCo 试验中达到 50% 成功，而直接手内控制难以协调手指接触。在移动操控中，混合控制在所评测的 RoboCasa365 上达到 38.7% 成功。在导航中，Astra 在我们的本地对照中领先：RxR 指令跟随成功率 92%，HM3D 目标搜索成功率 82%，但搜索路径常有大幅绕行。在步行控制中，稠密运动参考的生成仍不可靠：同一障碍赛道上连续五次尝试均未到达目标，尽管稳定性与前向进度有所改善。在人形步行-操控中，借助预训练全身控制器，Astra 在 30 项 HumanoidBench 任务中有 13 项超过基线方法。这些发现揭示了「有用的任务决策」与「可靠的物理控制」之间的差距。推理时延进一步制约实际控制：在每种条件各 50 个 RoboDojo 实例上，策略辅助与直接控制分别消耗 6.248 亿与 11.32 亿 token；一次 30 秒的步行仿真需要 250 次模型调用，平均每次 39.86 秒，推理期间物理仿真暂停。

Introduction

引言

Community experiments with Astra reveal possibilities ranging from 3D scene understanding and real-to-sim workflows [14] to robot manipulation [64]. These demonstrations motivate a systematic assessment of Astra as an embodied policy: Which tasks can it perform, where does it remain unreliable, and what resources do its decisions require? The assessment must account for both successes and failures.

社区围绕 Astra 的实验揭示了多种可能：从三维场景理解与 real-to-sim 工作流 [14]，到机器人操控 [64]。这些演示促使我们系统评估 Astra 作为具身策略：它能完成哪些任务，在何处仍不可靠，其决策又需要怎样的资源？评估必须同时计入成功与失败。

We evaluate GPT-6 Astra [32] across six domains: gripper manipulation, dexterous manipulation, mobile manipulation, navigation, locomotion, and humanoid loco-manipulation. Our primary aim is to characterize performance and capability boundaries through task outcomes, comparisons with existing methods, failure analysis, and inference demands. We then analyze how interfaces, whole-body controllers, and feedback shape these outcomes.

我们在六个领域评测 GPT-6 Astra [32]：夹爪操控、灵巧操控、移动操控、导航、步行控制，以及人形步行-操控。首要目标是通过任务结果、与既有方法的对照、失败分析与推理开销，刻画性能与能力边界。随后分析接口、全身控制器与反馈如何塑造这些结果。

Existing work spans spatial understanding [35, 16] and reasoning for robot actions [36]. Vision–language–action models such as $\pi_{0.5}$ map observations and instructions to actions learned from robot experience [34]. SayCan selects grounded skills [1], Code as Policies composes perception and control through programs [21], and Prompt a Robot to Walk and Natural Language as Policies explore numerical feedback control [50, 28]. Recent Astra evaluations report uneven manipulation performance across tasks [64]. Our evaluation covers both Astra-generated robot commands, termed Direct, and cooperation with learned action policies or whole-body controllers, termed Hybrid. Figure 1 summarizes these control paths.

既有工作覆盖空间理解 [35, 16] 与机器人动作推理 [36]。视觉–语言–动作模型如 $\pi_{0.5}$ 将观测与指令映射为从机器人经验中学到的动作 [34]。SayCan 选择有 grounding 的技能 [1]，Code as Policies 用程序组合感知与控制 [21]，Prompt a Robot to Walk 与 Natural Language as Policies 则探索数值反馈控制 [50, 28]。近期对 Astra 的评测报告了跨任务不均衡的操控表现 [64]。我们的评测同时覆盖 Astra 生成的机器人指令（称为 Direct），以及与已学动作策略或全身控制器的协作（称为 Hybrid）。图 1 概括了这些控制路径。

Direct or Hybrid

End-effector or joint targets; gripper and base commands

Inverse kinematics, joint or operational-space control

Gripper, dexterous, mobile manipulation

Direct

Forward steps, turns, and stopping decisions

Discrete navigation primitives

Navigation

Direct or Hybrid

Joint PD targets or dense motion references

PD controller or frozen whole-body controller

Locomotion

Hybrid

Velocity commands or sparse body targets

Frozen whole-body controller

Humanoid loco-manipulation

Feedback loop: observe $\rightarrow$ Astra generates or reviews $\rightarrow$ robot moves $\rightarrow$ observe

Figure 1: Control modes, interfaces, and low-level control. Direct uses Astra-generated commands with analytic control. Hybrid combines Astra with a learned task policy or whole-body controller: Astra reviews policy proposals or supplies commands or motion references to frozen controllers. Observations of the resulting motion inform subsequent decisions.

Direct 或 Hybrid

末端执行器或关节目标；夹爪与底盘指令

逆运动学，关节或操作空间控制

夹爪、灵巧、移动操控

Direct

前进、转向与停止决策

离散导航原语

导航

Direct 或 Hybrid

关节 PD 目标或稠密运动参考

PD 控制器或冻结的全身控制器

步行控制

Hybrid

速度指令或稀疏身体目标

冻结的全身控制器

人形步行-操控

反馈回路：观测 $\rightarrow$ Astra 生成或审阅 $\rightarrow$ 机器人运动 $\rightarrow$ 再观测

图 1：控制模式、接口与底层控制。Direct 使用 Astra 生成的指令配合解析控制。Hybrid 将 Astra 与已学任务策略或全身控制器结合：Astra 审阅策略提案，或向冻结控制器提供指令/运动参考。对所得运动的观测再指导后续决策。

The domains expose complementary demands. Gripper and dexterous manipulation test grasping, local correction, and changing contact. Mobile manipulation adds base–arm coordination and multistage goals. Navigation tests spatial decisions and stopping, while locomotion requires coordinated body motion. Humanoid loco-manipulation combines task progression with pretrained whole-body controllers. We retain each study’s metrics and protocols to make the scope of its conclusions explicit.

各领域暴露互补的需求。夹爪与灵巧操控检验抓取、局部修正与接触变化；移动操控额外要求底盘–机械臂协调与多阶段目标；导航检验空间决策与停止时机，步行则要求协调的身体运动；人形步行-操控把任务推进与预训练全身控制器结合起来。我们保留各研究自带的指标与协议，以明确其结论适用范围。

The results reveal substantial but uneven capabilities. Astra leads our local navigation comparisons on success and path efficiency, and Hybrid systems achieve higher aggregate performance in several manipulation studies. Yet direct in-hand control trails task-specific RL, dense-reference locomotion remains unreliable despite repeated attempts, and assistance can fail on cases completed by a standalone policy. Token use and inference latency qualify these outcomes and must be measured separately from motion duration.

结果呈现出可观但不均衡的能力。Astra 在本地导航对照中于成功率与路径效率领先，Hybrid 系统在若干操控研究中取得更高的总体表现。然而，直接手内控制落后于任务专用 RL，稠密参考步行尽管反复尝试仍不可靠，且辅助也可能在独立策略本可完成的案例上失败。Token 用量与推理时延限定了这些结果的解读，必须与运动时长分开度量。

To interpret these boundaries, we examine the division of control. Once a reasoning model can generate numerical actions, task success alone does not reveal which responsibilities it assumes, which capabilities interfaces and whole-body controllers supply, or how feedback connects them. In successful manipulation cases, Astra can prepare conditions for a policy’s next action without generating the full trajectory. Conversely, preserving a grasp can impede required contact changes, and delegating motion can retain substantial proposal-review costs. This analysis helps explain the measured strengths and limitations while distinguishing Astra’s decisions from the capabilities of the complete system.

为解释这些边界，我们考察控制职责的划分。一旦推理模型能生成数值动作，仅看任务成功并不能说明它承担了哪些责任、接口与全身控制器提供了哪些能力，或反馈如何把二者连起来。在成功的操控案例中，Astra 可为策略的下一步准备条件，而无需生成完整轨迹。反过来，维持抓取可能妨碍必要的接触变化，而把运动委托出去仍可能保留可观的提案审阅开销。这一分析有助于解释测得的优势与局限，并把 Astra 的决策与整套系统的能力区分开来。

Evaluation Framework

评测框架

How Astra Produces Robot Actions

Astra 如何产生机器人动作

An action is a command submitted to the robot’s control interface, such as a target pose, a joint increment, or a navigation move. Astra receives the observations allowed by the study, selects an action, and uses the resulting feedback to choose the next one. The interface checks command bounds, and the environment determines task success. Tools support calculations, image inspection, and notes.

动作（action） 是提交给机器人控制接口的指令，例如目标位姿、关节增量或导航移动。Astra 接收研究所允许的观测，选择一个动作，并用所得反馈决定下一步。接口检查指令边界，环境判定任务是否成功。工具可用于计算、图像检查与记笔记。

Direct uses Astra-generated robot commands without a learned action policy or whole-body controller. Inverse kinematics and proportional–derivative control can implement these commands. Hybrid adds a learned task policy or whole-body controller.

Direct 使用 Astra 生成的机器人指令，不依赖已学动作策略或全身控制器。逆运动学与比例–微分（PD）控制可实现这些指令。Hybrid 则额外加入已学任务策略或全身控制器。

Hybrid takes two forms in this report. In manipulation, Astra accepts, modifies, or replaces a task policy’s action proposals; RoboCasa also permits subgoal rewriting. In humanoid control, Astra supplies dense motion references, velocity commands, or sparse body targets to frozen whole-body controllers. Locomotion includes both direct joint PD targets and references executed by ScaleTrack. Figure 1 shows these control paths. Each study specifies what Astra decides, how commands become motion, and what feedback informs the next action.

本报告中 Hybrid 有两种形式。在操控中，Astra 接受、修改或替换任务策略的动作提案；RoboCasa 还允许改写子目标。在人形控制中，Astra 向冻结的全身控制器提供稠密运动参考、速度指令或稀疏身体目标。步行控制既包含直接关节 PD 目标，也包含由 ScaleTrack 执行的参考。图 1 展示了这些控制路径。每项研究都写明 Astra 决定什么、指令如何变成运动，以及何种反馈指导下一步动作。

Within a trial, Astra uses feedback to generate or review the next action. Some studies retain notes or refine task guidance between trials with model weights fixed. Separate development studies involve researcher changes to prompts, interfaces, or whole-body controller settings. These procedures are described with their results and detailed in Appendix A.

在一次试验内，Astra 用反馈生成或审阅下一步动作。部分研究在模型权重固定的前提下，于试验之间保留笔记或 修订 任务指引。另有独立的开发性研究涉及研究者对提示、接口或全身控制器设置的改动。这些流程随结果一并说明，细节见附录 A。

What the Comparisons Establish

对照能说明什么

Table 1 identifies the tasks, sample sizes, and comparison conditions. Where methods start from the same physical state, their outcomes can be compared on the same task instance. Each study specifies its observations, controllers, and inference budgets.

表 1 列出任务、样本量与对照条件。当方法从同一物理状态起步时，可在同一任务实例上比较结果。每项研究各自规定观测、控制器与推理预算。

Domain	Tasks and sample sizes	What is compared
Gripper manipulation	RoboDojo and RoboLab each cover 10 tasks with 5 results per task and method.	RoboDojo compares paired Direct and Hybrid runs with published policy scores. RoboLab compares Direct and Hybrid with three zero-shot policies. Control settings and evaluation procedures appear in Appendix B.
Dexterous manipulation	10 manipulation tasks with 5 cases each; 4 in-hand tasks with 5 initial states each.	Manipulation compares Direct, Hybrid, and a standalone policy on the same cases. In-hand control compares Astra and task-specific RL from identical physical states, with different observations and action rates.
Mobile manipulation	15 RoboCasa tasks with 5 episodes per task and method.	Direct, Hybrid, and a locally evaluated standalone policy share initial states and 20-step action segments. The Astra conditions differ in policy access, context management, and feedback guards.
Navigation	4 dataset subsets with 50 episodes per system in each.	Astra and released navigation policies use the same episode lists, scoring rules, and action budgets. Camera views and the conversion of outputs to navigation actions differ between systems.
Locomotion	5 sequential Astra attempts on one course. Two reference interfaces, each tested on 6 courses in 2 physics backends: 12 rollouts per interface.	Astra and PASSAGE [26] share the initial state, goal, and frozen tracker. Separate analytic tests compare five-point and whole-body reference interfaces without Astra.
Humanoid loco-manipulation	HumanoidBench covers 30 tasks. SIMPLE covers 6 L2 tasks with 10 scenes each.	HumanoidBench compares Astra returns with published DreamerV3, TD-MPC2, and SAC results. SIMPLE uses the official codebase and protocol for comparison.

Table 1: Evaluation scope and comparison conditions. Direct uses Astra-generated commands with analytic control; Hybrid adds a learned task policy or whole-body controller. Sample sizes are reported separately for each study.

领域	任务与样本量	比较内容
夹爪操控	RoboDojo 与 RoboLab 各覆盖 10 项任务，每任务每方法 5 次结果。	RoboDojo 将配对的 Direct 与 Hybrid 运行对照已发表策略分数；RoboLab 将 Direct、Hybrid 与三种零样本策略对照。控制设置与评测流程见附录 B。
灵巧操控	10 项操控任务各 5 例；4 项手内任务各 5 个初始状态。	操控对照在相同案例上比较 Direct、Hybrid 与独立策略；手内控制在相同物理起点上比较 Astra 与任务专用 RL，但观测与动作频率不同。
移动操控	15 项 RoboCasa 任务，每任务每方法 5 个 episode。	Direct、Hybrid 与本地评测的独立策略共享初始状态与 20 步动作段。Astra 条件在策略访问、上下文管理与反馈护栏上不同。
导航	4 个数据集子集，每系统每子集 50 个 episode。	Astra 与已发布导航策略使用相同 episode 列表、计分规则与动作预算。相机视角以及输出到导航动作的转换因系统而异。
步行控制	同一赛道上 Astra 连续 5 次尝试。两种参考接口各在 6 条赛道、2 种物理后端上测试：每接口 12 次 rollout。	Astra 与 PASSAGE [26] 共享初始状态、目标与冻结 tracker。另有不含 Astra 的解析测试，比较五点与全身参考接口。
人形步行-操控	HumanoidBench 覆盖 30 项任务；SIMPLE 覆盖 6 项 L2 任务，每任务 10 个场景。	HumanoidBench 将 Astra 回报与已发表的 DreamerV3、TD-MPC2、SAC 结果对照；SIMPLE 使用官方代码库与协议对照。

表 1：评测范围与对照条件。 Direct 使用 Astra 生成指令配合解析控制；Hybrid 额外加入已学任务策略或全身控制器。样本量按研究分别报告。

We report fixed-configuration evaluations and development trials separately. The former measure performance under specified conditions; the latter track changes in behavior as prompts, interfaces, or whole-body controller settings are refined. Transfer trials assess the resulting behavior on additional states.

我们分别报告固定配置评测与开发性试验。前者度量指定条件下的表现；后者跟踪提示、接口或全身控制器设置 修订 过程中行为的变化。迁移试验评估所得行为在额外状态上的表现。

Performance depends on the complete system, including the available commands, written guidance, and whole-body controller, even when Astra’s weights remain fixed. For example, the grasp interface determines which actions are possible, while gait calibration changes how commands produce motion.

即便 Astra 权重固定，表现仍取决于完整系统——可用指令、书面指引与全身控制器。例如，抓取接口决定哪些动作可行，而步态标定会改变指令如何产生运动。

Measuring Progress and Timing

进度与时序的度量

We distinguish completing a task from making partial progress or merely avoiding failure. The metrics must capture the intended change in physical state. We interpret rotation error together with angular speed, navigation success with path efficiency, and humanoid reward with observed progress and termination.

我们区分「完成任务」「取得部分进度」与「仅仅避免失败」。指标必须捕捉到预期的物理状态变化。旋转误差需结合角速度解读，导航成功需结合路径效率，人形回报需结合观察到的进度与终止。

Controller frequency and reasoning speed are separate quantities. A fast controller can apply actions while Astra takes much longer to choose the next command. When physics pauses during inference, the robot-motion clock excludes decision latency. Resource summaries distinguish robot control steps, action segments, model requests, and cached input, uncached input, and output tokens. Astra can review many policy proposals while changing few actions. Section 9.4 summarizes inference demands.

控制器频率与推理速度是不同的量。快速控制器可以在 Astra 仍在选择下一条指令时持续施加动作。当推理期间物理仿真暂停时，机器人运动时钟不含决策时延。资源汇总区分机器人控制步、动作段、模型请求，以及缓存输入、非缓存输入与输出 token。Astra 可审阅大量策略提案却只改动少量动作。第 9.4 节汇总推理开销。

Gripper Manipulation

夹爪操控

We compare two closed-loop control architectures for manipulation. Direct uses GPT 6 Astra alone: given images, proprioception, the task instruction, and execution history, it generates bimanual end-effector (EEF) targets and gripper commands, executes one to five control steps, and then observes the new state. Hybrid uses a learned policy, $\pi_{0.5}$, to propose a 50-step joint-space action sequence. GPT 6 Astra reviews the proposal and either executes its first one to fifteen steps or replaces it with a one-to-five-step EEF correction; the two branches are mutually exclusive. Both architectures use the same task descriptions, success criteria, GPT 6 Astra reasoning setting, and EEF execution interface.

我们比较两种闭环操控架构。Direct 单独使用 GPT-6 Astra：给定图像、本体感觉、任务指令与执行历史，它生成双臂末端执行器（EEF）目标与夹爪指令，执行 1–5 个控制步，再观测新状态。Hybrid 用已学策略 $\pi_{0.5}$ 提出 50 步关节空间动作序列；GPT-6 Astra 审阅该提案，或执行其前 1–15 步，或以 1–5 步 EEF 修正替换之——两支互斥。两种架构使用相同的任务描述、成功判据、GPT-6 Astra 推理设置与 EEF 执行接口。

Figure 2: Gripper manipulation results. (a,b) RoboDojo Score and final success on ten tasks. Public references are reweighted to the same task and scene mixture, whereas Direct and Hybrid use paired starts. Direct has 48 valid Score entries; success is computed over all 50 task instances. (c) RoboLab success rates. (d) Of the Hybrid control steps, 85.6% follow $\pi_{0.5}$ and 14.4% are generated or corrected by GPT 6 Astra.

图 2：夹爪操控结果。(a,b) 十项任务上的 RoboDojo Score 与最终成功率。公开参考结果已按相同任务与场景配比重加权；Direct 与 Hybrid 使用配对起点。Direct 有 48 条有效 Score；成功率按全部 50 个任务实例计算。(c) RoboLab 成功率。(d) Hybrid 控制步中，85.6% 跟随 $\pi_{0.5}$，14.4% 由 GPT-6 Astra 生成或修正。

Control and evaluation setup

控制与评测设置

The two architectures test complementary uses of GPT 6 Astra. Direct asks the model to construct the complete EEF action from the current state. Hybrid gives the model a candidate trajectory from $\pi_{0.5}$ and asks it to preserve the trajectory or make a local correction. In Hybrid, GPT 6 Astra can therefore change the target, prepare the contact geometry, or continue after the simulator reports that the task is not yet complete, while $\pi_{0.5}$ supplies most of the low-level motion. These architectures are evaluated separately on RoboDojo and RoboLab. Qualitative trajectories and the complete set of video cases are available in the accompanying web report.

两种架构检验 GPT-6 Astra 的互补用法。Direct 要求模型从当前状态构造完整 EEF 动作；Hybrid 给模型一条来自 $\pi_{0.5}$ 的候选轨迹，并要求保留该轨迹或做局部修正。因此在 Hybrid 中，GPT-6 Astra 可改变目标、准备接触几何，或在仿真器报告任务尚未完成时继续推进，而 $\pi_{0.5}$ 提供大部分底层运动。这两种架构分别在 RoboDojo 与 RoboLab 上评测。定性轨迹与完整视频案例见配套网页报告。

RoboDojo: paired bimanual evaluation

RoboDojo：配对双臂评测

Setting.

RoboDojo [8] is a bimanual manipulation benchmark covering semantic classification, sequence imitation, packing, construction, and deformable-object manipulation. We select ten tasks by stratifying the published $\pi_{0.5}$ success rates: six tasks from the lowest interval, two from the next, and one from each of the two higher intervals. This selection emphasizes tasks with room for improvement while retaining diverse interaction requirements. Each task is evaluated five times for each architecture. Direct and Hybrid use the same task instances, scene configurations, and random seeds. Number arrangement, packing, and clothes folding use two standard and three randomized scenes; the other tasks use five standard layouts. We report RoboDojo’s native final success rate and partial-completion Score. Hybrid uses the task-finetuned $\pi_{0.5}$ weights released for RoboDojo.

设置。

RoboDojo [8] 是双臂操控基准，覆盖语义分类、序列模仿、装箱、搭建与可变形物体操控。我们按已发表的 $\pi_{0.5}$ 成功率分层选取十项任务：最低区间六项、次低区间两项，以及两个较高区间各一项。该选取强调仍有提升空间的任务，同时保留多样的交互需求。每项任务对每种架构评测五次。Direct 与 Hybrid 使用相同任务实例、场景配置与随机种子。数字排列、装箱与叠衣服使用两个标准场景加三个随机化场景；其余任务使用五个标准布局。我们报告 RoboDojo 原生最终成功率与部分完成 Score。Hybrid 使用为 RoboDojo 发布的任务微调 $\pi_{0.5}$ 权重。

Results.

Hybrid succeeds on 24 of 50 instances (48%), compared with 13 of 50 (26%) for Direct. Its mean Score is 62.60, compared with 37.81 for Direct. The improvement is concentrated in tasks that require long-horizon coordination or contact-sensitive interaction: Hybrid scores 53 versus 0 on sequence imitation, 64 versus 12 on tower construction, 100 versus 40 on clothes folding, and 100 versus 36 on bottle disposal. Direct is higher on language-based classification (60 versus 38) and object classification (100 versus 71), showing that the Hybrid advantage is not uniform across tasks. In the 50 Hybrid trajectories, 36,576 of 42,750 executed control steps (85.6%) follow $\pi_{0.5}$, while 6,174 (14.4%) are generated or corrected by GPT 6 Astra. The model therefore intervenes selectively rather than replacing the learned policy.

结果。

Hybrid 在 50 例中成功 24 例（48%），Direct 为 13/50（26%）。其平均 Score 为 62.60，Direct 为 37.81。提升集中在需要长程协调或接触敏感交互的任务：序列模仿 Hybrid 53 对 Direct 0，搭塔 64 对 12，叠衣服 100 对 40，扔瓶入箱 100 对 36。Direct 在基于语言的分类（60 对 38）与物体分类（100 对 71）上更高，说明 Hybrid 优势并非跨任务均匀。在 50 条 Hybrid 轨迹中，42,750 个已执行控制步里有 36,576 步（85.6%）跟随 $\pi_{0.5}$，6,174 步（14.4%）由 GPT-6 Astra 生成或修正。因此模型是选择性介入，而非替换已学策略。

Reweighting the published RoboDojo results to the evaluated ten-task and scene mixture gives $\pi_{0.5}$ a success rate of 15.67% and a Score of 24.43. Table 2 lists the published references alongside our paired Direct and Hybrid runs.

将已发表 RoboDojo 结果按所评十项任务与场景配比重加权后，$\pi_{0.5}$ 成功率为 15.67%，Score 为 24.43。表 2 将公开参考与我们的配对 Direct、Hybrid 运行并列列出。

	Reweighted public references	Our evaluation
Task	DM0.5	Galaxea	Xiaomi R1	OpenWAM	$\pi_{0.5}$	Hybrid	Direct
Organize the table	44.00	46.33	57.67	62.50	23.33	60.00	30.00
Classify by language	0.47	1.07	2.00	1.33	0.60	38.00	60.00
Imitate a sorting sequence	1.80	1.67	2.50	2.90	1.60	53.00	0.00
Arrange the largest number	7.85	4.11	8.56	4.36	2.29	50.00	57.00
Pack objects into a box	14.72	17.12	18.69	20.83	18.36	50.00	50.00
Classify objects	26.83	10.33	17.20	5.53	24.67	71.00	100.00
Build a tower	55.20	82.93	52.60	52.53	37.73	64.00	12.00
Make a Kong in Mahjong	56.67	90.00	41.33	32.00	26.67	40.00	0.00
Fold clothes	28.96	32.75	41.49	51.31	29.12	100.00	40.00
Put bottles in a bin	81.70	96.30	97.70	94.03	79.93	100.00	36.00
Aggregate	31.82	38.26	33.97	32.73	24.43	62.60	37.81

Table 2: RoboDojo task-level Score. Public references are reweighted to the evaluation task and scene mixture. Direct and Hybrid are our paired evaluations; the Direct Score uses 48 valid entries, while success uses all 50 task instances.

	重加权公开参考	本评测
任务	DM0.5	Galaxea	Xiaomi R1	OpenWAM	$\pi_{0.5}$	Hybrid	Direct
Organize the table	44.00	46.33	57.67	62.50	23.33	60.00	30.00
Classify by language	0.47	1.07	2.00	1.33	0.60	38.00	60.00
Imitate a sorting sequence	1.80	1.67	2.50	2.90	1.60	53.00	0.00
Arrange the largest number	7.85	4.11	8.56	4.36	2.29	50.00	57.00
Pack objects into a box	14.72	17.12	18.69	20.83	18.36	50.00	50.00
Classify objects	26.83	10.33	17.20	5.53	24.67	71.00	100.00
Build a tower	55.20	82.93	52.60	52.53	37.73	64.00	12.00
Make a Kong in Mahjong	56.67	90.00	41.33	32.00	26.67	40.00	0.00
Fold clothes	28.96	32.75	41.49	51.31	29.12	100.00	40.00
Put bottles in a bin	81.70	96.30	97.70	94.03	79.93	100.00	36.00
Aggregate	31.82	38.26	33.97	32.73	24.43	62.60	37.81

表 2：RoboDojo 任务级 Score。 公开参考已按评测任务与场景配比重加权。Direct 与 Hybrid 为本配对评测；Direct 的 Score 使用 48 条有效记录，成功率使用全部 50 个任务实例。

RoboLab: zero-shot evaluation

RoboLab：零样本评测

Setting.

RoboLab [55] evaluates a single-arm Franka on ten tasks involving semantic pick-and-place, ordered block stacking, and mug reorientation. We report five trials per task and compare Direct, Hybrid, $\pi_{0.5}$, Cosmos3-Nano-Policy [30], and DreamZero [56]. This benchmark is evaluated with final task success only; RoboDojo’s partial-completion Score is not used. RoboLab provides no training data for the test tasks. The policy baselines therefore use DROID-trained weights for zero-shot transfer, unlike RoboDojo, where Hybrid uses task-finetuned $\pi_{0.5}$.

设置。

RoboLab [55] 在单臂 Franka 上评测十项任务，涉及语义抓放、有序搭积木与杯子重定向。每任务报告五次试验，并比较 Direct、Hybrid、$\pi_{0.5}$、Cosmos3-Nano-Policy [30] 与 DreamZero [56]。该基准只看最终任务成功，不使用 RoboDojo 的部分完成 Score。RoboLab 不为测试任务提供训练数据，因此策略基线使用 DROID 训练权重做零样本迁移；这与 RoboDojo 上 Hybrid 使用任务微调 $\pi_{0.5}$ 不同。

Results.

Direct succeeds on 49/50 trials (98%), with five successes on nine of the ten tasks. Hybrid succeeds on 46/50 (92%), while $\pi_{0.5}$, Cosmos3-Nano-Policy, and DreamZero succeed on 18/50 (36%), 18/50 (36%), and 17/50 (34%), respectively. Hybrid therefore improves over $\pi_{0.5}$ by 56 percentage points, but is three successful trajectories below Direct on this task subset.

结果。

Direct 在 50 次试验中成功 49 次（98%），十项任务中有九项取得五次全成功。Hybrid 成功 46/50（92%），而 $\pi_{0.5}$、Cosmos3-Nano-Policy 与 DreamZero 分别为 18/50（36%）、18/50（36%）与 17/50（34%）。因此 Hybrid 相对 $\pi_{0.5}$ 提升 56 个百分点，但在该任务子集上比 Direct 少三条成功轨迹。

Discussion: why the benchmark rankings differ

讨论：为何两个基准排名不同

RoboDojo and RoboLab favor different capabilities. RoboDojo contains bimanual, long-horizon, deformable, and contact-sensitive tasks. On these tasks, the learned policy supplies interaction patterns and coordinated motion, while GPT 6 Astra can correct the target, local geometry, or task-progress interpretation. This division of labor is consistent with Hybrid outperforming Direct on RoboDojo.

RoboDojo 与 RoboLab 偏爱不同能力。RoboDojo 含双臂、长程、可变形与接触敏感任务。在这些任务上，已学策略提供交互模式与协调运动，而 GPT-6 Astra 可修正目标、局部几何或任务进度解读。这一分工与 Hybrid 在 RoboDojo 上优于 Direct 相一致。

The RoboLab tasks are predominantly single-arm semantic pick-and-place problems, with additional ordered stacking and mug reorientation. Direct can already plan these operations from the current observation and reaches 98% success. In this setting, a candidate action from a zero-shot policy is not necessarily a useful prior: $\pi_{0.5}$ alone succeeds on only 36% of trials, so Hybrid may spend decisions correcting an action that is poorly adapted to the task. This suggests that the value of policy assistance depends on how well the learned motion prior fits the task.

RoboLab 任务以单臂语义抓放为主，另有有序堆叠与杯子重定向。Direct 已能从当前观测规划这些操作，并达到 98% 成功。在此设定下，零样本策略给出的候选动作未必是有用先验：$\pi_{0.5}$ 单独仅成功 36%，故 Hybrid 可能把决策花在修正并不贴合任务的动作上。这表明策略辅助的价值取决于已学运动先验与任务的匹配程度。

Dexterous Manipulation

灵巧操控

Dexterous manipulation requires both task-level planning, such as selecting a suitable grasp and deciding how to use it, and precise coordination of finger contacts during execution. To assess these capabilities, we evaluate Astra on end-to-end manipulation tasks, with optional policy assistance, and on in-hand control from an established grasp. The former tests acquiring and using a grasp; the latter tests changing finger contacts while maintaining object support.

灵巧操控既需要任务级规划（如选择合适抓取并决定如何使用），也需要执行中对指尖接触的精确协调。为评估这些能力，我们在端到端操控任务上评测 Astra（可选策略辅助），并在已建立抓取后评测手内控制。前者检验获取与使用抓取；后者检验在保持物体支撑的同时改变指尖接触。

Policy Assistance Across Ten Tasks

十项任务上的策略辅助

The first benchmark covers grasping, retrieval, placement, insertion, stacking, and rearrangement with simulated Sharpa hands [45]. One multitask $\pi_{0.5}$ policy is finetuned on 100 demonstrations per task (1,000 in total). For evaluation, Direct, standalone $\pi_{0.5}$, and Hybrid are each tested on the same five unseen cases per task, with simulation horizons of 20–60 s. Direct outputs wrist and finger targets; Hybrid reviews or corrects policy actions, executing prefixes of 1–16 steps or corrections of 1–5 steps at approximately 30 Hz. When the review or token budget is exhausted, the system falls back to $\pi_{0.5}$-only control.

第一项基准覆盖抓取、取回、放置、插入、堆叠与重排，使用仿真 Sharpa 手 [45]。一个多任务 $\pi_{0.5}$ 策略在每任务 100 条演示（共 1,000 条）上微调。评测时，Direct、独立 $\pi_{0.5}$ 与 Hybrid 各在相同的五个未见案例上测试，仿真时域 20–60 秒。Direct 输出腕与手指目标；Hybrid 审阅或修正策略动作，执行 1–16 步前缀或 1–5 步修正，约 30 Hz。当审阅或 token 预算耗尽时，系统回退为仅 $\pi_{0.5}$ 控制。

We evaluate performance using task completion scores on a 0–100 scale. The scores give partial credit for progress rather than measuring binary success, and we average them equally across tasks. Full credit requires task-specific verification of a maintained hold or a stable release. Appendix G provides details on observations, training, actions, and scoring.

我们用 0–100 的任务完成分数评测表现。分数对进度给部分分，而非二元成功，并在任务间等权平均。满分要求按任务核验「保持住」或「稳定释放」。观测、训练、动作与计分细节见附录 G。

As shown in Table 3, Hybrid achieves a mean Score of 61.6, compared with 44.2 for $\pi_{0.5}$ and 16.6 for Direct. Hybrid achieves a higher mean score than Direct on all ten tasks. Compared with $\pi_{0.5}$, it scores higher on eight tasks and ties on the remaining two. Gains over the policy are largest for mahjong storage at 40 points, upright egg placement at 32, and bottle/can sorting at 28. On bread insertion and nesting-doll ordering, Hybrid improves only slightly over $\pi_{0.5}$, with scores remaining at 28 and 24, respectively.

如表 3 所示，Hybrid 平均 Score 为 61.6，高于 $\pi_{0.5}$ 的 44.2 与 Direct 的 16.6。Hybrid 在全部十项任务上均高于 Direct；相对 $\pi_{0.5}$，有八项更高、两项持平。相对策略的最大提升出现在麻将收纳（+40）、竖放鸡蛋（+32）与瓶罐分拣（+28）。在面包插入与套娃排序上，Hybrid 仅略优于 $\pi_{0.5}$，分数仍为 28 与 24。

Task	$\pi_{0.5}$	Direct	Hybrid
Pot lift and hold	40.0	44.0	62.0
Headphones in box	42.0	22.0	42.0
Toy retrieval	100.0	22.0	100.0
Mug hanging	62.0	4.0	82.0
Mahjong tile storage	48.0	18.0	88.0
Bottles/cans sorting	72.0	14.0	100.0
Upright egg placement	6.0	12.0	38.0
Two-bowl stacking	46.0	20.0	52.0
Bread-slot insertion	10.0	8.0	28.0
Nesting-doll ordering	16.0	2.0	24.0
Overall mean	44.2	16.6	61.6

Table 3: Dexterous manipulation scores on a custom simulation benchmark. Scores measure subgoal completion on a 0–100 scale. The evaluation protocol and scoring details are provided in Appendix G.4.

任务	$\pi_{0.5}$	Direct	Hybrid
Pot lift and hold	40.0	44.0	62.0
Headphones in box	42.0	22.0	42.0
Toy retrieval	100.0	22.0	100.0
Mug hanging	62.0	4.0	82.0
Mahjong tile storage	48.0	18.0	88.0
Bottles/cans sorting	72.0	14.0	100.0
Upright egg placement	6.0	12.0	38.0
Two-bowl stacking	46.0	20.0	52.0
Bread-slot insertion	10.0	8.0	28.0
Nesting-doll ordering	16.0	2.0	24.0
Overall mean	44.2	16.6	61.6

表 3：自定义仿真基准上的灵巧操控分数。 分数按 0–100 度量子目标完成。评测协议与计分细节见附录 G.4。

Figure 3(a) categorizes Astra’s interventions by failure type. The most common address missed grasps or dropped objects and failed placements, followed by placement alignment and grip stabilization. In individual trajectories, Astra redirects wrists toward displaced objects, clears occlusions, refines placements, and resumes unfinished subgoals. These interventions account for only 11.98% of executed Hybrid steps (Figure 3(b)). Together with the task scores, this suggests that targeted changes to a small fraction of steps can yield substantial performance gains.

图 3(a) 按失败类型归类 Astra 的干预。最常见的是处理未抓取/掉落与放置失败，其次是放置对齐与握持稳定。在个别轨迹中，Astra 会把腕部重新指向移位物体、清除遮挡、修订放置并恢复未完成子目标。这些干预仅占已执行 Hybrid 步的 11.98%（图 3(b)）。结合任务分数可见，对一小部分步骤做针对性改动即可带来可观提升。

Even with these gains, grasp acquisition and completion checking remain the primary sources of failure. Direct can fail to establish a grasp when simultaneous finger closure pushes the object away or a weak two-finger lift leaves it poorly supported. Hybrid faces a similar difficulty when an object drops into an unfamiliar configuration: if the policy can no longer provide useful actions, Astra must generate a new grasp pose but may still fail to secure the object. These grasp failures suggest that Astra still relies heavily on a capable policy to generate viable action candidates for dexterous manipulation. In addition, Astra can misjudge task completion, as in an egg-placement case where it declares success while the egg remains on its side.

即便有这些提升，抓取获取与完成判定仍是主要失败来源。Direct 可能在同时合拢手指把物体推开，或弱双指提起导致支撑不足时建立不了抓取。当物体掉到陌生构型时，Hybrid 面临类似困难：若策略不再提供有用动作，Astra 必须生成新抓取位姿，但仍可能抓不稳。这些抓取失败表明，Astra 在灵巧操控上仍高度依赖有能力的策略来生成可行动作候选。此外 Astra 可能误判任务完成，例如在鸡蛋放置案例中，蛋仍侧躺却宣称成功。

Figure 3: Astra corrections in dexterous manipulation. (a) Primary reasons for Astra correction decisions across 50 Hybrid cases. (b) Executed steps comprise 88.02% unmodified $\pi_{0.5}$ actions and 11.98% Astra corrections.

图 3：灵巧操控中的 Astra 修正。(a) 50 个 Hybrid 案例中 Astra 修正决策的主要原因。(b) 已执行步骤中 88.02% 为未修改的 $\pi_{0.5}$ 动作，11.98% 为 Astra 修正。

Feedback-Guided Bimanual Manipulation in DexJoCo

DexJoCo 中的反馈引导双臂操控

The Hanoi disk-stacking task in DexJoCo [48] tests two ordered transfers with dual Franka Panda arms and Allegro hands: the right hand moves the medium disk to the destination peg, then the left hand places the small disk above it. Astra reviews a fixed $\pi_{0.5}$ policy trained for this task, accepting action prefixes or correcting wrist and finger targets using visual feedback. Each trial allows up to 1,500 control steps at 50 Hz, with at most 30 steps per action segment. Ten trials run in five successive pairs, starting from prior task experience. Written guidance is shared within each pair and refined between pairs, with model weights fixed. Image-history limits and compact references to older proposals are introduced during the run.

DexJoCo [48] 中的汉诺塔圆盘堆叠任务检验双 Franka Panda 臂与 Allegro 手上的两次有序转移：右手把中盘移到目标柱，再由左手把小盘放到其上。Astra 审阅为该任务训练的固定 $\pi_{0.5}$ 策略，接受动作前缀，或用视觉反馈修正腕与手指目标。每次试验最多 1,500 个控制步（50 Hz），每段动作最多 30 步。十次试验分成五对连续进行，从先前任务经验起步。书面指引在每对内共享、在对之间修订，模型权重固定。运行过程中还引入了图像历史限制与对旧提案的紧凑引用。

The system completes 5/10 trials under the native success criterion. Figure 4 places this result alongside published policy success rates on the same task. Of 11,320 executed steps, 73.6% use unmodified policy actions and 26.4% use Astra corrections. These results summarize the full sequence of experience-guided trials. The trajectories in Figure 5 show useful local interventions alongside persistent failures in grasp retention and final placement.

按原生成功判据，系统完成 5/10 次试验。图 4 将该结果与同任务上已发表策略成功率并列。在 11,320 个已执行步中，73.6% 使用未修改策略动作，26.4% 使用 Astra 修正。这些结果汇总了整段经验引导试验序列。图 5 的轨迹既显示有用的局部干预，也显示抓取保持与最终放置上的持续失败。

Figure 4: Success on bimanual disk stacking. Published baselines average three sets of 50 rand-obj episodes [48]; Astra + $\pi_{0.5}$ succeeds in 5/10 experience-guided trials.

图 4：双臂圆盘堆叠成功率。已发表基线为三组各 50 次 rand-obj episode 的平均 [48]；Astra + $\pi_{0.5}$ 在 10 次经验引导试验中成功 5 次。

A successful placement requires more than reaching the correct peg. In one successful trial, Astra delays a proposed release because the small disk remains above its support. Short downward corrections bring the disk into contact with the stack, after which Astra returns control to $\pi_{0.5}$ for release and withdrawal. The resulting hand clearance allows a separate check that the disk remains supported. This sequence illustrates Astra’s role in deciding when a physical precondition has been met, while the policy supplies the subsequent motion.

成功放置不止是到达正确柱子。在一次成功试验中，Astra 因小盘仍悬在支撑上方而推迟拟议的释放；短暂向下修正使圆盘与堆叠接触后，Astra 把控制交回 $\pi_{0.5}$ 以完成释放与抽手。由此产生的手部净空允许单独核验圆盘是否仍被支撑。该序列说明 Astra 负责判断物理前置条件是否满足，而策略提供后续运动。

The same strategy is not consistently effective. In a failed trial, repeated seating corrections leave the small disk tilted high on the destination peg at the step limit. Other failures involve losing a grasp during transport or failing to lift the small disk from its source. Astra’s own corrections can also hinder progress: in another successful trial, two wrist adjustments move the held disk away from the destination, whereas returning control to the policy restores transport. Effective cooperation therefore requires both timely intervention and recognition that the policy may offer the better recovery. The trials and five reviews consume 63.60 million recorded tokens, including cached input; only 0.136 million are used for review. Most token use arises during online control, even though the policy supplies most physical actions.

同一策略并非始终有效。在一次失败试验中，反复的落座修正使小盘在步数上限时仍高高倾斜于目标柱上。其他失败包括运输中丢抓，或未能从小盘源位置提起。Astra 自身的修正也可能妨碍进度：在另一次成功试验中，两次腕部调整把所持圆盘带离目标，而交回策略后运输得以恢复。因此有效协作既需要及时干预，也需要识别策略可能提供更好的恢复。试验与五次审阅共消耗 6,360 万记录 token（含缓存输入）；审阅仅用 13.6 万。尽管策略提供大部分物理动作，多数 token 仍花在在线控制上。

Figure 5: Bimanual disk stacking: successful placement and failed seating. Cropped wrist-camera observations show (a) Astra corrections followed by policy release and (b) persistent placement failure despite further corrections. Labels give control steps; outcomes follow the native success criterion.

图 5：双臂圆盘堆叠：成功放置与失败落座。裁剪的腕部相机观测显示 (a) Astra 修正后由策略释放，以及 (b) 尽管继续修正仍持续放置失败。标签为控制步；结果按原生成功判据。

Stable Grasping, Limited Rotation

抓取稳定，旋转仍受限

Once a grasp is established, success requires moving the object while maintaining support. We compare direct Astra joint commands with four frozen, task-specific RL policies to examine this demand. Sharpa tasks require continuous cylinder or cuboid rotation about world-frame $+Z$; Allegro tasks require cylinder translation, with or without an additional $-40^{\circ}$ long-axis rotation. Rotation tasks draw on ConTrack [22] and mjlab_hand [38]; translation follows tactile in-hand manipulation [58].

一旦建立抓取，成功还要求在保持支撑的同时移动物体。我们把 Astra 的直接关节指令与四个冻结的任务专用 RL 策略比较，以考察这一需求。Sharpa 任务要求圆柱或长方体绕世界系 $+Z$ 连续旋转；Allegro 任务要求圆柱平移，并可附加绕长轴 $-40^{\circ}$ 旋转。旋转任务借鉴 ConTrack [22] 与 mjlab_hand [38]；平移遵循触觉手内操控 [58]。

Each task uses five paired saved hand/object states and targets. Astra receives images and state and commands 22 Sharpa or 16 Allegro joints for one to five 20-Hz steps. RL uses its training observations and acts every step. Thus physical starts are paired, but observations and decision rates differ. Four cuboid starts overlap the RL initialization bank. Training and per-case records appear in Appendix H.

每项任务使用五组配对保存的手/物体状态与目标。Astra 接收图像与状态，对 22 个 Sharpa 或 16 个 Allegro 关节发出 1–5 个 20 Hz 步的指令。RL 使用其训练观测并逐步动作。因此物理起点配对，但观测与决策频率不同。四个长方体起点与 RL 初始化库重叠。训练与逐案例记录见附录 H。

An established grasp does not remove Astra’s difficulty with sustained object motion. RL tracks the rotation target within tolerance for 76.90% of cylinder steps and 63.50% of cuboid steps, versus 0.51% and 4.40% for Astra (Table 4). Astra reaches the horizon in three cylinder runs, drops the object once, and exhausts its decision budget once. The cuboid results isolate the distinction between retention and progress more clearly: both controllers retain the object through all five horizons, yet Astra rotates it too slowly. The gap therefore persists even when the grasp remains secure.

已建立抓取并不能消除 Astra 在持续物体运动上的困难。RL 在圆柱步中有 76.90%、在长方体步中有 63.50% 将旋转目标保持在容差内，Astra 仅 0.51% 与 4.40%（表 4）。Astra 在三次圆柱运行中到达时域终点，掉落一次，决策预算耗尽一次。长方体结果更清晰地隔离「保持」与「进度」：两种控制器在全部五个时域内都保住物体，但 Astra 转得太慢。因此即便抓取稳固，差距仍在。

Figure 6: Continuous rotation over five paired initial states. Rows show cylinder results over 20 s and cuboid results over 10 s. Columns report orientation error, axial speed over the preceding second, cumulative at-goal rate, and cumulative reward. Curves and shading show means and one sample SD among active runs. Dashed lines mark target speeds and the $0.1$-rad goal threshold. Periodic orientation error requires interpretation alongside axial speed.

图 6：五组配对初始状态下的连续旋转。行分别给出圆柱 20 秒与长方体 10 秒结果。列报告朝向误差、前一秒轴向速度、累计到目标率与累计回报。曲线与阴影为仍在运行轨迹的均值与一个样本标准差。虚线标出目标速度与 $0.1$ rad 目标阈值。周期性朝向误差需结合轴向速度解读。

The motion traces show how this gap develops. Astra’s cylinder speed stays near zero, while the cuboid turns gradually but falls behind the target (Figure 6). Representative cuboid sequences show little contact reconfiguration under Astra, whereas RL continues reorientation (Appendix H). This behavior suggests that preserving support is easier for Astra than reorganizing finger contacts to sustain motion. Periodic decreases in orientation error do not contradict this interpretation: the rotating target returns to an equivalent orientation after each revolution, so error must be read alongside axial speed.

运动轨迹显示差距如何形成。Astra 的圆柱速度接近零，长方体虽逐渐转动却落后于目标（图 6）。代表性长方体序列显示 Astra 几乎不做接触重配置，而 RL 持续重定向（附录 H）。这表明对 Astra 而言，保持支撑比重组指尖接触以维持运动更容易。朝向误差的周期性下降并不矛盾：旋转目标每转一圈回到等价朝向，故误差必须与轴向速度一并解读。

Task	Controller	Steps	Full	Budget	Drop	At-goal $\uparrow$	Error (rad) $\downarrow$	Speed MAE $\downarrow$
Cylinder, 20 s	Astra Direct	1,749	3/5	1/5	1/5	0.51%	1.626	0.960
	RL	2,000	5/5	–	0/5	76.90%	0.173	0.276
Cuboid, 10 s	Astra Direct	1,000	5/5	0/5	0/5	4.40%	0.742	0.143
	RL	1,000	5/5	–	0/5	63.50%	0.096	0.063

Table 4: Paired in-hand rotation results. Full, Budget, and Drop denote runs reaching the horizon, exhausting Astra’s decision limit, or losing the object, respectively. At-goal is the fraction of steps with quaternion geodesic error below $0.1\,\mathrm{rad}$. Error and Speed MAE are means over executed steps; terminated trajectories are not padded.

任务	控制器	步数	Full	Budget	Drop	At-goal $\uparrow$	Error (rad) $\downarrow$	Speed MAE $\downarrow$
Cylinder, 20 s	Astra Direct	1,749	3/5	1/5	1/5	0.51%	1.626	0.960
	RL	2,000	5/5	–	0/5	76.90%	0.173	0.276
Cuboid, 10 s	Astra Direct	1,000	5/5	0/5	0/5	4.40%	0.742	0.143
	RL	1,000	5/5	–	0/5	63.50%	0.096	0.063

表 4：配对手内旋转结果。 Full、Budget、Drop 分别表示到达时域终点、耗尽 Astra 决策限额、丢失物体。At-goal 为四元数测地误差低于 $0.1\,\mathrm{rad}$ 的步比例。Error 与 Speed MAE 为已执行步均值；已终止轨迹不填充。

Task	Controller	Endpoint	Position (mm) $\downarrow$	Rotation (deg) $\downarrow$	Success
Translation	Astra Direct	15 s	59.05	–	1/5
	RL	15 s	17.29	–	4/5
Translation + rotation	Astra Direct	120 decisions	47.35	32.72	0/5
	RL	matched steps	16.22	8.31	4/5
	RL	15 s	16.08	3.49	5/5

Table 5: Paired in-hand translation and reorientation results. Errors are measured at the stated terminal endpoint. Joint success requires both position error below $22.4\,\mathrm{mm}$ and long-axis rotation error below $10^{\circ}$ at the same endpoint.

任务	控制器	终点	Position (mm) $\downarrow$	Rotation (deg) $\downarrow$	Success
Translation	Astra Direct	15 s	59.05	–	1/5
	RL	15 s	17.29	–	4/5
Translation + rotation	Astra Direct	120 decisions	47.35	32.72	0/5
	RL	matched steps	16.22	8.31	4/5
	RL	15 s	16.08	3.49	5/5

表 5：配对手内平移与重定向结果。 误差在所述终端终点测量。联合成功要求同一终点处位置误差低于 $22.4\,\mathrm{mm}$ 且长轴旋转误差低于 $10^{\circ}$。

Translation reveals some useful direct control, but limited consistency across starts. Astra succeeds in 1/5 cases and RL in 4/5, with mean terminal errors of 59.05 and 17.29 mm (Table 5). Astra achieves lower position error in one paired case, showing that it can produce an effective displacement. Yet it retains the object in every translation run and still misses most targets. As in rotation, failure to reach the goal cannot be explained by object drops alone.

平移显示一定有用的直接控制，但跨起点一致性有限。Astra 成功 1/5，RL 成功 4/5，终端平均误差分别为 59.05 与 17.29 mm（表 5）。Astra 在一对案例中取得更低位置误差，说明它能产生有效位移。但每次平移运行都保住物体，却仍错过多数目标。与旋转一样，未达目标不能仅用掉落解释。

Figure 7: Per-case terminal errors for Allegro tasks. (a) Translation at 15 s. (b,c) Translation and rotation: Astra at its 120-decision endpoint, RL at matched control steps, and RL at 15 s. Shading and dashed lines mark the $22.4$-mm position and $10^{\circ}$ rotation success thresholds.

图 7：Allegro 任务的逐案例终端误差。(a) 15 秒平移。(b,c) 平移加旋转：Astra 在其 120 次决策终点，RL 在匹配控制步，以及 RL 在 15 秒。阴影与虚线标出 $22.4$ mm 位置与 $10^{\circ}$ 旋转成功阈值。

For combined translation and rotation, Astra succeeds in 0/5 cases, compared with 4/5 for RL at matched endpoints and 5/5 after 15 s. Success requires position error below 22.4 mm and rotation error below $10^{\circ}$ at the same endpoint. Astra’s 120-decision budget ends after 6.00–7.65 s, but RL already performs better over those same durations (Figure 7). A longer action horizon alone therefore does not explain the gap. The results favor task-specific learned control for reliable object motion.

对组合平移与旋转，Astra 成功 0/5，而 RL 在匹配终点为 4/5、15 秒后为 5/5。成功要求同一终点处位置误差低于 22.4 mm、旋转误差低于 $10^{\circ}$。Astra 的 120 次决策预算在 6.00–7.65 秒结束，但 RL 在相同时长内已表现更好（图 7）。因此仅靠更长动作时域无法解释差距。结果更支持用任务专用已学控制来获得可靠物体运动。

Contact Changes and Object Support

接触变化与物体支撑

Together, the two studies distinguish useful task correction from reliable contact coordination. Astra can redirect a learned policy when its actions no longer match the scene or instruction, but direct control remains unreliable both when acquiring a grasp and when moving within one. A stable grasp is only an intermediate condition: continued manipulation requires some fingers to release and reposition while others preserve support. The central limitation is sustaining this coordination as the object moves. Hybrid benefits from retaining learned motion generation while allowing Astra to guide recovery, yet recovery and completion still require verification.

两项研究合在一起，区分了「有用的任务修正」与「可靠的接触协调」。当策略动作不再匹配场景或指令时，Astra 可重定向已学策略；但在获取抓取以及在抓取内移动时，直接控制仍不可靠。稳定抓取只是中间条件：继续操控要求部分手指松开重定位，同时其他手指保持支撑。核心局限是在物体运动时维持这种协调。Hybrid 保留已学运动生成并允许 Astra 引导恢复，因而受益；但恢复与完成仍需核验。

Mobile Manipulation

移动操控

Mobile manipulation couples base placement, arm reachability, and object interaction. Using RoboCasa365 [29], we evaluate Astra Direct, Astra Hybrid with optional $\pi_{0.5}$, and standalone $\pi_{0.5}$.

移动操控把底盘放置、手臂可达性与物体交互耦合在一起。我们用 RoboCasa365 [29] 评测 Astra Direct、可选 $\pi_{0.5}$ 的 Astra Hybrid，以及独立 $\pi_{0.5}$。

Figure 8: Cooperation changes the RoboCasa capability profile. (a) Success on matched physical starts. Error bars show descriptive episode-level 95% Wilson intervals [52]. (b) Step shares from accepted proposals and Astra actions.

图 8：协作改变 RoboCasa 能力剖面。(a) 匹配物理起点上的成功。误差条为描述性 episode 级 95% Wilson 区间 [52]。(b) 来自已接受提案与 Astra 动作的步份额。

Control Conditions and Shared Starts

控制条件与共享初始状态

The fixed subset contains five atomic seen, five composite seen, and five composite unseen tasks, each with five seeds: 75 episodes per condition. Selection uses a seeded ordering without model outcomes. Conditions share initial-state fingerprints, the PandaOmron robot, controller, task horizon, and native success check. Seen/unseen refers to the policy’s task-training split. All layouts use split=pretrain.

固定子集含五项原子 seen、五项复合 seen 与五项复合 unseen 任务，各五个种子：每条件 75 个 episode。选取使用带种子排序，不看模型结果。各条件共享初始状态指纹、PandaOmron 机器人、控制器、任务时域与原生成功检查。Seen/unseen 相对策略的任务训练划分。全部布局使用 split=pretrain。

Astra receives three RGB views and end-effector, base, and gripper state, without object poses, contact truth, reference trajectories, or intermediate rewards. Direct supplies bounded arm, gripper, base, and torso commands through an operational-space controller. Segments normally execute 20 steps at 20 Hz; horizons range from 450 to 4,350 steps.

Astra 接收三个 RGB 视角以及末端、底盘与夹爪状态，不含物体位姿、接触真值、参考轨迹或中间奖励。Direct 通过操作空间控制器提供有界的臂、夹爪、底盘与躯干指令。动作段通常以 20 Hz 执行 20 步；时域从 450 到 4,350 步。

Hybrid adds a frozen multitask $\pi_{0.5}$ checkpoint without separate finetuning for these tasks. Astra can accept a 20-step prefix, construct actions, or rewrite the policy instruction as a subgoal. The standalone baseline runs the same checkpoint locally on the same initial states, using the task instruction and the same 20-step cadence. Direct and Hybrid also differ in context management and feedback guards, including context compaction and required observations between action calls, as detailed in Appendix I.4.

Hybrid 加入冻结的多任务 $\pi_{0.5}$ 检查点，不对这些任务单独微调。Astra 可接受 20 步前缀、构造动作，或把策略指令改写为子目标。独立基线在相同初始状态上本地运行同一检查点，使用任务指令与相同的 20 步节拍。Direct 与 Hybrid 在上下文管理与反馈护栏上也不同，包括上下文压缩以及动作调用之间的必需观测，详见附录 I.4。

Figure 9: RoboCasa outcomes from matched physical starts. Rows identify four task/seed pairs. Initial images come from the independent-policy run; subsequent columns show each condition's terminal view. Labels report native outcomes and executed steps. All images are saved observations; the coffee case succeeds only with the standalone policy.

图 9：匹配物理起点上的 RoboCasa 结果。行标识四组任务/种子。初始图像来自独立策略运行；后续列给出各条件终端视图。标签报告原生结果与已执行步。全部图像为保存的观测；咖啡案例仅独立策略成功。

Task group	$\pi_{0.5}$alone	Astra Direct	Astra Hybrid
Atomic seen	13/25 (52.0%)	7/25 (28.0%)	13/25 (52.0%)
Composite seen	3/25 (12.0%)	4/25 (16.0%)	7/25 (28.0%)
Composite unseen	1/25 (4.0%)	14/25 (56.0%)	9/25 (36.0%)
All tasks	17/75 (22.7%)	25/75 (33.3%)	29/75 (38.7%)

Table 6: RoboCasa completion on matched initial states. Each group contains five tasks with five episodes each; entries give successes/episodes and percentage. All conditions use the same 20-step feedback cadence and task horizons. Task groups describe the policy’s training coverage.

任务组	$\pi_{0.5}$ alone	Astra Direct	Astra Hybrid
Atomic seen	13/25 (52.0%)	7/25 (28.0%)	13/25 (52.0%)
Composite seen	3/25 (12.0%)	4/25 (16.0%)	7/25 (28.0%)
Composite unseen	1/25 (4.0%)	14/25 (56.0%)	9/25 (36.0%)
All tasks	17/75 (22.7%)	25/75 (33.3%)	29/75 (38.7%)

表 6：匹配初始状态上的 RoboCasa 完成情况。 每组五项任务各五个 episode；条目为成功数/episode 与百分比。全部条件使用相同的 20 步反馈节拍与任务时域。任务组描述策略的训练覆盖。

Uneven Gains from Policy Assistance

策略辅助带来的不均衡收益

Astra Hybrid completes 29/75 episodes, compared with 25/75 for Direct and 17/75 for standalone $\pi_{0.5}$ in Table 6. Hybrid exceeds Direct on atomic seen tasks, 52% versus 28%, and composite seen tasks, 28% versus 16%. Direct is stronger on unseen compositions, 56% versus 36%. Hybrid and standalone $\pi_{0.5}$ both reach 13/25 on atomic tasks, but succeed on different instances.

如表 6，Astra Hybrid 完成 29/75 episode，Direct 为 25/75，独立 $\pi_{0.5}$ 为 17/75。Hybrid 在原子 seen（52% 对 28%）与复合 seen（28% 对 16%）上高于 Direct；Direct 在未见复合上更强（56% 对 36%）。Hybrid 与独立 $\pi_{0.5}$ 在原子任务上同为 13/25，但成功实例不同。

Hybrid exceeds Direct on tasks seen during policy training but trails it on unseen compositions. These averages also conceal complementary successes. In Table 35, Hybrid alone succeeds on 13 paired starts and Direct alone on nine, with 16 shared successes. An exploratory exact McNemar test [10] gives $p=0.5235$.

Hybrid 在策略训练见过的任务上超过 Direct，在未见复合上落后。这些平均还掩盖了互补成功。表 35 中，仅 Hybrid 成功 13 个配对起点，仅 Direct 成功 9 个，共享成功 16 个。探索性精确 McNemar 检验 [10] 给出 $p=0.5235$。

Astra organizes grasping, transport, placement, and device operation through language decomposition and numerical control. In Hybrid, 66/75 episodes use rewritten instructions, which condition 72.6% of proposals. Figure 8 attributes 55.2% of executed steps to accepted policy actions and 44.8% to Astra. Rewriting changes what the policy is asked to achieve; numerical correction changes the robot commands, while context and feedback guards structure proposal review. The reported results evaluate these components jointly. Delegation still requires proposal review; Section 9.4 examines its inference demands.

Astra 通过语言分解与数值控制组织抓取、运输、放置与设备操作。在 Hybrid 中，66/75 episode 使用改写指令，条件化了 72.6% 的提案。图 8 将 55.2% 已执行步归于已接受策略动作，44.8% 归于 Astra。改写改变策略被要求达成什么；数值修正改变机器人指令；上下文与反馈护栏则组织提案审阅。所报告结果是对这些组件的联合评测。委托运动仍需审阅提案；第 9.4 节考察其推理开销。

Recovery and Lost Policy Successes

恢复能力与策略成功的丢失

Figure 9 illustrates these differences on matched starts. Only Hybrid succeeds on KettleBoiling/00, and only Direct succeeds on RecycleBottlesByType/01. On PickPlaceDrawerToCounter/04, both Hybrid and standalone $\pi_{0.5}$ succeed.

图 9 在匹配起点上说明这些差异。仅 Hybrid 在 KettleBoiling/00 成功，仅 Direct 在 RecycleBottlesByType/01 成功。在 PickPlaceDrawerToCounter/04 上，Hybrid 与独立 $\pi_{0.5}$ 都成功。

Conversely, only standalone $\pi_{0.5}$ succeeds on CoffeeSetupMug/03. Across all five seeds, both Astra conditions fail CoffeeSetupMug and WashLettuce, while the policy completes three and two cases, respectively. Policy access need not preserve success.

反过来，仅独立 $\pi_{0.5}$ 在 CoffeeSetupMug/03 成功。在全部五个种子上，两种 Astra 条件都未完成 CoffeeSetupMug 与 WashLettuce，而策略分别完成三例与两例。获得策略访问未必保住成功。

Visual feedback lets Astra detect some action errors, but recognizing a problem does not ensure a successful correction. Failure trajectories expose weaknesses in precise grasping and sustained contact, base–arm coordination while carrying objects, and tracking preconditions and object states across task stages. $\pi_{0.5}$ helps on some tasks but does not ensure reliable state verification or physical control.

视觉反馈让 Astra 能发现部分动作错误，但识别问题并不能保证成功修正。失败轨迹暴露了精确抓取与持续接触、携带物体时的底盘–臂协调，以及跨阶段跟踪前置条件与物体状态上的弱点。$\pi_{0.5}$ 在某些任务上有帮助，但不能保证可靠的状态核验或物理控制。

Navigation

导航

We evaluate Astra on instruction following in continuous environments (VLN-CE) and category-directed object search (ObjectNav) [19, 5], measuring both goal reaching and path efficiency. Astra chooses where to move and when to stop using RGB observations without pose or map input; the action interface applies discrete forward steps and turns.

我们在连续环境指令跟随（VLN-CE）与类别导向目标搜索（ObjectNav）[19, 5] 上评测 Astra，同时度量到达目标与路径效率。Astra 仅用 RGB 观测、不含位姿或地图输入来选择移动与停止时机；动作接口施加离散前进与转向。

RGB Navigation on Four Fixed Subsets

四个固定子集上的 RGB 导航

We select 50 validation-unseen episodes each from R2R and English-guide RxR, and 50 validation episodes each from MP3D ObjectNav v1 and HM3D ObjectNav v2 [3, 20, 7, 54]. Sampling uses seed 0 and balances scenes and ObjectNav categories.

我们从 R2R 与英文引导 RxR 各取 50 个 validation-unseen episode，并从 MP3D ObjectNav v1 与 HM3D ObjectNav v2 各取 50 个 validation episode [3, 20, 7, 54]。采样使用种子 0，并平衡场景与 ObjectNav 类别。

Astra receives one forward-facing $512\times 384$ RGB image, the instruction or object category, a step counter, and action feedback. GPS, compass, depth, semantic labels, goal coordinates, reference routes, and evaluation distances are withheld. A persistent episode session selects discrete moves or local waypoints. The adapter converts these to 0.25 m forward steps and $30^{\circ}$ turns without path planning, under a 500-primitive budget. Habitat [40] executes these primitives; VLN-CE enables simulator sliding, whereas ObjectNav disables it.

Astra 接收一张前向 $512\times 384$ RGB 图像、指令或物体类别、步计数器与动作反馈。GPS、罗盘、深度、语义标签、目标坐标、参考路线与评测距离均不提供。持久的 episode 会话选择离散移动或局部路点。适配器在 500 原语预算下将其转为 0.25 m 前进与 $30^{\circ}$ 转向，不做路径规划。Habitat [40] 执行这些原语；VLN-CE 启用仿真滑动，ObjectNav 关闭。

Success requires STOP within 3 m of the VLN-CE goal or strictly within 1 m of the ObjectNav annotated goal-viewpoint set. The latter is not distance to the object’s surface and does not require final-image visibility; HM3D records are rescored at this tolerance. We report success rate (SR), success weighted by path length (SPL), and VLN-CE route agreement through nDTW and sDTW [2, 15].

成功要求在 VLN-CE 目标 3 m 内 STOP，或严格落在 ObjectNav 标注目标视点集合 1 m 内。后者并非到物体表面的距离，也不要求终帧可见；HM3D 记录按此容差重计分。我们报告成功率（SR）、路径长度加权成功（SPL），以及 VLN-CE 路线一致性指标 nDTW 与 sDTW [2, 15]。

Task	Dataset	Split	Successes	SR $\uparrow$	SPL $\uparrow$	nDTW $\uparrow$	sDTW $\uparrow$
VLN-CE	R2R	Val-Unseen	39/50	78	65.27	72.20	59.35
VLN-CE	RxR (English)	Val-Unseen	46/50	92	77.25	84.73	80.42
ObjectNav	MP3D	Val	28/50	56	22.43	–	–
ObjectNav	HM3D	Val	41/50	82	43.69	–	–

Table 7: Navigation on four fixed subsets, 50 episodes each. Task, dataset, and split are listed separately. All metric values are percentages; RxR uses English guide instructions. ObjectNav has no instruction-specific reference trajectory.

任务	数据集	划分	成功数	SR $\uparrow$	SPL $\uparrow$	nDTW $\uparrow$	sDTW $\uparrow$
VLN-CE	R2R	Val-Unseen	39/50	78	65.27	72.20	59.35
VLN-CE	RxR (English)	Val-Unseen	46/50	92	77.25	84.73	80.42
ObjectNav	MP3D	Val	28/50	56	22.43	–	–
ObjectNav	HM3D	Val	41/50	82	43.69	–	–

表 7：四个固定子集上的导航，每子集 50 个 episode。 任务、数据集与划分分列。全部指标为百分比；RxR 使用英文引导指令。ObjectNav 无指令专用参考轨迹。

Figure 10: Navigation success, efficiency, and termination. (a) SR and SPL. (b) Successful STOP, failed STOP, or exhaustion of the 500-step budget. Each subset contains 50 episodes.

图 10：导航成功、效率与终止。(a) SR 与 SPL。(b) 成功 STOP、失败 STOP，或耗尽 500 步预算。每子集 50 个 episode。

Success and Search Efficiency

成功与搜索效率

Astra completes 39/50 R2R and 46/50 RxR episodes, with nDTW of 72.20% and 84.73%, showing goal reaching and route agreement in Table 7.

Astra 完成 39/50 个 R2R 与 46/50 个 RxR episode，nDTW 分别为 72.20% 与 84.73%，在表 7 中同时体现到达目标与路线一致性。

ObjectNav yields 28/50 MP3D and 41/50 HM3D successes, but SPL is only 22.43% and 43.69%. SPL discounts excess travel even when the final endpoint succeeds; the gap therefore reveals inefficient search that SR alone conceals. Figure 10 shows ten budget-limited MP3D episodes and five for HM3D; all evaluated VLN-CE episodes end with STOP.

ObjectNav 上 MP3D 成功 28/50、HM3D 成功 41/50，但 SPL 仅 22.43% 与 43.69%。即便终点成功，SPL 也会因多余行程而打折；因此该差距揭示了 SR 单独掩盖的低效搜索。图 10 显示 MP3D 有十个、HM3D 有五个预算耗尽 episode；全部评测的 VLN-CE episode 都以 STOP 结束。

A route instruction supplies spatial cues; an object category leaves the route to be discovered. Search thus requires deciding where to explore and when revisiting an area is no longer useful, in addition to grounding local observations. The subsets use different scenes, goals, instructions, and sliding settings.

路线指令提供空间线索；物体类别则把路线留给探索去发现。因此搜索除 grounding 局部观测外，还要决定探索何处、以及何时不再值得重访某区域。各子集在场景、目标、指令与滑动设置上不同。

		VLN-CE	ObjectNav
		R2R	RxR	MP3D	HM3D
System	Views	SR	SPL	SR	SPL	SR	SPL	SR	SPL
Astra	1	78.00	65.27	92.00	77.25	56.00	22.43	82.00	43.69
LightNav-0	1	58.00	54.14	72.00	64.08	20.00	8.39	66.00	37.75
Uni-NaVid 7B	1	30.00	27.76	34.00	25.67	8.00	5.46	30.00	20.48
Multi-view methods (separate observation regime)
OmniNav Flow	3	56.00	53.89	72.00	56.29	4.00	2.96	6.00	2.90
NavFoM	3	44.00	40.04	58.00	51.79	18.00	13.01	42.00	30.38
SPAN-Nav	3	70.00	61.71	72.00	56.19	–	–	–	–

Table 8: Local evaluation of released navigation policies on the same episode lists. Each dataset contains 50 episodes per system; values are percentages. The first three systems share the front-camera specification. OmniNav Flow uses three wider-angle views and is separated accordingly. All systems share task-specific scoring and primitive-action budgets, while their history processing, prompts, and action adapters differ.

		VLN-CE		ObjectNav
		R2R		RxR		MP3D		HM3D
System	Views	SR	SPL	SR	SPL	SR	SPL	SR	SPL
Astra	1	78.00	65.27	92.00	77.25	56.00	22.43	82.00	43.69
LightNav-0	1	58.00	54.14	72.00	64.08	20.00	8.39	66.00	37.75
Uni-NaVid 7B	1	30.00	27.76	34.00	25.67	8.00	5.46	30.00	20.48
多视角方法（独立观测体制）
OmniNav Flow	3	56.00	53.89	72.00	56.29	4.00	2.96	6.00	2.90
NavFoM	3	44.00	40.04	58.00	51.79	18.00	13.01	42.00	30.38
SPAN-Nav	3	70.00	61.71	72.00	56.19	–	–	–	–

表 8：在相同 episode 列表上对已发布导航策略的本地评测。 每数据集每系统 50 个 episode；数值为百分比。前三个系统共享前置相机规格。OmniNav Flow 使用三个更广角视角，故分开列出。全部系统共享任务专用计分与原语动作预算，但其历史处理、提示与动作适配器不同。

Local Comparisons and Failure Modes

本地对照与失败模式

Released LightNav-0, Uni-NaVid 7B, OmniNav Flow, NavFoM and SPAN-Nav policies are evaluated on the same 200 episodes with shared scoring and primitive budgets [49, 61, 53, 62, 23]. LightNav-0 and Uni-NaVid use Astra’s front-camera specification in place of their published camera settings; OmniNav Flow receives three wider-angle views. Local adapters convert LightNav-0 and OmniNav Flow outputs to the shared discrete actions. The OmniNav condition evaluates the released Flow policy without the full system’s slow exploration planner.

已发布的 LightNav-0、Uni-NaVid 7B、OmniNav Flow、NavFoM 与 SPAN-Nav 策略在相同 200 个 episode 上评测，共享计分与原语预算 [49, 61, 53, 62, 23]。LightNav-0 与 Uni-NaVid 使用 Astra 的前置相机规格替代其发表相机设置；OmniNav Flow 接收三个更广角视角。本地适配器将 LightNav-0 与 OmniNav Flow 输出转为共享离散动作。OmniNav 条件评测已发布 Flow 策略，不含完整系统的慢速探索规划器。

Under this local protocol, Astra has higher SR and SPL than the evaluated systems on all subsets with reported results in Table 8. Relative to LightNav-0, SR gains are 20, 20, 36, and 16 percentage points. LightNav-0 has slightly higher R2R nDTW, 72.77% versus 72.20%, while Astra has higher sDTW, 59.35% versus 48.89%. Appendix I.2 details the camera settings and action adapters.

在该本地协议下，Astra 在表 8 所有有报告结果的子集上 SR 与 SPL 均高于对照系统。相对 LightNav-0，SR 分别高 20、20、36 与 16 个百分点。LightNav-0 的 R2R nDTW 略高（72.77% 对 72.20%），而 Astra 的 sDTW 更高（59.35% 对 48.89%）。相机设置与动作适配器详见附录 I.2。

Figure 11: Two navigation successes and one budget-limited failure. Rows show initial, midpoint, and final front views beside the executed route on the Habitat navigation mesh. These evaluator-only maps were not shown to Astra.

图 11：两次导航成功与一次预算耗尽失败。行给出初始、中点与最终前视图，旁为 Habitat 导航网格上的执行路线。这些仅供评测者使用的地图未展示给 Astra。

Successful trajectories show Astra interpreting instructions, tracking landmark sequences, and recognizing target categories through multiple indoor stages. Figure 11 illustrates instruction following, search with detours, and budget exhaustion. Failure trajectories expose difficulties linking visual observations to specific locations, verifying goal completion, and using collision feedback or search history to change course. A plausible rationale for moving does not ensure correct goal judgment or spatial progress.

成功轨迹显示 Astra 解读指令、跟踪地标序列，并在多段室内过程中识别目标类别。图 11 展示指令跟随、带绕行的搜索，以及预算耗尽。失败轨迹暴露了把视觉观测联系到具体位置、核验目标完成，以及用碰撞反馈或搜索历史改道上的困难。移动理由听起来合理，并不保证正确的目标判断或空间进度。

See Appendix I for protocols and termination counts.

协议与终止计数见附录 I。

Locomotion

步行控制

Navigation primitives implement the motion Astra chooses. Locomotion exposes an additional demand: constructing the body motion itself. Here Astra generates dense motion references for a frozen tracker over repeated attempts on one obstacle course. Analytic tests examine compact interfaces without Astra.

导航原语实现 Astra 选定的运动。步行控制暴露额外需求：构造身体运动本身。此处 Astra 为冻结 tracker 在同一障碍赛道上反复尝试生成稠密运动参考。解析测试则在不含 Astra 的情况下考察紧凑接口。

Dense Motion on a Matched Course

匹配赛道上的稠密运动

We replace PASSAGE’s learned motion generator with Astra while retaining its scene-aligned $25\times 65$ representation and frozen G1 ScaleTrack controller [26, 60]. Each proposal specifies 0.5 s at 50 Hz: root height, projected gravity, planar velocity, yaw rate, and positions and velocities for 29 joints. Each call requires 1,625 consistent values.

我们用 Astra 替换 PASSAGE 的已学运动生成器，同时保留其场景对齐的 $25\times 65$ 表示与冻结的 G1 ScaleTrack 控制器 [26, 60]。每次提案指定 0.5 秒、50 Hz：根高度、投影重力、平面速度、偏航率，以及 29 个关节的位置与速度。每次调用需要 1,625 个一致数值。

On one open-clutter course, Astra and PASSAGE share the initial state, goal, elevation observation, local route guidance, tracker, and termination criteria. Success requires final planar error at most 0.5 m. Falls trigger resets; eight seconds with less than 0.10 m root displacement triggers no-progress termination. The supplied route leaves motion construction to Astra.

在一条开阔杂乱赛道上，Astra 与 PASSAGE 共享初始状态、目标、高程观测、局部路线指引、tracker 与终止判据。成功要求最终平面误差不超过 0.5 m。跌倒触发重置；根位移连续八秒小于 0.10 m 触发无进度终止。所提供路线把运动构造留给 Astra。

Astra makes five sequential attempts. The first is zero-shot; later attempts retain textual summaries of progress, falls, and collisions. Before the fifth, Astra also receives a retargeted walking clip from another course. Experience and motion context therefore change without weight updates. This is one adaptation trajectory, not five independent trials.

Astra 进行五次连续尝试。第一次为零样本；后续尝试保留进度、跌倒与碰撞的文本摘要。第五次之前，Astra 还收到来自另一赛道的重定向行走片段。因此经验与运动上下文在不更新权重的情况下变化。这是一条适应轨迹，而非五次独立试验。

Planner / Attempt	Outcome	Time (s)	Progress (m)	Final error (m)	Fall	Reached
PASSAGE	Goal reached	13.18	8.622	0.495	No	Yes
Astra Attempt 1	Fall	9.80	1.561	7.556	Yes	No
Astra Attempt 2	No progress	20.28	-0.024	9.141	No	No
Astra Attempt 3	No progress	8.00	0.012	9.104	No	No
Astra Attempt 4	No progress	8.48	0.159	8.958	No	No
Astra Attempt 5	Time horizon	30.00	2.598	6.519	No	No

Table 9: PASSAGE and sequential Astra attempts on one matched open-clutter course. Progress is the reduction in planar distance to the goal, so a negative value denotes motion away from it. The Astra rows are successive stages of one adaptation trajectory, not independent trials.

规划器 / 尝试	结果	Time (s)	Progress (m)	Final error (m)	Fall	Reached
PASSAGE	Goal reached	13.18	8.622	0.495	No	Yes
Astra Attempt 1	Fall	9.80	1.561	7.556	Yes	No
Astra Attempt 2	No progress	20.28	-0.024	9.141	No	No
Astra Attempt 3	No progress	8.00	0.012	9.104	No	No
Astra Attempt 4	No progress	8.48	0.159	8.958	No	No
Astra Attempt 5	Time horizon	30.00	2.598	6.519	No	No

表 9：PASSAGE 与 Astra 在同一匹配开阔杂乱赛道上的连续尝试。 Progress 为到目标平面距离的减少，负值表示远离目标。Astra 行是一条适应轨迹的相继阶段，而非独立试验。

Persistent Locomotion Failures

持续的步行失败

PASSAGE reaches the goal in 13.18 s. Astra initially advances 1.561 m before falling; the next three attempts remain upright with little useful progress. Table 9 shows that the fifth attempt stays upright for 30 s, travels 3.833 m, and reduces goal distance by 2.598 m. It still ends 6.519 m from the goal, before the principal obstacle field.

PASSAGE 在 13.18 秒到达目标。Astra 最初前进 1.561 m 后跌倒；随后三次尝试保持直立但几乎没有有用进度。表 9 显示第五次尝试直立 30 秒，行驶 3.833 m，把到目标距离减少 2.598 m，仍停在距目标 6.519 m 处、尚未进入主要障碍区。

After five rounds of trial and error, Astra still does not achieve reliable locomotion through this dense-reference interface. Later attempts improve stability and produce some forward motion, but body and limb coordination is inconsistent. Across attempts, Astra uses motion feedback while the walking clip, gait timing, and amplitude are adjusted together.

五轮试错之后，Astra 仍未能通过该稠密参考接口实现可靠步行。后期尝试改善了稳定性并产生一些前向运动，但身体与肢体协调不一致。跨尝试中，Astra 使用运动反馈，同时一并调整行走片段、步态时序与幅度。

The interface helps explain the difficulty. Joint limits alone do not make a reference coherent: root velocity, limb posture, and their evolution must describe compatible motion. A tracker can follow a reference yet produce in-place stepping, unwanted contacts, or a fall. A supplied route simplifies planning but does not supply coordinated body motion.

接口有助于解释困难。仅有关节限位并不能使参考自洽：根速度、肢体姿态及其演化必须描述兼容运动。Tracker 可以跟随参考却仍产生原地踏步、多余接触或跌倒。提供路线简化了规划，但并不提供协调的身体运动。

Timing is also limiting. The final 30 s episode uses 250 synchronous calls averaging 39.86 s of recorded latency, versus approximately 0.08 s for PASSAGE planning. Physics pauses during inference. The inference budget permits five attempts.

时序同样构成限制。最终 30 秒 episode 使用 250 次同步调用，平均记录时延 39.86 秒，而 PASSAGE 规划约 0.08 秒。推理期间物理仿真暂停。推理预算允许五次尝试。

Compact-Reference Diagnostics without Astra

不含 Astra 的紧凑参考量诊断

We additionally test two compact-reference interfaces with analytic generators, without Astra, to identify failures in converting references into robot motion. Both use six-frame link-pose sequences for the same 29-DoF robot. Five-point specifies the pelvis, wrists, and ankles; WholeBody-14 adds the torso and bilateral hips, knees, shoulders, and elbows.

我们另外用不含 Astra 的解析生成器测试两种紧凑参考接口，以识别把参考转为机器人运动时的失败。二者都对同一 29-DoF 机器人使用六帧连杆位姿序列。Five-point 指定骨盆、腕与踝；WholeBody-14 另加躯干以及双侧髋、膝、肩、肘。

The deterministic generators run on three open-clutter and three low-ceiling courses in two physics backends: 12 rollouts per interface. Strict success requires an upright robot, final error at most 0.2 m, a five-second hold, tracked points within the course, and no obstacle force above 5 N. Goal + stop requires only arrival and a hold.

确定性生成器在三条开阔杂乱与三条低天花赛道、两种物理后端上运行：每接口 12 次 rollout。严格成功要求机器人直立、最终误差不超过 0.2 m、保持五秒、跟踪点在赛道内，且障碍力不超过 5 N。Goal + stop 仅要求到达并保持。

Interface	$N$	Strict success	Goal + stop	Falls	Final error (m)	Travel (m)	Peak contact (N)	Non-foot contact (N)	Five-point RMSE (m)
Five-point	12	0	4 (33.3%)	1 (8.3%)	3.441	5.745	1853	644	0.161
WholeBody-14	12	0	3 (25.0%)	4 (33.3%)	5.334	3.866	2293	1562	0.197

Table 10: Paired analytic interface feasibility study over six courses and two physics backends. Each course–backend cell contains one rollout. Contact magnitudes are useful failure diagnostics, but are not treated as calibrated measurements across simulators. These rollouts do not use Astra.

Interface	$N$	Strict success	Goal + stop	Falls	Final error (m)	Travel (m)	Peak contact (N)	Non-foot contact (N)	Five-point RMSE (m)
Five-point	12	0	4 (33.3%)	1 (8.3%)	3.441	5.745	1853	644	0.161
WholeBody-14	12	0	3 (25.0%)	4 (33.3%)	5.334	3.866	2293	1562	0.197

表 10：六条赛道与两种物理后端上的配对解析接口可行性研究。 每个赛道–后端单元一次 rollout。接触幅值是有用的失败诊断，但不视为跨仿真器的标定测量。这些 rollout 不使用 Astra。

Neither interface achieves strict success in Table 10. Five-point reaches and holds the endpoint in 4/12 rollouts with one fall; WholeBody-14 does so in 3/12 with four falls. Five-point has lower mean endpoint and tracking errors. Low ceilings remain difficult, with only 1/6 and 0/6 rollouts reaching the endpoint.

表 10 中两种接口均未达到严格成功。Five-point 在 4/12 次 rollout 到达并保持终点，跌倒一次；WholeBody-14 为 3/12，跌倒四次。Five-point 的平均终点与跟踪误差更低。低天花仍然困难，分别仅 1/6 与 0/6 到达终点。

The comparison favors the tested Five-point generator–interface pair descriptively, without isolating dimensionality because the generators differ. Large obstacle contacts also show that trackable motion is not necessarily safe motion.

对照在描述上更偏向所测试的 Five-point 生成器–接口组合；由于生成器不同，并未隔离维度因素。较大的障碍接触也表明：可跟踪的运动未必是安全运动。

Humanoid Loco-Manipulation

人形步行-操控

We study Hybrid humanoid control, with Astra directing pretrained whole-body controllers. In HumanoidBench [44], Astra commands frozen Humanoid-GPT [37] policies across a broad task suite. In SIMPLE [51], it uses a frozen ScaleBFM [60] whole-body controller with task instructions refined through experience, examining manipulation and the retention of successful behavior. Astra selects task commands; whole-body controllers coordinate motion.

我们研究 Hybrid 人形控制：由 Astra 指挥预训练全身控制器。在 HumanoidBench [44] 上，Astra 在广泛任务套件上指挥冻结的 Humanoid-GPT [37] 策略。在 SIMPLE [51] 上，它使用冻结的 ScaleBFM [60] 全身控制器，并通过经验修订任务指令，考察操控以及对成功行为的保持。Astra 选择任务指令；全身控制器协调运动。

HumanoidBench Evaluation Setup

HumanoidBench 评测设置

We evaluate the 30 executable HumanoidBench tasks in the pinned registry, excluding the two Highbar variants. Evaluation uses seeds 1 and 2 for each task. Torque replays independently verify episodes.

我们评测 pinned registry 中 30 项可执行 HumanoidBench 任务，排除两个 Highbar 变体。每任务使用种子 1 与 2。扭矩回放独立核验 episode。

Astra receives RGB, privileged state, task geometry, and guidance. It selects walking velocities and yaw rate, or sparse pelvis and wrist references. Frozen whole-body controllers run at 50 Hz over 500-Hz physics and joint control. Wrist targets couple to the body and depend on the control mode.

Astra 接收 RGB、特权状态、任务几何与指引。它选择行走速度与偏航率，或稀疏骨盆与腕部参考。冻结全身控制器以 50 Hz 运行于 500 Hz 物理与关节控制之上。腕部目标与身体耦合，并取决于控制模式。

Table 11 compares Astra with published methods on eight tasks; Figure 12 covers all 30. Appendix E gives complete Astra results and evaluation details.

表 11 在八项任务上比较 Astra 与已发表方法；图 12 覆盖全部 30 项。完整 Astra 结果与评测细节见附录 E。

Task	DreamerV3	TD-MPC2	SAC	Astra	Threshold
Maze	272.3	244.3	144.8	1358.8	1200
Reach	7580.9	7316.1	4565.1	11430.2	12000
Walk	800.2	782.0	31.7	848.7	700
Run	633.8	93.3	5.0	642.1	700
Crawl	878.8	957.4	330.0	971.9	700
Stair	131.1	70.4	14.1	272.5	700
Push	-1251.9	-258.7	-97.9	877.3	700
Door	213.0	274.7	39.4	142.4	600

Table 11: HumanoidBench returns against published methods. Entries are mean returns; bold marks the highest value per task. DreamerV3, TD-MPC2, and SAC results follow HumanoidBench [44]. Threshold denotes the native task criterion. Appendix E documents result sources and evaluation protocols.

Task	DreamerV3	TD-MPC2	SAC	Astra	Threshold
Maze	272.3	244.3	144.8	1358.8	1200
Reach	7580.9	7316.1	4565.1	11430.2	12000
Walk	800.2	782.0	31.7	848.7	700
Run	633.8	93.3	5.0	642.1	700
Crawl	878.8	957.4	330.0	971.9	700
Stair	131.1	70.4	14.1	272.5	700
Push	-1251.9	-258.7	-97.9	877.3	700
Door	213.0	274.7	39.4	142.4	600

表 11：HumanoidBench 回报对照已发表方法。 条目为平均回报；加粗为每任务最高。DreamerV3、TD-MPC2 与 SAC 结果遵循 HumanoidBench [44]。Threshold 为原生任务判据。附录 E 记录结果来源与评测协议。

Figure 12: Results across 30 HumanoidBench tasks. Green bars show Astra mean returns and sample SDs. Left bars reproduce published method returns [44]. Axes are task-specific; dashed lines mark native thresholds. Appendix E gives exact values and reference protocols.

图 12：30 项 HumanoidBench 任务结果。绿色条为 Astra 平均回报与样本标准差。左侧条复现已发表方法回报 [44]。坐标轴因任务而异；虚线标出原生阈值。精确数值与参考协议见附录 E。

Command Selection with Pretrained Whole-Body Controllers

借助预训练全身控制器的指令选择

Maze provides a clear example of adapting commands to state. Astra slows near intersections, turns, and resumes forward motion using supplied geometry. Maze achieves a mean return of 1358.8, with the evaluated trajectories reaching native stage 4. Reach achieves 11430.2, below the 12000 threshold, with individual returns straddling it.

Maze 清楚展示了按状态适应指令：Astra 在路口附近减速、转向，并用所给几何恢复前进。Maze 平均回报 1358.8，所评轨迹达到原生阶段 4。Reach 为 11430.2，低于 12000 阈值，个别回报在阈值两侧。

Walk combines posture calibration with startup and speed guidance, achieving a mean return of 848.7. Run uses a five-point gait and reaches 642.1; both runs stay upright but below threshold. Crawl uses a low, forward-pitched gait and reaches 971.9, with about 27.39 m displacement per run. Astra selects speed and heading, while calibrated generators supply posture and limb coordination. Performance reflects Astra’s decisions and whole-body controllers.

Walk 结合姿态标定与启动、速度指引，平均回报 848.7。Run 使用五点步态，达到 642.1；两次运行均保持直立但低于阈值。Crawl 使用低姿前倾步态，达到 971.9，每次约 27.39 m 位移。Astra 选择速度与航向，标定生成器提供姿态与肢体协调。表现反映 Astra 决策与全身控制器。

Push: initial state (0)

Base staged (175)

Contact approach (267)

Native success (323)

Figure 13: Decisions and physical response in three separate experiments. Top left: Maze trajectories and supplied checkpoints; both runs reach stage 4. Top right: a scripted Walk frequency sweep with fixed weights and command speed; the dashed line marks the native threshold. Bottom: experience-conditioned Push, seed 7, ending at 4.81 cm box error. Parentheses give control steps.

Push：初始状态 (0)

底盘就位 (175)

接触接近 (267)

原生成功 (323)

图 13：三个独立实验中的决策与物理响应。左上：Maze 轨迹与所给检查点；两次运行均达阶段 4。右上：权重与指令速度固定的脚本化 Walk 频率扫描；虚线为原生阈值。下方：经验条件化 Push，种子 7，终点箱误差 4.81 cm。括号内为控制步。

Astra achieves the highest mean native return among the compared methods: 581.9, versus 338.2 for TD-MPC2, 42.7 for SAC, and $-41.9$ for DreamerV3, averaging task means equally across all 30 tasks in Figure 12. Astra also exceeds the published returns of DreamerV3 on 16 tasks, TD-MPC2 on 19, and SAC on 25. They exceed the best of these three references on 13 tasks and tie on Kitchen [44]. Six Astra task means reach native thresholds: Stand, Walk, Maze, Crawl, Push, and Sit Simple.

在图 12 全部 30 项任务等权平均任务均值后，Astra 在对照方法中取得最高平均原生回报：581.9，相对 TD-MPC2 的 338.2、SAC 的 42.7 与 DreamerV3 的 $-41.9$。Astra 还在 16 项上超过 DreamerV3 已发表回报，19 项超过 TD-MPC2，25 项超过 SAC；相对这三者最佳，有 13 项更高，Kitchen 持平 [44]。六项 Astra 任务均值达到原生阈值：Stand、Walk、Maze、Crawl、Push 与 Sit Simple。

Calibration also affects what a command accomplishes. In the scripted Walk sweep, changing gait-clock frequency from 1.2 to 1.8 Hz raises mean return from 705.72 to 746.60 across eight paired seeds without changing weights, command speed, or feedback rule. Actual forward speed is about 1.48 m/s for a 1.25 m/s command. Return and measured speed together describe the effect of controller calibration.

标定也会改变一条指令实际达成什么。在脚本化 Walk 扫描中，把步态时钟频率从 1.2 调到 1.8 Hz，在八个配对种子上把平均回报从 705.72 提到 746.60，且不改权重、指令速度或反馈规则。对 1.25 m/s 指令，实际前向速度约 1.48 m/s。回报与测得速度共同描述控制器标定的效应。

Contact Failures Between Task Stages

任务阶段之间的接触失败

Push illustrates useful preparation and limited transfer. A demonstration-guided configuration succeeds on three development seeds and one transfer seed fixed in advance with box errors below 5 cm; two development trajectories reproduce supplied demonstrations. On the evaluation cases, staging and contact guidance support successful pushing in 277 and 357 controls, at 4.89 and 4.97 cm error.

Push 展示了有用的准备与有限的迁移。演示引导配置在三个开发种子与一个事先固定的迁移种子上成功，箱误差低于 5 cm；两条开发轨迹复现所给演示。在评测案例上，就位与接触指引支持在 277 与 357 个控制步内成功推动，误差 4.89 与 4.97 cm。

A separate scripted diagnostic clarifies the interaction: changing entry alone or contact references alone fails, while changing both reaches 4.97 cm error. A hand offset depends on the body’s configuration. The records in Figure 13 and Appendix C show why base preparation and contact must be coordinated: an effective offset can fail from a different entry state.

单独的脚本化诊断澄清了交互：只改进入方式或只改接触参考都会失败，两者都改则达到 4.97 cm 误差。手部偏移取决于身体构型。图 13 与附录 C 的记录说明为何底盘准备与接触必须协调：有效偏移在不同进入状态下可能失效。

Door exposes the next difficulty. Episodes rotate the hatch without opening a usable passage; scripted finger and body variants achieve partial opening without passage, and Astra also fails to complete the task. Reaching a mechanism is insufficient if contact fails under the loading and body movement needed to continue. Stair achieves a mean return of 272.5 with terrain-aware references, but one run falls after limited progress and the other retreats. Cube and Window lose objects early; Kitchen returns zero.

Door 暴露下一层困难。episode 能转动舱门却打不开可用通道；脚本化手指与身体变体达到部分开启但无通道，Astra 也未完成任务。若在继续所需的载荷与身体运动下接触失败，仅到达机构不够。Stair 在地形感知参考下平均回报 272.5，但一次有限进度后跌倒，另一次后退。Cube 与 Window 早期丢物；Kitchen 回报为零。

These failures occur between task stages, when changes in posture or contact disrupt the next action. Whole-body controllers alone do not resolve them. SIMPLE examines whether guidance drawn from experience helps Astra manage these transitions during grasping, transport, and handover.

这些失败发生在任务阶段之间：姿态或接触变化打断下一步动作。全身控制器 单独 无法解决。SIMPLE 考察来自经验的指引能否帮助 Astra 在抓取、运输与交接中管理这些过渡。

Refining Task Guidance through Experience in SIMPLE

在 SIMPLE 中借经验修订任务指引

The SIMPLE study examines whether experience can help Astra coordinate grasping, transport, and handover. Using visual and proprioceptive feedback, Astra selects actions for a G1 humanoid, while ScaleBFM supplies whole-body motion. After each trial, Astra reviews the outcome and revises the instructions and notes used in subsequent attempts. This adaptation changes the written guidance while keeping both models’ weights fixed.

SIMPLE 研究考察经验能否帮助 Astra 协调抓取、运输与交接。Astra 用视觉与本体感觉反馈为 G1 人形选择动作，ScaleBFM 提供全身运动。每次试验后，Astra 审阅结果并修订后续尝试所用的指令与笔记。该适应改变书面指引，同时保持两模型权重固定。

We evaluate Astra using the official SIMPLE codebase and L2 evaluation protocol, covering six tasks with ten scenes each. Astra succeeds in 50 of 60 trials, achieving an overall success rate of 83.3% (Table 12). Performance is strongest in tabletop and picking tasks, while mobile pick/place remains more difficult. Appendix E.2 details the protocol and separate retention checks.

我们用官方 SIMPLE 代码库与 L2 评测协议评测 Astra，覆盖六项任务、每任务十个场景。Astra 在 60 次试验中成功 50 次，总体成功率 83.3%（表 12）。桌面与拾取任务最强，移动抓放仍更难。协议与单独的保持性检查见附录 E.2。

Task	Successes	Rate
Handover	8/10	80%
Mobile pick/place	7/10	70%
Tabletop	9/10	90%
XMove bend pick	9/10	90%
XMove pick	9/10	90%
Bend	8/10	80%
Total	50/60	83.3%

Table 12: Astra with ScaleBFM on SIMPLE L2. Success under the official evaluation protocol, with ten scenes per task.

Task	Successes	Rate
Handover	8/10	80%
Mobile pick/place	7/10	70%
Tabletop	9/10	90%
XMove bend pick	9/10	90%
XMove pick	9/10	90%
Bend	8/10	80%
Total	50/60	83.3%

表 12：SIMPLE L2 上 Astra 配合 ScaleBFM。 按官方评测协议计成功，每任务十个场景。

Why Handover Remains Fragile

为何交接仍然脆弱

Handover reveals a limitation hidden by the aggregate score: native success does not require the carton to remain stable after complete release. In the observed failures, finger closure or carton tilt can be mistaken for a secure grasp while the table or the other hand still carries the load. Moving the receiving hand can then shift the supplying wrist as the body adjusts. A grasp that appears adequate at one stage may therefore fail during the next.

交接揭示了总分掩盖的局限：原生成功并不要求纸箱在完全释放后仍保持稳定。在观察到的失败中，手指合拢或纸箱倾斜可能被误认为稳固抓取，而桌子或另一只手仍在承重。移动接收手时，身体调整可能带动供货腕。因此某一阶段看似足够的抓取，可能在下一阶段失败。

The grasp procedure addresses this problem by checking support before advancing. It instructs Astra to separate grasping from lifting, withdraw the assisting hand, and check that the entire carton rises and moves with the supplying hand. If it merely rotates or the body keeps leaning forward, the procedure calls for restoring support and trying again. These instructions specify physical checks; their reliability still depends on Astra’s observations and actions.

抓取流程通过在推进前检查支撑来应对该问题。它指示 Astra 把抓取与提起分开、撤回辅助手，并检查整个纸箱是否随供货手升起并移动。若只是旋转或身体持续前倾，流程要求恢复支撑再试。这些指令规定物理检查；其可靠性仍取决于 Astra 的观测与动作。

The handover guidance does not consistently preserve successful behavior across repeated attempts. In a sequence of instruction-adaptation trials, a successful handover is followed by a failed attempt after further reflection. A fixed procedure tested on three previously successful scenes passes native success and an additional hold check on only two. The recovery trials in Appendix E.2 also fail on all three valid attempts despite longer budgets. A diagnosis need not yield correction or repeatable success.

交接指引并不能在重复尝试间一致地保持成功行为。在一串指令适应试验中，一次成功交接之后，进一步反思后的尝试失败。在三个先前成功场景上测试的固定流程，仅两次同时通过原生成功与额外保持检查。附录 E.2 的恢复试验在全部三次有效尝试上也失败，尽管预算更长。诊断未必带来修正或可重复成功。

Figure 14: Interactive scenes from egocentric video. Source frames and reconstructed geometry for two apple-transfer cases. Object and support layout is recovered, while table edges and furniture remain approximate.

图 14：来自第一人称视频的交互场景。两例苹果转移的源帧与重建几何。物体与支撑布局被恢复，桌边与家具仍为近似。

Scene and Motion from Egocentric Video

从第一人称视频得到场景与运动

Astra also builds and refines a pipeline that converts egocentric videos into interactive scenes and humanoid motion references. It assembles perception and retargeting tools, inspects their outputs, and corrects geometric and temporal inconsistencies. Camera estimates, object roles, and support relations define the scene; declared camera or support heights provide metric scale. Motion references specify pelvis, wrist, and ankle poses together with hand closure. They are generated from video-derived geometry or through EgoAllo body estimation [57], SMPL-X fitting [33], and motion retargeting [4].

Astra 还构建并修订一条把第一人称视频转为交互场景与人形运动参考的管线。它组装感知与重定向工具，检查其输出，并修正几何与时间不一致。相机估计、物体角色与支撑关系定义场景；声明的相机或支撑高度提供度量尺度。运动参考指定骨盆、腕与踝姿态以及手闭合；可由视频几何生成，或经 EgoAllo 身体估计 [57]、SMPL-X 拟合 [33] 与运动重定向 [4]。

In a kitchen with motion-capture-calibrated cameras, image-based refinement reduces scene Chamfer-L1 error from 35.93 to 24.73 mm; incorporating CAD dimensions reduces it further to 12.14 mm. Transferring to other recordings exposes the importance of object placement: reusing a bottle position displaced by a few centimeters causes grasp failures. Figure 14 shows reconstructions from separate everyday videos. Across three apple-transfer cases, complete-video references support one complete interaction: opening a bin by its pedal, releasing the apple inside, and allowing the lid to close. The other two cases still lose the apple during lifting despite local scene adjustments.

在装有动捕标定相机的厨房中，基于图像的修订把场景 Chamfer-L1 误差从 35.93 mm 降到 24.73 mm；纳入 CAD 尺寸进一步降到 12.14 mm。迁移到其他录制暴露了物体放置的重要性：复用偏移数厘米的瓶子位置会导致抓取失败。图 14 展示来自不同日常视频的重建。三例苹果转移中，完整视频参考支持一次完整交互：用踏板开箱、把苹果放入并让盖关闭。另两例尽管有局部场景调整，仍在提起时丢失苹果。

The central challenge is preserving interaction structure through reconstruction and retargeting. Support surfaces, container interiors, and movable mechanisms must remain usable, while motion references must connect approach, grasp, transport, and release in the correct order. The successful bin interaction preserves the approach and pedal contact before release; local grasp corrections alone leave the task incomplete. Astra can identify and revise these dependencies, but geometric plausibility does not ensure a successful physical interaction. The in-the-wild study consumed 217 million tokens across development, evaluation, and associated documentation, highlighting the cost of per-video refinement.

核心挑战是在重建与重定向中保持交互结构。支撑面、容器内部与可动机构必须仍可用，运动参考必须以正确顺序连接接近、抓取、运输与释放。成功的开箱交互在释放前保持了接近与踏板接触；仅做局部抓取修正会使任务不完整。Astra 能识别并修订这些依赖，但几何上说得通并不能保证物理交互成功。野外研究在开发、评测与相关文档上消耗 2.17 亿 token，凸显了按视频修订的成本。

Discussion

讨论

Across the six domains, Astra shows strong navigation results and useful manipulation capabilities, alongside persistent difficulties in contact coordination, locomotion, and completion verification. To interpret this uneven performance, we examine how control is divided and why feedback does not consistently turn useful local decisions into successful, efficient task behavior.

跨六个领域，Astra 展现出较强的导航结果与有用的操控能力，同时在接触协调、步行与完成核验上存在持续困难。为解释这种不均衡表现，我们考察控制如何划分，以及反馈为何不能稳定地把有用的局部决策变成成功、高效的任务行为。

Setting	Recorded resource use	Scope and interpretation
RoboDojo, 50 task instances per condition	Hybrid / Direct: 624.8M / 1,132.3M total tokens; 607.6M / 1,107.3M cached input; 15.90M / 22.91M uncached input; 1.31M / 2.09M output	Earlier attempts excluded. Cached input dominates the totals.
RoboCasa, 75 episodes per condition	Direct / Hybrid: 7,910 / 8,941 recorded model requests; median 85 / 108 per episode	Counts cover saved request logs, including recovery history, rather than all inference calls.
Dense locomotion, final attempt	250 synchronous calls for 30 s of simulated robot motion; mean model latency 39.86 s per call	Physics pauses during inference; PASSAGE planning takes approximately 0.08 s per call.

Table 13: Inference demands across studies. M denotes million tokens; rounded totals may not sum exactly. Units and recording scope are listed for each study. Full manipulation accounting appears in Tables 17 and 37.

设置	记录的资源用量	范围与解读
RoboDojo，每条件 50 个任务实例	Hybrid / Direct：总计 624.8M / 1,132.3M token；缓存输入 607.6M / 1,107.3M；非缓存输入 15.90M / 22.91M；输出 1.31M / 2.09M	排除更早尝试。缓存输入主导总量。
RoboCasa，每条件 75 个 episode	Direct / Hybrid：记录模型请求 7,910 / 8,941；中位每 episode 85 / 108	计数覆盖保存的请求日志（含恢复历史），而非全部推理调用。
稠密步行，最终尝试	30 秒仿真机器人运动对应 250 次同步调用；平均模型时延每调用 39.86 秒	推理期间物理暂停；PASSAGE 规划约每调用 0.08 秒。

表 13：各研究的推理开销。 M 表示百万 token；四舍五入后合计可能不完全相等。各研究列出单位与记录范围。完整操控核算见表 17 与表 37。

Capability Depends on the Complete Control Path

能力取决于完整控制路径

An action acquires physical meaning through its interface and controller. A navigation move invokes a discrete motion primitive; a dense motion reference requires compatible body and joint targets; a whole-body controller command invokes learned coordination. Evaluating Astra therefore requires specifying both the commands it constructs and the motion other components supply. The varied tasks and controllers provide complementary evidence about the complete control paths.

动作通过接口与控制器获得物理含义。导航移动调用离散运动原语；稠密运动参考要求兼容的身体与关节目标；全身控制器指令调用已学协调。因此评测 Astra 必须同时写明它构造的指令，以及其他组件提供的运动。多样任务与控制器为完整控制路径提供互补证据。

The in-hand results reveal a tension between stability and progress: retaining the current grasp can prevent the release and repositioning needed for continued rotation. Navigation presents a different problem, since a recognizable landmark need not establish the correct location or justify stopping. Control responsibility thus includes deciding which state changes matter and checking that they occur. These failures require different diagnostics and explanations.

手内结果揭示稳定与进度之间的张力：保持当前抓取可能妨碍继续旋转所需的松开与重定位。导航呈现不同问题：可识别的地标未必确立正确位置或证明应当停止。因此控制职责包括决定哪些状态变化要紧，并核验它们是否发生。这些失败需要不同的诊断与解释。

Intervention and Verified Progress

干预与经核验的进度

Useful cooperation requires both repairing failures and preserving behavior that already works. In the manipulation cases, Astra can improve the conditions under which a policy acts by redirecting a wrist or preparing contact, without constructing the subsequent trajectory. This division succeeds unevenly: RoboCasa Hybrid completes some instances that defeat the standalone policy and fails on others the policy completes. The trajectories illustrate useful roles for reasoning, while the aggregate comparisons measure the combined effects of policy access, action timing, and feedback management.

有用的协作既要修复失败，也要保住已经奏效的行为。在操控案例中，Astra 可通过重定向腕部或准备接触来改善策略作用条件，而无需构造后续轨迹。这一分工成功并不均衡：RoboCasa Hybrid 完成一些独立策略失败的实例，也在策略能完成的其他实例上失败。轨迹说明了推理的有用角色，而总体对照度量的是策略访问、动作时序与反馈管理的综合效应。

Detecting an error, proposing a correction, and verifying its effect are separate demands. A new grasp still needs clearance and support; an adjusted placement still needs a correct completion check. Action-source fractions describe who supplies commands, but neither the number of corrections nor the plausibility of their explanations measures their value. That value rests on the ensuing physical outcome.

发现错误、提出修正与核验其效果是不同需求。新抓取仍需净空与支撑；调整后的放置仍需正确的完成检查。动作来源比例描述谁提供指令，但修正次数或其解释的可信度都不能度量其价值。价值取决于随后的物理结果。

Retaining Success across Trials

跨试验保持成功

Feedback can improve a procedure on a known case without making it reliable across attempts. The Push development records include a successful trial followed by a failed repeat. Diagnosis, written guidance, and sustained control thus need separate assessment: a sound explanation can identify what went wrong while leaving the next action inadequate.

反馈可以在已知案例上改进流程，却不一定在重复尝试间可靠。Push 开发记录包含一次成功试验后紧跟失败重试。因此诊断、书面指引与持续控制需要分开评估：正确解释能指出哪里出错，却仍可能留下不足的下一步动作。

This distinction matters when interpreting improvement with fixed model weights. Changes to guidance, interfaces, or whole-body controller calibration can change system performance. Repeated trials reveal whether the resulting behavior persists; transfer to new states tests whether the guidance extends beyond the cases that produced it.

在模型权重固定时解读改进，这一区分很重要。对指引、接口或全身控制器标定的改动可以改变系统表现。重复试验揭示所得行为是否持续；迁移到新状态则检验指引是否超出产生它的那些案例。

Resource Costs and Capability Limits

资源开销与能力边界

A capability is demonstrated with a particular budget. Long input histories and repeated review increase resource use, while paused physics separates simulated motion time from inference time. Table 13 reports recorded resource use.

能力是在特定预算下被展示的。长输入历史与反复审阅增加资源用量，而暂停物理把仿真运动时间与推理时间分开。表 13 报告记录的资源用量。

The two manipulation studies illustrate different effects of cooperation. RoboDojo Hybrid reduces recorded tokens by 44.8%, yet retains substantial uncached input and output. RoboCasa Hybrid reduces Astra-authored action segments but records more model requests than Direct. Delegating motion therefore does not necessarily delegate the work of reviewing it. Inference efficiency depends on review frequency as well as action generation.

两项操控研究说明协作的不同效应。RoboDojo Hybrid 使记录 token 减少 44.8%，但仍保留可观的非缓存输入与输出。RoboCasa Hybrid 减少 Astra 撰写的动作段，但记录的模型请求多于 Direct。因此委托运动并不必然委托审阅工作。推理效率既取决于审阅频率，也取决于动作生成。

Dense locomotion exposes a more immediate limit: a decision takes far longer than the motion executed before the next observation. Pausing physics makes that delay tolerable in the experiment; a physical scene cannot be assumed to wait. Practical control therefore requires assessing inference delays alongside the rate at which physical feedback changes.

稠密步行暴露更直接的限制：一次决策远长于下一次观测前已执行的运动。暂停物理使该时延在实验中可容忍；物理场景不能假定会等待。因此实际控制需要把推理时延与物理反馈变化速率一并评估。

Conclusion

结论

We systematically evaluate GPT-6 Astra as an embodied policy across six domains. Astra shows strengths in navigation and task-oriented manipulation, correcting targets and contact conditions while cooperating with learned policies; pretrained whole-body controllers also support humanoid loco-manipulation. However, direct in-hand control and dense-reference locomotion remain unreliable, and policy assistance can fail on tasks that the standalone policy solves. These findings expose a gap between generating useful action decisions and reliably producing and verifying their physical effects. Learned policies and whole-body controllers help bridge this gap, but effective cooperation still requires Astra to recognize when to intervene and when to return control to the policy. Substantial token use and inference latency further constrain practical control, even when learned policies supply most physical actions.

我们在六个领域系统评测 GPT-6 Astra 作为具身策略。Astra 在导航与面向任务的操控上表现较强，能在与已学策略协作时修正目标与接触条件；预训练全身控制器也支持人形步行-操控。然而，直接手内控制与稠密参考步行仍不可靠，策略辅助也可能在独立策略本可解决的任务上失败。这些发现暴露了「生成有用动作决策」与「可靠地产生并核验其物理效果」之间的差距。已学策略与全身控制器有助于弥合差距，但有效协作仍要求 Astra 识别何时干预、何时把控制交回策略。即便已学策略提供大部分物理动作，可观的 token 用量与推理时延仍进一步制约实际控制。

Contributors

贡献者

Each group lists names alphabetically by surname.

每组按姓氏字母顺序列名。

Xiaoqian Cheng, Jiayi Su, Mi Yan, Yixin Zheng

Jiahang Liu, Ruochen Xu, Tianyu Xu

Yu Deng, Lihe Ding, Shaocong Dong, Xiangjun Gao, Sikai Liang, Qingtao Liu, Zhe Xu, Siming Yan

Haozhe Jia, Zhoujian Li, Yunrui Lian, Chenghuai Lin, Yuxuan Ma, Xudong Xu, Zhikai Zhang, Weiyi Zhu

Xuchuan Chen, Zekai Li, Dairu Liu, Zekun Qi, Ruixi Yu, Jinlu Zhang, Yintianrun Zhang

Yu Deng, Jiahang Liu, Qingtao Liu, Yuxuan Ma, Zekun Qi, Jiayi Su, Ruochen Xu, Siming Yan

He Wang, Li Yi, Zhizheng Zhang

References

参考文献

[1] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691 , 2022. https://arxiv.org/abs/2204.01691 .

[2] Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, and Amir R. Zamir. On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757 , 2018a. https://arxiv.org/abs/1807.06757 .

[3] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018b. https://arxiv.org/abs/1711.07280 .

[4] Joao Pedro Araujo, Yanjie Ze, Pei Xu, Jiajun Wu, and C. Karen Liu. Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking. arXiv preprint arXiv:2510.02252 , 2025. https://arxiv.org/abs/2510.02252 .

[5] Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. ObjectNav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006.13171 , 2020. https://arxiv.org/abs/2006.13171 .

[6] Yihan Cao, Jiazhao Zhang, Zhinan Yu, Shuzhen Liu, Zheng Qin, Qin Zou, Bo Du, and Kai Xu. CogNav: Cognitive process modeling for object goal navigation with LLMs. arXiv preprint arXiv:2412.10439 , 2024. https://arxiv.org/abs/2412.10439v2 .

[7] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Nießner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3D: Learning from RGB-D Data in Indoor Environments. arXiv preprint arXiv:1709.06158 , 2017. https://arxiv.org/abs/1709.06158 .

[8] Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Haoran Lu, Weijie Wan, Baijun Chen, et al. RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies. arXiv preprint arXiv:2607.04434 , 2026. https://arxiv.org/abs/2607.04434 .

[9] Olive Jean Dunn. Multiple Comparisons among Means. Journal of the American Statistical Association , 56(293):52–64, 1961. 10.1080/01621459.1961.10482090 . https://doi.org/10.1080/01621459.1961.10482090 .

[10] Morten W Fagerland, Stian Lydersen, and Petter Laake. The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional. BMC Medical Research Methodology , 13(1), 2013. 10.1186/1471-2288-13-91 . https://doi.org/10.1186/1471-2288-13-91 .

[11] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv preprint arXiv:1801.01290 , 2018. https://arxiv.org/abs/1801.01290 .

[12] Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering Diverse Domains through World Models. arXiv preprint arXiv:2301.04104 , 2023. https://arxiv.org/abs/2301.04104 .

[13] Nicklas Hansen, Hao Su, and Xiaolong Wang. TD-MPC2: Scalable, Robust World Models for Continuous Control. arXiv preprint arXiv:2310.16828 , 2023. https://arxiv.org/abs/2310.16828 .

[14] HKU SAIL. Real2Sim_GPT6_ASTRA: Three-view robot RGB scene reconstruction and action replay. Community implementation, 2026. https://github.com/hku-sail/Real2Sim_GPT6_ASTRA . Accessed September 25, 2026.

[15] Gabriel Ilharco, Vihan Jain, Alexander Ku, Eugene Ie, and Jason Baldridge. General evaluation for instruction conditioned navigation using dynamic time warping. arXiv preprint arXiv:1907.05446 , 2019. https://arxiv.org/abs/1907.05446 .

[16] Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. OmniSpatial: Towards comprehensive spatial reasoning benchmark for vision language models. In International Conference on Learning Representations (ICLR) , 2026. https://openreview.net/forum?id=6nZKT2rL0H .

[17] O. Khatib. A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE Journal on Robotics and Automation , 3(1):43–53, 1987. 10.1109/JRA.1987.1087068 . https://doi.org/10.1109/JRA.1987.1087068 .

[18] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, et al. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset. arXiv preprint arXiv:2403.12945 , 2024. https://arxiv.org/abs/2403.12945 .

[19] Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision-and-language navigation in continuous environments. arXiv preprint arXiv:2004.02857 , 2020. https://arxiv.org/abs/2004.02857 .

[20] Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding. arXiv preprint arXiv:2010.07954 , 2020. https://arxiv.org/abs/2010.07954 .

[21] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , 2023. https://arxiv.org/abs/2209.07753 .

[22] Yutong Liang, Quanquan Peng, Ri-Zhao Qiu, and Xiaolong Wang. ConTrack: Constrained hand motion tracking with adaptive trade-off control. In European Conference on Computer Vision (ECCV) , 2026. https://arxiv.org/abs/2606.03177 .

[23] Jiahang Liu, Tianyu Xu, Jiawei Chen, Lu Yue, Jiazhao Zhang, Zhiyong Wang, Minghan Li, Qisheng Zhao, Anqi Li, Qi Su, et al. Span-nav: Generalized spatial awareness for versatile vision-language navigation. arXiv preprint arXiv:2603.09163 , 2026a.

[24] Zixuan Liu, Ruoyi Qiao, Chenrui Tie, Xuanwei Liu, Yunfan Lou, Chongkai Gao, Zhixuan Xu, and Lin Shao. ContactExplorer: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation. arXiv preprint arXiv:2603.10971 , 2026b. https://arxiv.org/abs/2603.10971 .

[25] Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. arXiv preprint arXiv:1711.05101 , 2017. https://arxiv.org/abs/1711.05101 .

[26] Yuxuan Ma, Zicheng Zeng, Chunlin Peng, Zhoujian Li, Zetong Zhao, Zhikai Zhang, Yunrui Lian, Han Xue, Sikai Liang, Weiyi Zhu, Mulin Chen, Chenghuai Lin, Jiayu Zeng, Yanwei An, Songan Zhang, Jiayuan Gu, Jilong Wang, Jingbo Wang, He Wang, and Li Yi. Passage: Scaling scene-aligned motion learning for perceptive humanoid traversal in cluttered environments, 2026. https://arxiv.org/abs/2609.18732 .

[27] Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, et al. Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning. arXiv preprint arXiv:2108.10470 , 2021. https://arxiv.org/abs/2108.10470 .

[28] Yusuke Mikami, Andrew Melnik, Jun Miura, and Ville Hautamäki. Natural language as policies: Reasoning for coordinate-level embodied control with LLMs. arXiv preprint arXiv:2403.13801 , 2024. https://arxiv.org/abs/2403.13801 .

[29] Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddukuri, and Yuke Zhu. RoboCasa365: A large-scale simulation framework for training and benchmarking generalist robots. arXiv preprint arXiv:2603.04356 , 2026. https://arxiv.org/abs/2603.04356 .

[30] NVIDIA. Cosmos3-Nano-Policy-DROID. Official model card, 2026. https://huggingface.co/nvidia/Cosmos3-Nano-Policy-DROID . Accessed September 25, 2026.

[31] NVIDIA, Mayank Mittal, Pascal Roth, James Tigue, Antoine Richard, Octi Zhang, Peter Du, Antonio Serrano-Muñoz, et al. Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. arXiv preprint arXiv:2511.04831 , 2025. https://arxiv.org/abs/2511.04831 .

[32] OpenAI. GPT-6 Astra: A new generation of intelligence. Model release, 2026. https://openai.com/index/gpt-6-astra/ . Accessed September 25, 2026.

[33] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive Body Capture: 3D Hands, Face, and Body from a Single Image. arXiv preprint arXiv:1904.05866 , 2019. https://arxiv.org/abs/1904.05866 .

[34] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, et al. π 0.5 \pi_{0.5} : a Vision-Language-Action Model with Open-World Generalization. arXiv preprint arXiv:2504.16054 , 2025. https://arxiv.org/abs/2504.16054 .

[35] Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. ShapeLLM: Universal 3D object understanding for embodied interaction. In European Conference on Computer Vision (ECCV) , 2024. https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/05974.pdf .

[36] Zekun Qi, Wenyao Zhang, Yufei Ding, Runpei Dong, XinQiang Yu, Jingwen Li, Lingyun Xu, Baoyu Li, Xialin He, Guofan Fan, Jiazhao Zhang, Jiawei He, Jiayuan Gu, Xin Jin, Kaisheng Ma, Zhizheng Zhang, He Wang, and Li Yi. SoFar: Language-grounded orientation bridges spatial reasoning and object manipulation. In Advances in Neural Information Processing Systems , volume 38, pages 76367–76412, 2025. 10.52202/085713-2565 .

[37] Zekun Qi, Xuchuan Chen, Dairu Liu, Chenghuai Lin, Yunrui Lian, Sikai Liang, Zhikai Zhang, Yu Guan, et al. Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking. arXiv preprint arXiv:2606.03985 , 2026. https://arxiv.org/abs/2606.03985 .

[38] Ruoyi Qiao and contributors. mjlab_hand: Dexterous manipulation tasks built on mjlab. Code repository, 2026. https://github.com/ruoyiqiao/mjlab_hand . Accessed September 25, 2026.

[39] Stan Salvador and Philip Chan. Toward accurate dynamic time warping in linear time and space. Intelligent Data Analysis , 11(5):561–580, 2007. 10.3233/IDA-2007-11508 . https://doi.org/10.3233/IDA-2007-11508 .

[40] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, et al. Habitat: A Platform for Embodied AI Research. arXiv preprint arXiv:1904.01201 , 2019. https://arxiv.org/abs/1904.01201 .

[41] John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv preprint arXiv:1506.02438 , 2015. https://arxiv.org/abs/1506.02438 .

[42] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347 , 2017. https://arxiv.org/abs/1707.06347 .

[43] Clemens Schwarke, Mayank Mittal, Nikita Rudin, David Hoeller, and Marco Hutter. RSL-RL: A Learning Library for Robotics Research. arXiv preprint arXiv:2509.10771 , 2025. https://arxiv.org/abs/2509.10771 .

[44] Carmelo Sferrazza, Dun-Ming Huang, Xingyu Lin, Youngwoon Lee, and Pieter Abbeel. HumanoidBench: Simulated humanoid benchmark for whole-body locomotion and manipulation. arXiv preprint arXiv:2403.10506 , 2024. https://arxiv.org/abs/2403.10506 .

[45] Sharpa Robotics. Sharpa Wave dexterous hand. [Online], n.d. https://www.sharpa.com/pages/wave . Accessed: September 20, 2026.

[46] Emanuel Todorov, Tom Erez, and Yuval Tassa. MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 5026–5033, 2012. 10.1109/IROS.2012.6386109 . https://doi.org/10.1109/IROS.2012.6386109 .

[47] Charles Wampler. Manipulator Inverse Kinematic Solutions Based on Vector Formulations and Damped Least-Squares Methods. IEEE Transactions on Systems, Man, and Cybernetics , 16(1):93–101, 1986. 10.1109/TSMC.1986.289285 . https://doi.org/10.1109/TSMC.1986.289285 .

[48] Hanwen Wang, Weizhi Zhao, Xiangyu Wang, Siyuan Huang, He Lin, Boyuan Zheng, Rongtao Xu, Gang Wang, Yao Mu, He Wang, Lue Fan, Hongsheng Li, Zhaoxiang Zhang, and Tieniu Tan. DexJoCo: A benchmark and toolkit for task-oriented dexterous manipulation on MuJoCo, 2026a. https://arxiv.org/abs/2605.16257v2 .

[49] Shaoan Wang, Aocheng Luo, Fei Huang, Jingyi Xu, Xiaoyang Wang, Yueyu Wang, Qianli Ma, Fan Yang, Ran Mei, Jia Wei, et al. LightNav-0: Eliciting VLM spatial intelligence for generalist embodied navigation. arXiv preprint arXiv:2608.30935 , 2026b. https://arxiv.org/abs/2608.30935 .

[50] Yen-Jen Wang, Bike Zhang, Jianyu Chen, and Koushil Sreenath. Prompt a robot to walk with large language models. In 2024 IEEE 63rd Conference on Decision and Control (CDC) , 2024. https://arxiv.org/abs/2309.09969 .

[51] Songlin Wei, Zhenhao Ni, Jie Liu, Zhenyu Zhao, Junjie Ye, Hongyi Jing, Junkai Xia, Xiawei Liu, Michael Leong, Liang Heng, Di Huang, and Yue Wang. SIMPLE: Simulation-based policy learning and evaluation for humanoid loco-manipulation. arXiv preprint arXiv:2606.08278 , 2026. https://arxiv.org/abs/2606.08278 .

[52] Edwin B. Wilson. Probable Inference, the Law of Succession, and Statistical Inference. Journal of the American Statistical Association , 22(158):209–212, 1927. 10.1080/01621459.1927.10502953 . https://doi.org/10.1080/01621459.1927.10502953 .

[53] Xinda Xue, Junjun Hu, Minghua Luo, Shichao Xie, Jintao Chen, Zixun Xie, Kuichen Quan, Wei Guo, Mu Xu, and Zedong Chu. OmniNav: A unified framework for prospective exploration and visual-language navigation. arXiv preprint arXiv:2509.25687 , 2025. https://arxiv.org/abs/2509.25687 .

[54] Karmesh Yadav, Ram Ramrakhya, Santhosh Kumar Ramakrishnan, Theo Gervet, John Turner, Aaron Gokaslan, Noah Maestre, Angel Xuan Chang, Dhruv Batra, Manolis Savva, et al. Habitat-matterport 3d semantics dataset. arXiv preprint arXiv:2210.05633 , 2022. https://arxiv.org/abs/2210.05633 .

[55] Jenai Xuning Yang, Rishit Dagli, Alex Zook, Hugo Hadfield, Ankit Goyal, Stan Birchfield, Fabio Ramos, and Jonathan Tremblay. RoboLab: A high-fidelity simulation benchmark for analysis of task generalist policies. arXiv preprint arXiv:2604.09860 , 2026. https://arxiv.org/abs/2604.09860 .

[56] Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, et al. World Action Models are Zero-shot Policies. arXiv preprint arXiv:2602.15922 , 2026. https://arxiv.org/abs/2602.15922 .

[57] Brent Yi, Vickie Ye, Maya Zheng, Yunqi Li, Lea Müller, Georgios Pavlakos, Yi Ma, Jitendra Malik, and Angjoo Kanazawa. Estimating Body and Hand Motion in an Ego-sensed World. arXiv preprint arXiv:2410.03665 , 2024. https://arxiv.org/abs/2410.03665 .

[58] Jessica Yin, Haozhi Qi, Jitendra Malik, James Pikul, Mark Yim, and Tess Hellebrekers. Learning in-hand translation using tactile skin with shear and normal force sensing. In 2025 IEEE International Conference on Robotics and Automation (ICRA) , pages 5850–5856. IEEE, 2025. https://arxiv.org/abs/2407.07885 .

[59] Kevin Zakka, Qiayuan Liao, Brent Yi, Louis Le Lay, Koushil Sreenath, and Pieter Abbeel. mjlab: A lightweight framework for gpu-accelerated robot learning. arXiv preprint arXiv:2601.22074 , 2026. https://arxiv.org/abs/2601.22074 .

[60] Weishuai Zeng, Kangning Yin, Xiaojie Niu, Shunlin Lu, Weixiang Zhong, Jiahe Chen, Feiyu Jia, Xiao Chen, et al. Scaling Behavior Foundation Model for Humanoid Robots. arXiv preprint arXiv:2607.15163 , 2026. https://arxiv.org/abs/2607.15163 .

[61] Jiazhao Zhang, Kunyu Wang, Shaoan Wang, Minghan Li, Haoran Liu, Songlin Wei, Zhongyuan Wang, Zhizheng Zhang, and He Wang. Uni-NaVid: A video-based vision-language-action model for unifying embodied navigation tasks. Robotics: Science and Systems , 2025. https://arxiv.org/abs/2412.06224 .

[62] Jiazhao Zhang, Anqi Li, Yunpeng Qi, Minghan Li, Jiahang Liu, Shaoan Wang, Haoran Liu, Gengze Zhou, Yuze Wu, Xingxing Li, et al. Embodied navigation foundation model. In International Conference on Learning Representations , volume 2026, pages 127293–127322, 2026a. https://arxiv.org/abs/2509.12129 .

[63] Jiazhao Zhang, Gengze Zhou, Hale Yin, Yiyang Huang, Zixing Lei, Qihang Peng, Haoqi Yuan, Jie Zhang, et al. Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System. arXiv preprint arXiv:2606.18112 , 2026b. https://arxiv.org/abs/2606.18112 .

[64] Wenbo Zhang, Kaixuan Wang, Yutao Ouyang, Xiaoyu Huang, Liyang Li, Kailun Su, Weiyang Jin, Wenhao Chai, et al. An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond. arXiv preprint arXiv:2609.24170 , 2026c. https://arxiv.org/abs/2609.24170 .

[65] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. On the Continuity of Rotation Representations in Neural Networks. arXiv preprint arXiv:1812.07035 , 2018. https://arxiv.org/abs/1812.07035 .

[66] Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Abhishek Joshi, Kevin Lin, Abhiram Maddukuri, Soroush Nasiriany, and Yifeng Zhu. robosuite: A Modular Simulation Framework and Benchmark for Robot Learning. arXiv preprint arXiv:2009.12293 , 2020. https://arxiv.org/abs/2009.12293 .

Appendix A Additional Interface Details

附录 A 额外接口细节

This appendix specifies the gripper and humanoid interfaces summarized in Section 2. Figure 15 illustrates how these systems connect action proposals, whole-body controllers, and permitted feedback.

本附录说明第 2 节概括的夹爪与人形接口。图 15 展示这些系统如何连接动作提案、全身控制器与允许的反馈。

Figure 15: Action and development loops. Path 01 illustrates Direct control; paths 02 and 03 illustrate Hybrid with a task policy or whole-body controller. The action loop returns only study-permitted observations and feedback; task evaluation is a separate branch. The outer loop represents researcher and development-agent changes to guidance, tools, or interfaces before evaluation.

图 15：动作与开发回路。路径 01 示意 Direct 控制；路径 02 与 03 示意带任务策略或全身控制器的 Hybrid。动作回路仅返回研究所允许的观测与反馈；任务评测是单独分支。外层回路表示评测前研究者与开发智能体对指引、工具或接口的改动。

Gripper Interfaces

夹爪接口

RoboDojo supplies head and bilateral wrist images, a 14-dimensional proprioceptive state, measured end-effector poses, the instruction, execution history, progress, step budget, and completion rules. Hidden object states, future trajectories, rollback, and hidden planners are excluded. Policy images remain $640\times 480$; Astra generally views previews with a 480-pixel long side, with full-resolution images accessible.

RoboDojo 提供头部与双侧腕部图像、14 维本体感觉状态、测得的末端位姿、指令、执行历史、进度、步预算与完成规则。隐藏物体状态、未来轨迹、回滚与隐藏规划器均排除。策略图像保持 $640\times 480$；Astra 通常查看长边 480 像素的预览，也可访问全分辨率图像。

Direct specifies bimanual position, orientation, and gripper targets. Bounded damped least-squares inverse kinematics [47] converts them to joint commands, with 5 cm translation and 0.35 rad rotation safeguards. Segments last one to five 25-Hz control steps. Hybrid exposes a $50\times 14$ task-finetuned $\pi_{0.5}$ proposal through forward kinematics. Astra accepts a prefix of one to fifteen steps or replaces it with a one-to-five-step correction. Both use xhigh reasoning and the same native goals and safeguards. Many tasks require both arms home.

Direct 指定双臂位置、朝向与夹爪目标。有界阻尼最小二乘逆运动学 [47] 将其转为关节指令，并设 5 cm 平移与 0.35 rad 旋转护栏。动作段持续 1–5 个 25 Hz 控制步。Hybrid 通过正运动学暴露 $50\times 14$ 的任务微调 $\pi_{0.5}$ 提案。Astra 接受 1–15 步前缀，或以 1–5 步修正替换。二者均使用 xhigh 推理与相同原生目标与护栏。许多任务要求双臂回位。

RoboLab uses a single-arm Franka and the same division between direct targets and policy assistance. Its $\pi_{0.5}$ weights are DROID-trained [18], without finetuning on demonstrations from the test tasks. Appendix B gives the evaluation procedure and decision budgets.

RoboLab 使用单臂 Franka，并采用相同的直接目标与策略辅助划分。其 $\pi_{0.5}$ 权重为 DROID 训练 [18]，未在测试任务演示上微调。评测流程与决策预算见附录 B。

Humanoid Interfaces

人形接口

The HumanoidBench interface uses Humanoid-GPT whole-body controllers [37]. It supplies body-frame forward/lateral velocity and yaw rate to a frozen walking policy, or pelvis-height and wrist offsets to a frozen sparse five-point tracker. Feet and keypoint orientations keep nominal references. Walking and tracking policies operate at 50 Hz over 500-Hz native physics and joint proportional–derivative control. IK and PD implement the commanded motion.

HumanoidBench 接口使用 Humanoid-GPT 全身控制器 [37]。它向冻结行走策略提供身体坐标系前后/侧向速度与偏航率，或向冻结稀疏五点 tracker 提供骨盆高度与腕部偏移。足部与关键点朝向保持标称参考。行走与跟踪策略以 50 Hz 运行于 500 Hz 原生物理与关节 PD 控制之上。IK 与 PD 实现所指令的运动。

Sparse targets are coupled whole-body references, re-entering sparse mode resets the anchor, and walking changes arm posture. Observations include privileged state and task geometry. The task suite also includes task-specific access such as bounded finger closure for Door.

稀疏目标是耦合的全身参考；重新进入稀疏模式会重置锚点，行走会改变手臂姿态。观测包括特权状态与任务几何。任务套件还包含任务专用访问，例如 Door 的有界手指闭合。

Run, Crawl, and Stair use continuous five-point gait generators. Astra chooses speed, heading, and observation interval; calibrated generators supply posture, phase, and limb references. Stair additionally uses measured support heights and lateral foot-placement feedback. Walk combines cadence and leg-extension calibration with startup guidance. Push uses staging, contact guidance, and bounded hand control. The dense ScaleTrack locomotion study uses a separate robot–tracker protocol.

Run、Crawl 与 Stair 使用连续五点步态生成器。Astra 选择速度、航向与观测间隔；标定生成器提供姿态、相位与肢体参考。Stair 额外使用测得支撑高度与侧向落足反馈。Walk 结合步频与腿伸展标定以及启动指引。Push 使用就位、接触指引与有界手控。稠密 ScaleTrack 步行研究使用单独的机器人–tracker 协议。

Experience and Evaluation Scope

经验与评测范围

Completion and timing conventions follow Section 2; replay verification is documented in Appendix C. The interfaces explicitly supply the state variables listed above to Astra.

完成与计时约定遵循第 2 节；回放核验见附录 C。接口明确向 Astra 提供上文所列状态变量。

Astra’s weights remain fixed; guidance may include calibrated examples, feedback rules, or motion clips.

Astra 权重保持固定；指引可包含标定示例、反馈规则或运动片段。

Appendix B Detailed Manipulation Results

附录 B 详细操控结果

RoboDojo

Table 14 expands the aggregate results in Section 3.2. Direct Score averages the 48 episodes with valid Scores; its success rate uses all 50 instances. Published policy scores are reweighted to the same task and scene mixture. Direct and Hybrid share scene configurations and seeds.

表 14 展开第 3.2 节的汇总结果。Direct Score 对 48 个有有效 Score 的 episode 取平均；成功率使用全部 50 个实例。已发表策略分数按相同任务与场景配比重加权。Direct 与 Hybrid 共享场景配置与种子。

	Native Score	Success rate (%)
Task	Direct	Hybrid	Direct	Hybrid
Organize Table	30.0	60.0	0	0
Classify Objects By Language	60.0	38.0	40	20
Imitate Sorting Sequence	0.0	53.0	0	40
Arrange Largest Number	57.0	50.0	40	40
Pack Objects Into Box	50.0	50.0	20	20
Classify Objects	100.0	71.0	100	60
Build Tower	12.0	64.0	0	60
Make Kong	0.0	40.0	0	40
Fold Clothes	40.0	100.0	40	100
Put Bottles Into Dustbin	36.0	100.0	20	100
Aggregate	37.81	62.60	26	48

Table 14: RoboDojo results. Each architecture has five evaluation instances per task. Direct and Hybrid are paired on scene configuration and recorded seeds.

	Native Score		Success rate (%)
Task	Direct	Hybrid	Direct	Hybrid
Organize Table	30.0	60.0	0	0
Classify Objects By Language	60.0	38.0	40	20
Imitate Sorting Sequence	0.0	53.0	0	40
Arrange Largest Number	57.0	50.0	40	40
Pack Objects Into Box	50.0	50.0	20	20
Classify Objects	100.0	71.0	100	60
Build Tower	12.0	64.0	0	60
Make Kong	0.0	40.0	0	40
Fold Clothes	40.0	100.0	40	100
Put Bottles Into Dustbin	36.0	100.0	20	100
Aggregate	37.81	62.60	26	48

表 14：RoboDojo 结果。 每种架构每任务五个评测实例。Direct 与 Hybrid 在场景配置与记录种子上配对。

RoboLab

Task	Direct	Hybrid	$\pi_{0.5}$	Cosmos	DreamZero
Blocks into bin	5/5	5/5	0/5	2/5	0/5
Pumpkins in clutter	5/5	4/5	0/5	0/5	0/5
Butter on raisin box	5/5	5/5	0/5	1/5	2/5
Stack blocks in order	5/5	4/5	0/5	0/5	0/5
Reorient red mug	5/5	4/5	2/5	1/5	1/5
Larger raisin box into bin	4/5	4/5	3/5	0/5	0/5
Sauce bottle into crate	5/5	5/5	2/5	5/5	5/5
Canned food into bin	5/5	5/5	4/5	4/5	4/5
Yogurt into bowl	5/5	5/5	2/5	0/5	0/5
Rubik’s cube into bowl	5/5	5/5	5/5	5/5	5/5
Total	49/50	46/50	18/50	18/50	17/50

Table 15: RoboLab task success. Each method contributes five outcomes per task. Cosmos denotes Cosmos3-Nano-Policy [30]; DreamZero follows [56].

Task	Direct	Hybrid	$\pi_{0.5}$	Cosmos	DreamZero
Blocks into bin	5/5	5/5	0/5	2/5	0/5
Pumpkins in clutter	5/5	4/5	0/5	0/5	0/5
Butter on raisin box	5/5	5/5	0/5	1/5	2/5
Stack blocks in order	5/5	4/5	0/5	0/5	0/5
Reorient red mug	5/5	4/5	2/5	1/5	1/5
Larger raisin box into bin	4/5	4/5	3/5	0/5	0/5
Sauce bottle into crate	5/5	5/5	2/5	5/5	5/5
Canned food into bin	5/5	5/5	4/5	4/5	4/5
Yogurt into bowl	5/5	5/5	2/5	0/5	0/5
Rubik’s cube into bowl	5/5	5/5	5/5	5/5	5/5
Total	49/50	46/50	18/50	18/50	17/50

表 15：RoboLab 任务成功。 每方法每任务贡献五个结果。Cosmos 指 Cosmos3-Nano-Policy [30]；DreamZero 遵循 [56]。

RoboLab permits repeat attempts for Astra. Two Direct Blocks-in-Bin attempts use a 500-decision budget; the standard budget is 180. Simulation horizons and native success criteria remain fixed. Each baseline contributes its first five runs per task, ordered by run, episode, and environment ID, with the initial states, task versions, and control settings of those runs.

RoboLab 允许 Astra 重复尝试。两次 Direct Blocks-in-Bin 尝试使用 500 次决策预算；标准预算为 180。仿真时域与原生成功判据保持固定。每个基线按 run、episode 与 environment ID 排序贡献其每任务前五次运行，并沿用那些运行的初始状态、任务版本与控制设置。

Appendix C Focused Experiments, Resource Use, and Reproducibility

附录 C 聚焦实验、资源用量与可复现性

Focused Push and Walk Records

聚焦 Push 与 Walk 记录

These focused studies use separately frozen configurations. The table distinguishes development seeds from transfer seeds fixed in advance; these runs are separate from the task-suite results in Section 8.

这些聚焦研究使用分别冻结的配置。表区分开发种子与事先固定的迁移种子；这些运行独立于第 8 节任务套件结果。

Configuration	Seed role	Seed	Native return	Controls	Physical outcome
Focused Push	Development	0	872.58	396	4.66 cm; success
Focused Push	Development	3	866.75	423	4.78 cm; success
Focused Push	Development	5	848.01	457	4.85 cm; success
Focused Push	Transfer	7	900.37	323	4.81 cm; success
Walk, 1.8 Hz	Development	1	743.54	1000	No fall
Walk, 1.8 Hz	Development	2	748.92	1000	No fall
Walk, 1.8 Hz	Transfer	8	743.18	1000	No fall
Walk, 1.8 Hz	Transfer	9	748.54	1000	No fall

Table 16: Focused online Astra runs under separately frozen configurations. Push uses calibrated examples, with seeds 3 and 5 reproducing supplied physical trajectories; its physical outcome gives the final box-to-goal error and native success. Walk uses a gait frequency calibrated offline and a supplied heading-feedback rule. These outcomes are not substituted into the 30-task suite.

Configuration	Seed role	Seed	Native return	Controls	Physical outcome
Focused Push	Development	0	872.58	396	4.66 cm; success
Focused Push	Development	3	866.75	423	4.78 cm; success
Focused Push	Development	5	848.01	457	4.85 cm; success
Focused Push	Transfer	7	900.37	323	4.81 cm; success
Walk, 1.8 Hz	Development	1	743.54	1000	No fall
Walk, 1.8 Hz	Development	2	748.92	1000	No fall
Walk, 1.8 Hz	Transfer	8	743.18	1000	No fall
Walk, 1.8 Hz	Transfer	9	748.54	1000	No fall

表 16：分别冻结配置下的聚焦在线 Astra 运行。 Push 使用标定示例，种子 3 与 5 复现所给物理轨迹；物理结果给出最终箱到目标误差与原生成功。Walk 使用离线标定的步态频率与所给航向反馈规则。这些结果不并入 30 任务套件。

A separate scripted Push diagnostic tests whether contact must be calibrated jointly with the state at which the robot enters the contact phase. On seed 5, the final box-to-goal error is 27.02 cm with unchanged entry and references, 36.81 cm with changed entry only, 23.11 cm with changed contact references only, and 4.97 cm when both are changed. Heading and velocity also vary slightly between conditions.

单独的脚本化 Push 诊断检验接触是否必须与进入接触阶段时的状态联合标定。在种子 5 上，入口与参考不变时最终箱到目标误差为 27.02 cm；仅改入口为 36.81 cm；仅改接触参考为 23.11 cm；两者都改为 4.97 cm。航向与速度在条件间也略有变化。

Independent Reach Records

独立 Reach 记录

Two Reach runs from a separate Walk/Reach/Door batch measure target dwell time. Seeds 0 and 2 finish all 1000 controls with native returns of 15675.19 and 15380.91, respectively, both above the 12000 threshold. The hand is within the 5 cm target radius for 11.36 s and 12.78 s. These records use separate archived instructions and execution settings; they are not part of the task suite’s Reach sample.

来自单独 Walk/Reach/Door 批次的两次 Reach 运行度量目标停留时间。种子 0 与 2 完成全部 1000 个控制，原生回报分别为 15675.19 与 15380.91，均高于 12000 阈值。手在 5 cm 目标半径内分别停留 11.36 秒与 12.78 秒。这些记录使用单独归档的指令与执行设置；不属于任务套件的 Reach 样本。

RoboDojo Execution and Token Accounting

RoboDojo 执行与 token 核算

Table 17 provides exact counts for Section 9.4. Simulated duration excludes inference latency and can be shortened by early failure.

表 17 给出第 9.4 节的精确计数。仿真时长不含推理时延，并可因提前失败而缩短。

Metric	Hybrid	Direct
Executed control steps	42,750	38,221
Executed action segments	3,776	7,729
Mean simulated duration per slot (s)	34.20	30.58
Total tokens, including cached input	624,762,828	1,132,343,772
Cached input tokens	607,555,840	1,107,349,760
Uncached input tokens	15,901,963	22,907,448
Output tokens	1,305,025	2,086,564

Table 17: RoboDojo action and token accounting. Mean simulated duration is the total control-step count multiplied by 0.04 s and divided by 50 slots. Token totals include cached input and exclude earlier attempts.

Metric	Hybrid	Direct
Executed control steps	42,750	38,221
Executed action segments	3,776	7,729
Mean simulated duration per slot (s)	34.20	30.58
Total tokens, including cached input	624,762,828	1,132,343,772
Cached input tokens	607,555,840	1,107,349,760
Uncached input tokens	15,901,963	22,907,448
Output tokens	1,305,025	2,086,564

表 17：RoboDojo 动作与 token 核算。 平均仿真时长为总控制步数乘以 0.04 秒再除以 50 个槽位。Token 总计含缓存输入，并排除更早尝试。

Source Snapshots and Verification

源快照与核验

The audit reports 270 checked hashes across 90 online HumanoidBench episodes and checks on ten further runs. Reproduction requires the archived manifests, prompts, execution traces, and robot runtime. Replay verifies recorded dynamics and rewards.

审计报告覆盖 90 个在线 HumanoidBench episode 的 270 个已检查哈希，以及对另外十次运行的检查。复现需要归档的清单、提示、执行轨迹与机器人运行时。回放核验记录的动力学与回报。

Appendix D Manipulation Tasks and Behavioral Cases

附录 D 操控任务与行为案例

Task Coverage

任务覆盖

Table organization

Language classification

Sequence imitation

Number arrangement

Packing

Object classification

Tower construction

Mahjong

Clothes folding

Bottle disposal

Figure 16: The ten RoboDojo manipulation tasks. Task images from our evaluation website are reproduced without alteration. They illustrate the environments rather than evaluation outcomes; the Direct and Hybrid results in Table 2 use the paired episodes.

整理桌面

语言分类

序列模仿

数字排列

装箱

物体分类

搭塔

麻将

叠衣服

扔瓶

图 16：十项 RoboDojo 操控任务。评测网站上的任务图像原样复现，用于说明环境而非评测结果；表 2 中的 Direct 与 Hybrid 结果使用配对 episode。

Behavioral Case Index

行为案例索引

The 21 recorded cases cover five aspects of manipulation: cases 1–4 concern goal alignment; 5–8 contact preparation; 9–12 terminal-state checking; 13–16 Hybrid contact and feedback failures; and 17–21 Direct strategies and local recovery. The cases illustrate behaviors observed in the recorded trajectories. Both Direct and Hybrid receive robot state, and the interpretations refer to complete trajectories rather than still images.

21 个记录案例覆盖操控的五个方面：案例 1–4 关注目标对齐；5–8 接触准备；9–12 终端状态检查；13–16 Hybrid 接触与反馈失败；17–21 Direct 策略与局部恢复。案例说明记录轨迹中观察到的行为。Direct 与 Hybrid 都接收机器人状态，解读针对完整轨迹而非静帧。

Appendix E Humanoid Loco-Manipulation: Supporting Results

附录 E 人形步行-操控：支撑结果

Table 18 expands the HumanoidBench study in Section 8, reporting Astra returns across all 30 tasks. Task-specific return thresholds and observed physical outcomes are reported separately. The separate SIMPLE development records follow below.

表 18 展开第 8 节的 HumanoidBench 研究，报告全部 30 项任务上的 Astra 回报。任务专用回报阈值与观察到的物理结果分开报告。单独的 SIMPLE 开发记录见下文。

Figure 12 reproduces published returns from HumanoidBench Table V [44]: DreamerV3 [12] and SAC [11] use 10M training steps, and TD-MPC2 [13] uses 2M. We use the average returns in Table V, reported over three seeds, rather than the maximum returns in Table VI. The comparison counts a task only when Astra’s reported mean is strictly higher than the reference value; Kitchen is a tie at zero.

图 12 复现 HumanoidBench 表 V [44] 的已发表回报：DreamerV3 [12] 与 SAC [11] 使用 1000 万训练步，TD-MPC2 [13] 使用 200 万。我们使用表 V 中三个种子上报告的平均回报，而非表 VI 的最大回报。对照仅在 Astra 报告均值严格高于参考值时计入任务；Kitchen 在零处分持平。

Task	Threshold	Astra
Stand	800	954.2 $\pm$ 0.1
Walk	700	848.7 $\pm$ 5.9
Run	700	642.1 $\pm$ 2.8
Kitchen	4	0.0 $\pm$ 0.0
Maze	1200	1358.8 $\pm$ 7.4
Hurdle	700	183.0 $\pm$ 0.3
Cube	370	6.3 $\pm$ 0.3
Bookshelf Simple	2000	666.3 $\pm$ 7.6
Bookshelf Hard	2000	572.8 $\pm$ 201.0
Crawl	700	971.9 $\pm$ 0.2
Window	650	6.1 $\pm$ 0.7
Spoon	650	319.1 $\pm$ 8.6
Door	600	142.4 $\pm$ 7.8
Push	700	877.3 $\pm$ 59.6
Reach	12000	11430.2 $\pm$ 2089.0
Basketball	1200	24.0 $\pm$ 2.9
Truck	3000	787.3 $\pm$ 87.1
Package	1500	-5148.0 $\pm$ 1288.1
Cabinet	2500	274.1 $\pm$ 110.0
Sit Simple	750	793.0 $\pm$ 19.1
Sit Hard	750	277.5 $\pm$ 283.5
Balance Simple	800	53.1 $\pm$ 6.7
Balance Hard	800	48.3 $\pm$ 1.8
Stair	700	272.5 $\pm$ 44.0
Slide	700	141.1 $\pm$ 45.9
Pole	700	628.7 $\pm$ 10.5
Room	400	143.9 $\pm$ 27.6
Insert Normal	350	50.6 $\pm$ 1.5
Insert Small	350	46.7 $\pm$ 8.8
Powerlift	800	84.6 $\pm$ 53.7

Table 18: All 30 evaluated HumanoidBench tasks. Entries are means $\pm$ sample SD over two seeds. Bold values meet or exceed the listed return threshold.

Task	Threshold	Astra
Stand	800	954.2 $\pm$ 0.1
Walk	700	848.7 $\pm$ 5.9
Run	700	642.1 $\pm$ 2.8
Kitchen	4	0.0 $\pm$ 0.0
Maze	1200	1358.8 $\pm$ 7.4
Hurdle	700	183.0 $\pm$ 0.3
Cube	370	6.3 $\pm$ 0.3
Bookshelf Simple	2000	666.3 $\pm$ 7.6
Bookshelf Hard	2000	572.8 $\pm$ 201.0
Crawl	700	971.9 $\pm$ 0.2
Window	650	6.1 $\pm$ 0.7
Spoon	650	319.1 $\pm$ 8.6
Door	600	142.4 $\pm$ 7.8
Push	700	877.3 $\pm$ 59.6
Reach	12000	11430.2 $\pm$ 2089.0
Basketball	1200	24.0 $\pm$ 2.9
Truck	3000	787.3 $\pm$ 87.1
Package	1500	-5148.0 $\pm$ 1288.1
Cabinet	2500	274.1 $\pm$ 110.0
Sit Simple	750	793.0 $\pm$ 19.1
Sit Hard	750	277.5 $\pm$ 283.5
Balance Simple	800	53.1 $\pm$ 6.7
Balance Hard	800	48.3 $\pm$ 1.8
Stair	700	272.5 $\pm$ 44.0
Slide	700	141.1 $\pm$ 45.9
Pole	700	628.7 $\pm$ 10.5
Room	400	143.9 $\pm$ 27.6
Insert Normal	350	50.6 $\pm$ 1.5
Insert Small	350	46.7 $\pm$ 8.8
Powerlift	800	84.6 $\pm$ 53.7

表 18：全部 30 项已评 HumanoidBench 任务。 条目为两个种子上的均值 $\pm$ 样本标准差。加粗值达到或超过所列回报阈值。

Additional Physical Diagnostics

额外物理诊断

Walk.

The four 1.8 Hz runs are listed in Appendix C. The task-suite results use posture calibration together with startup and speed guidance.

Walk.

四次 1.8 Hz 运行列于附录 C。任务套件结果使用姿态标定以及启动与速度指引。

Run and Crawl.

Run finishes 1,000 controls without falls, with mean forward center-of-mass speeds of 3.85 and 3.83 m/s and displacements of 77.05 and 76.50 m. Crawl maintains mean forward speed near 1.37 m/s. Both Crawl runs stay within the lateral band, and neither records head height below 0.2 m. A standing-based diagnostic flags the intended pitched posture; native task terms and progress are more informative.

Run and Crawl.

Run 完成 1,000 个控制且无跌倒，平均前向质心速度为 3.85 与 3.83 m/s，位移 77.05 与 76.50 m。Crawl 平均前向速度约 1.37 m/s。两次 Crawl 运行均保持在侧向带内，且均未记录头高低于 0.2 m。基于站立的诊断会标出预期的前倾姿态；原生任务项与进度更有信息量。

Stair.

Seed 1 reaches root X of 1.55 m before falling at control 970; seed 2 reaches 1.06 m, retreats, and finishes behind its starting X position.

Stair.

种子 1 在控制步 970 跌倒前根 X 达到 1.55 m；种子 2 达到 1.06 m 后后退，结束于起点 X 之后。

Push.

The Push runs are described in Section 8.3; Appendix C gives the separate focused runs and exact entry/contact diagnostic outcomes.

Push.

Push 运行见第 8.3 节；附录 C 给出单独聚焦运行以及精确的进入/接触诊断结果。

SIMPLE Development and Retention Checks

SIMPLE 开发与保持性检查

Control and observations.

The SIMPLE implementation uses G1 with Dex3 hands, MuJoCo physics and native scoring, and Isaac Sim rendering. Astra runs with xhigh reasoning and commands both wrists and both ankles, together with independent finger actions. Frozen ScaleBFM executes the whole-body targets. Astra receives external, head, and wrist RGB, joint encoders, forward kinematics, and a base-state estimate. The depth-assisted variants also use relative depth inferred from RGB. Object ground truth, simulator contact diagnostics, and scoring signals are excluded from both action selection and reflection.

控制与观测。

SIMPLE 实现使用带 Dex3 手的 G1、MuJoCo 物理与原生计分，以及 Isaac Sim 渲染。Astra 以 xhigh 推理运行，指挥双腕与双踝以及独立手指动作。冻结 ScaleBFM 执行全身目标。Astra 接收外部、头部与腕部 RGB、关节编码器、正运动学与底盘状态估计。深度辅助变体还使用由 RGB 推断的相对深度。物体真值、仿真器接触诊断与计分信号均不进入动作选择与反思。

Experience representation.

Two text documents support adaptation: Skill contains task instructions, and Memory contains cross-episode notes. Both are separate from shared control instructions and the robot interface. After each episode, Astra uses its observations and actions to revise these documents. Astra and ScaleBFM remain frozen.

经验表示。

两份文本文档支持适应：Skill 含任务指令，Memory 含跨 episode 笔记。二者均独立于共享控制指令与机器人接口。每个 episode 后，Astra 用其观测与动作修订这些文档。Astra 与 ScaleBFM 保持冻结。

Single-scene development.

Across ten tasks, 58 attempts are recorded, including interruptions. The task-development endpoints meet native success on nine tasks; eight also meet execution, audit, and additional hold requirements. Handover triggers native success but times out before the hold check. Basket carrying receives a favorable qualitative assessment of its action sequence but fails native scoring. Neither is counted as a complete success. These outcomes summarize development rather than 58 independent trials.

单场景开发。

跨十项任务记录 58 次尝试（含中断）。任务开发终点在九项上达到原生成功；其中八项还满足执行、审计与额外保持要求。Handover 触发原生成功但在保持检查前超时。提篮搬运对其动作序列得到正面定性评估，但未通过原生计分。二者均不计入完整成功。这些结果汇总开发过程，而非 58 次独立试验。

Separate Handover studies.

Table 19 separates grasp stabilization, fallen-carton recovery, and instruction adaptation. The stabilization variants pass native success and the additional hold check in 2/5 and 3/5 trials, respectively. The latter changes both instructions and relative-depth input. Recovery starts with the carton already fallen and allows a longer budget; its results are not pooled with upright-carton trials. In the retention check, one fixed procedure fails on scene00 and passes both criteria on scene02 and scene08. The instruction-adaptation trials use changing guidance.

单独的交接研究。

表 19 分开报告抓取稳定、倒箱恢复与指令适应。稳定变体分别在 2/5 与 3/5 试验中通过原生成功与额外保持检查；后者同时改动指令与相对深度输入。恢复从纸箱已倒下开始并允许更长预算；其结果不与直立纸箱试验合并。保持性检查中，一个固定流程在 scene00 失败，在 scene02 与 scene08 通过两项判据。指令适应试验使用变化的指引。

Stage	Condition	Valid trials	Success + hold
Grasp stabilization	Grasp entry, enclosure, and support transfer	5	2/5
Depth-assisted grasp	Relative depth and side-entry grasp guidance	5	3/5
Fallen-carton recovery	Table-edge support, regrasping, and handover	1	0/1
Depth-assisted recovery	Relative depth and recovery notes	2	0/2
Trajectory-informed guidance	Scene 1; instructions derived from contrasting trajectories	1	1/1
Post-trial reflection	Scene 1; guidance informed by the preceding trial	1	0/1
Difficult-scene check	Scene 9; guidance informed by trial feedback	1	0/1
Frozen retention check	Scenes 00, 02, 08; one fixed candidate	3	2/3

Table 19: SIMPLE Handover development records. Native success and success with the additional hold check have identical counts in these trials and are reported together. Rows describe separate development conditions and are excluded from the L2 total in Table 12.

Stage	Condition	Valid trials	Success + hold
Grasp stabilization	Grasp entry, enclosure, and support transfer	5	2/5
Depth-assisted grasp	Relative depth and side-entry grasp guidance	5	3/5
Fallen-carton recovery	Table-edge support, regrasping, and handover	1	0/1
Depth-assisted recovery	Relative depth and recovery notes	2	0/2
Trajectory-informed guidance	Scene 1; instructions derived from contrasting trajectories	1	1/1
Post-trial reflection	Scene 1; guidance informed by the preceding trial	1	0/1
Difficult-scene check	Scene 9; guidance informed by trial feedback	1	0/1
Frozen retention check	Scenes 00, 02, 08; one fixed candidate	3	2/3

表 19：SIMPLE 交接开发记录。 这些试验中原生成功与带额外保持检查的成功计数相同，一并报告。各行描述单独开发条件，不计入表 12 的 L2 总计。

L2 evaluation protocol.

We follow the official SIMPLE codebase and evaluation rules, using the same L2 protocol as the benchmark baselines. The evaluation comprises six tasks with ten scenes each; 50 of 60 trials meet the official success criteria, yielding an overall success rate of 83.3%. The separate Handover development and retention studies are excluded from this total.

L2 评测协议。

我们遵循官方 SIMPLE 代码库与评测规则，使用与基准基线相同的 L2 协议。评测含六项任务、每任务十个场景；60 次试验中 50 次满足官方成功判据，总体成功率 83.3%。单独的交接开发与保持性研究不计入该总计。

Appendix F Harness Development and Repeatability

附录 F Harness 开发与可重复性

These development cases supplement Section 8.3. Each retains its own configuration and reasoning setting; none belongs to the frozen evaluation suite.

这些开发案例补充第 8.3 节。各自保留自己的配置与推理设置；均不属于冻结评测套件。

Contact-Conditioned Guidance

接触条件化指引

Contact-conditioned Push guidance reduces the prompt from 129 to 90 lines, replacing unconditional replay of a calibrated example with checks on contact, measured box displacement, and remaining error. Controller code, observations, action limits, native rules, and the 1.0 s walking stop delay remain unchanged. On development seed 3, return changes from $-148.56$ to 863.90 and final box error from 29.45 to 4.88 cm, with success after 428 controls. Seed 0 also succeeds; the additional seed 5 fails at 500 controls with 27.02 cm error.

接触条件化 Push 指引把提示从 129 行减到 90 行，用对接触、测得箱位移与剩余误差的检查，替换对标定示例的无条件回放。控制器代码、观测、动作限位、原生规则与 1.0 秒行走停止延迟保持不变。在开发种子 3 上，回报从 $-148.56$ 变为 863.90，最终箱误差从 29.45 cm 变为 4.88 cm，428 个控制后成功。种子 0 也成功；额外的种子 5 在 500 个控制处失败，误差 27.02 cm。

The archived replay reports record no state, reward, or native-information discrepancies. Each condition has one online attempt on known development seeds, with the prompt changes described above.

归档回放报告未记录状态、回报或原生信息差异。每个条件在已知开发种子上各有一次在线尝试，提示改动如上所述。

A separate Push request selects ten controls while noting rising lateral box speed and the need to observe before overshoot. Its explicit body and wrist references show state-informed action timing. The record captures local anticipation from observed state changes.

另一次 Push 请求在选取十个控制时注意到侧向箱速上升，并需要在过冲前观察。其显式身体与腕部参考显示了基于状态的动作时序。记录捕捉到由观测到的状态变化带来的局部预判。

A Saved Diagnosis and a Failed Repeat

一次保存的诊断与一次失败重试

After a seed-2 Push failure, an online tool event records Astra writing NOTES.md: repeated lifting leaves the hand below the tabletop, without useful pushing contact. The subsequent guidance calls for outward lifting on the robot side and checks clearance before extension.

种子 2 的 Push 失败后，一次在线工具事件记录 Astra 写入 NOTES.md：反复提起使手低于桌面，没有有用的推动接触。随后指引要求在机器人一侧向外提起，并在伸出前检查净空。

This procedure succeeds once in 381 controls with return 803.86, then fails on a repeat after 500 controls with return $-259.83$. Task and composed-skill hashes match; full transmitted-prompt hashes differ because the workspace path changes. All three workers use high reasoning. The archive attributes the written diagnosis to Astra; it does not identify the author of the subsequent guidance change.

该流程在 381 个控制内成功一次，回报 803.86；随后重试在 500 个控制后失败，回报 $-259.83$。任务与组合技能哈希匹配；完整传输提示哈希因工作区路径变化而不同。三名 worker 均使用 high 推理。归档将书面诊断归于 Astra；未指明后续指引改动的作者。

Finger Control Does Not Solve Door

手指控制解决不了 Door

Developers extend Door with bounded finger closure and observations of finger positions and contact geometry. One online Astra episode issues five full-closure commands and completes 1,000 controls, but returns 144.38 against a threshold of 600, with no passage and maximum door angle approximately 0.0195 rad. Guidance, observations, and action access change together in this condition.

开发者用有界手指闭合以及手指位置与接触几何观测扩展 Door。一次在线 Astra episode 发出五次完全闭合指令并完成 1,000 个控制，但回报仅 144.38（阈值为 600），无通道，最大门角约 0.0195 rad。该条件中指引、观测与动作访问一并改动。

Provenance and Evidence Availability

出处与证据可得性

The development archive is organized under sources/harness-study/, with provenance, hashes, source excerpts, prompt versions, and replay reports. Its collection-time commit, db82157bf9a64143433ec75e824a8cb7f89e2b8f, identifies the inspected checkout. Verification requirements are given in Appendix C.

开发归档组织于 sources/harness-study/，含出处、哈希、源摘录、提示版本与回放报告。其采集时提交 db82157bf9a64143433ec75e824a8cb7f89e2b8f 标识所检查的检出。核验要求见附录 C。

Appendix G Dexterous Manipulation: Data, Interfaces, and Evaluation

附录 G 灵巧操控：数据、接口与评测

This appendix specifies the task suite, demonstration data, control interfaces, and evaluation protocol for Section 4. Aggregate scores are reported in Table 3.

本附录说明第 4 节的任务套件、演示数据、控制接口与评测协议。汇总分数见表 3。

Task Suite and Demonstration Data

任务套件与演示数据

Table 20 lists instructions and simulation budgets for the ten tasks; Figure 17 shows their initial configurations.

表 20 列出十项任务的指令与仿真预算；图 17 展示其初始配置。

(a) Headphones in box

(b) Toy retrieval

(c) Mahjong placement

(d) Bottle/can sorting

(e) Upright egg placement

(f) Two-bowl stacking

(g) Bread insertion

(h) Nesting-doll ordering

Figure 17: Example initial states for dexterous manipulation. Panels show eight benchmark tasks, with one initial state each.

(a) Headphones in box

(b) Toy retrieval

(c) Mahjong placement

(d) Bottle/can sorting

(e) Upright egg placement

(f) Two-bowl stacking

(g) Bread insertion

(h) Nesting-doll ordering

图 17：灵巧操控的示例初始状态。各面板展示八项基准任务，每项一个初始状态。

Task	Task instruction	$K$	$t_{\max}$ (s)	Verification (s)
Pot lift and hold	Lift the pot at least 10 centimeters above the table and hold it steadily without dropping it.	1	30	3
Headphones in box	Pick the headset and place it into the box.	1	20	2
Toy retrieval	Take the toy out of the bin and place it on the table.	2	20	2
Mug hanging	Hang the mug on the mug rack without requiring handle-specific alignment.	1	25	2
Two mahjong tiles	Place both mahjongs into the basket.	2	30	2
Bottles/cans sorting	Place bottles into the front-left basket and cans into the front-right basket.	4	60	2
Upright egg placement	Move the egg from the blue platform to the orange platform and leave it upright in the holder.	2	40	2
Two-bowl stacking	Stack the two bowls nested upright on the black mousemat.	2	30	2
Bread-slot insertion	Place both bread slices into the toaster, one slice in each slot.	2	45	2
Nesting-doll ordering	Arrange the four nesting dolls in a row from left to right, from smallest to largest.	2	60	2

Table 20: Dexterous manipulation tasks and evaluation horizons. Task instructions reproduce the prompts used for training and evaluation. $K$ is the number of subgoals used for partial-credit scoring. $t_{\max}$ and the continuous verification window are measured in simulation time. Subgoal predicates are specified in Table 22.

任务	任务指令	$K$	$t_{\max}$ (s)	Verification (s)
Pot lift and hold	将锅至少抬离桌面 10 厘米并稳定持住，不要掉落。	1	30	3
Headphones in box	拿起耳机并放入盒子。	1	20	2
Toy retrieval	把玩具从箱中取出放到桌上。	2	20	2
Mug hanging	把杯子挂到杯架上，不要求手柄特定对齐。	1	25	2
Two mahjong tiles	把两块麻将都放入篮中。	2	30	2
Bottles/cans sorting	瓶子放入左前篮，罐子放入右前篮。	4	60	2
Upright egg placement	把蛋从蓝色平台移到橙色平台，并使其直立留在托座中。	2	40	2
Two-bowl stacking	把两个碗嵌套直立堆叠在黑色鼠标垫上。	2	30	2
Bread-slot insertion	把两片面包分别放入烤面包机的两个槽。	2	45	2
Nesting-doll ordering	把四个套娃从左到右按从小到大排成一行。	2	60	2

表 20：灵巧操控任务与评测时域。 任务指令复现训练与评测所用提示。$K$ 为部分计分所用子目标数。$t_{\max}$ 与连续核验窗口按仿真时间计。子目标谓词见表 22。

Demonstrations use the instructions in Table 20; their counts and training configuration appear in Table 21.

演示使用表 20 中的指令；其数量与训练配置见表 21。

Initial configurations.

The task configurations vary object locations and, where applicable, object yaw and initial hand posture. Initial hand postures are drawn from the first frames of the corresponding task’s demonstrations. The evaluation consequently measures execution across specified scene configurations with the same task objectives and assets. The fixed evaluation subset is described in Section G.4.

初始配置。

任务配置变化物体位置，并在适用时变化物体偏航与初始手姿。初始手姿取自对应任务演示的首帧。因此评测在指定场景配置上度量执行，任务目标与资产相同。固定评测子集见第 G.4 节。

Multitask Policy Finetuning

多任务策略微调

Standalone $\pi_{0.5}$ and Hybrid share one policy finetuned on the pooled demonstrations from all ten tasks. Table 21 specifies initialization, AdamW optimization [25], and execution settings for this shared policy.

独立 $\pi_{0.5}$ 与 Hybrid 共享一个在全部十项任务汇总演示上微调的策略。表 21 给出该共享策略的初始化、AdamW 优化 [25] 与执行设置。

Observations, Numerical Actions, and Execution

观测、数值动作与执行

Policy observations.

We use Sharpa dexterous hands. The simulation uses MuJoCo 3.11.0 [46]. The $\pi_{0.5}$ policy receives RGB images from the head and both wrist cameras together with the task instruction and a 62-dimensional state vector comprising the two wrist poses and finger-joint angles. Each image is resized from $640\times 360$ to $224\times 224$ with aspect-ratio-preserving padding.

策略观测。

我们使用 Sharpa 灵巧手。仿真使用 MuJoCo 3.11.0 [46]。$\pi_{0.5}$ 策略接收头部与双腕相机 RGB，以及任务指令和 62 维状态向量（含双腕位姿与手指关节角）。每张图像从 $640\times 360$ 调整到 $224\times 224$，并做保持宽高比的填充。

Policy action representation.

Each $\pi_{0.5}$ action comprises two wrist poses, each encoded by 3 position values and a 6D rotation representation [65], and 22 joint angles per hand. The resulting action dimension is $2\times(3+6)+2\times 22=62$.

策略动作表示。

每个 $\pi_{0.5}$ 动作含两个腕部位姿（各由 3 个位置值与 6D 旋转表示 [65] 编码）以及每手 22 个关节角。所得动作维度为 $2\times(3+6)+2\times 22=62$。

Execution and model budgets.

The baseline discards each unexecuted prediction suffix before observing and replanning. Table 21 gives execution cadences, Hybrid corrections, and budget-exhaustion behavior. Task limits in Table 20 count simulated time; model calls and inference wall time are separate.

执行与模型预算。

基线在观测与重规划前丢弃未执行的预测后缀。表 21 给出执行节拍、Hybrid 修正与预算耗尽行为。表 20 的任务限额按仿真时间计；模型调用与推理墙钟时间分开。

Configuration	Specification
Demonstration budget	100 trajectories per task; ten tasks; 1,000 trajectories in total.
S1 adaptation	One multi-task $\pi_{0.5}$ policy, shared by the standalone baseline and Hybrid.
Direct action responsibility	Astra specifies wrist/arm targets and finger-joint targets.
Hybrid action responsibility	$\pi_{0.5}$ supplies manipulation actions; Astra can edit wrist and finger targets.
S1 initialization and training	$\pi_{0.5}$ base initialization; full-parameter finetuning; 50,000 optimization steps; global batch size 256.
S1 optimizer	AdamW with $\beta=(0.9,0.95)$, $\epsilon=10^{-8}$, weight decay $10^{-10}$, and gradient-norm clipping at 1.0.
S1 learning rate	Peak $2.5\times 10^{-5}$; cosine decay to $2.5\times 10^{-6}$.
S1 numerical actions	$50\times 62$ output; world-frame wrist poses with 6D rotations and absolute finger-joint targets; wrist increments restored to absolute poses before execution.
$\pi_{0.5}$ baseline execution	30 Hz; execute the first 16 steps of each 50-step prediction before replanning; physics paused during inference.
Hybrid observations and actions	Astra receives the task instruction, synchronized head and wrist RGB views, measured wrist poses, 22 joint angles per hand, hand keypoints, and the remaining simulation budget. A persistent episode conversation retains observation and action history; each decision also includes the previous execution record, a 50-step $\pi_{0.5}$ wrist preview, and hand-motion summaries. Astra selects an unchanged $\pi_{0.5}$ action prefix or specifies bounded wrist and finger corrections, which are converted into 62-dimensional control targets.
Hybrid execution and prompting	Control runs at approximately 30 Hz, with one reset-settling step; $\pi_{0.5}$ action prefixes execute for 1–16 steps and Astra corrections for 1–5 steps before re-observation. Astra uses xhigh reasoning, with prompts and model configuration archived per run. Takeover requires observed execution failure or misaligned $\pi_{0.5}$ intent; control returns to $\pi_{0.5}$ once recovery permits. Each run can set a maximum number of reviewed action chunks and a token budget. Under the default fallback, either limit ends Astra’s involvement, and $\pi_{0.5}$ continues alone until the simulator terminates the episode. Simulation time pauses while Astra deliberates.

Table 21: Dexterous manipulation implementation details.

配置	说明
Demonstration budget	每任务 100 条轨迹；十项任务；共 1,000 条。
S1 adaptation	一个多任务 $\pi_{0.5}$ 策略，由独立基线与 Hybrid 共享。
Direct action responsibility	Astra 指定腕/臂目标与手指关节目标。
Hybrid action responsibility	$\pi_{0.5}$ 提供操控动作；Astra 可编辑腕与手指目标。
S1 initialization and training	$\pi_{0.5}$ 基座初始化；全参数微调；50,000 优化步；全局 batch size 256。
S1 optimizer	AdamW，$\beta=(0.9,0.95)$，$\epsilon=10^{-8}$，weight decay $10^{-10}$，梯度范数裁剪 1.0。
S1 learning rate	峰值 $2.5\times 10^{-5}$；余弦衰减至 $2.5\times 10^{-6}$。
S1 numerical actions	$50\times 62$ 输出；世界系腕位姿（6D 旋转）与绝对手指关节目标；腕增量在执行前还原为绝对位姿。
$\pi_{0.5}$ baseline execution	30 Hz；每次 50 步预测先执行前 16 步再重规划；推理期间物理暂停。
Hybrid observations and actions	Astra 接收任务指令、同步头/腕 RGB、测得腕位姿、每手 22 关节角、手关键点与剩余仿真预算。持久 episode 对话保留观测与动作历史；每次决策还含先前执行记录、50 步 $\pi_{0.5}$ 腕预览与手部运动摘要。Astra 选择未改动的 $\pi_{0.5}$ 动作前缀，或指定有界腕/指修正并转为 62 维控制目标。
Hybrid execution and prompting	控制约 30 Hz，含一步复位稳定；$\pi_{0.5}$ 前缀执行 1–16 步，Astra 修正 1–5 步后再观测。Astra 使用 xhigh 推理，提示与模型配置按运行归档。接管要求观测到执行失败或 $\pi_{0.5}$ 意图错位；一旦可恢复则交回 $\pi_{0.5}$。每次运行可设最大审阅动作块数与 token 预算。默认回退下，任一限额结束 Astra 介入，$\pi_{0.5}$ 单独继续直至仿真终止 episode。Astra 思考时仿真时间暂停。

表 21：灵巧操控实现细节。

Evaluation Protocol and Scoring

评测协议与计分

Fixed evaluation cases.

We evaluate each task on five fixed cases, yielding 50 evaluation cases across the ten tasks. All three methods are evaluated on the same case set.

固定评测案例。

我们在每项任务上评测五个固定案例，十项任务共 50 个评测案例。三种方法在同一案例集上评测。

Continuous execution and verification.

Each task has a simulation-time limit $t_{\max}$, chosen to cover at least 1.5 times the longest demonstration and twice the median demonstration duration. Verification is performed during continuous execution. For the nine placement and rearrangement tasks, full completion requires all task subgoals to hold, the manipulated objects to be released, and their positions to remain stable for 2 s. Release is verified from contact data showing zero contacts between an object and the hand collision geometries. Pot lifting instead requires the pot to remain at least 10 cm above its initial height, in contact with a hand and out of contact with the table, for a 3 s stable-holding window.

连续执行与核验。

每项任务有仿真时间上限 $t_{\max}$，选取为至少覆盖最长演示的 1.5 倍与演示时长中位数的两倍。核验在连续执行中进行。对九项放置与重排任务，完整完成要求全部任务子目标成立、被操控物体已释放，且其位置稳定保持 2 秒。释放由接触数据核验：物体与手碰撞几何之间接触为零。抬锅则要求锅相对初始高度至少 10 cm、与手接触且不与桌接触，稳定持住窗口为 3 秒。

The positional stability threshold is 1 cm over the verification window. The documented window definition measures each object’s maximum root-position displacement from the start of that window; it is not a sum of frame-to-frame displacements. An interrupted verification condition restarts the window. Successful verification permits early completion; otherwise the episode is scored at termination. There is no separate stage in which the robot is frozen to validate an earlier placement.

位置稳定阈值为核验窗口内 1 cm。所记录的窗口定义度量各物体相对该窗口起点的最大根位置位移，而非帧间位移之和。核验条件中断则窗口重启。成功核验允许提前完成；否则在终止时计分。没有单独把机器人冻结以验证更早放置的阶段。

Task subgoals.

Table 22 defines each predicate, including geometric tolerances, release requirements, and dependencies between subgoals.

任务子目标。

表 22 定义各谓词，包括几何容差、释放要求以及子目标之间的依赖。

Task	Subgoal predicates and task-specific verification conditions
Pot lift and hold	The pot root is at least 10 cm above its initial height. The 0.8 tier requires this lift condition at termination. Full-score verification requires continued hand contact, no pot–table contact, and stability for 3 s.
Headphones in box	The root projects inside the box-bottom polygon; at least 50% of the vertically constrained minimum bounding-box volume lies in the vertical prism extending upward from that polygon; and hand contact is absent. The prism has no upper height limit, so a released headset resting across the rim can qualify.
Toy retrieval	Two independent subgoals: (i) the root projects outside the box-bottom polygon; (ii) the toy contacts the table and its root height is at most the table height plus its reference resting height plus 3 cm. Reference resting height is the initial root height above the box-bottom reference plane.
Mug hanging	Conjunctive subgoal: lift from the initial root height $>4.5$ cm, root height above the table $>13.5$ cm, and root-to-rack horizontal distance $<9$ cm. Handle alignment is unnecessary.
Two mahjong tiles	One subgoal per tile: the root projects inside the basket interior (half-widths 10.85 and 6.67 cm), and hand contact is absent. No constraint is imposed on the tiles' vertical positions. Full credit requires both released tiles to remain stable for 2 s.
Bottles/cans sorting	One subgoal per object: the root projects inside its assigned basket interior and its height relative to the basket reference plane is in $[-6,10]$ cm. Bottles belong in the front-left basket and cans in the front-right basket; placement in the other basket receives no credit.
Upright egg placement	Two nested subgoals: (i) the root is within 2 cm horizontally and 1.5 cm vertically of the target seat; (ii) condition (i) holds and the egg’s local $z$ axis is within $30^{\circ}$ of world vertical. Upright orientation at the source provides no subgoal credit.
Two-bowl stacking	Two independent subgoals: (i) the bowls’ horizontal separation is at most 2 cm and vertical separation at least 0.5 cm, each bowl tilts at most $45^{\circ}$, and the lower bowl at most $7^{\circ}$; (ii) the lower bowl’s center lies inside the coaster-local rectangle $x\in[-12.005,12.005]$ cm, $y\in[-9.601,9.601]$ cm, with no edge tolerance.
Bread-slot insertion	One subgoal per distinct occupied slot: a bread root projects within the slot polygon, its height above the slot reference plane is in $[2,8]$ cm, and at least one of its local $\pm x$ or $\pm y$ axes lies within $30^{\circ}$ of world upward. Two slices in one slot count once; either end may face upward. Full-score verification requires both slices to be released.
Nesting-doll ordering	Two subgoals: (i) the four roots span at most 6 cm along world $x$; (ii) in increasing size order, every adjacent pair has a signed world-$y$ separation of at least 4 cm.

Table 22: Task-specific completion criteria. Positions refer to object roots unless stated otherwise. Full credit additionally requires continuous stability within 1 cm and the verification windows in Table 20; all tasks require release except pot holding. Headset and mahjong release conditions also apply to their subgoal predicates.

任务	子目标谓词与任务专用核验条件
Pot lift and hold	锅根至少高于初始高度 10 cm。0.8 档要求终止时满足该抬升条件。满分核验要求持续手接触、无锅–桌接触，并稳定 3 秒。
Headphones in box	根投影落在盒底多边形内；竖直约束的最小包围盒体积至少 50% 位于自该多边形向上延伸的竖直棱柱内；且无手接触。棱柱无高度上限，因此跨在边缘上的已释放耳机也可合格。
Toy retrieval	两个独立子目标：(i) 根投影在盒底多边形外；(ii) 玩具接触桌面，且根高度不超过桌高加参考静息高度加 3 cm。参考静息高度为相对盒底参考面的初始根高度。
Mug hanging	合取子目标：相对初始根高度抬升 $>4.5$ cm，相对桌面的根高度 $>13.5$ cm，根到杯架水平距离 $<9$ cm。不需要手柄对齐。
Two mahjong tiles	每块一个子目标：根投影在篮内（半宽 10.85 与 6.67 cm），且无手接触。不对竖直位置设约束。满分要求两块已释放麻将稳定 2 秒。
Bottles/cans sorting	每物体一个子目标：根投影在指定篮内，相对篮参考面的高度在 $[-6,10]$ cm。瓶子属左前篮、罐子属右前篮；放错篮不得分。
Upright egg placement	两个嵌套子目标：(i) 根在目标座水平 2 cm、竖直 1.5 cm 内；(ii) 条件 (i) 成立且蛋局部 $z$ 轴与世界竖直夹角在 $30^{\circ}$ 内。源处直立朝向不给子目标分。
Two-bowl stacking	两个独立子目标：(i) 碗水平间距至多 2 cm、竖直间距至少 0.5 cm，每碗倾斜至多 $45^{\circ}$，下碗至多 $7^{\circ}$；(ii) 下碗中心落在杯垫局部矩形 $x\in[-12.005,12.005]$ cm、$y\in[-9.601,9.601]$ cm 内，无边缘容差。
Bread-slot insertion	每个被占用槽一个子目标：面包根投影在槽多边形内，相对槽参考面高度在 $[2,8]$ cm，且其局部 $\pm x$ 或 $\pm y$ 轴之一与世界向上夹角在 $30^{\circ}$ 内。两片同槽只计一次；任一端朝上均可。满分核验要求两片都已释放。
Nesting-doll ordering	两个子目标：(i) 四个根沿世界 $x$ 跨度至多 6 cm；(ii) 按尺寸递增，每对相邻的有符号世界-$y$ 间距至少 4 cm。

表 22：任务专用完成判据。 除非另有说明，位置指物体根。满分还要求在 1 cm 内连续稳定以及表 20 的核验窗口；除持锅外全部任务要求释放。耳机与麻将的释放条件也适用于其子目标谓词。

Partial-credit scores.

Let $K_{t}$ be the number of subgoals for task $t$, and let $k$ be the number satisfied at termination. The documented score tiers use an internal scale $s\in[0,1]$:

部分计分。

设任务 $t$ 的子目标数为 $K_{t}$，终止时满足数为 $k$。所记录的分数档使用内部尺度 $s\in[0,1]$：

• • $s=1$ when all subgoals and the full continuous verification window are satisfied;
• • $s=0.8$ when all subgoals hold at termination but full verification has not been completed;
• • $s=0.8k/K_{t}$ for partial completion, where $0<k<K_{t}$;
• • $s=0.1$ when no subgoal holds at termination but a manipulated object was lifted by 5 cm during execution;
• • $s=0$ for no effective progress or an applicable invalidating termination.