Fabrica: Dual-Arm Assembly of General Multi-Part Objects via Integrated Planning and Learning

Fabrica：面向通用多零件物体的双臂装配——规划与学习一体化

Yunsheng Tian † † thanks: Correspondence to: yunsheng@mit.edu , hui.xylo.li@autodesk.com , and wojciech@mit.edu Affiliation:  MIT CSAIL    Joshua Jacob Affiliation:  MIT CSAIL    Yijiang Huang Affiliation:  ETH Zurich    Jialiang Zhao Affiliation:  MIT CSAIL    Edward Gu Affiliation:  MIT CSAIL    Pingchuan Ma Affiliation:  MIT CSAIL    Annan Zhang Affiliation:  MIT CSAIL    Farhad Javid Affiliation:  Autodesk Research    Branden Romero Affiliation:  MIT CSAIL    Sachin Chitta Affiliation:  Autodesk Research    Shinjiro Sueda Affiliation:  Texas A&M University    Hui Li Affiliation:  Autodesk Research    Wojciech Matusik Affiliation:  MIT CSAIL

Abstract

摘要

Multi-part assembly poses significant challenges for robots to execute long-horizon, contact-rich manipulation with generalization across complex geometries. We present Fabrica, a dual-arm robotic system capable of end-to-end planning and control for autonomous assembly of general multi-part objects. For planning over long horizons, we develop hierarchies of precedence, sequence, grasp, and motion planning with automated fixture generation, enabling general multi-step assembly on any dual-arm robots. The planner is made efficient through a parallelizable design and is optimized for downstream control stability. For contact-rich assembly steps, we propose a lightweight reinforcement learning framework that trains generalist policies across object geometries, assembly directions, and grasp poses, guided by equivariance and residual actions obtained from the plan. These policies transfer zero-shot to the real world and achieve 80% successful steps. For systematic evaluation, we propose a benchmark suite of multi-part assemblies resembling industrial and daily objects across diverse categories and geometries. By integrating efficient global planning and robust local control, we showcase the first system to achieve complete and generalizable real-world multi-part assembly without domain knowledge or human demonstrations. Project website: fabrica.csail.mit.edu

多零件装配要求机器人完成长时程、富接触的操作，并在复杂几何上具备泛化能力，难度很高。我们提出 Fabrica：一套双臂机器人系统，能对通用多零件物体做端到端规划与控制，实现自主装配。面向长时程规划，我们建立优先级、装配顺序、抓取与运动规划的层次结构，并自动生成夹具，从而在任意双臂机器人上完成通用多步装配；规划器可并行加速，并针对下游控制稳定性做了优化。面向富接触装配步，我们提出轻量强化学习框架：在等变性与来自规划的残差动作引导下，跨物体几何、装配方向与抓取位姿训练通才策略。这些策略可零样本迁移到真实世界，单步成功率约 80%。为系统评测，我们构建了覆盖工业与日用、类别与几何多样的多零件装配基准。通过高效全局规划与稳健局部控制，我们展示了无需领域知识或人类示教、即可完成完整且可泛化真实多零件装配的首个系统。项目主页：fabrica.csail.mit.edu

Fig. 1: Our proposed dual-arm robotic system demonstrates adaptive manipulation and assembly capabilities for diverse multi-part objects. The system combines offline task-oriented planning and optimization to address sequencing, grasping, and motion planning for long-horizon assembly tasks. For robust online control, it utilizes guidance from the offline plan to learn assembly skills that generalize effectively across diverse object geometries, assembly paths, and grasp poses.

图 1：所提出的双臂系统在多样多零件物体上展示自适应操作与装配能力。系统结合离线面向任务的规划与优化，解决长时程装配中的排序、抓取与运动规划；面向稳健在线控制，它利用离线规划的引导学习装配技能，并在多样物体几何、装配路径与抓取位姿上有效泛化。

Keywords: Assembly, Planning, Reinforcement Learning, Benchmark

关键词： 装配、规划、强化学习、基准

1 Introduction

1 引言

Multi-part assemblies are prevalent in home and industrial settings. Robotic assembly of multi-part objects presents a longstanding challenge: long-term planning to map CAD models to robot programs and robust control skills to achieve high precision and adaptivity during contact-rich interactions. However, most assembly robots today are programmed manually with specially designed infrastructures, and the program is executed repetitively using a stiff controller. As a result, they take substantial time to adapt to new production demands and are highly sensitive to uncertainties.

多零件装配在家庭与工业场景中都很常见。机器人装配多零件物体是长期难题：既要把 CAD 映射成机器人程序做长期规划，又要在富接触交互中以高精度与自适应性完成控制。但当前多数装配机器人仍靠人工编程与专用工装，再用刚性控制器重复执行，因此难以快速适应新产线需求，也对不确定性极为敏感。

Despite recent progress in sim-to-real transfer of contact-rich part insertion skills [1, 2, 3], current robotic systems are still not capable of assembling general multi-part objects. Prior research has primarily focused on two-part, top-down insertion using a single robot arm, but multi-part assembly requires diverse insertion and grasping poses and a bi-manual operation that frequently changes which part to hold to counter-balance the insertion force from the other hand. This presents new challenges to planning and control. First, jointly finding an assembly-hold sequence, physically stable grasps, and collision-free robot motion presents a hybrid (discrete-continuous) optimization problem in a large search space. Second, control policies for part insertion must be robust to misalignment and uncertainty, while being able to generalize across a wide range of part geometries.

尽管富接触零件插入的仿真到现实迁移近来有进展 [1, 2, 3]，现有系统仍难以装配通用多零件物体。既往工作主要关注单臂、两零件、自上而下的插入；而多零件装配需要多样的插入与抓取位姿，以及频繁切换「哪只手持件、哪只手插入」以平衡另一只手插入力的双臂协作。这给规划与控制带来新挑战。其一，同时寻找装配–持件顺序、物理稳定抓取与无碰撞运动，是一个搜索空间巨大的混合（离散–连续）优化问题。其二，插入控制策略必须对错位与不确定性稳健，并在广泛零件几何上泛化。

We tackle these challenges by building a general planning and control system for flexible, dual-arm assembly of multi-part objects, with zero-shot sim-to-real transfer. Our contributions include:

我们通过构建面向柔性双臂多零件装配的通用规划与控制系统来应对上述挑战，并实现零样本仿真到现实迁移。贡献包括：

Algorithms: We propose a hierarchical dual-arm planner to plan and optimize the assembly-hold sequence, grasps, and robot motion. For contact-rich steps, we learn generalist reinforcement learning (RL) policies utilizing equivariant representations guided by planned motion to achieve robustness.

算法： 提出层次化双臂规划器，规划并优化装配–持件顺序、抓取与机器人运动。对富接触步骤，学习在规划运动引导下、利用等变表示的通才强化学习（RL）策略以获得稳健性。

Systems: We build a real-world system that can map a CAD assembly model to robot execution that alternates between tracking planned motions and reactive control policies. To our knowledge, this is the first system that autonomously achieves all phases of a multi-part assembly problem: from automatic pickup fixture design, to sequence, grasps, and motion planning, to insertion. Our system is tested on commonly used robotics hardware and can be generalized to different dual-arm robots.

系统： 构建真实系统，能把 CAD 装配模型映射为在跟踪规划运动与反应式控制策略之间交替的机器人执行。据我们所知，这是首个从自动取件夹具设计，到顺序、抓取与运动规划，再到插入，自主覆盖多零件装配全部阶段的系统。系统在常用机器人硬件上验证，并可推广到不同双臂机器人。

Benchmarks: We design a benchmark suite of 7 multi-part assemblies ranging from 5 to 9 parts, and our system can assemble them robustly in both simulation and real-world system.

基准： 设计包含 7 个多零件装配（5–9 个零件）的基准套件；系统在仿真与真实世界中均能稳健装配。

2 Related Work

2 相关工作

Prior work on multi-part assembly is heavily focused on planning assembly sequences and paths, including geometric reasoning [4], sampling-based motion planners [5, 6, 7], and RL for combinatorial sequence search [8, 9]. Recently, physics-based motion planning [10] has shown success in assembling many complex parts with tight clearances. In addition, realistic kinematic and dynamic constraints have been considered in sequence planning for real-world robot setups [11, 12, 13]. However, planning alone struggles with execution uncertainties, and stability- or efficiency-optimal plans remain underexplored. While robust and efficient robotic systems have been built for tasks like assembling IKEA chairs [14], LEGO blocks [15], and structural elements [16, 17, 18, 19, 20, 21, 22], these are domain-specific and lack generalizability. In contrast, our planner generalizes across diverse multi-part assemblies, employs hierarchical structure and parallelization for efficiency, and explicitly optimizes stability to enhance downstream control robustness.

多零件装配的既往工作高度集中在装配顺序与路径规划，包括几何推理 [4]、基于采样的运动规划 [5, 6, 7]，以及用于组合顺序搜索的 RL [8, 9]。近来基于物理的运动规划 [10] 已能在紧公差下装配许多复杂零件。此外，面向真实机器人设置的顺序规划也开始考虑更真实的运动学与动力学约束 [11, 12, 13]。但仅靠规划难以应对执行不确定性，稳定性或效率最优的计划也仍欠探索。虽有面向宜家椅 [14]、乐高 [15] 与结构件 [16, 17, 18, 19, 20, 21, 22] 的稳健高效系统，它们多属领域专用、泛化有限。相较之下，我们的规划器可跨多样多零件装配泛化，以层次结构与并行化提升效率，并显式优化稳定性以增强下游控制稳健性。

Even with given assembly plans, executing contact-rich assembly remains challenging due to tight clearances, system uncertainties, and the need for generalization. RL has shown promise in addressing these issues, combining motion planning with policy learning from CAD models or supervised trajectories [23, 24], and leveraging accurate simulations for motion generation and policy training [11, 25]. Sim-to-real transfer [26, 3, 27, 2] and real-world RL [28] have enabled high-precision insertion, while some efforts [29] explore multi-step tasks. However, they all primarily work under simplified settings like top-down insertions and fixed grasps, which are insufficient for multi-part assembly where side-way insertions or tilted grasps are necessary. Imitation learning approaches [30, 31] support complex multi-step skill learning but lack robustness and generality. While spatial-equivariant techniques have improved generalization in other domains [32, 33, 34, 35, 36], they remain underexplored for assembly. Notably, Seo et al. [37] learn an SE(3)-equivariant gain scheduling policy, but without varying grasps or geometries. Existing benchmarks focus on narrow tasks [38, 39, 40, 41, 42], limiting the evaluation of generalization. In contrast, we demonstrate that combining planning with equivariant generalist policies, for the first time, enables multi-part assembly over diverse geometries, paths, and grasps without any human demonstration.

即便已有装配计划，执行富接触装配仍因紧公差、系统不确定性与泛化需求而困难。RL 在此显示出潜力：把运动规划与来自 CAD 或监督轨迹的策略学习结合 [23, 24]，并利用高精度仿真做运动生成与策略训练 [11, 25]。仿真到现实迁移 [26, 3, 27, 2] 与真实世界 RL [28] 已实现高精度插入，也有工作探索多步任务 [29]。但它们主要仍在自上而下插入、固定抓取等简化设定下工作，不足以覆盖需要侧向插入或倾斜抓取的多零件装配。模仿学习 [30, 31] 支持复杂多步技能，但稳健性与通用性不足。空间等变技术在其他领域改善了泛化 [32, 33, 34, 35, 36]，在装配中仍欠探索。Seo 等 [37] 学习 SE(3) 等变增益调度策略，但不涉及变化抓取或几何。现有基准多聚焦窄任务 [38, 39, 40, 41, 42]，限制对泛化的评估。相较之下，我们首次展示：把规划与等变通才策略结合，无需任何人类示教，即可在多样几何、路径与抓取上完成多零件装配。

3 Planning Multi-Step Dual-Arm Assembly

3 规划多步双臂装配

Figure 2: System overview. Fabrica takes part meshes and hardware configurations as inputs. It plans sequences, grasps, fixture designs, and motions through a multi-stage planner, and learns RL policies for all insertion steps, which are deployed together on real robots to complete the assembly.

图 2：系统总览。Fabrica 以零件网格与硬件配置为输入；经多阶段规划器规划顺序、抓取、夹具设计与运动，并为所有插入步骤学习 RL 策略，一并部署到真实机器人上完成装配。

Given a $n$-part assembly with parts indexed by $o\in O$, we compute a plan to manipulate all parts from the initial poses $p_{O}^{0}$ to the goal poses $p_{O}^{G}\in SE(3)$ under all physical constraints. We focus on sequential, collaborative manipulation that alternates between robot $R_{a}$ assembling a part and another robot $R_{h}$ holding a part to stabilize the sub-assembly. Then, we train control policies for precise contact-rich assembly steps. Finally, our system execution alternates between open-loop planned motions and closed-loop reactive policies. Figure 2 provides an overview of the system.

给定 $n$ 零件装配，零件索引为 $o\in O$，我们计算计划，把所有零件从初始位姿 $p_{O}^{0}$ 操纵到目标位姿 $p_{O}^{G}\in SE(3)$，并满足全部物理约束。我们聚焦顺序式协作操作：机器人 $R_{a}$ 装配某一零件，与另一机器人 $R_{h}$ 持件以稳定子装配之间交替。随后为精确富接触装配步训练控制策略。最终执行在开环规划运动与闭环反应式策略之间交替。图 2 给出系统总览。

We formulate planning as optimizing assembly-hold sequences $\phi$, grasps $\sigma$, and robot motions $\pi$:

我们将规划表述为优化装配–持件序列 $\phi$、抓取 $\sigma$ 与机器人运动 $\pi$：

$$\min_{\phi,\sigma,\pi}E\big(\Phi_{i=1}^{n}\vec{f}(\phi_{1:i},\sigma_{i-1:i},\pi_{i})\big)\quad\text{s.t.}\quad C_{\text{prec}}(\phi)\leq 0,\hskip 1.99997ptC_{\text{kin}}(\phi,\sigma,\pi)=0,\hskip 1.99997ptC_{\text{col}}(\phi,\sigma,\pi)\leq 0 \tag{1}$$

The sequence $\phi=[o_{a,1},o_{h,2},o_{a,2},\cdots,o_{h,n},o_{a,n}]$ is an ordering of parts to be held ($h$) and assembled ($a$), with $\sigma=[g_{a,1},g_{h,2},g_{a,2},\cdots,g_{h,n},g_{a,n}]$ including grasp $g\in SE(3)$ for each step. The robot motion $\pi$ is divided based on the mode families [43, 44] and skills: $\pi=[\underbrace{\tau_{a,1}^{f},\tau_{a,1}^{g},\tau_{a,1}^{a}}_{\pi[a,1]},\underbrace{\tau_{h,2}^{f}}_{\pi[h,2]},\underbrace{\tau_{a,2}^{f},\tau_{a,2}^{g},\tau_{a,2}^{a}}_{\pi[a,2]},\cdots]$ where each assembly task $\pi[a,i]$ for $R_{a}$ contains (1) a transit motion $\tau_{a,i}^{f}$ with its hand free, (2) a transfer motion $\tau^{g}_{a,i}$ grasping an object, and (3) an assembly motion $\tau_{a,i}^{a}$ for part insertion. A hold task $\pi[h,i]$ only involves a hold transfer motion $\tau_{h,i}^{f}$. The cost function composes step-wise objective vectors $\vec{f}$ that evaluate the quality of each step, $\Phi:\mathbb{R}^{|\vec{f}|\times n}\rightarrow\mathbb{R}^{|\vec{f}|}$ aggregates objectives across steps (e.g., sum or max), and $E$ maps the result to a scalar (e.g., weighted sum). $C_{\text{prec}},C_{\text{kin}},C_{\text{col}}$ represent part precedence, kinematic, and collision constraints respectively. Please refer to App. B for detailed constraint formulation.

序列 $\phi=[o_{a,1},o_{h,2},o_{a,2},\cdots,o_{h,n},o_{a,n}]$ 是持件（$h$）与装配（$a$）零件的排序，$\sigma=[g_{a,1},g_{h,2},g_{a,2},\cdots,g_{h,n},g_{a,n}]$ 包含每步抓取 $g\in SE(3)$。机器人运动 $\pi$ 按模式族 [43, 44] 与技能划分：$\pi=[\underbrace{\tau_{a,1}^{f},\tau_{a,1}^{g},\tau_{a,1}^{a}}_{\pi[a,1]},\underbrace{\tau_{h,2}^{f}}_{\pi[h,2]},\underbrace{\tau_{a,2}^{f},\tau_{a,2}^{g},\tau_{a,2}^{a}}_{\pi[a,2]},\cdots]$，其中 $R_{a}$ 的每个装配任务 $\pi[a,i]$ 包含（1）空手转移运动 $\tau_{a,i}^{f}$，（2）抓取物体后的搬运运动 $\tau^{g}_{a,i}$，以及（3）用于零件插入的装配运动 $\tau_{a,i}^{a}$。持件任务 $\pi[h,i]$ 仅含持件转移运动 $\tau_{h,i}^{f}$。代价函数由逐步目标向量 $\vec{f}$ 组成，用于评估每步质量；$\Phi:\mathbb{R}^{|\vec{f}|\times n}\rightarrow\mathbb{R}^{|\vec{f}|}$ 跨步聚合目标（如求和或取最大），$E$ 再映射为标量（如加权和）。$C_{\text{prec}},C_{\text{kin}},C_{\text{col}}$ 分别表示零件优先级、运动学与碰撞约束。详细约束表述见附录 B。

Solving for feasible or even optimal solutions in a joint manner is intractable. We present a hierarchical approach to decompose it into simpler subproblems for efficient computation with optimality guarantees under assumptions (A1)-(A5). The complete pesudocode can be found in App. C.

联合求可行甚至最优解通常不可行。我们提出层次方法，在假设 (A1)–(A5) 下将其分解为更简单的子问题以高效计算并给出最优性保证。完整伪代码见附录 C。

3.1 Part Precedence Planning

3.1 零件优先级规划

To evaluate constraint $C_{\text{prec}}$, we propose an algorithm to determine the complete precedence relationships for assembling all parts $O$. First, we define precedence tier as a group of parts that can be removed independently of one another. Tiers are ordered so that parts in earlier tiers must be disassembled before those in later ones. To iteratively construct all tiers, we use a physics-based motion planner [10] to find all parts that can be disassembled without interfering with the rest, which are grouped into the current tier. We then remove these parts and repeat the process on the remaining assembly until each part is assigned to a tier. Next, we build a precedence graph $G_{\text{prec}}$ that encodes the minimal set of ordering constraints that any collision-free assembly sequence must follow. Each node in $G_{\text{prec}}$ is a part, and a directed edge $o_{i}\rightarrow o_{j}$ means that $o_{i}$ must be assembled before $o_{j}$ because $o_{j}$ blocks the (dis)assembly path $\tau_{o_{i}}$ planned during tier generation. For each part $o$, we define its precedence set $O_{\text{prec}}[o]=\{o^{\prime}|(o^{\prime}\rightarrow o)\in G_{\text{prec}}\}$ as all parts that must be assembled before it.

为评估约束 $C_{\text{prec}}$，我们提出算法以确定装配全部零件 $O$ 的完整优先级关系。首先定义优先级层：可彼此独立拆卸的一组零件。层按序排列，较早层中的零件必须先于较晚层拆卸。为迭代构造所有层，我们用基于物理的运动规划器 [10] 找出可在不干扰其余部分的前提下拆卸的全部零件，归入当前层；再移除这些零件，对剩余装配重复，直到每个零件都分配到一层。接着构建优先级图 $G_{\text{prec}}$，编码任何无碰撞装配序列必须遵守的最小排序约束集。$G_{\text{prec}}$ 中每个节点是零件，有向边 $o_{i}\rightarrow o_{j}$ 表示 $o_{i}$ 必须先于 $o_{j}$ 装配，因为 $o_{j}$ 阻挡了层生成阶段为 $o_{i}$ 规划的（拆）装配路径 $\tau_{o_{i}}$。对每个零件 $o$，定义其优先级集合 $O_{\text{prec}}[o]=\{o^{\prime}|(o^{\prime}\rightarrow o)\in G_{\text{prec}}\}$，即必须先于它装配的全部零件。

3.2 Dual-Arm Grasp Filtering

3.2 双臂抓取过滤

We aim to identify valid grasp pairs $\mathcal{G}^{a\times h}[o_{a},o_{h}]$ for each assembly-hold part pair $(o_{a},o_{h})$ which support insertion and holding without colliding with preceding parts of $o_{a}$ and $o_{h}$. Since searching the full 6-DoF space is infeasible, we assume feasible grasps exist in a dense, finite set of grasps, see (A5). Because each grasp must be checked for collisions with the current subassembly $\psi_{1:i}$, doing this online results in repeated and expensive checks against many part combinations. To speed up, we precompute valid grasps offline by sampling grasp candidates and performing parallelized inverse kinematics (IK) and collision checks for both arms. In practice, we simulate the robot following the part’s disassembly path $\tau_{o}$ and reject it if the motion collides with the precedence set $O_{\text{prec}}[o]$. All collisions and motions are recorded for reuse in later stages. Similarly, we check if a grasp can securely hold a part while disassembling other parts without collision. Finally, for each part pair $(o_{a},o_{h})$, we filter feasible grasps $(g_{a},g_{h})$ by checking interarm collisions under computed IK.

我们旨在为每个装配–持件零件对 $(o_{a},o_{h})$ 识别有效抓取对 $\mathcal{G}^{a\times h}[o_{a},o_{h}]$，使其在不与 $o_{a}$、$o_{h}$ 的前驱零件碰撞的前提下支持插入与持件。完整搜索 6-DoF 空间不可行，故假设在稠密有限抓取集中存在可行抓取，见 (A5)。由于每个抓取都要对当前子装配 $\psi_{1:i}$ 做碰撞检查，在线进行会对许多零件组合重复昂贵检查。为加速，我们离线预计算有效抓取：采样候选抓取，并对双臂并行做逆运动学（IK）与碰撞检查。实践中，我们让机器人沿零件拆卸路径 $\tau_{o}$ 运动，若与优先级集合 $O_{\text{prec}}[o]$ 碰撞则拒绝；所有碰撞与运动记录供后续复用。类似地，检查抓取能否在拆卸其他零件时稳固持件且无碰撞。最后，对每个零件对 $(o_{a},o_{h})$，在计算得到的 IK 下检查臂间碰撞，过滤可行抓取 $(g_{a},g_{h})$。

3.3 Dual-Arm Sequence-Grasp Optimization

3.3 双臂顺序–抓取优化

With all valid grasps computed, we now solve for the optimal sequence $\phi^{*}$ and grasps $\sigma^{*}$ in Eq. 2. We formulate this as a state-space search problem and construct a directed state tree $T_{G}$, where each node represents a partial assembly state $s=(t,O_{r},o,g)$ consisting of robot task $t$ (assemble or hold), assembled parts $O_{r}$, part being grasped $o$, and grasp pose $g$. Starting from root nodes (complete assembly), we recursively expand the tree by alternating between assembling and holding, pruning states that violate constraints. Valid transitions must also respect precomputed grasp feasibility between successive steps. All collision and motion feasibility checks are reused from the earlier filtering stage. Each transition is scored by a grasp stability vector $\vec{f}$, capturing objectives such as supportiveness of the held part, frequency of grasp switches, torque stability, and contact area, which are designed to be lightweight yet effective for downstream control. We apply dynamic programming (DP) to propagate the best cumulative scores through the tree and identify the optimal solution. Please find more details in App. C.3.

在算得全部有效抓取后，我们求解式 2 中的最优序列 $\phi^{*}$ 与抓取 $\sigma^{*}$。将其表述为状态空间搜索，并构造有向状态树 $T_{G}$：每个节点表示部分装配状态 $s=(t,O_{r},o,g)$，包含机器人任务 $t$（装配或持件）、已装配零件 $O_{r}$、正在抓取的零件 $o$ 与抓取位姿 $g$。从根节点（完整装配）出发，在装配与持件之间交替递归扩展树，剪除违反约束的状态。有效转移还必须满足相继步骤间预计算的抓取可行性。碰撞与运动可行性检查复用此前过滤阶段的结果。每条转移由抓取稳定性向量 $\vec{f}$ 打分，刻画持件支撑性、抓取切换频率、力矩稳定性与接触面积等目标——设计为轻量却对下游控制有效。我们用动态规划（DP）在树中传播最佳累积分数并找出最优解。更多细节见附录 C.3。

3.4 Grasp-Aware Pickup Fixture Generation

3.4 感知抓取的取件夹具生成

For precise pickup, we develop a software-hardware co-design approach to automatically generate a fixture that stabilizes and orients each part for top-down pickup, based on planned grasps in Sec. 3.3. This removes the need for reorientation or regrasping between pickup and assembly, allowing the system to focus on the core assembly challenges. We first determine each part’s pickup pose in the world frame, with its orientation defined by the rotation from the assembly grasp to a top-down grasp. We then compute pickup positions by packing parts on the XY plane to avoid collisions between parts and the gripper. To reduce material usage and workspace area, we model this as a bin-packing problem and solve it with an iterative algorithm that alternates between packing, collision checking, and resolution. Finally, we generate the fixture by creating mold cavities based on the part geometries and poses, ensuring stable placement. See details in App. C.4 and examples in Fig. 3.

为精确取件，我们采用软硬件协同设计，基于第 3.3 节规划的抓取，自动生成稳定并定向每个零件以便自上而下取件的夹具。这样无需在取件与装配之间再定向或重抓，使系统聚焦核心装配挑战。我们先确定每个零件在世界系中的取件位姿，其朝向由装配抓取到自上而下抓取的旋转定义。再在 XY 平面上打包零件位置，避免零件与夹爪碰撞。为降低材料用量与工作空间面积，将其建模为装箱问题，用在打包、碰撞检查与消解之间交替的迭代算法求解。最后基于零件几何与位姿生成模具型腔以得到夹具，保证稳定放置。细节见附录 C.4，示例见图 3。

Figure 3: Top: benchmark assemblies. Bottom: the auto-generated pickup fixtures in Sec. 3.4.

图 3：上：基准装配。下：第 3.4 节自动生成的取件夹具。

3.5 Motion Planning for Transit and Transfer

3.5 转移与搬运的运动规划

Finally, RRT-Connect [45] plans for all remaining transit and transfer motions, i.e., $\tau^{f}_{a,i},\tau^{g}_{a,i},\tau^{f}_{h,i}$, which can be parallelized since all start and goal states of motions are provided from earlier stages.

最后，用 RRT-Connect [45] 规划全部剩余转移与搬运运动，即 $\tau^{f}_{a,i},\tau^{g}_{a,i},\tau^{f}_{h,i}$；由于所有运动的起终点已由前序阶段给出，这些规划可以并行。

4 Learning General Single-Step Assembly Policy

4 学习通用单步装配策略

Once the full assembly plan is computed, the next challenge is to track it reliably in the real world. We use a hybrid controller that alternates between tracking the pre-planned transit and transfer motions, and an RL-based reactive controller for contact-rich assembly steps. The controller must generalize across variations in object geometry, grasp poses, and assembly directions. To this end, we design a lightweight yet highly effective RL framework for training a generalist assembly policy. Given a pre-planned insertion path $\tau_{o}$, our goal is to guide the part from its noisy initial pose $\hat{\tau}_{o}[0]$ to the goal pose $\hat{\tau}_{o}[1]$, accounting for uncertainty in grasp and mating part geometry. We frame this as a Markov decision process (MDP) and learn a policy $\pi:\mathcal{O}\to\mathbb{P}(\mathcal{A})$ that maximizes expected rewards over time $\mathbb{E}_{\pi}[\sum_{t=0}^{T-1}\gamma^{t}r(s_{t})]$. We use proximal policy optimization (PPO) [46] to train a stochastic policy in simulation, which is then transferred to the real world without additional fine-tuning.

完整装配计划算完后，下一步是在真实世界可靠跟踪它。我们使用混合控制器：在跟踪预规划转移/搬运运动与面向富接触装配步的基于 RL 的反应式控制器之间交替。控制器必须在物体几何、抓取位姿与装配方向的变化上泛化。为此我们设计轻量却高效的 RL 框架以训练通才装配策略。给定预规划插入路径 $\tau_{o}$，目标是把零件从带噪声的初始位姿 $\hat{\tau}_{o}[0]$ 引导到目标位姿 $\hat{\tau}_{o}[1]$，并计入抓取与配合件几何的不确定性。我们将其表述为马尔可夫决策过程（MDP），学习策略 $\pi:\mathcal{O}\to\mathbb{P}(\mathcal{A})$ 以最大化期望回报 $\mathbb{E}_{\pi}[\sum_{t=0}^{T-1}\gamma^{t}r(s_{t})]$。用近端策略优化（PPO） [46] 在仿真中训练随机策略，再无额外微调迁移到真实世界。

4.1 Path-Centric Coordinate Transformation

4.1 以路径为中心的坐标变换

Humans naturally reuse the same assembly skills across different objects, regardless of their poses or motions. We emulate this ability by designing a problem space transformation [47] with SE(3)-equivariance, which maps all straight-line assembly motions in the world frame into equivalent top-down insertions in the task frame, allowing the RL agent to perceive them in a unified way.

人类会把同一装配技能复用到不同物体上，而不论其位姿或运动。我们通过带 SE(3) 等变性的问题空间变换 [47] 模仿这一点：把世界系中所有直线装配运动映射为任务系中等价的自上而下插入，使 RL 智能体以统一方式感知它们。

Formally speaking, given an assembly path $\tau$ with a pre-assembly position $p_{d}$ and the assembled position $p_{a}$, we define a path-centric transformation $\mathbf{T}$ conditioned on $\tau$ such that $\mathbf{T}(p_{d})=(0,0,0)$ and $\mathbf{T}(p_{d})-\mathbf{T}(p_{a})=(0,0,\|p_{d}-p_{a}\|)$. Thus, the agent’s observation is the transformed position of the part being assembled (under unknown noise), and its action is the transformed delta position, i.e., the ideal position change in the path-centric frame. Thanks to equivariance, the agent only needs to learn top-down insertions and can omit orientation from both observations and actions, simplifying the learning setup. We use a task-space impedance (TSI) controller to enable smooth and compliant insertions, with impedance gains similarly transformed into the path-centric frame to maintain consistent behavior across different assembly directions. This design ensures that the observation and action spaces are minimal yet essential, facilitating generalizability among different assembly tasks, and is transferable to different robot arms and end effectors.

形式化地，给定装配路径 $\tau$，其预装配位置为 $p_{d}$、装配完成位置为 $p_{a}$，我们定义以路径为条件的路径中心变换 $\mathbf{T}$，使得 $\mathbf{T}(p_{d})=(0,0,0)$ 且 $\mathbf{T}(p_{d})-\mathbf{T}(p_{a})=(0,0,\|p_{d}-p_{a}\|)$。于是智能体观测是被装配零件经变换后的位置（含未知噪声），动作是变换后的位置增量，即路径中心系中的理想位置变化。凭等变性，智能体只需学习自上而下插入，并可从观测与动作中省略朝向，从而简化学习设置。我们用任务空间阻抗（TSI）控制器实现平滑柔顺插入，阻抗增益同样变换到路径中心系，以在不同装配方向上保持一致行为。该设计使观测与动作空间既最小又必要，促进不同装配任务间的泛化，并可迁移到不同机械臂与末端执行器。

4.2 Plan-Guided Residual Action

4.2 规划引导的残差动作

We find that guidance from the planned open-loop action helps learning by injecting prior knowledge about the coarse assembly direction. Thus, we adopt the idea of residual action [48, 31] in RL, where the policy outputs only the corrective action on top of the open-loop action, allowing the policy to focus on refining the assembly rather than learning the full assembly behavior. In practice, residual action warm-starts policy learning and typically leads to faster and better convergence.

我们发现，规划开环动作的引导能注入关于粗装配方向的先验，有助于学习。因此在 RL 中采用残差动作 [48, 31]：策略只在开环动作之上输出修正，从而把学习重点放在细调装配，而不是从零学习完整装配行为。实践中，残差动作能热启动策略学习，通常带来更快更好的收敛。

4.3 Minimalist Reward Design and Sim-to-Real Transfer

4.3 极简奖励设计与仿真到现实迁移

Surprisingly, our insertion reward is simply the negative L2 distance from the current part position to the goal position. This form is dense and sufficient enough for learning effective local insertion policies as our initial state is the pre-assembled pose given by the planner, which is already in the proximity of the goal thus does not require expensive exploration or complex reward engineering.

出人意料的是，我们的插入奖励就是当前零件位置到目标位置的负 L2 距离。该形式稠密，且足以学习有效的局部插入策略：因为初始状态就是规划器给出的预装配位姿，已靠近目标，无需昂贵探索或复杂奖励工程。

Due to the sim-to-real gap from misaligned dynamics, we adopt 1) domain randomization with 3mm-noised initialization on object pose during training and 2) Policy-Level Action Integrator (PLAI) [26] during deployment to ease the sim-to-real transfer of RL policies, which improves action consistency by incrementally applying policy outputs to the last desired state instead of the current state. PLAI applies policy actions as $s_{t+1}^{d}=s_{t}^{d}\oplus\Pi(o_{t})$, where $s_{t}^{d}$ represents the desired state at time $t$, $\Pi(o_{t})$ is the policy action computed based on the current observation $o_{t}$, and $\oplus$ denotes the composition operation, instead of the nominal approach $s_{t+1}^{d}=s_{t}\oplus\Pi(o_{t})$ which is prone to error accumulation.

由于动力学不对齐带来的仿真到现实差距，我们采用：（1）训练时对物体位姿做 3 mm 噪声初始化的域随机化；（2）部署时使用策略级动作积分器（PLAI） [26] 以缓解 RL 策略的仿真到现实迁移——它把策略输出增量施加到上一期望状态而非当前状态，从而提升动作一致性。PLAI 按 $s_{t+1}^{d}=s_{t}^{d}\oplus\Pi(o_{t})$ 施加动作，其中 $s_{t}^{d}$ 是时刻 $t$ 的期望状态，$\Pi(o_{t})$ 是基于当前观测 $o_{t}$ 的策略动作，$\oplus$ 表示复合运算；相对名义做法 $s_{t+1}^{d}=s_{t}\oplus\Pi(o_{t})$，后者更容易误差累积。

5 Experiments

5 实验

5.1 Benchmark Suite and Experimental Setup

5.1 基准套件与实验设置

We develop a diverse benchmark suite spanning furniture, toys, and industrial equipment, which includes beam (5 parts), plumber block (5 parts), car (6 parts), gamepad (6 parts), cooling manifold (7 parts), duct (8 parts), and stool (9 parts). These assemblies cover various geometries and connection types found in real-world applications, with both top-down and sideway insertions, and are feasible for dual-arm robots with parallel grippers. For planning in simulation, we demonstrate on several different robots, including Franka Emika Panda, UFactory xArm7, and UR5e with different grippers. We use Panda for systematic evaluations of policy training and real-world execution. See more details on the experimental setup and hyper-parameters in App. D.

我们构建覆盖家具、玩具与工业设备的多样基准套件，包括 beam（5 零件）、plumber block（5 零件）、car（6 零件）、gamepad（6 零件）、cooling manifold（7 零件）、duct（8 零件）与 stool（9 零件）。这些装配覆盖真实应用中的多种几何与连接类型，含自上而下与侧向插入，且对带平行夹爪的双臂机器人可行。仿真规划在多种机器人上演示，包括 Franka Emika Panda、UFactory xArm7，以及配备不同夹爪的 UR5e。系统评测策略训练与真实执行时使用 Panda。实验设置与超参数详见附录 D。

5.2 Planning Multi-Step Assembly in Simulation

5.2 仿真中的多步装配规划

Efficiency: Table 2 shows the breakdown of planning time by stages for different assemblies. Our overall speed is on the order of minutes to solve for optimal plans given efficient parallelization.

效率： 表 2 给出不同装配各阶段规划时间分解。借助高效并行化，求解最优计划的整体耗时约在分钟量级。

Optimality: Table 2 shows the objective scores of optimized sequences surpassing the random ones by a large margin with priorities from $f_{1}$ to $f_{4}$ (see App. C.3 for details on the score definitions).

最优性： 表 2 显示优化序列的目标分数大幅优于随机序列，优先级从 $f_{1}$ 到 $f_{4}$（分数定义见附录 C.3）。

Generality: Please see App. A for visual demonstrations of planning with different robot arms (Panda, xArm7, UR5e) and grippers. Our planning framework is general to any given hardware.

通用性： 使用不同机械臂（Panda、xArm7、UR5e）与夹爪的规划可视化见附录 A。我们的规划框架对给定硬件具有通用性。

Assembly Runtime (s) Prec∗ Grasp∗ Seq Fixture Motion Total Beam 19.7 35.7 0.2 1.7 115.9 173.2 Plumber 21.6 31.8 12.6 0.9 118.9 185.7 Car 23.4 52.9 0.9 1.9 127.4 206.4 Gamepad 22.2 37.2 4.5 3.6 117.4 184.8 Manifold 20.9 162.7 5.2 1.9 149.6 340.4 Duct 93.9 102.3 208.1 2.5 185.9 592.7 Stool 57.2 109.1 8.0 4.0 324.5 502.7

Assembly Objective Values (Optimal / Random) $f_{1}\uparrow$ $f_{2}\downarrow$ $f_{3}\downarrow$ $f_{4}\uparrow$ Beam 4.00 / 0.88 4.00 / 0.88 0.48 / 0.39 119.8 / 6.8 Plumber 4.00 / 1.27 2.00 / 1.79 0.18 / 0.33 293.0 / 93.4 Car 4.00 / 1.72 1.00 / 1.64 0.21 / 0.94 140.6 / 35.2 Gamepad 5.00 / 1.76 1.00 / 1.56 1.40 / 0.63 428.6 / 52.1 Manifold 6.00 / 2.62 1.00 / 2.17 0.18 / 0.69 12.5 / 18.7 Duct 6.00 / 3.00 1.00 / 2.88 0.09 / 0.39 536.0 / 101.3 Stool 8.00 / 3.43 6.00 / 3.24 0.03 / 0.54 322.2 / 87.9

Table 1: Planning runtime breakdown of each assembly. Stages marked with * are parallelized, while others have yet to be parallelized.

装配 运行时间 (s) Prec∗ Grasp∗ Seq Fixture Motion 合计 Beam 19.7 35.7 0.2 1.7 115.9 173.2 Plumber 21.6 31.8 12.6 0.9 118.9 185.7 Car 23.4 52.9 0.9 1.9 127.4 206.4 Gamepad 22.2 37.2 4.5 3.6 117.4 184.8 Manifold 20.9 162.7 5.2 1.9 149.6 340.4 Duct 93.9 102.3 208.1 2.5 185.9 592.7 Stool 57.2 109.1 8.0 4.0 324.5 502.7

装配 目标值（最优 / 随机） $f_{1}\uparrow$ $f_{2}\downarrow$ $f_{3}\downarrow$ $f_{4}\uparrow$ Beam 4.00 / 0.88 4.00 / 0.88 0.48 / 0.39 119.8 / 6.8 Plumber 4.00 / 1.27 2.00 / 1.79 0.18 / 0.33 293.0 / 93.4 Car 4.00 / 1.72 1.00 / 1.64 0.21 / 0.94 140.6 / 35.2 Gamepad 5.00 / 1.76 1.00 / 1.56 1.40 / 0.63 428.6 / 52.1 Manifold 6.00 / 2.62 1.00 / 2.17 0.18 / 0.69 12.5 / 18.7 Duct 6.00 / 3.00 1.00 / 2.88 0.09 / 0.39 536.0 / 101.3 Stool 8.00 / 3.43 6.00 / 3.24 0.03 / 0.54 322.2 / 87.9

表 1：各装配规划运行时间分解。标 * 的阶段已并行化，其余尚未并行。

5.3 Learning Single-Step Assembly in Simulation

5.3 仿真中的单步装配学习

Method % of Successful Steps without Intervention (Simulation) Beam Plumber Block Car Gamepad Cooling Manifold Duct Stool Open-Loop Tracking 21.48 24.22 2.34 2.34 3.91 18.75 0.00 Part Specialist Policy (PS) 98.63 84.08 90.82 87.60 94.63 100.00 78.91 Assembly Specialist Policy (AS) 99.12 97.46 70.12 88.87 95.02 96.58 76.66 Assembly Generalist Policy (AG) 98.83 81.64 60.55 71.48 89.06 89.84 58.59

Table 3: % of successful steps without intervention in simulation evaluations.

方法 无干预成功步比例 %（仿真） Beam Plumber Block Car Gamepad Cooling Manifold Duct Stool Open-Loop Tracking 21.48 24.22 2.34 2.34 3.91 18.75 0.00 Part Specialist Policy (PS) 98.63 84.08 90.82 87.60 94.63 100.00 78.91 Assembly Specialist Policy (AS) 99.12 97.46 70.12 88.87 95.02 96.58 76.66 Assembly Generalist Policy (AG) 98.83 81.64 60.55 71.48 89.06 89.84 58.59

表 3：仿真评测中无干预成功步比例 %。

We use Isaac Gym [49] for training RL policies and performing simulation evaluations, with the PPO code from RL Games [50]. Table 3 presents the average % of successful steps for assembling our benchmark assemblies in simulation across 1024 random trials using different methods: 1) Open-Loop Tracking: A baseline that strictly follows the pre-planned path without feedback correction. 2) Part Specialist Policy (PS): Policies trained on individual pairs of parts. 3) Assembly Specialist Policy (AS): Policies trained on all parts within a single assembly. 4) Assembly Generalist Policy (AG): Policies trained on all parts from all assemblies in our suite, aiming for broad generalization.

我们用 Isaac Gym [49] 训练 RL 策略并做仿真评测，PPO 代码来自 RL Games [50]。表 3 给出在基准装配上、跨 1024 次随机试验、不同方法的平均成功步比例：1) 开环跟踪（Open-Loop Tracking）：严格跟随预规划路径、无反馈修正的基线。2) 零件专才策略（PS）：在单对零件上训练。3) 装配专才策略（AS）：在单个装配内全部零件上训练。4) 装配通才策略（AG）：在套件全部装配的全部零件上训练，追求广泛泛化。

The results show that open-loop tracking exhibits the lowest success rates across all assemblies, indicating its limitations in handling uncertainties and variations. The AS policy demonstrates competitive performance as the PS policy, suggesting that a shared policy across different parts in an assembly can generalize well. It may sacrifice some part-specific optimization, but can transfer the knowledge between similar parts. The AG policy, while slightly less effective than the other RL counterparts, still demonstrates robust performance, suggesting that learning a single shared policy across different assemblies is promising, given the equivariant representations. Furthermore, the success rates vary across different assemblies, with simpler assemblies like the Beam and Cooling Manifold achieving higher performance across all methods, while more complex assemblies such as the Gamepad and Stool exhibit lower success rates due to their intricate geometries and constraints.

结果显示开环跟踪在所有装配上成功率最低，说明其难以处理不确定性与变化。AS 策略与 PS 策略表现接近，表明装配内共享策略可以较好泛化：可能牺牲部分零件专用优化，但能在相似零件间迁移知识。AG 策略虽略弱于其他 RL 对照，仍表现稳健，说明在等变表示下学习跨装配共享策略是可行的。此外成功率随装配而变：Beam、Cooling Manifold 等较简单装配在各方法上更高；Gamepad、Stool 等因几何与约束更复杂而更低。

5.4 Executing Multi-Step Assembly in Real World

5.4 真实世界中的多步装配执行

Table 4 shows the % of successful steps on benchmark assemblies evaluated in the real world (step-wise statistics), and Table 5 shows the multi-step cumulative success rates with 0/1/2 total interventions for failure recovery (overall statistics). All numbers are averaged across three complete multi-step real experiments, which translate to thousands of total assembly steps. We deploy stochastic policies with state-based success detection and allow up to three trials per step until success. For qualitative results on real-world multi-step executions, please refer to Fig. 1 and App. A.

表 4 给出真实世界基准装配上无干预成功步比例（逐步统计），表 5 给出允许 0/1/2 次失败恢复干预时的多步累计成功率（整体统计）。所有数字为三次完整多步真实实验的平均，对应数千次装配步。我们部署带基于状态成功判定的随机策略，每步最多三次尝试直至成功。真实多步执行的定性结果见图 1 与附录 A。

Ours: We use both AS and AG policies for real-world comparisons. For AG, we perform out-of-distribution (OOD) evaluations by training 7 generalist policies, where each one is trained on the other 6 benchmark assemblies (excluding the test assembly). Remarkably, these OOD generalist policies still achieved comparable performance to specialist policies trained directly on the test assembly, which indicates that through our approach, insertion strategies learned from a diverse set of assemblies can effectively transfer to novel, unseen assemblies.

本文方法： 真实对比同时使用 AS 与 AG 策略。对 AG，我们做分布外（OOD）评测：训练 7 个通才策略，每个在其余 6 个基准装配上训练（排除测试装配）。值得注意的是，这些 OOD 通才策略仍能达到与直接在测试装配上训练的专才策略相当的表现，说明通过我们的方法，从多样装配学到的插入策略可以有效迁移到未见装配。

Baseline: Since Fabrica is the first to assemble general multi-part objects with only CAD input, identifying a comparable SOTA baseline is challenging. The closest work is ASAP [11], which performs single-arm kinematic feasibility search without sequence/grasp optimization or closed-loop control. To compare, we adapted it by planning with dual-arm and adding our RL policy. Without optimized part sequencing and grasping, ASAP performed substantially worse, often struggling even with two interventions, which emphasizes the contributions of our planning optimizations.

基线： 由于 Fabrica 是首个仅用 CAD 输入装配通用多零件物体的工作，难以找到完全对等的 SOTA 基线。最接近的是 ASAP [11]：做单臂运动学可行性搜索，无顺序/抓取优化或闭环控制。为对比，我们将其改为双臂规划并加上我们的 RL 策略。缺少优化的零件排序与抓取后，ASAP 明显更差，即便允许两次干预也常卡住，凸显我们规划优化的贡献。

Ablation: To understand how much optimizing part sequences and grasps helps, we conduct ablation studies on our method by removing those optimizations respectively. We observed that suboptimal sequencing often caused instability due to inadequate support of critical neighboring parts and more part drifts due to unnecessary re-grasps. Meanwhile, suboptimal grasp selection frequently caused part slippage due to insufficient contact area or inadequate resistance to external torques. Thus, our planner inherently accounts for control-level uncertainties, and results demonstrate that selecting effective part sequences and grasps significantly enhances assembly reliability. For control, we observed that our path-centric transformation is crucial for generalizing across varying assembly directions. Policies trained without it perform significantly worse when multiple directions are involved. For more studies on the effects of path-centric transformation and residual actions introduced in Sec 4, and success increase w.r.t. the number of trials per step, please see App. E.

消融： 为理解优化零件顺序与抓取的帮助有多大，我们分别去掉这些优化做消融。观察到次优排序常因关键邻接零件支撑不足导致不稳，以及不必要重抓带来更多零件漂移；次优抓取选择则常因接触面积不足或抗外力矩能力不够导致打滑。因此规划器本身已计入控制层不确定性，结果表明选择有效零件顺序与抓取显著提升装配可靠性。控制方面，路径中心变换对跨不同装配方向泛化至关重要；不使用它时，涉及多方向的策略明显更差。关于第 4 节路径中心变换与残差动作、以及每步尝试次数对成功提升的更多研究，见附录 E。

Failures: We observed a noticeable gap between simulation and real-world performances. Thus, we present a detailed analysis of common failure cases shown in the right figure: a) Small parts slip between gripper pads during insertion attempts; b) Cumulative error accrued during the assembly of large assemblies increases the displacement error of final part insertions; c) The holding gripper is not modeled during RL training, causing unexpected part obstructions in the real world; d) Unstabilized parts shift previously assembled parts during insertion. We assume a 3mm noise in simulation given that the base part is stably held. However, many sources of real-world error lead to much more significant errors than simulated, which are non-trivial challenges for future work. Due to these failures, all methods achieve near-zero multi-step success rates without intervention due to inherently challenging steps causing consistent failures. However, with minimal interventions, our method significantly outperforms others, reaching 81% success with one intervention and 95% with two.

失败： 我们观察到仿真与真实表现存在明显差距。右图给出常见失败案例分析：a) 小零件在插入尝试时从夹爪垫缝滑出；b) 大型装配过程中累积误差增大最终零件插入的位移误差；c) RL 训练未建模持件夹爪，真实中出现意外遮挡；d) 未稳定零件在插入时推动已装配零件。仿真中假设 3 mm 噪声且基座零件被稳定持住；但真实误差源往往远大于仿真，是未来工作的非平凡挑战。由于这些失败，所有方法在无干预时多步成功率接近零——某些固有难题步会持续失败。但在极少干预下，我们的方法显著优于其他：一次干预约 81% 成功，两次约 95%。

Method % of Successful Steps without Intervention (Real World) Beam Plumber Block Car Gamepad Cooling Manifold Duct Stool Overall Ours AS 75 83 80 87 72 71 92 80 AG (OOD) 67 75 93 80 72 81 92 80 Baseline ASAP (Adapted) 50 42 67 33 55 52 75 55 Open-Loop Tracking 42 25 20 20 17 14 21 23 Ablation w/o Part Seq Optim 75 75 73 40 61 71 92 71 w/o Grasp Optim 42 83 47 60 67 67 75 64 w/o Path-Centric Transform 67 67 20 53 78 38 92 61

Ours

Baseline

Ablation

Table 4: % of successful steps without intervention in real-world evaluations.

方法 无干预成功步比例 %（真实） Beam Plumber Block Car Gamepad Cooling Manifold Duct Stool 总体 Ours AS 75 83 80 87 72 71 92 80 AG (OOD) 67 75 93 80 72 81 92 80 Baseline ASAP (Adapted) 50 42 67 33 55 52 75 55 Open-Loop Tracking 42 25 20 20 17 14 21 23 Ablation w/o Part Seq Optim 75 75 73 40 61 71 92 71 w/o Grasp Optim 42 83 47 60 67 67 75 64 w/o Path-Centric Transform 67 67 20 53 78 38 92 61

Ours

Baseline

Ablation

表 4：真实世界评测中无干预成功步比例 %。

Method Multi-Step Cumulative Success Rate with 0/1/2 Interventions (%) (Real World) Beam Plumber Block Car Gamepad Cooling Manifold Duct Stool Overall Ours AS 0/100/100 33/100/100 0/100/100 33/100/100 0/67/67 0/0/100 33/100/100 15/81/95 AG (OOD) 0/67/100 0/100/100 67/100/100 0/100/100 0/33/100 0/67/100 33/100/100 10/81/100 Baseline ASAP (Adapted) 0/0/100 0/0/67 0/33/100 0/0/0 0/0/33 0/0/0 0/0/100 0/5/57 Open-Loop Tracking 0/0/67 0/0/33 0/0/0 0/0/0 0/0/0 0/0/0 0/0/0 0/0/14 Ablation w/o Part Seq Optim 0/100/100 33/67/100 0/67/100 0/0/0 0/0/67 0/0/100 33/100/100 10/48/81 w/o Grasp Optim 0/0/67 33/100/100 0/0/33 0/0/100 0/0/100 0/0/67 0/0/100 5/14/81 w/o Path-Centric Transform 0/67/100 0/67/100 0/0/0 0/0/67 0/67/100 0/0/0 33/100/100 5/43/67

Ours

Baseline

Ablation

Table 5: Multi-step cumulative success rate with 0/1/2 interventions in real-world evaluations.

方法 允许 0/1/2 次干预的多步累计成功率 (%)（真实） Beam Plumber Block Car Gamepad Cooling Manifold Duct Stool 总体 Ours AS 0/100/100 33/100/100 0/100/100 33/100/100 0/67/67 0/0/100 33/100/100 15/81/95 AG (OOD) 0/67/100 0/100/100 67/100/100 0/100/100 0/33/100 0/67/100 33/100/100 10/81/100 Baseline ASAP (Adapted) 0/0/100 0/0/67 0/33/100 0/0/0 0/0/33 0/0/0 0/0/100 0/5/57 Open-Loop Tracking 0/0/67 0/0/33 0/0/0 0/0/0 0/0/0 0/0/0 0/0/0 0/0/14 Ablation w/o Part Seq Optim 0/100/100 33/67/100 0/67/100 0/0/0 0/0/67 0/0/100 33/100/100 10/48/81 w/o Grasp Optim 0/0/67 33/100/100 0/0/33 0/0/100 0/0/100 0/0/67 0/0/100 5/14/81 w/o Path-Centric Transform 0/67/100 0/67/100 0/0/0 0/0/67 0/67/100 0/0/0 33/100/100 5/43/67

Ours

Baseline

Ablation

表 5：真实世界评测中允许 0/1/2 次干预的多步累计成功率。

6 Conclusion

6 结论

We presented Fabrica, a dual-arm robotic system that innovates and integrates global hierarchical planning with local generalist policy learning for autonomous multi-part assembly. To support reproducible and rigorous evaluation, we introduced a comprehensive benchmark suite covering diverse multi-part assemblies. Fabrica is the first to demonstrate robust and generalizable performance across a wide range of real-world assembly tasks. We discuss limitations and future work in Sec 7.

我们提出 Fabrica：把全局层次规划与局部通才策略学习结合并创新的双臂机器人系统，用于自主多零件装配。为支持可复现、严谨的评测，我们引入覆盖多样多零件装配的综合基准套件。Fabrica 首次在广泛真实装配任务上展示稳健可泛化表现。局限与未来工作见第 7 节。

7 Limitations and Future Work

7 局限与未来工作

While Fabrica shows promising results for autonomous multipart assembly, there remain several limitations and opportunities for future extension.

尽管 Fabrica 在自主多零件装配上结果可观，仍有若干局限与可扩展方向。

Assumptions: The assumptions we make in this problem formulation are the following:

假设： 问题表述中的假设如下：

(A1) Insertion-only assembly: We assume that the mating between two parts only involves an insertion motion, without requiring skills like screwing or sliding.

(A1) 仅插入装配：假设两零件配合只涉及插入运动，不需要拧紧或滑动等技能。

(A2) No subassembly reorientation: The final assembled positions of all parts are assumed to be given and fixed during the assembly process. This means that no further movement or re-orientation is allowed once a part is assembled.

(A2) 子装配不再定向：假设全部零件的最终装配位姿给定且在装配过程中固定。这意味着零件一旦装配完成，不允许再移动或再定向。

(A3) Monotonic assembly: Each part is only moved once, without considering regrasps, in-hand manipulation, or handovers.

(A3) 单调装配：每个零件只移动一次，不考虑重抓、手内操作或交接。

(A4) No force and torque constraints for the robots: We assume all parts are light compared to the robot payload.

(A4) 机器人无力/力矩约束：假设相对机器人负载，全部零件都较轻。

(A5) A finite grasp set for each part: $\forall g\in\sigma,g\in\mathcal{G}[o],|\mathcal{G}[o]|=N$, where $\mathcal{G}[o]$ can be computed by any grasp generator.

(A5) 每个零件有有限抓取集：$\forall g\in\sigma,g\in\mathcal{G}[o],|\mathcal{G}[o]|=N$，其中 $\mathcal{G}[o]$ 可由任意抓取生成器计算。

The above assumptions leave areas such as handling heavier parts, managing grasp slippage, and performing other operations such as screwing or sliding unaddressed. Incorporating these capabilities would significantly improve the robustness and applicability of Fabrica in more complex and diverse assembly tasks.

上述假设使更重零件处理、抓取打滑管理，以及拧紧或滑动等其他操作仍未覆盖。纳入这些能力将显著提升 Fabrica 在更复杂、更多样装配任务中的稳健性与适用性。

Dexterity: Moreover, the current setup enforces a fixed part pose once assembled. This is in contrast with the more dexterous human assembly behavior, where one would constantly reorient the partial assembly so that the parts are easily reachable. However, allowing reorientation would introduce additional planning overhead and more uncertainty for control due to potential subassembly instability. Addressing it will enable a more dexterous robotic system that can handle large assemblies that are beyond the reach of the current system, e.g., a large tabletop with parts on both sides.

灵巧性： 此外，当前设置在零件装配后强制固定其位姿。这与更灵巧的人类装配行为不同——人会不断再定向部分装配，使零件更易够到。但允许再定向会带来额外规划开销，并因潜在子装配不稳而增加控制不确定性。解决它将使系统更灵巧，能处理超出当前可达范围的大型装配，例如两侧都有零件的大桌面。

Hardware capability: Compared to the existing multipart assembly dataset with thousands of objects [10], our current benchmark is limited in its size and diversity. This is because we want to ensure that the benchmark tasks are achievable by commonly used parallel grippers, but these grippers have a limited grasp width and thus cannot establish stable antipodal grasps for parts with large and complex geometries. However, to broaden the assembly capability, we can envision either a multi-finger hand or a multi-tool system in which the robots can switch specialized grippers according to the part geometry, and our planning and control system could be adapted to this setting.

硬件能力： 相较含数千物体的现有多零件装配数据集 [10]，我们当前基准在规模与多样性上仍有限。原因是我们希望保证任务可用常用平行夹爪完成，但这些夹爪抓取宽度有限，难以对大而复杂几何建立稳定对跖抓取。为拓宽装配能力，可设想多指手或多工具系统：机器人按零件几何切换专用夹爪，我们的规划与控制系统也可适配该设定。

Perception: Integrating vision systems for alignment feedback could greatly improve the accuracy and adaptability of the assembly process. By incorporating perception, the system could enable direct bin-picking, allowing it to grasp parts from random, unknown initial poses instead of requiring a specialized pickup fixture. However, bin-picking remains a well-known challenge in industry, particularly in terms of robustness and generalizability across arbitrary part geometries and physical properties. Addressing these challenges requires substantial research and development efforts, but would significantly expand the practical applications of our approach.

感知： 集成视觉系统做对齐反馈，可大幅提升装配精度与适应性。引入感知后，系统可直接料箱拣选，从随机未知初始位姿抓取零件，而不必依赖专用取件夹具。但料箱拣选在工业中仍是公认难题，尤其在任意零件几何与物理属性上的稳健性与泛化。应对这些挑战需要大量研发，但会显著扩展本方法的实际应用。

Data collection: Collecting real-world assembly data is challenging due to the task’s long-horizon, contact-rich nature and the high cost of acquiring or fabricating diverse assembly assets. Fabrica addresses these barriers by enabling fully autonomous data collection in simulation and requiring only minimal human intervention in the real world. In future work, we aim to leverage this capability to generate large-scale, diverse datasets of assembly trajectories. These datasets can facilitate broader research in generalizable policy learning, sim-to-real transfer, and foundation models for robotic manipulation. Moreover, because assembly is one of the most constrained and demanding manipulation tasks, learning from assembly data has the potential to positively transfer to a wide range of general-purpose manipulation skills.

数据采集： 真实装配数据因任务长时程、富接触，以及获取或制作多样装配资产成本高而难以采集。Fabrica 通过在仿真中完全自主采集、在真实中仅需极少人工干预来降低这些门槛。未来我们希望借此生成大规模多样装配轨迹数据集，推动可泛化策略学习、仿真到现实迁移，以及机器人操作基础模型的研究。此外，装配是约束最强、要求最高的操作任务之一，从装配数据中学习有望正向迁移到广泛的通用操作技能。

Acknowledgments

This work is funded by Autodesk, and in part by NSF 1846368 & 2313076. Yijiang Huang is supported by the SNSF Ambizione program. The authors would like to thank Bingjie Tang and Lars Ankile for insightful discussions during the RL environment setup, Xiang Zhang and Yotto Koga for valuable advice on the early physical setup, the members of Ted Adelson’s lab at MIT for their support with the physical Panda robot, and the MIT SuperCloud and Lincoln Laboratory for HPC resources.

致谢

本工作由 Autodesk 资助，并部分得到 NSF 1846368 与 2313076 支持。Yijiang Huang 获 SNSF Ambizione 项目支持。作者感谢 Bingjie Tang 与 Lars Ankile 在 RL 环境搭建中的深入讨论，感谢 Xiang Zhang 与 Yotto Koga 对早期物理搭建的宝贵建议，感谢 MIT Ted Adelson 实验室成员对实体 Panda 机器人的支持，以及 MIT SuperCloud 与 Lincoln Laboratory 提供的 HPC 资源。

References

参考文献

[1] J. Xu, S. Kim, T. Chen, A. R. Garcia, P. Agrawal, W. Matusik, and S. Sueda. Efficient tactile simulation with differentiability for robotic manipulation. In 6th Annual Conference on Robot Learning , 2022.

[2] X. Zhang, M. Tomizuka, and H. Li. Bridging the sim-to-real gap with dynamic compliance tuning for industrial insertion. In ICRA , 2024.

[3] B. Tang, I. Akinola, J. Xu, B. Wen, A. Handa, K. Van Wyk, D. Fox, G. S. Sukhatme, F. Ramos, and Y. Narang. Automate: Specialist and generalist assembly policies over diverse geometries. In Robotics: Science and Systems , 2024.

[4] D. Halperin, J.-C. Latombe, and R. H. Wilson. A general framework for assembly planning: The motion space approach. Algorithmica , 26(3):577–601, 2000.

[5] S. Sundaram, I. Remmler, and N. M. Amato. Disassembly sequencing using a motion planning approach. In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164) , volume 2, pages 1475–1480. IEEE, 2001.

[6] D. T. Le, J. Cortés, and T. Siméon. A path planning approach to (dis) assembly sequencing. In 2009 IEEE International Conference on Automation Science and Engineering , pages 286–291. IEEE, 2009.

[7] X. Zhang, R. Belfer, P. G. Kry, and E. Vouga. C-space tunnel discovery for puzzle path planning. ACM Transactions on Graphics (TOG) , 39(4):104–1, 2020.

[8] N. Funk, G. Chalvatzaki, B. Belousov, and J. Peters. Learn2assemble with structured representations and search for robotic architectural construction. In Conference on Robot Learning , pages 1401–1411. PMLR, 2022.

[9] S. K. S. Ghasemipour, S. Kataoka, B. David, D. Freeman, S. S. Gu, and I. Mordatch. Blocks assemble! learning to assemble with large-scale structured reinforcement learning. In International Conference on Machine Learning , pages 7435–7469. PMLR, 2022.

[10] Y. Tian, J. Xu, Y. Li, J. Luo, S. Sueda, H. Li, K. D. Willis, and W. Matusik. Assemble them all: Physics-based planning for generalizable assembly by disassembly. ACM Transactions on Graphics (TOG) , 41(6):1–11, 2022.

[11] Y. Tian, K. D. Willis, B. Al Omari, J. Luo, P. Ma, Y. Li, F. Javid, E. Gu, J. Jacob, S. Sueda, et al. Asap: Automated sequence planning for complex robotic assembly with physical feasibility. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 4380–4386. IEEE, 2024.

[12] I. Rodrıguez, K. Nottensteiner, D. Leidner, M. Kaßecker, F. Stulp, and A. Albu-Schäffer. Iteratively refined feasibility checks in robotic assembly sequence planning. IEEE Robotics and Automation Letters , 4(2):1416–1423, 2019.

[13] X. Zhu, D. K. Jha, D. Romeres, L. Sun, M. Tomizuka, and A. Cherian. Multi-level reasoning for robotic assembly: From sequence inference to contact selection. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 816–823. IEEE, 2024.

[14] F. Suárez-Ruiz, X. Zhou, and Q.-C. Pham. Can robots assemble an ikea chair? Science Robotics , 3(17):eaat6385, 2018.

[15] L. Nägele, A. Hoffmann, A. Schierl, and W. Reif. Legobot: Automated planning for coordinated multi-robot assembly of lego structures. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 9088–9095. IEEE, 2020.

[16] Y. Huang, C. R. Garrett, I. Ting, S. Parascho, and C. T. Mueller. Robotic additive construction of bar structures: Unified sequence and motion planning. Construction Robotics , 5:115–130, 2021a.

[17] Y. Huang, P. Y. V. Leung, C. Garrett, F. Gramazio, M. Kohler, and C. Mueller. The new analog: A protocol for linking design and construction intent with algorithmic planning for robotic assembly of complex structures. In Proceedings of the 6th Annual ACM Symposium on Computational Fabrication , pages 1–17, 2021b.

[18] Z. Wang, F. Kennel-Maushart, Y. Huang, B. Thomaszewski, and S. Coros. A temporal coherent topology optimization approach for assembly planning of bespoke frame structures. ACM Transactions on Graphics (TOG) , 42(4):1–13, 2023.

[19] C.-J. Liang, S.-C. Kang, and M.-H. Lee. Ras: a robotic assembly system for steel structure erection and assembly. International Journal of Intelligent Robotics and Applications , 1:459–476, 2017.

[20] K. Dörfler, T. Sandy, M. Giftthaler, F. Gramazio, M. Kohler, and J. Buchli. Mobile robotic brickwork: automation of a discrete robotic fabrication process using an autonomous mobile robot. Robotic Fabrication in Architecture, Art and Design 2016 , pages 204–217, 2016.

[21] A. A. Apolinarska, M. Pacher, H. Li, N. Cote, R. Pastrana, F. Gramazio, and M. Kohler. Robotic assembly of timber joints using reinforcement learning. Automation in Construction , 125:103569, 2021.

[22] A. Kramberger, A. Kunic, I. Iturrate, C. Sloth, R. Naboni, and C. Schlette. Robotic assembly of timber structures in a human-robot collaboration setup. Frontiers in Robotics and AI , 8:768038, 2022.

[23] G. Thomas, M. Chien, A. Tamar, J. A. Ojea, and P. Abbeel. Learning robotic assembly from cad. In 2018 IEEE International Conference on Robotics and Automation (ICRA) , pages 3524–3531. IEEE, 2018.

[24] Y. Fan, J. Luo, and M. Tomizuka. A learning framework for high precision industrial assembly. In 2019 International Conference on Robotics and Automation (ICRA) , pages 811–817. IEEE, 2019.

[25] Y. Narang, K. Storey, I. Akinola, M. Macklin, P. Reist, L. Wawrzyniak, Y. Guo, A. Moravanszky, G. State, M. Lu, et al. Factory: Fast contact for robotic assembly. arXiv preprint arXiv:2205.03532 , 2022.

[26] B. Tang, M. A. Lin, I. Akinola, A. Handa, G. S. Sukhatme, F. Ramos, D. Fox, and Y. Narang. Industreal: Transferring contact-rich assembly tasks from simulation to reality. arXiv preprint arXiv:2305.17110 , 2023.

[27] X. Zhang, C. Wang, L. Sun, Z. Wu, X. Zhu, and M. Tomizuka. Efficient sim-to-real transfer of contact-rich manipulation skills with online admittance residual learning. In Conference on Robot Learning , pages 1621–1639. PMLR, 2023.

[28] J. Luo, Z. Hu, C. Xu, Y. L. Tan, J. Berg, A. Sharma, S. Schaal, C. Finn, A. Gupta, and S. Levine. Serl: A software suite for sample-efficient robotic reinforcement learning. arXiv preprint arXiv:2401.16013 , 2024.

[29] M. Noseworthy, B. Tang, B. Wen, A. Handa, N. Roy, D. Fox, F. Ramos, Y. Narang, and I. Akinola. Forge: Force-guided exploration for robust contact-rich manipulation under uncertainty. arXiv preprint arXiv:2408.04587 , 2024.

[30] L. Ankile, A. Simeonov, I. Shenfeld, and P. Agrawal. Juicer: Data-efficient imitation learning for robotic assembly. arXiv , 2024a.

[31] L. Ankile, A. Simeonov, I. Shenfeld, M. Torne, and P. Agrawal. From imitation to refinement – residual rl for precise assembly, 2024b. URL https://arxiv.org/abs/2407.16677 .

[32] H. Huang, D. Wang, A. Tangri, R. Walters, and R. Platt. Leveraging symmetries in pick and place. The International Journal of Robotics Research , page 02783649231225775, 2024.

[33] A. Simeonov, Y. Du, A. Tagliasacchi, J. B. Tenenbaum, A. Rodriguez, P. Agrawal, and V. Sitzmann. Neural descriptor fields: Se (3)-equivariant object representations for manipulation. In 2022 International Conference on Robotics and Automation (ICRA) , pages 6394–6400. IEEE, 2022.

[34] A. Simeonov, Y. Du, Y.-C. Lin, A. R. Garcia, L. P. Kaelbling, T. Lozano-Pérez, and P. Agrawal. Se (3)-equivariant relational rearrangement with neural descriptor fields. In Conference on Robot Learning , pages 835–846. PMLR, 2023.

[35] J. Yang, C. Deng, J. Wu, R. Antonova, L. Guibas, and J. Bohg. Equivact: Sim (3)-equivariant visuomotor policies beyond rigid object manipulation. arXiv preprint arXiv:2310.16050 , 2023.

[36] D. Wang, S. Hart, D. Surovik, T. Kelestemur, H. Huang, H. Zhao, M. Yeatman, J. Wang, R. Walters, and R. Platt. Equivariant diffusion policy. arXiv preprint arXiv:2407.01812 , 2024.

[37] J. Seo, N. P. Prakash, X. Zhang, C. Wang, J. Choi, M. Tomizuka, and R. Horowitz. Contact-rich se (3)-equivariant robot manipulation task learning via geometric impedance control. IEEE Robotics and Automation Letters , 2023.

[38] W. Lian, T. Kelch, D. Holz, A. Norton, and S. Schaal. Benchmarking off-the-shelf solutions to robotic assembly tasks. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 1046–1053. IEEE, 2021.

[39] L. Fan, Y. Zhu, J. Zhu, Z. Liu, O. Zeng, A. Gupta, J. Creus-Costa, S. Savarese, and L. Fei-Fei. Surreal: Open-source reinforcement learning framework and robot manipulation benchmark. In Conference on robot learning , pages 767–782. PMLR, 2018.

[40] Y. Lee, E. S. Hu, and J. J. Lim. Ikea furniture assembly environment for long-horizon complex manipulation tasks. In 2021 ieee international conference on robotics and automation (icra) , pages 6343–6349. IEEE, 2021.

[41] M. Heo, Y. Lee, D. Lee, and J. J. Lim. Furniturebench: Reproducible real-world benchmark for long-horizon complex manipulation. In Robotics: Science and Systems , 2023.

[42] J. Luo, C. Xu, F. Liu, L. Tan, Z. Lin, J. Wu, P. Abbeel, and S. Levine. Fmb: a functional manipulation benchmark for generalizable robotic learning. arXiv preprint arXiv:2401.08553 , 2024.

[43] R. Alami, J.-P. Laumond, and T. Siméon. Two manipulation planning algorithms. In WAFR Proceedings of the workshop on Algorithmic foundations of robotics , pages 109–125. AK Peters, Ltd. Natick, MA, USA, 1994.

[44] T. Siméon, J.-P. Laumond, J. Cortés, and A. Sahbani. Manipulation planning with probabilistic roadmaps. The International Journal of Robotics Research , 23(7-8):729–746, 2004.

[45] J. J. Kuffner and S. M. LaValle. Rrt-connect: An efficient approach to single-query path planning. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065) , volume 2, pages 995–1001. IEEE, 2000.

[46] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017.

[47] K. Doshi, M. Bagatella, and S. Coros. Problem space transformations for generalisation in behavioural cloning. arXiv preprint arXiv:2411.04056 , 2024.

[48] T. Silver, K. Allen, J. Tenenbaum, and L. Kaelbling. Residual policy learning. arXiv preprint arXiv:1812.06298 , 2018.

[49] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac gym: High performance gpu-based physics simulation for robot learning, 2021.

[50] D. Makoviichuk and V. Makoviychuk. rl-games: A high-performance framework for reinforcement learning. https://github.com/Denys88/rl_games , May 2021.

[51] A. Lodi, S. Martello, and M. Monaci. Two-dimensional packing problems: A survey. European journal of operational research , 141(2):241–252, 2002.

[52] J. Jylänki. A thousand ways to pack the bin-a practical approach to two-dimensional rectangle bin packing, 2010. URL https://raw.githubusercontent.com/juj/RectangleBinPack/master/RectangleBinPack.pdf .

Appendix A Qualitative Results

附录 A 定性结果

Fig. 1 and Fig. 2 show qualitative results of our system assembling a variety of multi-part objects using different robot arms, both in simulation and in the real world. Once the hardware configuration is provided, our planner works seamlessly with a wide range of robot arms and end-effectors without requiring additional tuning.

图 1 与图 2 展示系统用不同机械臂、在仿真与真实世界中装配多种多零件物体的定性结果。一旦提供硬件配置，规划器即可与广泛机械臂与末端执行器无缝配合，无需额外调参。

The assembly process begins with picking up parts from the fixture, then transferring them to the assembly area for either holding or insertion. The system determines when to switch roles between the two arms, for example by handing over a part or changing the holding grasp, in order to maintain stability and accuracy. After each insertion, the robot returns to retrieve the next part, and this process continues until the assembly is complete. Finally, both arms return to their initial positions.

装配过程从夹具取件开始，再转移到装配区进行持件或插入。系统决定双臂何时切换角色——例如交接零件或更换持件抓取——以保持稳定与精度。每次插入后，机器人返回取下一零件，直至装配完成。最后双臂回到初始位姿。

All plans are generated automatically, including grasp selection, arm coordination, motion generation, and fixture design. The only manual input required is the initial setup of the workcell and placement of the hardware.

全部计划自动生成，包括抓取选择、双臂协调、运动生成与夹具设计。唯一需要人工的是工作单元初始搭建与硬件放置。

For real-world execution, we demonstrate robust sim-to-real transfer across long-horizon assembly tasks, with consistent step-by-step correspondence between simulation and physical execution. The system maintains geometric and temporal alignment across all stages of the assembly, including grasping, insertion, and part switching. This highlights the reliability of our simulation-informed planning and policy execution.

真实执行中，我们在长时程装配任务上展示稳健的仿真到现实迁移，仿真与物理执行逐步对应一致。系统在抓取、插入与零件切换等各阶段保持几何与时间对齐，凸显仿真知情规划与策略执行的可靠性。

Figure 1: Step-by-step rendered assembly executions on different assemblies with different robots.

图 1：不同装配、不同机器人上的逐步渲染装配执行。

Figure 2: Step-by-step real-world assembly executions on different assemblies with Panda robots, with side-by-side correspondences between simulation and real.

图 2：Panda 机器人上不同装配的逐步真实执行，并给出仿真与真实的并排对应。

Appendix B Problem Formulation

附录 B 问题表述

As in Sec. 3, we formulate planning as optimizing assembly-hold sequences $\phi$, grasps $\sigma$, and robot motions $\pi$:

如第 3 节，我们将规划表述为优化装配–持件序列 $\phi$、抓取 $\sigma$ 与机器人运动 $\pi$：

$$\min_{\phi,\sigma,\pi}E\big(\Phi_{i=1}^{n}\vec{f}(\phi_{1:i},\sigma_{i-1:i},\pi_{i})\big)\quad\text{s.t.}\quad C_{\text{prec}}(\phi)\leq 0,\hskip 1.99997ptC_{\text{kin}}(\phi,\sigma,\pi)=0,\hskip 1.99997ptC_{\text{col}}(\phi,\sigma,\pi)\leq 0 \tag{2}$$

A more detailed breakdown of constraints $C_{\text{prec}}(\phi)$, $C_{\text{kin}}(\phi)$, $C_{\text{col}}(\phi)$ is shown below.

约束 $C_{\text{prec}}(\phi)$、$C_{\text{kin}}(\phi)$、$C_{\text{col}}(\phi)$ 的更细分解如下。

$$\displaystyle\min_{\phi,\sigma,\pi}\ \\ \displaystyle\vec{F}(\phi,\sigma) \\ \displaystyle\forall i\in[1,n], \\ \displaystyle C_{\text{prec}}(\phi_{1:i})\leq 0 \\ \displaystyle C_{\text{kin}}(o_{a,i},g_{a,i},\tau_{a,i}^{g}(0),p^{0}_{o_{i}})=0 \\ \displaystyle C_{\text{kin}}(o_{a,i},g_{a,i},\tau_{a,i}^{f}(1),p^{G}_{o_{i}})=0 \\ \displaystyle C_{\text{kin}}(o_{h,i},g_{h,i},\tau_{h,i}^{f}(1),p^{G}_{o_{i}})=0 \\ \displaystyle C_{\text{col}}(\phi_{1:i},\pi[a,i],\pi[h,i])\leq 0 \tag{3a}$$

where each motion $\tau:[0,1]\rightarrow\mathcal{Q}$ is a time-parametrized joint trajectory in the robot’s configuration space $\mathcal{Q}$. Eq. 3a are the precedence constraints that ensure the partially assembled parts are connected and do not collapse under gravity. Eqs. 3b, 3c and 3d ensure the robot configuration, grasp, and the grasped object pose are kinematically consistent when picking, assembling, and holding. Eq. 3e ensures that both robots’ trajectories do not collide with previously assembled parts and other static obstacles.

其中每条运动 $\tau:[0,1]\rightarrow\mathcal{Q}$ 是机器人构型空间 $\mathcal{Q}$ 中的时间参数化关节轨迹。式 3a 是优先级约束，保证部分已装配零件相连且不在重力下坍塌。式 3b、3c 与 3d 保证取件、装配与持件时机器人构型、抓取与被抓物体位姿运动学一致。式 3e 保证双臂轨迹不与已装配零件及其他静态障碍碰撞。

The key planning stages in Sec. 3 are further summarized here.

第 3 节关键规划阶段在此再作摘要。

• • Section 3.1: Starting from a multi-part mesh model, we construct a precedence graph representing a minimum constraint set for any feasible sequence, considering only collision among parts.

• • 第 3.1 节：从多零件网格模型出发，构造优先级图，表示任意可行序列的最小约束集，仅考虑零件间碰撞。

• • Section 3.2: To reduce online collision checking overhead during search, we pre-compute a discretized, collision-free grasp set for assembling and holding for all part pairs. Each feasible grasp is associated with a corresponding robot trajectory.

• • 第 3.2 节：为降低搜索中的在线碰撞检查开销，我们为所有零件对预计算离散、无碰撞的装配与持件抓取集。每个可行抓取关联对应机器人轨迹。

• • Section 3.3: We leverage the precedence graph and precomputed grasp pairs to expand a state tree that contains all feasible part-grasp sequences and then search for an optimal part-grasp sequence that minimizes a grasp stability cost.

• • 第 3.3 节：利用优先级图与预计算抓取对扩展包含全部可行零件–抓取序列的状态树，再搜索使抓取稳定性代价最小的最优零件–抓取序列。

• • Section 3.4: After the grasps are determined, we develop an automatic design algorithm to generate a pickup fixture, so that the planned grasp can be achieved easily without the need of a re-grasp.

• • 第 3.4 节：抓取确定后，开发自动设计算法生成取件夹具，使规划抓取易于实现而无需重抓。

• • Section 3.5: With the assembly sequence fixed and all robot configurations determined for kinematic switches, i.e., pick-up, assembly, and hold, we plan for all transit and transfer motions.

• • 第 3.5 节：装配顺序固定、取件/装配/持件等运动学切换处的机器人构型均已确定后，规划全部转移与搬运运动。

Appendix C Algorithmic Details

附录 C 算法细节

We now present detailed mathematical formulations of each algorithm introduced in Sec. 3.

下面给出第 3 节各算法的详细数学表述。

C.1 Part Precedence Planning

C.1 零件优先级规划

Algorithm 1 and the following paragraphs provide the details of the part precedence planning algorithm.

算法 1 及随后段落给出零件优先级规划算法的细节。

Precedence Tier Generation

Initially, an empty list of tiers $T_{\text{prec}}$ is initialized. We use $O_{r}$ to represent all the remaining assembled parts, which starts from all parts $O$. Although $O_{r}$ is not empty, the algorithm constructs a new tier $t_{\text{prec}}$ by evaluating each part $o_{i}\in O_{r}$ to determine the feasibility of disassembly. Using motion planning, a disassembly path $q_{i}$ is computed for $o_{i}$. Specifically, we apply Assemble-Them-All [10], a physics-based method for efficient disassembly motion planning given the highly constrained search space, and determine success based on a given timeout. If $o_{i}$ can be disassembled, the pair $(o_{i},q_{i})$ is added to $t_{\text{prec}}$. Once all feasible parts are processed, the constructed tier $t_{\text{prec}}$ is added to $T_{\text{prec}}$, and all $o_{i}\in t_{\text{prec}}$ is removed from $O_{r}$. This process repeats until all parts are assigned to tiers.

优先级层生成

首先初始化空的层列表 $T_{\text{prec}}$。用 $O_{r}$ 表示剩余已装配零件，初始为全部零件 $O$。只要 $O_{r}$ 非空，算法就通过评估每个 $o_{i}\in O_{r}$ 的拆卸可行性构造新层 $t_{\text{prec}}$。用运动规划为 $o_{i}$ 计算拆卸路径 $q_{i}$。具体采用 Assemble-Them-All [10]——在高度约束搜索空间中做高效拆卸运动规划的基于物理方法——并按给定超时判定成功。若 $o_{i}$ 可拆卸，则将 $(o_{i},q_{i})$ 加入 $t_{\text{prec}}$。处理完全部可行零件后，把 $t_{\text{prec}}$ 加入 $T_{\text{prec}}$，并从 $O_{r}$ 移除其中全部 $o_{i}$。重复直至所有零件分入各层。

Precedence Graph Generation

The graph construction phase generates a directed graph $G_{\text{prec}}$ that encodes precedence constraints among parts. An empty set $O_{e}$ is initialized to track parts in earlier tiers (i.e., parts that are supposed to be disassembled earlier). For each tier $t_{\text{prec}}\in T_{\text{prec}}$, the algorithm processes every part $o_{i}\in t_{\text{prec}}$ by checking its disassembly path $\tau_{o_{i}}$ for collisions with parts in earlier tiers $O_{e}$. A collision check function identifies the set of colliding parts $O_{c}\subseteq O_{e}$. For each $o_{c}\in O_{c}$, an edge $(o_{i},o_{c})$ is added to $G_{\text{prec}}$, indicating that $o_{i}$’s disassembly depends on $o_{c}$’s prior removal. After processing all parts in $t_{\text{prec}}$, the disassembled parts $O_{t}$ are added to $O_{e}$. The algorithm proceeds until all parts in all precedence tiers are added to the precedence graph.

优先级图生成

图构造阶段生成编码零件间优先级约束的有向图 $G_{\text{prec}}$。初始化空集 $O_{e}$ 以跟踪较早层中的零件（即应更早拆卸的零件）。对每一层 $t_{\text{prec}}\in T_{\text{prec}}$，算法处理每个 $o_{i}\in t_{\text{prec}}$：检查其拆卸路径 $\tau_{o_{i}}$ 与较早层零件 $O_{e}$ 的碰撞。碰撞检查函数识别碰撞零件集 $O_{c}\subseteq O_{e}$。对每个 $o_{c}\in O_{c}$，向 $G_{\text{prec}}$ 添加边 $(o_{i},o_{c})$，表示 $o_{i}$ 的拆卸依赖于 $o_{c}$ 先被移除。处理完 $t_{\text{prec}}$ 中全部零件后，将拆卸零件 $O_{t}$ 加入 $O_{e}$。算法继续直至所有优先级层中的零件都进入优先级图。

1: Input: All parts $O$ with goal poses $p_{O}^{G}$
2: Output: Directed graph $G_{\text{prec}}$ representing precedence constraints
3: Initialize an empty list of tiers: $T_{\text{prec}}\leftarrow\texttt{[]}$
4: Initialize an empty directed graph: $G_{\text{prec}}\leftarrow\text{DiGraph}()$
5: Initialize all remaining parts $O_{r}\leftarrow O$
6: while $O_{r}\neq\emptyset$ do $\triangleright$ Tier generation
7:   Initialize an empty tier: $t_{\text{prec}}\leftarrow\{\}$
8:   for each part $o_{i}\in O_{r}$ do
9:    $\tau_{o_{i}}\leftarrow\text{DisassemblyPath}(o_{i},O_{r})$
10:    if feasible $\tau_{o_{i}}$ is found then
11:      $t_{\text{prec}}\leftarrow t_{\text{prec}}\cup\{(o_{i},\tau_{o_{i}})\}$
12:   $O_{r}\leftarrow O_{r}\setminus\{o_{i}\mid(o_{i},\tau_{o_{i}})\in t_{\text{prec}}\}$
13:   $T_{\text{prec}}.\text{Append}(t_{\text{prec}})$
14: Initialize an empty set of parts in earlier tiers: $O_{e}\leftarrow\emptyset$
15: for each tier $t_{\text{prec}}\in T_{\text{prec}}$ do $\triangleright$ Graph construction
16:   Let $O_{t}\leftarrow\{o_{i}\mid(o_{i},\tau_{o_{i}})\in t_{\text{prec}}\}$ $\triangleright$ Parts in tier $t_{\text{prec}}$
17:   for each $(o_{i},\tau_{o_{i}})\in t_{\text{prec}}$ do
18:    $G_{\text{prec}}.\text{AddNode}(o_{i},\text{path}=\tau_{o_{i}})$
19:    $O_{c}\leftarrow\text{CheckPathCollision}(o_{i},\tau_{o_{i}},O_{e})$ $\triangleright$ $O_{c}$: colliding parts in $O_{e}$
20:    for each $o_{c}\in O_{c}$ do
21:      $G_{\text{prec}}.\text{AddEdge}(o_{i},o_{c})$
22:   $O_{e}\leftarrow O_{\text{e}}\cup O_{t}$ $\triangleright$ Update parts in earlier tiers
23: return $G_{\text{prec}}$

Algorithm 1 Part Precedence Planning

1: 输入： 全部零件 $O$ 及其目标位姿 $p_{O}^{G}$
2: 输出： 表示优先级约束的有向图 $G_{\text{prec}}$
3: 初始化空层列表：$T_{\text{prec}}\leftarrow\texttt{[]}$
4: 初始化空有向图：$G_{\text{prec}}\leftarrow\text{DiGraph}()$
5: 初始化全部剩余零件 $O_{r}\leftarrow O$
6: while $O_{r}\neq\emptyset$ do $\triangleright$ 层生成
7:   初始化空层：$t_{\text{prec}}\leftarrow\{\}$
8:   for 每个零件 $o_{i}\in O_{r}$ do
9:    $\tau_{o_{i}}\leftarrow\text{DisassemblyPath}(o_{i},O_{r})$
10:    if 找到可行 $\tau_{o_{i}}$ then
11:      $t_{\text{prec}}\leftarrow t_{\text{prec}}\cup\{(o_{i},\tau_{o_{i}})\}$
12:   $O_{r}\leftarrow O_{r}\setminus\{o_{i}\mid(o_{i},\tau_{o_{i}})\in t_{\text{prec}}\}$
13:   $T_{\text{prec}}.\text{Append}(t_{\text{prec}})$
14: 初始化较早层零件空集：$O_{e}\leftarrow\emptyset$
15: for 每一层 $t_{\text{prec}}\in T_{\text{prec}}$ do $\triangleright$ 图构造
16:   令 $O_{t}\leftarrow\{o_{i}\mid(o_{i},\tau_{o_{i}})\in t_{\text{prec}}\}$ $\triangleright$ 层 $t_{\text{prec}}$ 中的零件
17:   for 每个 $(o_{i},\tau_{o_{i}})\in t_{\text{prec}}$ do
18:    $G_{\text{prec}}.\text{AddNode}(o_{i},\text{path}=\tau_{o_{i}})$
19:    $O_{c}\leftarrow\text{CheckPathCollision}(o_{i},\tau_{o_{i}},O_{e})$ $\triangleright$ $O_{c}$：$O_{e}$ 中的碰撞零件
20:    for 每个 $o_{c}\in O_{c}$ do
21:      $G_{\text{prec}}.\text{AddEdge}(o_{i},o_{c})$
22:   $O_{e}\leftarrow O_{\text{e}}\cup O_{t}$ $\triangleright$ 更新较早层零件
23: return $G_{\text{prec}}$

算法 1 零件优先级规划

C.2 Dual-Arm Grasp Filtering

C.2 双臂抓取过滤

This section provides a detailed description of the sub-steps involved in the grasp filtering algorithm.

本节详细描述抓取过滤算法中的子步骤。

Single-Pose Grasp Feasibility Check

Alg. 2 evaluates whether a specific grasp configuration $g$ is feasible for a target part $o$ in its current pose $p_{o}$. The algorithm first determines the set of preceding parts $O_{\text{prec}}$ from the precedence graph $G_{\text{prec}}$. For each gripper aperture ($a_{\text{grasp}}$ and $a_{\text{release}}$), collision checks are performed involving the robot body, the gripper, the target part $o$, the preceding parts $O_{\text{prec}}$, and the environment obstacles $E$. If any collision occurs between them, the grasp $g$ is not feasible. Additional collision checks are performed between the gripper and other non-preceding parts $O_{\text{all}}\setminus O_{\text{prec}}$, and collision information is added to the grasp. Such collisions are not hard constraints because the non-preceding parts may get assembled later than the target part depending on the specific assembly sequence. In this case, the collision does not matter, but the information has to be recorded to find collision-free assembly sequences in the later stage.

单位姿抓取可行性检查

算法 2 评估特定抓取配置 $g$ 对目标零件 $o$ 在其当前位姿 $p_{o}$ 下是否可行。算法先从优先级图 $G_{\text{prec}}$ 确定前驱零件集 $O_{\text{prec}}$。对每个夹爪开口（$a_{\text{grasp}}$ 与 $a_{\text{release}}$），在机器人本体、夹爪、目标零件 $o$、前驱零件 $O_{\text{prec}}$ 与环境障碍 $E$ 之间做碰撞检查。若任一对发生碰撞，则抓取 $g$ 不可行。另外还检查夹爪与其他非前驱零件 $O_{\text{all}}\setminus O_{\text{prec}}$ 的碰撞，并把碰撞信息写入抓取。这类碰撞不是硬约束，因为非前驱零件可能按具体装配序列晚于目标零件装配；此时碰撞无关紧要，但信息必须记录，以便后续寻找无碰撞装配序列。

1: Input: Grasp $g$, robot $R$, target part $o$ with pose $p_{o}$, all parts $o\in O$ with their corresponding goal poses $p_{o}^{G}$, precedence graph $G_{\text{prec}}$
2: Output: Feasibility (True/False), updated grasp $g$ with collision and IK information
3: $O_{\text{prec}}\leftarrow G_{\text{prec}}.\text{PrecedingParts}(o)$
4: $E\leftarrow$ environment obstacles
5: $q_{g}^{R}\leftarrow\text{IK}(R,g,o,p_{o})$ $\triangleright$ IK for robot to grasp part $o$ under pose $p_{o}$ with grasp $g$
6: if feasible $q_{g}^{R}$ is found then
7:   Set robot $R$ configuration to $q_{g}^{R}$
8:   for each gripper aperture $a\in\{a_{\text{grasp}},a_{\text{release}}\}$ do
9:    Set gripper with aperture $a$
10:    if $\text{CheckCollision}(R,o,O_{\text{prec}},E)$ then
11:      return False, $g$
12:    $\text{CheckCollision}(R,O\setminus O_{\text{prec}})$
13:   Record collision and IK information to $g$
14:   return True, $g$
15: else
16:   return False, $g$

Algorithm 2 Single-Pose Grasp Feasibility Check

1: 输入： 抓取 $g$，机器人 $R$，目标零件 $o$ 及其位姿 $p_{o}$，全部零件 $o\in O$ 及其目标位姿 $p_{o}^{G}$，优先级图 $G_{\text{prec}}$
2: 输出： 可行性（True/False），带碰撞与 IK 信息的更新抓取 $g$
3: $O_{\text{prec}}\leftarrow G_{\text{prec}}.\text{PrecedingParts}(o)$
4: $E\leftarrow$ 环境障碍
5: $q_{g}^{R}\leftarrow\text{IK}(R,g,o,p_{o})$ $\triangleright$ 机器人在位姿 $p_{o}$ 下以抓取 $g$ 抓取零件 $o$ 的 IK
6: if 找到可行 $q_{g}^{R}$ then
7:   将机器人 $R$ 构型设为 $q_{g}^{R}$
8:   for each 夹爪开口 $a\in\{a_{\text{grasp}},a_{\text{release}}\}$ do
9:    设置夹爪开口为 $a$
10:    if $\text{CheckCollision}(R,o,O_{\text{prec}},E)$ then
11:      return False, $g$
12:    $\text{CheckCollision}(R,O\setminus O_{\text{prec}})$
13:   将碰撞与 IK 信息记录到 $g$
14:   return True, $g$
15: else
16:   return False, $g$

算法 2 单位姿抓取可行性检查

Assembling Grasp Feasibility Check

Alg. 3 determines the feasibility of using a grasp $g$ to disassemble a target part $o$ along its disassembly path $\tau_{o}$, derived from $G_{\text{prec}}$. For each pose $p_{o,t}$ along $\tau_{o}$, the grasp is transformed accordingly and its feasibility is validated using Alg. 2. The aggregated collision and IK information across all poses is stored in $g$. The grasp is feasible if all poses along $\tau_{o}$ are validated.

装配抓取可行性检查

算法 3 判定用抓取 $g$ 沿来自 $G_{\text{prec}}$ 的拆卸路径 $\tau_{o}$ 拆卸目标零件 $o$ 是否可行。对 $\tau_{o}$ 上每个位姿 $p_{o,t}$，相应变换抓取并用算法 2 验证可行性。跨全部位姿聚合的碰撞与 IK 信息存入 $g$。若 $\tau_{o}$ 上全部位姿均通过验证，则抓取可行。

1: Input: Grasp $g$, robot $R$, target part $o$, all parts $O$ with goal poses $p_{O}^{G}$, precedence graph $G_{\text{prec}}$
2: Output: Feasibility (True/False), updated grasp $g$ with collision and IK information
3: $\tau_{o}\leftarrow G_{\text{prec}}.\text{GetPath}(o)$ $\triangleright$ Disassembly path of part $o$
4: for each part pose $p_{o,t}\in\tau_{o}$ do $\triangleright$ Disassembling $o$ while grasping $o$
5:   Transform grasp $g$ according to $p_{o}^{t}$ to obtain $g_{t}$
6:   feas, $g_{t}\leftarrow$ CheckGraspFeas($g_{t}$, $o$, $p_{o,t}$, …) $\triangleright$ Alg. 2
7:   if not feas then
8:    return False, $g$
9:   Gather collision and IK information from $g_{t}$ to $g$
10: return True, $g$

Algorithm 3 Assembling Grasp Feasibility Check

1: 输入： 抓取 $g$，机器人 $R$，目标零件 $o$，全部零件 $O$ 及其目标位姿 $p_{O}^{G}$，优先级图 $G_{\text{prec}}$
2: 输出： 可行性（True/False），带碰撞与 IK 信息的更新抓取 $g$
3: $\tau_{o}\leftarrow G_{\text{prec}}.\text{GetPath}(o)$ $\triangleright$ 零件 $o$ 的拆卸路径
4: for each 零件位姿 $p_{o,t}\in\tau_{o}$ do $\triangleright$ 抓取 $o$ 的同时拆卸 $o$
5:   按 $p_{o}^{t}$ 变换抓取 $g$ 得到 $g_{t}$
6:   feas, $g_{t}\leftarrow$ CheckGraspFeas($g_{t}$, $o$, $p_{o,t}$, …) $\triangleright$ 算法 2
7:   if not feas then
8:    return False, $g$
9:   从 $g_{t}$ 收集碰撞与 IK 信息到 $g$
10: return True, $g$

算法 3 装配抓取可行性检查

Holding Grasp Feasibility Check

Alg. 4 evaluates whether a grasp $g$ can securely hold a part $o$ while allowing other parts $o_{i}$ to be disassembled. The feasibility of $g$ is first validated using Alg. 2.

持件抓取可行性检查

算法 4 评估抓取 $g$ 能否在允许其他零件 $o_{i}$ 被拆卸的同时稳固持住零件 $o$。先用算法 2 验证 $g$ 的可行性。

1: Input: Grasp $g$, robot $R$, target part $o$, all parts $O$ with goal poses $p_{O}^{G}$, precedence graph $G_{\text{prec}}$
2: Output: Feasibility (True/False), updated grasp $g$ with collision and IK information
3: feas, $g\leftarrow$ CheckGraspFeas($g$, $o$, $p_{o}^{G}$, …) $\triangleright$ Alg. 2
4: if not feas then
5:   return False, $g$
6: for each part $o_{i}\in O_{\text{all}}\setminus\{o\}$ do
7:   $q_{o_{i}}\leftarrow G_{\text{prec}}.\text{GetPath}(o_{i})$ $\triangleright$ Disassembly path of $o_{i}$
8:   for each part pose $p_{o_{i},t}\in q_{o_{i}}$ do $\triangleright$ Disassembling $o_{i}$ while grasping $o$
9:    CheckCollision($R$, $o_{i}$) and gather collision information to $g$
10: return True, $g$

Algorithm 4 Holding Grasp Feasibility Check

1: 输入： 抓取 $g$，机器人 $R$，目标零件 $o$，全部零件 $O$ 及其目标位姿 $p_{O}^{G}$，优先级图 $G_{\text{prec}}$
2: 输出： 可行性（True/False），带碰撞与 IK 信息的更新抓取 $g$
3: feas, $g\leftarrow$ CheckGraspFeas($g$, $o$, $p_{o}^{G}$, …) $\triangleright$ 算法 2
4: if not feas then
5:   return False, $g$
6: for each 零件 $o_{i}\in O_{\text{all}}\setminus\{o\}$ do
7:   $q_{o_{i}}\leftarrow G_{\text{prec}}.\text{GetPath}(o_{i})$ $\triangleright$ $o_{i}$ 的拆卸路径
8:   for each 零件位姿 $p_{o_{i},t}\in q_{o_{i}}$ do $\triangleright$ 抓取 $o$ 的同时拆卸 $o_{i}$
9:    CheckCollision($R$, $o_{i}$) 并将碰撞信息收集到 $g$
10: return True, $g$

算法 4 持件抓取可行性检查

Grasp Pair Filtering

Putting the assembling and holding feasibility checks together, Alg. 5 generates and filters the dual-arm grasp pairs. For each part $o_{i}$, a set of candidate grasp poses $\{g_{k}\}_{k=1}^{N_{g}}$ is generated. Each grasp is evaluated for assembling and holding feasibility using Algs. 3 and 4, respectively, and feasible grasps are stored in $\mathcal{G}^{a}[o_{i}]$ and $\mathcal{G}^{h}[o_{i}]$. Finally, iterating through all assembly-hold part pairs, the set of feasible assembly-hold grasp pairs $\mathcal{G}^{a\times h}[o_{a},o_{h}]$ only contains those that do not lead to collisions between the two robots.

抓取对过滤

综合装配与持件可行性检查，算法 5 生成并过滤双臂抓取对。对每个零件 $o_{i}$，生成候选抓取位姿集 $\{g_{k}\}_{k=1}^{N_{g}}$。每个抓取分别用算法 3 与 4 评估装配与持件可行性，可行者存入 $\mathcal{G}^{a}[o_{i}]$ 与 $\mathcal{G}^{h}[o_{i}]$。最后遍历全部装配–持件零件对，可行装配–持件抓取对集合 $\mathcal{G}^{a\times h}[o_{a},o_{h}]$ 仅保留不会导致双臂碰撞的那些。

1: Input: All parts $O$ with goal poses $p_{O}^{G}$, robots $R_{a},R_{h}$, precedence graph $G_{\text{prec}}$
2: Output: Assembling grasps $\mathcal{G}^{a}$, holding grasps $\mathcal{G}^{h}$, assembling-holding grasp pairs $\mathcal{G}^{a\times h}$
3: Initialize empty dictionaries $\mathcal{G}^{a},\mathcal{G}^{h},\mathcal{G}^{a\times h}\leftarrow\{:\},\{:\},\{:\}$
4: for each part $o_{i}$ in $O$ do $\triangleright$ Feasible grasp generation
5:   $\mathcal{G}^{a}[o_{i}],\mathcal{G}^{h}[o_{i}]\leftarrow\{\},\{\}$
6:   Generate $N_{g}$ grasp poses $\{g_{k}\}_{k=1}^{N_{g}}$ on part $o_{i}$
7:   for each grasp pose $g_{k}$ do
8:    $\text{feas}_{a},g_{a}\leftarrow\text{CheckAssemGraspFeas}(g_{k},R_{a},o_{i},...)$
9:    if $\text{feas}_{a}$ then
10:      $\mathcal{G}^{a}[o_{i}]\leftarrow\mathcal{G}^{a}[o_{i}]\cup\{g_{a}\}$
11:    $\text{feas}_{h},g_{h}\leftarrow\text{CheckHoldGraspFeas}(g_{k},R_{h},o_{i},...)$
12:    if $\text{feas}_{h}$ then
13:      $\mathcal{G}^{h}[o_{i}]\leftarrow\mathcal{G}^{h}[o_{i}]\cup\{g_{h}\}$
14: for each part $o_{a}\in O$ do $\triangleright$ Feasible grasp pair filtering
15:   for each part $o_{h}\in O$ do
16:    if $o_{a}\in G_{\text{prec}}.\text{PrecedingParts}(o_{h})$ then
17:      continue    $\mathcal{G}^{a\times h}[(o_{a},o_{h})]\leftarrow\{\}$
18:    for each grasp $g_{a}\in\mathcal{G}^{a}[o_{a}]$ do
19:      for each grasp $g_{h}\in\mathcal{G}^{h}[o_{h}]$ do
20:       Set robot $R_{a}$ configuration to $q_{g_{a}}^{R_{a}}$
21:       Set robot $R_{h}$ configuration to $q_{g_{h}}^{R_{h}}$
22:       if not CheckCollision$(R_{a},R_{h})$ then
23:         $\mathcal{G}^{a\times h}[(o_{a},o_{h})]\leftarrow\mathcal{G}^{a\times h}[(o_{a},o_{h})]\cup\{(g_{a},g_{h})\}$
24: return $\mathcal{G}^{a},\mathcal{G}^{h},\mathcal{G}^{a\times h}$

Algorithm 5 Dual-Arm Grasp Pair Filtering