Transporter Networks: Rearranging the Visual World for Robotic Manipulation

Transporter Networks：为机器人操作重排视觉世界（Transporter Networks: Rearranging the Visual World for Robotic Manipulation）

Andy Zeng    Pete Florence    Jonathan Tompson    Stefan Welker    Jonathan Chien Affiliation:  Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong Affiliation:  Ayzaan Wahid, Vikas Sindhwani, Johnny Lee Affiliation:  Robotics at Google

Andy Zeng Pete Florence Jonathan Tompson Stefan Welker Jonathan Chien Maria Attarian Travis Armstrong Ivan Krasin Dan Duong Ayzaan Wahid Vikas Sindhwani Johnny Lee
所属机构：Robotics at Google

Abstract

摘要

Robotic manipulation can be formulated as inducing a sequence of spatial displacements: where the space being moved can encompass an object, part of an object, or end effector. In this work, we propose the Transporter Network, a simple model architecture that rearranges deep features to infer spatial displacements from visual input – which can parameterize robot actions. It makes no assumptions of objectness (e.g. canonical poses, models, or keypoints), it exploits spatial symmetries, and is orders of magnitude more sample efficient than our benchmarked alternatives in learning vision-based manipulation tasks: from stacking a pyramid of blocks, to assembling kits with unseen objects; from manipulating deformable ropes, to pushing piles of small objects with closed-loop feedback. Our method can represent complex multi-modal policy distributions and generalizes to multi-step sequential tasks, as well as 6DoF pick-and-place. Experiments on 10 simulated tasks show that it learns faster and generalizes better than a variety of end-to-end baselines, including policies that use ground-truth object poses. We validate our methods with hardware in the real world. Experiment videos and code will be made available at https://transporternets.github.io

机器人操作可表述为诱导一串空间位移：被移动的空间可以是物体、物体的一部分，或末端执行器。本文提出 Transporter Network：一种简单模型架构，通过重排深度特征，从视觉输入推断空间位移，从而参数化机器人动作。它不对「物体性」做假设（例如标准姿态、模型或关键点），利用空间对称性，在学习基于视觉的操作任务上比我们对照的基线样本效率高出数个数量级：从堆叠积木金字塔，到用未见过的物体组装工具套件；从操作可变形绳索，到用闭环反馈推动小物堆。方法能表示复杂的多峰策略分布，并泛化到多步序列任务以及 6DoF 抓放。在 10 个仿真任务上，它比多种端到端基线（含使用真值物体位姿的策略）学得更快、泛化更好。我们在真实硬件上验证了方法。实验视频与代码见 https://transporternets.github.io

Keywords: Deep Learning, Vision, Manipulation

关键词： 深度学习、视觉、操作

1 Introduction

1 引言

End-to-end models that map directly from pixels to actions hold the capacity to learn complex manipulation skills, but are known to require copious amounts of data [1, 2]. Integrating object-centric assumptions – e.g., object keypoints [3, 4, 5, 6], embeddings [7, 8], or dense descriptors [9, 10, 11] – has been shown to improve sample efficiency [10]. However, these representations often impose data collection burdens (i.e., configuring scenes with specific singulated objects) and still struggle to address challenging scenarios with unseen classes of objects, occluded objects, highly deformable objects, or piles of small objects [12]. This naturally leads us to ask: is there structure that we can incorporate into our end-to-end models to improve their learning efficiency, without imposing any of the limitations or burdens of explicit object representations?

从像素直接映射到动作的端到端模型具有学习复杂操纵技能的能力,但已知需要大量的数据[1,2]. 综合以对象为中心的假设——例如对象键点[3,4,5,6],嵌入式[7,8],或密集的描述符[9,10,11]——已被证明可以提高样本效率[10]. 然而,这些表述往往会给数据收集带来负担(即将场景配置为特定的单独设定的物体),并且仍然以隐形的物体类别,隐形物体,极易变形的物体,或小物体堆积而成[12]来努力解决具有挑战性的情景. 这自然导致我们问:我们能否在端到端模型中纳入结构,以提高其学习效率,而不强加任何明确对象表述的限制或负担?

In this work, we propose the Transporter Network, a simple end-to-end model architecture that preserves spatial structure for vision-based manipulation, without object-centric assumptions:

在这部作品中,我们提出Transporter Network(Transporter Network),这是一个简单的端对端模型架构,它保留了空间结构用于基于视觉的操纵,没有以对象为中心的假设:

• • Manipulation involves rearranging things, which can be thought of as executing a sequence of spatial displacements: where the space being moved (i.e., transported) can encompass an object(s), part of an object, or end effector. We formulate vision for manipulation as estimating these displacements. Transporter Networks directly optimize for this by learning to 1) attend to a local region, and 2) predict its target spatial displacement via deep feature template matching – which then parameterizes robot actions for manipulation. This formulation enables high-level perceptual reasoning about which visual cues are important, and how they should be rearranged in a scene – the distributions of which can be learned from demonstrations.
• • Transporter Networks preserve the 3D spatial structure of the visual input. Prior end-to-end models [1, 2] often use convolutional architectures with raw images, in which valuable spatial information can be lost to perspective distortions. Our method uses 3D reconstruction to project visual data onto a spatially-consistent representation as input, with which it is able to better exploit equivariance [13, 14] for inductive biases that are present within the geometric symmetries [15] of the data for more efficient learning.

• 操纵涉及重新排列事物,这可以被认为是执行一系列空间转移:所移动的空间(即运输的空间)可以包括物体、物体的一部分或终端效应器。 我们为操纵制定设想,以估计这些流离失所情况。 Transporter Network通过学习来直接优化,1)关注一个局部区域,2)通过深度特性模板匹配预测其目标空间位移 — — 然后将机器人动作参数化用于操纵. 这种提法使得高层次的感知推理能够了解哪些视觉提示很重要,以及如何在某一场景中重新排列这些提示 — — 其分布可以从演示中得知。
• 运输网络维护视觉输入的3D空间结构。 之前的端到端模型[1,2]经常使用带有原始图像的演化结构,其中有价值的空间信息可能会丢失给视角扭曲. 我们的方法是用三维重建将视觉数据投射到空间一致的表示上,作为输入,它能够更好地利用等效性[13,14],用于在数据的几何对称性[15]中存在的诱导偏差,以便更有效地学习。

In our experiments, Transporter Networks exhibit superior sample efficiency on a number of tabletop manipulation tasks that involve changing the state of the robot’s environment in a purposeful manner: from stacking a pyramid of blocks, to assembling kits with unseen objects; from manipulating deformable ropes, to pushing piles of small objects with closed-loop feedback. Transporter Networks excel in modeling multi-modal spatial action distributions, and by construction generalize across rotations and translations of objects. They do not require any prior knowledge of the objects to be manipulated – they rely only on information contained within partial RGB-D data from demonstrations, and are capable of generalizing to new objects and configurations, and for some tasks, one-shot learning from a single demonstration.

在我们的实验中,Transporter Network在涉及有目的地改变机器人环境状况的一些桌面操作任务上表现出了优越的样本效率:从堆积一块金字塔,到用隐形物体组装工具箱;从操纵可变形的绳索,到用闭路反馈推压一堆小物体。 Transporter Network在模拟多模式空间行动分布方面表现突出,通过建筑将物体的旋转和翻译普遍化。 它们不需要事先对被操纵的对象有任何了解 — — 它们只依赖于演示中部分RGB-D数据所包含的信息,并且能够概括为新的对象和配置,对于一些任务,从单一演示中学习一弹一弹.

Our main contribution is a new perspective on the role of spatial structure and its capacity to improve end-to-end learning for vision-based manipulation. We propose a simple model architecture that learns to attend to a local region and predict its spatial displacement, while retaining the spatial structure of the visual input. On 10 unique tabletop manipulation tasks, Transporter Networks trained from scratch are capable of achieving greater than 90% success on most tasks with objects in new configurations using 100 expert demonstrations, while other end-to-end alternatives struggle to generalize with the same amount of data. We also develop an extension to 6DoF tasks by combining 3DoF Transporter Networks with continuous regression to handle the remaining degrees of freedom. To facilitate further research in vision-based manipulation, we plan release code and open-source Ravens, our new simulated benchmark with all tasks. Ravens features a Gym-like API [16] with a built-in stochastic oracle to evaluate the sample efficiency of imitation learning methods.

我们的主要贡献是重新审视空间结构的作用及其改进端到端学习的能力,以便进行基于愿景的操纵。 我们提出一个简单的模型架构,学习关注一个局部区域并预测其空间位移,同时保留视觉输入的空间结构. 在10个独特的桌面操作任务上,从零开始训练的运算器网络能够用100个专家演示的新的配置对象在大多数任务上取得超过90%的成功,而其他端到端的替代品则会用同样数量的数据进行归纳. 我们还通过将3DOF Transporter Networks与持续回归相结合来开发扩展至6DoF任务,以处理剩余自由度. 为了促进基于视觉的操纵的进一步研究,我们计划发布代码和开源Ravens,这是我们所有任务的新模拟基准. Ravens具有类似Gym的API[16]的特征,带有内置的纹理甲骨文,用来评价模仿学习方法的样本效率.

Figure 1: A Transporter Network is a simple model architecture that attends to a local region and predicts its spatial displacement (b) from visual input – which can parameterize robot actions. It is sample efficient in learning complex vision-based manipulation tasks: inserting blocks into fixtures (a), sequential pick-and-place in Towers of Hanoi (c), assembling kits with unseen objects (d), palletizing boxes (e), stacking a pyramid of blocks (f), manipulating rope (g), and pushing piles of small objects with closed-loop feedback (h) – and is practical to deploy on real production robots (k, m).

图1:A Transporter Network是一个简单的模型架构,它关注一个局部区域,并预测其空间位移(b)来自视觉输入——它可以参数化机器人动作. 它在学习复杂的基于视觉的操纵任务方面是高效的样本:在固定装置(a)中插入块,在河内塔(c)中依次选择和放置,用隐形物体组装包(d),托盘箱(e),堆叠一个金字塔(f),操纵绳(g),用闭路回馈(h)将小物体堆积起来(k,m)——并实用地部署在真正的生产机器人上。

2 Related Work

2 相关工作

Object-centric Representations. Considerable research in vision for manipulation has been devoted to object detectors and pose estimators [17, 18, 19, 20, 21]. These methods often require object-specific training data, making them difficult to scale to applications with unseen objects. Alternative representations including keypoints [3, 4, 5, 6] and dense descriptors [9, 10, 11] have shown to be capable of category-level generalization and representing deformable objects, but still struggle to represent scenes with an unknown number of objects (e.g., piles of small objects) or occluded objects. End-to-end models for vision-based manipulation learn faster when integrated with such object-centric representations [10], providing sample-efficiency benefits but retaining the data-collection and representational constraints of object-centricness. In this work, we show that it is possible to achieve sample efficient end-to-end learning without object-centric representations – enabling the same model architecture to generalize to tasks with unseen objects, variable numbers of objects, deformable objects, and piles of small objects.

** 以客观为中心的表述。 ** 对操纵的视觉进行了大量研究,研究对象探测器和外观测量器[17、18、19、20、21]。 这些方法往往需要针对特定对象的训练数据,使得它们难以与不可见对象的应用进行规模化. 包括键点[ 你说什么, 4, 5, 6] 和稠密的描述符[9, 10, 11] 在内的替代表示符已经证明能够进行分类级的概括,并代表可变形物体,但仍然挣扎着用未知数量的物体(如小物体堆积)或隐形物体来代表场景. 以视觉为基础的操纵的端到端模型在与这种以物体为中心的表示法结合时学习得更快[10],提供了样本效率效益,但保留了以物体为中心的数据收集和表现限制. 在这项工作中,我们显示,在没有以对象为中心的表达方式的情况下,可以实现样本高效的端到端学习 — — 使同样的模型架构能够通用到包含不可见对象,对象的可变数量,可变对象,以及小对象堆积的任务.

Pick-and-Place has had a long history in robotics, motivated by industrial needs. Classic systems use pose estimation of known objects [17, 18], as well as scripted planning and motion control [22]. While robust in structured settings (e.g., manufacturing), these systems are difficult to deploy in unstructured settings (e.g., logistics, agriculture, household) – leading to renewed interest in leveraging learned models to achieve general pick-and-place policies [23, 24, 25, 26, 27, 28, 29, 30] that can handle unseen objects. We benchmark against the more recent Form2Fit [24] on multiple tasks, and demonstrate that Transporter Networks are better able to handle tasks that require precise placing, multi-step sequencing, and closed-loop visual feedback.

** Pick-and-Place**在机器人方面有着悠久的历史,其动机是工业需求. 经典系统使用对已知物体进行估计[17,18],以及脚本规划和运动控制[22]. 这些系统虽然在结构化环境(如制造业)中很强大,但很难在结构化环境(如物流,农业,家庭)中部署,导致人们重新关注利用学到的模型来实现一般的选址政策[23, 24, 25, 26, 27, 28, 29, 30] 能够处理看不见的对象. 我们参照最近Form2Fit [24] 对多个任务进行基准测试,并表明Transporter Network更有能力处理需要精确定位,多步骤排序和闭路视反馈的任务.

3 Method

3 方法

Consider the problem of learning pick-and-place actions $\mathbf{a}_{t}$ with a robot from visual observations $\mathbf{o}_{t}$:

考虑学习选择和放置动作的问题。

$$f(\mathbf{o}_{t})\rightarrow\mathbf{a}_{t}=(\mathcal{T}_{\textrm{pick}},\mathcal{T}_{\textrm{place}})\in\mathcal{A}$$

where $\mathcal{T}_{\textrm{pick}}$ is the pose of the end effector used to pick an object, and $\mathcal{T}_{\textrm{place}}$ to place the object. Both poses can be defined in SE(2) or SE(3), depending on the task and degrees of freedom available to the end effector. We can generalize beyond pick-and-place by considering manipulation with other motion primitives [22] that are also parameterized by two end effector poses. For example, we can parameterize pushing, where $\mathcal{T}_{\textrm{pick}}$ is instead the pose of the end effector at the start of the push, and $\mathcal{T}_{\textrm{place}}$ is where it moves to complete the push. In this work, we consider tasks that can be completed by a sequence of two-pose motion primitives. In the following section, we begin by describing our approach in the context of planar pick-and-place. We then discuss extensions to 6DoF pick-and-place, sequential multi-step tasks, and motion-primitive-based manipulation of deformable objects and piles of objects.

其中$\mathcal{T}_{\textrm{pick}}$是用于选择对象的终端效应器的姿势,而$\mathcal{T}_{\textrm{place}}$是用于定位对象的. 这两种姿势都可以在SE(2)或SE(3)中定义,这取决于最终效果器可得到的任务和自由度. 我们可以通过考虑用其他运动原始体来操纵, 来概括到选择和位置以外。 [22] 这些原始体同样被两个终端效应器所构成的参数化。 例如,我们可以参数化推力,其中$\mathcal{T}_{\textrm{pick}}$是推力开始时端效应器的姿势,而$\mathcal{T}_{\textrm{place}}$是它移动完成推力的地方. 在这部作品中,我们考虑了可以由两pose运动原始物序列完成的任务. 在下面一节中,我们首先介绍我们在平板取地方面的做法。 然后我们讨论扩展到6DoF采样和位置,顺序的多步任务,以及基于运动的对变形物体和堆积物体的操纵.

3.1 Learning to Transport

3.1 学习传输

Figure 2: Simple planar pick-and-place task where (i) there is a distribution of successful pick poses, and (ii) for each successful pick pose, there is a corresponding distribution of successful place poses.

图2: 简单平板挑位置任务,其中(一) 成功挑姿势分布,(二) 对每个成功挑姿势,有相应的成功挑姿势分布.

Consider the planar pick-and-place problem on the right, where $\mathcal{T}_{\textrm{pick}},\mathcal{T}_{\textrm{place}}\in\mathbb{R}^{2}$ are 2D coordinates. In this task, the goal is to pick up the red block with an immobilizing grasp (e.g., with a suction gripper), then place it into the fixture. This simple setting reveals two fundamental aspects of the pick-and-place problem: (i) there may be a distribution of successful pick poses (from which $\mathcal{T}_{\textrm{pick}}$ may be sampled e.g., on the surface of the block), and (ii) for each successful pick pose, there is a corresponding distribution of successful place poses (from which $\mathcal{T}_{\textrm{place}}$ may be sampled).

考虑右侧的平板取地问题,其中$\mathcal{T}_{\textrm{pick}},\mathcal{T}_{\textrm{place}}\in\mathbb{R}^{2}$为2D坐标. 在这个任务中,目标是用不动的抓取器(例如,用吸力钳)来捡起红色块,然后将其放入固定器中. 这种简单的设定揭示了采摘和选址问题的两个基本方面:(一) 可能存在成功的采摘表层分布(从中可以对$\mathcal{T}_{\textrm{pick}}$进行采样,例如在区块表面),(二) 对于每一个成功的采摘表层,都有相应的成功地层分布(从中可以对$\mathcal{T}_{\textrm{place}}$进行采样).

The goal of Transporter Networks is to recover these distributions from visual observations alone – without assumptions of objectness. While assigning canonical poses to the block and fixture might simplify the problem, there are many benefits to avoiding such object-centric representations – including the capacity to handle unseen objects, deformable objects, and uncountable piles of small objects. Hence we assume no prior information about the objects (e.g., no 3D models, poses, class categories, keypoints, etc.).

Transporter Networks的目标是单独从视觉观察中恢复这些分布——而不假设对象. 虽然给块和固定物分配划线姿势可能会简化问题,但避免这种以物体为中心的表述有许多好处 — — 包括处理看不见物体、变形物体和无法计算的小物体堆积的能力。 因此,我们假定没有关于物体的事先信息(例如,没有立体模型、姿势、类类别、关键点等)。

Transporter Networks decompose the problem into (i) picking and (ii) pick-conditioned placing:

赋权网(Transporter Networks)将问题分解为(一)采摘和(二)采摘有条件的放置:

$$f_{\textrm{pick}}(\mathbf{o}_{t})\rightarrow\mathcal{T}_{\textrm{pick}}\qquad\qquad\qquad\qquad f_{\textrm{place}}(\mathbf{o}_{t},\mathcal{T}_{\textrm{pick}})\rightarrow\mathcal{T}_{\textrm{place}}$$

We present a solution to both with Transporter Networks – but in this work, our primary contribution is on (ii), pick-conditioned placing via transporting. We first begin by briefly describing (i), picking.

我们向Transporter Network(Transporter Networks)提出解决方案, 我们首先简要说明(一),选择。

Learning Picking. Our visual observation $\mathbf{o}_{t}$ is a projection of the scene (e.g., reconstructed from RGB-D images), defined on a regular grid of pixels $\{(u,v)\}$ at timestep $t$ of a sequential rearrangement task. Through camera-to-robot calibration, we can correspond each pixel in $\mathbf{o}_{t}$ to a picking action at that location: $\mathcal{T}_{\textrm{pick}}\sim(u,v)\in\mathbf{o}_{t}$. The distribution of successful picks over pixels in $\mathbf{o}_{t}$ can be multi-modal in nature – particularly in cases when there are multiple instances of the same object in the scene, or when there are symmetries in the shapes of objects. As in prior works [31, 32, 33], Transporter Networks use fully convolutional networks (FCNs, commonly used in vision for image segmentation tasks [34]) to model an action-value function $\mathcal{Q}_{\textrm{pick}}((u,v)|\mathbf{o}_{t})$ that correlates with picking success (architecture in Sec. 3.2):

** 学习选择。 ** 我们的视觉观察$\mathbf{o}_{t}$是场景的投影(例如从RGB-D图像中重建),在按顺序重排任务的$t$时间步上定义在像素的常规网格上. 通过相机对机器人校准,我们可以将$\mathbf{o}_{t}$中的每个像素对应到该地点的采摘动作:$\mathcal{T}_{\textrm{pick}}\sim(u,v)\in\mathbf{o}_{t}$. 在$\mathbf{o}_{t}$中成功拾取像素的分布在性质上可以是多模式的——特别是在场景中出现多个相同物体的情况,或者物体形状的对称时. 与先前的作品[31,32,33]一样,Transporter Network使用全转录网络(FCN,在图像分割任务视觉中通常使用[34])来建模与选取成功相关的动作值函数$\mathcal{Q}_{\textrm{pick}}((u,v)|\mathbf{o}_{t})$(第3.2节中的architecture):

$$\mathcal{T}_{\textrm{pick}}=\argmax_{(u,v)}\ \mathcal{Q}_{\textrm{pick}}((u,v)|\mathbf{o}_{t})$$

FCNs are translationally equivariant by nature, which synergizes with spatial action representations: if an object to be picked in the scene is translated, then the picking pose also translates. Formally, equivariance here can be characterized as $f_{\textrm{pick}}(g\circ\mathbf{o}_{t})=g\circ f_{\textrm{pick}}(\mathbf{o}_{t})$, where $g$ is a translation. Spatial equivariance has previously been shown to improve learning efficiency for vision-based picking [29, 31, 32, 33, 35]. To this end, FCNs excel in modeling complex multi-modal picking distributions anchored on visual features.

FCN在翻译上按性质是等效的,它与空间动作表现的协同:如果在场景中选择的物体被翻译,那么采样也会翻译. 形式上,这里的等效性可以被定性为$f_{\textrm{pick}}(g\circ\mathbf{o}_{t})=g\circ f_{\textrm{pick}}(\mathbf{o}_{t})$,其中$g$是一个翻译. 空间均匀性以前已被证明可以提高基于视觉的选取的学习效率[29、31、32、33、35]。 为此,法国竞争网络在模拟基于视觉特征的复杂多式采样分销方面表现突出。

Spatially Consistent Visual Representations. The benefits of spatial equivariance are most pronounced when observations $\mathbf{o}_{t}$ are spatially consistent, meaning that the appearance of an object remains constant across different camera views, as well as across spatial transforms of rigid objects. This is in contrast to perspective images, in which the appearance of an object could scale or distort depending on the camera view and lens properties. In practice, spatial consistency also simplifies data augmentation during training, since applying rigid transforms to $\mathbf{o}_{t}$ can yield data that appears as if objects in the scene were physically reconfigured. As with perspective images, however, spatially consistent representations still retain challenges from partial visibility and occlusion. As in [33], we convert RGB-D images into a spatially consistent form by unprojecting to a 3D point cloud, then rendering into an orthographic projection, in which each pixel $(u,v)$ represents a fixed window of 3D space (Fig. 3a). Unlike in prior work, we use this spatially consistent representation for the transport operation, which we describe next.

** 空间一致的视觉表现。 ** 当观测$\mathbf{o}_{t}$在空间上保持一致时,空间均匀性的好处最为显著,这意味着一个物体的外观在不同相机视图之间以及刚性物体的空间变换之间保持不变. 这与透视图像形成对比,在透视图像中,一个物体的外观会视相机视图和透视属性而成规模或扭曲. 在实践中,空间一致性还简化了培训期间的数据增强,因为对$\mathbf{o}_{t}$进行刚性变换可以产生数据,似乎现场的物体被物理重组。 然而,与视角图象一样,空间上一致的表述仍然保留着局部可见度和隐蔽性带来的挑战。 如[33]中,我们通过不预测到3D 大概点云将RGB-D图像转换成空间一致的形式,然后渲染成正图投影,其中每个像素$(u,v)$代表一个3D空间的固定窗口(图3a). 与以前的工作不同,我们使用这种空间上一致的表示来进行我们接下来描述的运输业务。

Figure 3: In this setting (a) where the task is to pick up the red block with an immobilizing grasp (e.g., suction) and place it into the fixture, the goal of Transporter Networks is to recover the distribution of successful picks (b), and distribution of successful placements (c) conditioned on a sampled pick. For pick-conditioned placing (c), deep feature template matching occurs with a local crop around the sampled pick as the exemplar. Rotations of the crop around the pick are used to decode the best placing rotation. Our method preserves rotation and translation equivariance for efficient learning.

图3:在这一设置中(a)的任务是用不动的夹子(如吸吸)捡起红色块并将其放置在固定处,Transporter Network的目标是恢复成功采摘的分布(b)和成功投放的分布(c)以采样采摘为条件. 对于选取条件的放置(c),深层特征模板与被选取物周围的局部作物进行比对,作为示例. 采摘周围作物的旋转用来解码最好的放置旋转. 我们的方法保留了旋转和翻译的等效性,以便有效地学习。

Learning Pick-Conditioned Placing via Transporting. Spatially consistent visual representations enable us to perform visuo-spatial transporting, in which dense pixel-wise features from a partial crop $\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$ centered on $\mathcal{T}_{\textrm{pick}}$ are rigidly transformed then overlaid on top of another partial crop $\mathbf{o}_{t}[\tau]$ centered on a candidate place pose $\tau$, where $\mathbf{o}_{t}$ is the observation before the pick. Intuitively, if the crop encompasses an object, then this operation can be thought of as imagining the object at another location in the scene. Fig. 3c shows this in the context of overlaying the red block across the scene to find its placement in the fixture.

** 通过运输学习预选档**。 空间连贯的视觉表现使我们能够进行空间运输,其中以$\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$为中心的部分作物$\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$的密集像素特性被硬化,然后在以候选地点为中心的另一部分作物$\mathbf{o}_{t}[\tau]$的上方覆盖,形成$\tau$,其中$\mathbf{o}_{t}$是采摘前的观测。 直觉上,如果作物包含一个物体,那么这个操作可以被认为是在现场的另一个地点想象物体. 图3c显示这一点的背景是覆盖整个场景的红色块,以发现其位置在固定处.

In this work, the goal of Transporter Networks is to transport our partial crop $\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$ densely across a set of poses $\{\tau_{i}\}$ to search for its best placement, i.e. the $\mathbf{o}_{t}[\tau_{i}]$ with the highest feature correlation. This operation is analogous to the search function in visual tracking literature [36, 37, 38] where $\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$ is the exemplar and $\mathbf{o}_{t}$ is the search area, or in retrieval systems literature where $\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$ is the query and $\mathbf{o}_{t}$ is the key. We formulate this as a template matching problem, using cross-correlation with dense feature embeddings $\psi(\cdot)$ and $\phi(\cdot)$ from two deep models:

在这项工作中,Transporter Network的目标是将我们部分作物$\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$密集地穿过一组构件$\{\tau_{i}\}$,以寻找其最佳位置,即具有最高特征相关性的$\mathbf{o}_{t}[\tau_{i}]$. 这一操作类似于视觉跟踪文献[36,37,38]中的搜索功能,其中$\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$是实例,$\mathbf{o}_{t}$是搜索区域,或者在检索系统文献中$\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}]$是查询,$\mathbf{o}_{t}$是关键. 我们用密度特性嵌入的交叉对映$\psi(\cdot)$和$\phi(\cdot)$从两个深层模型中得出:

$$\mathcal{Q}_{\textrm{place}}(\tau|\mathbf{o}_{t},\mathcal{T}_{\textrm{pick}})=\psi(\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}])*\phi(\mathbf{o}_{t})[\tau],\qquad\mathcal{T}_{\textrm{place}}=\argmax_{\{\tau_{i}\}}\ \mathcal{Q}_{\textrm{place}}(\tau_{i}|\mathbf{o}_{t},\mathcal{T}_{\textrm{pick}}) \tag{1}$$

where $\mathcal{Q}_{\textrm{place}}(\tau|\mathbf{o}_{t},\mathcal{T}_{\textrm{pick}})$ is an action-value function that correlates with placing success, and $\tau$ covers the space of all possible placing poses – which is equivalent to the space of 2D translations across $\mathbf{o}_{t}$ in our first simple red block and fixture example. Since $\mathbf{o}_{t}$ is spatially consistent, the structure of dense features $\psi(\mathbf{o}_{t})$ and $\phi(\mathbf{o}_{t})$ are also spatially consistent. A key property is that Eq. 1 is invariant to $\mathcal{T}_{\textrm{pick}}$ as long as cross-correlation via transform $\mathcal{T}_{\textrm{place}}$ produces the desired imagined overlay. This enables learning pick-conditioned placing from few examples (Fig. 5), without having to train over all possible picks [27]. Note that it is the deep features, not the RGB-D pixels, which are overlaid, and that the influence of the overlay extends beyond the crop, since for each dense pixel in feature space, the receptive field is large.

其中$\mathcal{Q}_{\textrm{place}}(\tau|\mathbf{o}_{t},\mathcal{T}_{\textrm{pick}})$是一个与放置成功相关的动作值函数,$\tau$覆盖了所有可能放置的空间 — 这相当于我们第一个简单的红块和固定实例中$\mathbf{o}_{t}$的 2D 翻译的空间 。 由于$\mathbf{o}_{t}$在空间上是一致的,所以密度特征$\psi(\mathbf{o}_{t})$和$\phi(\mathbf{o}_{t})$的结构在空间上也是一致的. 一个关键属性是,只要通过变换的$\mathcal{T}_{\textrm{place}}$实现的交叉对映,Eq. 1即为变量至$\mathcal{T}_{\textrm{pick}}$. 这使学习能够从少数例子(图5)中选取条件,而不必对所有可能选取的选取进行训练[27]。 请注意,是深层的特征,而不是RGB-D像素,是覆盖的,而覆盖的影响也超越了作物,因为对于特征空间中每个密集的像素来说,接受场是巨大的.

Learning with Planar Rotations in SE(2). Our first example uses only pick-and-place actions in the space of 2D translations – however, $\mathcal{T}_{\textrm{pick}}$ and $\mathcal{T}_{\textrm{place}}$ may also represent rotations as in Fig. 3. In the planar SE(2) case, we discretize the space of SO(2) rotations into $k$ bins, then rotate the input visual observation $\mathbf{o}_{t}$ for each bin. $\mathbf{o}_{t}$ is thereby defined on a grid of voxels $\{(u,v,w)_{i}\}$ where $w$ lies on the $k$-discretized axis of rotations, and $\mathcal{T}_{\textrm{pick}}$ and $\mathcal{T}_{\textrm{place}}$ are represented discretely on this grid in SE(2). In practice, this is achieved by running the FCN $k$ times in parallel, once for each rotated $\mathbf{o}_{t}$, sharing model weights across all rotations.

** 学习SE的轮班课程(2)。 ** 我们的第一例只使用2D翻译空间中的拾取和放置动作,然而,$\mathcal{T}_{\textrm{pick}}$和$\mathcal{T}_{\textrm{place}}$也可能像Fig一样代表旋转. 3. 在Planar SE(2)的情况下,我们把SO(2)旋转的空间磁盘化为$k$ bin,然后将输入视觉观测XMATH3XX旋转为每个 bin. $\mathbf{o}_{t}$因此被定义在一个 voxels $\{(u,v,w)_{i}\}$上,其中$w$位于$k$- 离散的旋转轴上,$\mathcal{T}_{\textrm{pick}}$和$\mathcal{T}_{\textrm{place}}$在SE(2)中在这个网格上单独代表. 实际上,这是通过平行运行FCN $k$,每次旋转一次$\mathbf{o}_{t}$,在所有旋转中共享模型权重来实现的.

Figure 4: In this 6-DoF variant of the task presented in Fig. 3, the fixture location varies as well in the out-of-plane rotations ($r_{x}$, $r_{y}$), and height ($z$). Transporter Networks can address this task through multi-stage inference.

图4:图3所列任务的6-DoF变体中,固定位置在飞机外旋转($r_{x}$, $r_{y}$)和高度($z$)方面也各不相同。 Transporter Network可以通过多阶段推论来解决这项任务.

Learning with Full Rigid Body Transformations in SE(3). While the previous sections discussed learning a $\mathcal{Q}$ function over a discretized space, extending to the full SE(3) case to handle all six degrees of freedom of rigid objects becomes infeasible to sufficiently discretize the 6-dimensional space – due to the curse of dimensionality. However, interestingly we find the transport operation to be an enabling building block into higher dimensions, using a multi-stage approach to extend into tasks which require SE(3) placing. First, the three SE(2) degrees-of-freedom are addressed as in the previous section, producing an estimated $\hat{\mathcal{T}}^{\text{SE(2)}}_{\textrm{place}}$. Then, we extend the cross-correlation transport operation – rather than using one channel over SE(2) to classify the correct discretized action, we use three channels over SE(2) to regress the remaining rotational ($r_{x},r_{y}$) and translational ($z$-height) degrees of freedom. We use a $\psi^{\prime}$ and $\phi^{\prime}$ which are identical to the $\psi$ and $\phi$ used for SE(2) classification, but with three separate cross-correlations ($*_{3}$) from subsets of channels, and also add learnable nonlinearity after the cross-correlation via a three-headed MLP network, $f(\cdot)$: $r_{x},\ r_{y},\ z=f\big(\psi^{\prime}(\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}])*_{3}\phi^{\prime}(\mathbf{o}_{t})[\hat{\mathcal{T}}^{\text{SE(2)}}_{\textrm{place}}]\big)$. This hybrid discrete/continuous approach affords a number of nice properties: the discrete SE(2) step is good at representing complex multi-modal distributions in those degrees of freedom, and once an approximate SE(2) alignment has been overlaid via the transport operation, this acts as an attention mechanism to aid the model in precise continuous regression. To handle multi-modality in the continuous degrees of freedom, we can regress mixture densities. In practice we have found that multiple approaches can work well once approximate SE(2) alignment has been applied, see the Appendix for more. Multi-stage SE(3) classification, using Eq. 1 iteratively, is also possible.

** 在SE(3)中学习全刚体转换。 ** 虽然前几节讨论了在盘状空间上学习$\mathcal{Q}$函数的问题,但由于维度的诅咒,将处理所有六度刚性物体自由度的SE(3)案件扩展到整个SE(3),因此无法将六维空间充分盘状。 然而,有趣的是,我们发现运输业务是一个重要的组成部分,它采用多阶段办法,将需要SE(3)安排的任务扩大到更高层面。 首先,与上一节一样,讨论三个SE(2)自由度,估计得出$\hat{\mathcal{T}}^{\text{SE(2)}}_{\textrm{place}}$。 然后,我们延长了交叉交配运输业务 — — 而不是使用一个频道在SE(2)上对正确的盘状动作进行分类,而是使用三个频道在SE(2)上对剩下的旋转($r_{x},r_{y}$)和翻译($z$-h8)自由度进行倒退. 我们使用的是$\psi^{\prime}$和$\phi^{\prime}$,它们与SE(2)分类所用的$\psi$和$\phi$完全相同,但从通道子集中分出三个独立的交叉关系($*_{3}$),并且通过三头MLP网络在交叉关系后加入可学习的非线性,$f(\cdot)$:$r_{x},\ r_{y},\ z=f\big(\psi^{\prime}(\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}])*_{3}\phi^{\prime}(\mathbf{o}_{t})[\hat{\mathcal{T}}^{\text{SE(2)}}_{\textrm{place}}]\big)$. 这种混合的离散/连续方法具有一些良好的特性:离散的SE(2)步骤在自由度上能很好地代表复杂的多式联运分布,一旦大约SE(2)的配合通过运输作业得到*覆盖,这便成为在精确的连续回归中协助模型的注意机制。 为了在自由的持续程度上处理多式联运问题,我们可以使混合密度倒退。 在实践中,我们发现,一旦采用了大致的SE(2)调整办法,多种办法就能够很好地发挥作用,详见附录。 多阶段的SE(3)分类,反复使用Eq. 1也是可能的.

Learning Sequential Pick-and-Place. Consider the challenging example with 3-disk Towers of Hanoi in Fig. 1c, where the task is to sequentially move 3 disks from the first tower to the third without placing a larger disk over a smaller disk. Completing this task requires correctly sequencing 7 pick-and-place actions. In this setup with a unique coloring of the base and disks, the task is Markovian. Hence actions can be directly conditioned on visual contextual cues, e.g., the next pick-and-place depends on the position (or absence due to occlusion) of observed objects in the scene. Our method is stateless, but experiments show that it can learn such sequencing behaviors through visual feedback. To enable this, it is important to increase the receptive field [39] of the FCNs to encompass most of the visual observation $\mathbf{o}_{t}$. Extending Transporter Networks with memory to handle non-Markovian tasks would be interesting future work.

** 学习顺序选择和位置。 ** 考虑在图1c的河内三盘塔这个具有挑战性的例子,其中的任务是将三盘从第一塔依次移动到第三塔,而不将更大的盘子放在一个较小的盘子上. 完成这项任务需要正确排序7个选址动作。 在此设置中,具有独特的基座和磁盘颜色,任务是马尔科维安. 因此动作可以直接以视觉上下文提示为条件,例如,下一个采样和位置取决于所观察物体在现场的位置(或由于被隔离而缺席). 我们的方法是无国籍的,但实验表明它可以通过视觉反馈来学习这样的测序行为. 为此,必须增加FCN的可接受场[39],以涵盖大多数视觉观测$\mathbf{o}_{t}$。 延长Transporter Network的内存,以便处理非马尔科维亚人的任务,将是今后令人感兴趣的工作。

Manipulation Beyond Pick-and-Place: Deformables and Piles of Objects. Beyond pick-and-place of rigid objects, Transporter Networks can generalize to tasks that can be completed with two-pose primitives. For example, they can learn to sequentially rearrange a deformable rope such that it connects the two end points of an incomplete 3-sided square (Fig. 1e), or sequentially push piles of small objects into a desired target set (Fig. 1h) using a spatula-shaped end effector. In both cases, our experiments (Sec. 4) demonstrate that they can learn closed-loop behaviors using visual feedback to complete these tasks. Both tasks are characterized by dynamics that cannot be modeled with rigid displacements – however, our results suggest that rigid displacements can serve as a useful prior for non-rigid ones [40].

** 管理 " 选择和位置之外 " :变形物和物体的枕头。 ** 除了刚性物体的采选和位置之外,Transporter Networks还可以概括为以双pose原始完成的任务. 例如,他们可以学会依次重排可变形绳,从而将不完整的3边方形(图1e)的两个端点连接起来,或者依次将堆积的小物体推向理想的目标集(图1h),使用溅射形的终端效应器. 在这两种情况下,我们的实验(Sec. 4)都表明,他们可以利用视觉反馈来学习闭路行为来完成这些任务. 这两项任务的特点是不能以僵硬的流离失所现象为模式的动态,然而,我们的结果表明,僵硬的流离失所现象对于非僵硬的流离失所现象来说是有用的[40]。

3.2 Network Architecture

3.2 网络架构

Observation Space: For tabletop manipulation, our visual observation $\mathbf{o}_{t}$ is an orthographic top-down view of a $0.5{\times}1$m tabletop workspace, generated in simulation by fusing $480{\times}640$ RGB-D images captured with calibrated cameras using known intrinsics and extrinsics. The top-down image $\mathbf{o}_{t}$ has a pixel resolution of $160{\times}320$ – each pixel represents a $3.125{\times}3.125$mm vertical column of 3D space in the workspace. The image $\mathbf{o}_{t}$ contains both color (RGB) and scalar height-from-bottom (H) information, which enables the deep models to learn features that are rich in both visual texture and geometric shape.

观测 空间:关于桌面操作,我们的视觉观测$\mathbf{o}_{t}$是$0.5{\times}1$ m平面工作空间的正向上下视图,在模拟中利用已知的内在和外显相机用校准相机捕获的$480{\times}640$ RGB-D图像生成. 自上而下的图像$\mathbf{o}_{t}$有像素分辨率$160{\times}320$– 每个像素代表工作空间中3D空间的$3.125{\times}3.125$mm 垂直列. 图像$\mathbf{o}_{t}$既包含颜色(RGB),也包含自下而上(H)的scalar高度信息,使深层模型能够学习既具有视觉纹理又具有几何形状丰富的特征.

Picking: Our picking model is a single feed-forward FCN that takes as input the visual observation $\mathbf{o}_{t}\in\mathbb{R}^{H{\times}W{\times}4}$ and outputs dense pixel-wise values that correlate with picking success: $\mathcal{V}_{\textrm{pick}}\in\mathbb{R}^{H{\times}W}=\textrm{softmax}(\mathcal{Q}_{\textrm{pick}}((u,v)|\mathbf{o}_{t}))$. Our picking model is an hourglass encoder decoder architecture: a 43-layer residual network (ResNet) [41] with 12 residual blocks and 8-stride (3 2-stride convolutions in the encoder and 3 bilinear-upsampling layers in the decoder), followed by image-wide softmax. Each convolutional layer after the first is equipped with dilation [42], and interleaved with ReLU activations [43] before the last layer. 8-stride was chosen to balance between maximizing receptive field coverage per pixel prediction, while minimizing loss of resolution in the latent mid-level features of the network.

** 选择**: 我们的选取模型是一个单一的向导FCN,它以视觉观察$\mathbf{o}_{t}\in\mathbb{R}^{H{\times}W{\times}4}$作为输入,并输出与选取成功相关的密集像素偏振值:$\mathcal{V}_{\textrm{pick}}\in\mathbb{R}^{H{\times}W}=\textrm{softmax}(\mathcal{Q}_{\textrm{pick}}((u,v)|\mathbf{o}_{t}))$. 我们的采样模型是一个小时玻璃编码器解码器架构:一个43层的残余网络(ResNet)[41],有12个剩余块和8个阶梯(编码器为3个阶梯2层,解码器为3个线性叠加层),之后是图像宽软max. 第一层之后的每一个卷积层都配备了拓扑[42],并在最后一层之前与ReLU激活(英语:ReLU actions)互换[43]. 选择了8步,以平衡兼顾最大限度地扩大每个像素预测的可接受性,同时尽量减少网络潜在中层特征中的分辨率损失。

Placing: Our placing model is a two-stream feed-forward FCN that takes as input the visual observation $\mathbf{o}_{t}\in\mathbb{R}^{H{\times}W{\times}4}$ and outputs two dense feature maps: query features $\psi(\mathbf{o}_{t})\in\mathbb{R}^{H{\times}W{\times}d}$ and key features $\phi(\mathbf{o}_{t})\in\mathbb{R}^{H{\times}W{\times}d}$, where $d$ is the feature dimensionality. Our placing model shares a similar hourglass encoder decoder architecture as the picking model: each stream is an 8-stride 43-layer ResNet, but without non-linear activations in the last layer. A partial crop $\psi(\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}])\in\mathbb{R}^{c{\times}c{\times}d}$ of query features with size $c$ centered around $\mathcal{T}_{\textrm{pick}}=\argmax{\mathcal{V}_{\textrm{pick}}}$ is transformed by $\tau$ (which covers the space of possible placement configurations), then cross-correlated (Eq. 1) with the key feature map $\phi(\mathbf{o}_{t})$ to output dense pixel-wise values that correlate with placing success: $\mathcal{V}_{\textrm{place}}\in\mathbb{R}^{H{\times}W{\times}k}=\textrm{softmax}(\mathcal{Q}_{\textrm{place}}(\tau|\mathbf{o}_{t},\mathcal{T}_{\textrm{pick}}))$ where $\tau$ is discretized into $k=36$ angles (multiples of 10*∘). The translation and rotation $\tau$ which gives the highest correlation with the partial crop, via $\psi(\mathbf{o}{t}[\mathcal{T}{\textrm{pick}}])\phi(\mathbf{o}{t})[\tau],$gives$\mathcal{T}{\textrm{place}}=\argmax{\mathcal{V}_{\textrm{place}}}$. Note that this operation can be made fast if implemented with highly optimized matrix multiplication (where the crop is flattened) or as a convolution (where the crop is the convolving kernel). Total inference time (with both picking and placing models) amounts to 200ms on an Nvidia GTX 2080 GPU.

** 安置**: 我们的定位模型是一个双流向导FCN,它以视觉观测$\mathbf{o}_{t}\in\mathbb{R}^{H{\times}W{\times}4}$作为输入,输出两个密集的特征图:查询特征$\psi(\mathbf{o}_{t})\in\mathbb{R}^{H{\times}W{\times}d}$和关键特征$\phi(\mathbf{o}_{t})\in\mathbb{R}^{H{\times}W{\times}d}$,其中$d$是特征维度. 我们的放置模型共享一个类似采样模型的小时玻璃编码器解码器架构:每条流是8梯43层的ResNet,但在最后一层没有非线性激活. 以$\mathcal{T}_{\textrm{pick}}=\argmax{\mathcal{V}_{\textrm{pick}}}$为中心大小的$c$查询特性的一部分作物$\psi(\mathbf{o}_{t}[\mathcal{T}_{\textrm{pick}}])\in\mathbb{R}^{c{\times}c{\times}d}$由$\tau$(涵盖可能的放置配置的空间)转化而来,然后是交叉相关(Eq. 1),关键特性图$\phi(\mathbf{o}_{t})$输出密集的像素偏向值,与放置成功相关: $\mathcal{V}_{\textrm{place}}\in\mathbb{R}^{H{\times}W{\times}k}=\textrm{softmax}(\mathcal{Q}_{\textrm{place}}(\tau|\mathbf{o}_{t},\mathcal{T}_{\textrm{pick}}))$,其中$\tau$被圆盘化成$k=36$角(倍数为10*∘). 翻译和旋转的$\tau$通过$\psi(\mathbf{o}{t}[\mathcal{T}{\textrm{pick}}])\phi(\mathbf{o}{t})[\tau],$使$\mathcal{T}{\textrm{place}}=\argmax{\mathcal{V}_{\textrm{place}}}$与部分作物具有最高的相关性. 请注意,如果采用高度优化的矩阵乘法(在作物被扁平的情况下)或作为进化(在作物是结合内核的情况下),这种操作可以快速进行. 在Nvidia GTX 2080 GPU上,总推断时间(有采样和放置模型)为200ms。

3.3 Training: Learning from Demonstrations

3.3 训练：从演示中学习

We assume access to a dataset $\mathcal{D}=\{\zeta_{1},\zeta_{2},...,\zeta_{n}\}$ of $n$ expert demonstrations, where each trajectory $\zeta_{i}=\{(\mathbf{o}_{0},\mathbf{a}_{0}),(\mathbf{o}_{1},\mathbf{a}_{1}),...\}$ is a sequence of one or more observation-action pairs $(\mathbf{o}_{t},\mathbf{a}_{t})$. In our experiments (Sec. 4), we evaluate Transporter Networks in their ability to learn from $n$ = 1, 10, 100, or 1,000 demonstrations per task. Our method does not take any task-specific information as input, and is trained on one task per model (future work may investigate multi-task). During training, we uniformly sample observation-action pairs from the dataset $\mathcal{D}$, from which each action $\mathbf{a}_{t}$ can be unpacked into two training labels: $\mathcal{T}_{\textrm{pick}}$ and $\mathcal{T}_{\textrm{place}}$, used to generate binary one-hot pixel maps $Y_{\textrm{pick}}\in\mathbb{R}^{H{\times}W}$ and $Y_{\textrm{place}}\in\mathbb{R}^{H{\times}W{\times}k}$ respectively. Our training loss is simply the cross-entropy between these one-hot pixel maps and the outputs of the picking and placing models: $\mathcal{L}=-\mathbb{E}_{Y_{\textrm{pick}}}[\textrm{log}\,\mathcal{V}_{\textrm{pick}}]-\mathbb{E}_{Y_{\textrm{place}}}[\textrm{log}\mathcal{V}_{\textrm{place}}]$. While we only have labels for a single pixel per dense probability map, gradients are passed to all other pixels via image-wide softmax. We find this loss to be capable of learning multi-modal non-isotropic spatial action distributions. For SE(3) models involving regression, we use a Huber loss on each regression channel. Validation performance generally converges after a few hours of training on a single commodity GPU.

我们假设可以访问$n$专家演示的数据集$\mathcal{D}=\{\zeta_{1},\zeta_{2},...,\zeta_{n}\}$,其中每个轨迹$\zeta_{i}=\{(\mathbf{o}_{0},\mathbf{a}_{0}),(\mathbf{o}_{1},\mathbf{a}_{1}),...\}$是一个或多个观测动作对的序列$(\mathbf{o}_{t},\mathbf{a}_{t})$. 在我们的实验(第4节)中,我们评价Transporter Network从$n$ = 1, 10, 100, 或1000个演示任务中学习的能力. 我们的方法并不将任何特定任务的信息作为投入,而是就每个模式的一项任务进行培训(未来的工作可以调查多个任务)。 我们得在训练期间从数据集$\mathcal{D}$中统一取样观测-动作对子,从中可以将每个动作$\mathbf{a}_{t}$拆解成两个训练标签:$\mathcal{T}_{\textrm{pick}}$和$\mathcal{T}_{\textrm{place}}$,分别用于生成二元一热像素地图$Y_{\textrm{pick}}\in\mathbb{R}^{H{\times}W}$和$Y_{\textrm{place}}\in\mathbb{R}^{H{\times}W{\times}k}$. 我们的训练损失只是这些一热像素地图与采摘和放置模型的输出之间的交叉交织:$\mathcal{L}=-\mathbb{E}_{Y_{\textrm{pick}}}[\textrm{log}\,\mathcal{V}_{\textrm{pick}}]-\mathbb{E}_{Y_{\textrm{place}}}[\textrm{log}\mathcal{V}_{\textrm{place}}]$. 虽然我们每个密度概率图只有单个像素的标签,但梯度通过图像宽软max传递给所有其他像素. 我们认为,这种损失能够学习多种模式的非同位素空间行动分布。 对于涉及回归的SE(3)模型,我们在每个回归通道上使用Huber损失. 验证性能一般在单一商品GPU的几小时训练后趋同.

4 Results

4 结果

We execute experiments in both simulated and real settings to evaluate Transporter Networks across various tasks. Our goals are threefold: 1) investigate whether preserving visuo-spatial structure within Transporter Networks improves their sample efficiency and generalization, 2) compare them to common baselines for end-to-end vision-based manipulation, and 3) demonstrate that they are practical to deploy on real robots.

我们在模拟和真实的环境下进行实验,评价各种任务的Transporter Network。 我们的目标有三:1)调查在运输网内保持相对空间结构是否提高了样本效率和概括性;2)将它们与基于端到端的视线操纵的共同基线进行比较;3)表明它们实际部署在真正的机器人上.

4.1 Simulation Experiments

4.1 仿真实验

	precise	multimodal	multi-step	unseen
Task	placing	placing	sequencing	objects
block-insertion (Fig. 3)	✓	✗	✗	✗
place-red-in-green	✗	✓	✗	✗
towers-of-hanoi (Fig. 7)	✓	✓	✓	✗
align-box-corner	✓	✓	✗	✓
stack-block-pyramid∗	✓	✓	✓	✗
palletizing-boxes*∗§*	✓	✓	✓	✗
assembling-kits*∗§* [24]	✓	✓	✗	✓
packing-boxes*∗§*	✓	✓	✓	✓
manipulating-rope*∗†*	✓	✓	✓	✗
sweeping-piles*∗†* [12]	✓	✓	✓	✗

Table 1: Each task in Ravens is characterized by a unique set of attributes.

	准确	多式联运	多步骤	看不见
馥 任务	目前正在设置	目前正在设置	顺序	对象敷设
块插入(图3)	✓	✗	✗	✗
红绿色	✗	✓	✗	✗
(图7)	✓	✓	✓	✗
对齐框角	✓	✓	✗	✓
堆栈-块-金字塔*	✓	✓	✓	✗
托盘化框QQ	✓	✓	✓	✗
组装 - Kits {} [24]	✓	✓	✗	✓
包装箱	✓	✓	✓	✓
操纵rope {}	✓	✓	✓	✗
[12] (中文(简体) ).	✓	✓	✓	✗

表1:雷文斯的每一项任务都有一套独特的属性特征.

We use behavior cloning simulation experiments to compare with baselines. In this work, we do not use simulation for sim-to-real transfer – rather only as a means to provide a consistent and controlled environment for fair comparisons and ablations. Ravens, our simulated benchmark learning environment built with PyBullet [44], consists of a Universal Robot UR5e with a suction gripper overlooking a $0.5{\times}1$m tabletop workspace, with 3 simulated 640x480 RGB-D cameras pointing towards the workspace: one stationed in front (facing the robot), one on the left shoulder of the robot arm, and one on the right shoulder. Our method does not require 3 cameras – having more cameras only improves visual coverage. For example, our real experiments only use 1 camera per robot.

我们用行为克隆模拟实验来比较基线. 在这项工作中,我们并不将模拟用于模拟实际转移 — — 而是仅仅作为一种为公平比较和模拟提供连贯和可控环境的手段。 Ravens,我们用PyBullet搭建的模拟基准学习环境[44],由Universal Robot UR5e组成,其吸附式抓手俯瞰着一个$0.5{\times}1$米的台顶工作空间,3台模拟640x480 RGB-D相机指向工作空间:1台驻扎在前面(面对机器人),1台在机器人手臂的左肩,1台在右肩. 我们的方法不需要三台摄像机, 例如,我们真正的实验每个机器人只使用一台相机.

Tasks and Metrics. In simulation, we benchmark on 10 discrete-time tabletop manipulation tasks, some which require closed-loop visual feedback for multi-step sequencing. We position-control the robot end effector (suction cup, or spatula for pushing) with two-pose motion primitives [22]. For each task, all objects (including target zones) are randomly positioned and oriented in the workspace during training and testing. Each task comes with a scripted oracle that provides expert demonstrations by randomly sampling the distribution of successful $\mathcal{T}_{\textrm{pick}}$ and $\mathcal{T}_{\textrm{place}}$ – samples of these are made available for the learner. Some tasks include multi-modality and permutations – for example, in stacking, a successful 6-block pyramid is invariant to the permutation of blocks within each row, but not between rows. Performance is evaluated with a metric from 0 (failure) to 100 (success). Partial credit is assigned during tasks that require sequencing multiple actions. We report results with the models that have achieved highest validation performance during training, averaged over 100 unseen test runs for each task. More details in the Appendix.

** 任务和计量。 ** 在模拟中,我们以10个离散时间台式计算机操纵任务作为基准,有些任务需要闭路视反馈来进行多步测序. 我们定位控制机器人端效应器(吸杯,或用于推力的溅射)与双pose运动原始体[22]. 对于每一项任务,所有物体(包括目标区)在训练和测试期间随机定位和面向工作空间. 每个任务都有一个脚本甲骨文,通过随机抽样提供专家演示成功$\mathcal{T}_{\textrm{pick}}$和$\mathcal{T}_{\textrm{place}}$的分布,这些样本提供给学习者。 一些任务包括多模式和表层——例如,在堆叠中,一个成功的6块金字塔对每行内的区块的表层不具有变化性,但不会在行间. 业绩评价的尺度为0(失败)至100(成功)。 在需要确定多个行动顺序的任务中,将分配部分功劳。 我们报告在培训期间获得最高验证性能的模型的结果,每项任务平均进行100多次未见的测试。 详见附录.

Baseline Methods. Form2Fit [24], as mentioned in Related Work, is different from our method in that it does not perform visuo-spatial transporting, but instead uses a matching network. ConvMLP is a common model architecture in the end-to-end literature [1] that uses a series of convolutional layers, followed by spatial-softmax and a multi-layer perceptron (MLP) – here, we use this to regress the sequence of two SE(2) poses. Form2Fit and ConvMLP use the same input images as Transporter Networks, but to get a sense of the difficulty of our tasks, we also supply two baselines which consume ground truth state (object poses and 3D bounding boxes) as input – i.e., assuming perfect object poses. GT-State MLP is an MLP which consumes these and regresses two SE(2) poses, while 2-step GT-State MLP first regresses the first SE(2) pose, then adds this to the observation vector and regresses the second SE(2) pose . To represent multi-modality, ConvMLP, GT-State MLP, and 2-step GT-State MLP regress mixture densities [45]. All methods, including ours, use identical data augmentation, with random SE(2) world-frame transforms.

** 碱性方法** Form2Fit [24],如相关工作中提到的,与我们的方法不同,因为它不进行空间运输,而是使用匹配的网络。 ConvMLP是端对端文献[1]中常见的模型架构,它使用了一系列的卷积层,然后是空间-软面和多层感官(MLP)——在这里,我们用它来倒退两个SE(2)的姿势的序列. Form2Fit和ConvMLP使用与Transporter Networks相同的输入图像,但为了了解我们任务的难度,我们还提供了两个消耗地面真实状态的基线(对象姿态和3D边框)作为输入——即假设完美的物体姿势. GT-State MLP是一种MLP,它消耗这些物质并递减了两个SE(2)的姿势,而2-Step GT-State MLP首先递减了第一个SE(2)的姿势,然后在观测矢量中加入这个姿势,递减了第二个SE(2)的姿势. 为了代表多式联运,ConvMLP、GT-State MLP和两步GT-State MLP将混合物密度递减[45]。 所有方法,包括我们的方法,都使用相同的数据增强,随机SE(2)世界范围变换.

Results: Sample Efficiency and Generalization on Benchmark Tasks

** 成果:基准任务的抽样效率和一般化**

Tab. 2 shows sample efficiency of baselines trained from stochastic demonstrations for each task, and evaluated on unseen test settings, with random rotations and translations of objects (including target zones). The benchmark is difficult – most baselines, while capable of over-fitting to the demonstration training set, generalize poorly with only 1,000 demonstrations. In general, Transporter Networks achieve orders of magnitude more sample efficiency than the image-based alternatives, and also provides better sample efficiency than multi-layer perceptrons trained with ground truth state. More analysis in the Appendix.

标签。 2 显示每个任务从随机演示中培训的基线的样本效率,并在不见的测试设置上加以评价,随机旋转和翻译对象(包括目标区)。 基准是困难的 — — 大多数基线虽然能够过拟合示范培训,但一般情况下只有1000个示范。 一般来说,Transporter Networks比基于图像的替代品更能达到数量级的样本效率,也比经过地面真实状态训练的多层感知器提供更好的样本效率. 附录中更多的分析.

	block-insertion	place-red-in-green	towers-of-hanoi	align-box-corner	stack-block-pyramid
Method	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
Transporter Network	100	100	100	100	84.5	100	100	100	73.1	83.9	97.3	98.1	35.0	85.0	97.0	98.3	13.3	42.6	56.2	78.2
Form2Fit [24]	17.0	19.0	23.0	29.0	83.4	100	100	100	3.6	4.4	3.7	7.0	7.0	2.0	5.0	16.0	19.7	17.5	18.5	32.5
Conv. MLP	0.0	5.0	6.0	8.0	0.0	3.0	25.5	31.3	0.0	1.0	1.9	2.1	0.0	2.0	1.0	1.0	0.0	1.8	1.7	1.7
GT-State MLP	4.0	52.0	96.0	99.0	0.0	0.0	3.0	82.2	10.7	10.7	6.1	5.3	47.0	29.0	29.0	59.0	0.0	0.2	1.3	15.3
GT-State MLP 2-Step	6.0	38.0	95.0	100	0.0	0.0	19.0	92.8	22.0	6.4	5.6	3.1	49.0	12.0	43.0	55.0	0.0	0.8	12.2	17.5
	palletizing-boxes	assembling-kits	packing-boxes	manipulating-rope	sweeping-piles
	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
Transporter Network	63.2	77.4	91.7	97.9	28.4	78.6	90.4	94.6	56.8	58.3	72.1	81.3	21.9	73.2	85.4	92.1	52.4	74.4	71.5	96.1
Form2Fit [24]	21.6	42.0	52.1	65.3	3.4	7.6	24.2	37.6	29.9	52.5	62.3	66.8	11.9	38.8	36.7	47.7	13.2	15.6	26.7	38.4
Conv. MLP	31.4	37.4	34.6	32.0	0.0	0.2	0.2	0.0	0.3	9.5	12.6	16.1	3.7	6.6	3.8	10.8	28.2	48.4	44.9	45.1
GT-State MLP	0.6	6.4	30.2	30.1	0.0	0.0	1.2	11.8	7.1	1.4	33.6	56.0	5.5	11.5	43.6	47.4	7.2	20.6	63.2	74.4
GT-State MLP 2-Step	0.6	9.6	32.8	37.5	0.0	0.0	1.6	4.4	4.0	3.5	43.4	57.1	6.0	8.2	41.5	58.7	9.7	21.4	66.2	73.9

Table 2: Baseline comparisons. Task success rate (mean %) vs. # of demonstration episodes (1, 10, 100, or 1000) used in training.

	块插入	红绿色	哈诺伊塔	对齐框角	叠块金字塔
方法	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
运输网	(单位:千美元)	(单位:千美元)	(单位:千美元)	(单位:千美元)	** 84.5** với联合国。	(单位:千美元)	(单位:千美元)	(单位:千美元)	** 第73.1段**	页:1	页:1	** 98.1 **	35.0	** 85.0** (单位:千美元)	页:1	页:1	13.3	页:1	** 56.2 **	页:1
窗体2Fit [24]	17.0	19.0	23.0	29.0	83.4	(单位:千美元)	(单位:千美元)	(单位:千美元)	3.6	4.4	3.7	7.0	7.0	2.0	5.0	16.0	页:1	17.5	18.5	32.5
MLP 联盟	0.0	5.0	6.0	8.0	0.0	3.0	25.5	31.3	0.0	1.0	1.9	2.1	0.0	2.0	1.0	1.0	0.0	1.8	1.7	1.7
GT 状态 MLP	4.0	52.0	96.0	99.0	0.0	0.0	3.0	82.2	10.7	10.7	6.1	5.3	47.0	29.0	29.0	59.0	0.0	0.2	1.3	15.3
GT 状态 MLP 2- Step	6.0	38.0	95.0	(单位:千美元)	0.0	0.0	19.0	92.8	22.0	6.4	5.6	3.1	页:1	12.0	43.0	55.0	0.0	0.8	12.2	17.5
	托盘化框	组装减缓贫穷人	包装箱	操纵绳	扫描页
	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
运输网	页:1	页:1	91.7 国家	页:1	页:1	你是怎么决定的?	90.4 (单位:千美元)	** 94.6 **	页:1	** 58.3 **	页:1	页:1	页:1	** 第73.2段**	** 85.4** (单位:千美元)	92.1 国家	52.4 (单位:千美元)	页:1	页:1	** 第96.1段 **
窗体2Fit [24]	21.6	42.0	52.1	65.3	3.4	7.6	24.2	37.6	29.9	52.5	62.3	66.8	11.9	38.8	36.7	47.7	13.2	15.6	26.7	38.4
MLP 联盟	31.4	37.4	34.6	32.0	0.0	0.2	0.2	0.0	0.3	9.5	12.6	16.1	3.7	6.6	3.8	10.8	28.2	48.4	44.9	45.1
GT 状态 MLP	0.6	6.4	30.2	30.1	0.0	0.0	1.2	11.8	7.1	1.4	33.6	56.0	5.5	11.5	43.6	47.4	7.2	20.6	63.2	74.4
GT 状态 MLP 2- Step	0.6	9.6	32.8	37.5	0.0	0.0	1.6	4.4	4.0	3.5	43.4	57.1	6.0	8.2	41.5	58.7	9.7	21.4	66.2	73.9

表2:** 基线比较。 ** 培训中使用的演示节目(1、10、100或1000)中的任务成功率(平均百分比)对#。

Generalizing to Unseen Objects. Three of our benchmark tasks involve generalizing to unseen objects (see Tab. 1). In our variant of the kit assembly task with unseen objects (shown in Fig. 8), the gap in performance between Form2Fit and Transporter Networks is rather large. On a simpler set of unseen objects that have more primitive geometries (e.g., circular disks, squares), which reflect the distribution of objects used in the original paper [24], Form2Fit achieves 96.3% task success. This suggests that Form2Fit descriptors have the capacity to express larger differences in geometry, but are less capable of matching higher resolution information than our deep template-matching based model, with the same amount of data.

• 泛指不明物体。 * 我们的三项基准任务涉及概括到看不见的对象(见附件1)。 在用隐形物体装配工具箱任务的变体(在图8中显示)中,Form2Fit和Transporter Networks之间的性能差距相当大. 在较简单的一组具有更原始的几何美图(如圆盘,方块)的隐形物体上,这反映了原始纸中使用的对象的分布[24],Form2Fit实现了96.3%的任务成功. 这表明Form2Fit描述符具有表达几何学上较大差异的能力,但比起我们的深层模板比对模型,数据数量相同,更不能匹配更高的分辨率信息.

Learning Sequential Manipulation with Closed-Loop Visual Feedback. In this work, Transporter Networks are stateless models that react only to information presented as visual input during the current timestep. However, our experiments show that the models have the capacity to learn visual feedback: they make use of contextual visual cues to determine which step of the task they are in, and use that to condition action-value predictions. For example, when a stack of blocks falls over, they can re-build the stack of blocks as if they had just started the task. If they accidentally push granular media out of the target zone during sweeping, they then push it back in (Fig. 5). We hypothesize that equivariance to rotations and translations enable them to learn these recovery behaviors even with little data on multi-step tasks (see Tab. 1). We show examples of these emerging behaviors in our supplementary video.

• 利用闭环视觉反馈学习序列操纵。 在本作中,Transporter Networks是无国籍的模型,仅对当前时间步内作为视觉输入而呈现的信息作出反应. 然而,我们的实验表明,这些模型具有学习视觉反馈的能力:它们利用上下文的视觉提示来决定它们所承担任务的哪个步骤,并利用它来设定动作价值预测的条件。 例如,当堆积的块掉下来时,它们可以像刚刚开始任务一样重建堆积的块. 如果他们在扫射过程中不慎将颗粒介质推出目标区,那么就将其推回(图5)。 我们假设,对旋转和翻译的等效性使他们能够学习这些恢复行为,即使关于多步骤任务的数据很少(见Tab.1)。 我们在补充视频中展示了这些新兴行为的例子。

	deterministic	stochastic
Method	1	10	100	1000	1	10	100	1000
Transporter Network	100	100	100	100	100	100	100	100
Form2Fit [24]	100	100	100	100	100	30.0	45.0	45.0
Conv. MLP	7.0	73.0	81.0	69.0	11.0	31.0	52.0	68.0
GT-State MLP	100	100	100	100	21.0	77.0	100	100
GT-State MLP 2-Step	100	100	100	100	53.0	70.0	100	93.0

Table 3: Simplified 2DoF (no rotation) block-insertion is harder to learn with demonstrations from a stochastic oracle than a deterministic one.

	决定性	结构化
方法	1	10	100	1000	1	10	100	1000
运输网	100	100	100	100	100	100	100	100
窗体2Fit [24]	100	100	100	100	100	30.0	45.0	45.0
共和党	7.0	73.0	81.0	69.0	11.0	31.0	52.0	68.0
G掌管状态 MLP	100	100	100	100	21.0	77.0	100	100
GT 状态 ML在这一问题上	100	100	100	100	53.0	70.0	100	93.0

Of 表3:简化的2DoF(不旋转)块插入比决定性的要难于通过演示来学习。

Sample Complexity in Simplified Environments. Consider a simplified translation-only block-insertion task illustrated in Fig. 3, where no rotations are needed, and the block is initialized to the same location, with only the fixture location varying between environments. We investigate two variants of experts in this setting: (a) one that provides deterministic demonstrations where $\mathcal{T}_{\textrm{pick}}$, $\mathcal{T}_{\textrm{place}}$ are fixed relative to the block and fixture respectively, and (b) one that provides stochastic demonstrations where $\mathcal{T}_{\textrm{pick}}$, $\mathcal{T}_{\textrm{place}}$ are uniformly random sampled from the distribution of successful actions. Intuitively, the stochastic demonstrator reflects a more realistic setting in which the expert is inconsistent. Tab. 3 displays sample complexity results on these tasks. Most baselines perform well with deterministic demonstrations, but begin to struggle with stochastic ones. Transporter Networks, by construction, work well in both cases. We show additional extrapolation experiments in the Appendix.

• 简化环境中的样本复杂性。 考虑在图3中说明的简化的只翻译块插入任务,其中不需要旋转,块被初始化到同一个位置,只有固定位置在环境上有所不同. 我们调查了两种专家在此情况下的变体:(a) 一种是确定性演示,其中$\mathcal{T}_{\textrm{pick}}$、$\mathcal{T}_{\textrm{place}}$分别相对于区块和固定物固定;(b) 另一种是确定性演示,其中$\mathcal{T}_{\textrm{pick}}$、$\mathcal{T}_{\textrm{place}}$从成功行动的分布中统一随机抽样。 直观地说,斯图克式的显示器反映了专家不一致的更现实的环境. Tab. 3显示了这些任务的样本复杂性结果. 多数基线在决定性的演示中表现良好,但开始与诡异的示威作斗争。 Transporter Network,通过建筑,在这两种情况下都运作良好. 我们在附录中显示更多的推断实验。

	6DoF block-insertion
	1	10	100	1000
Transporter Net. SE(3)	38.0	76.0	84.0	91.0
GT-State MLP 3-Step SE(3)	0.0	1.0	1.0	5.0

Table 4: 6DoF block-insertion: our method generalizes with fewer demonstrations than GT-State MLPs.

	6DoF 块插入
	1	10	100	1000
运力网. SE(3)	38.0	76.0	84.0	91.0
GT- 状态 MLP 3- Step SE (3)	0.0	1.0	1.0	5.0

表 4: 6DoF 块插入:我们的方法比GT状态的 MLP 的演示较少。

Learning 6DoF Pick-and-Place. We also test the SE(3) formulation discussed in Sec. 3.1 in the 6DoF environment shown in Fig. 4 with unseen target fixture configurations and stochastic demonstrations. On this more challenging task, our model demonstrates considerably better sample efficiency than the image-based baselines achieved on the strictly-easier 3DoF-only block-insertion task (Tab. 2 and 4). Additionally, while the hybrid discrete and continuous SE(3) model is not able to 1-shot generalize as the discrete-only SE(2) model could, we still observe substantially better performance than GT-State MLP models (Tab. 4) which struggle in the larger observation and action spaces. For further analysis of this task and models, see the Appendix.

学习6DoF 选择和定位 我们还在图4显示的6DoF环境中测试第3.1节讨论的SE(3)配方,采用隐蔽的目标固定配置和花样演示。 在这项更具挑战性的任务中,我们的模型显示出比严格简化的3DoF-只插入块任务(Tab.2和4)上实现的基于图像的基线要高得多的样本效率。 此外,虽然混合离散和连续的SE(3)模型无法像离散的SE(2)模型那样进行1射一般化,但我们仍然观察到比GT状态的MLP模型(Tab. 4)更好的性能,这些模型在较大的观测和行动空间中挣扎。 关于对这项任务和模式的进一步分析,见附录。

Figure 5: Transporter Networks can predict the desired spatial displacements of piles of small objects (left), which informs how to push them towards a target set. Our experiments show that it can learn visual feedback – enabling it to recover from mistakes (e.g., top row: pushing objects out of the target set at $t=11$), or sequentially rearrange a deformable rope (middle row). Since it learns pick-conditioned placing, it can also stack plates with varying initial predicted pick locations from only 10 demonstrations on a real robot (bottom row), without needing any object-specific representations.

图5:Transporter Network可以预测小物体堆积(左)的预期空间迁移情况,这说明如何将它们推向目标设定。 我们的实验表明,它可以学习视觉反馈 — — 使其能够从错误中恢复(例如:上行:将物体推出$t=11$设定的目标),或者依次重新安排可变形绳(中行). 由于它学会了选取条件的放置,它也可以堆叠板块,上面只有10个实际机器人(下行)的演示,最初预测的选取位置也各不相同,不需要任何特定物体的表示.

4.2 Real-World Experiments

4.2 真实世界实验

Task	Test %	# Samples
Mouthwash Kit Assembly	98.9	8141
Sweeping Piles of Go Pieces	98.3	6759

Table 5: Transporter Networks test performance on real robots with human demonstration data.

任务	发送信息	样本
洗口袋	98.9	8141
扫荡飞碟	98.3	6759

表5:运兵网用人示范数据对真机器人进行性能测试.

We validate Transporter Networks on both the kit assembly and the sweeping tasks (shown in Fig. 1k and 1m) using real UR5e robots. For kit assembly, the robot uses a suction gripper and an industrial Photoneo PhoXi camera (for high resolution and accurate IR-depth). For sweeping, the robot uses a brush end effector and a low-cost consumer-grade Azure Kinect (for RGB-depth). Despite COVID-19 lockdowns preventing physical access, we were still able to perform real experiments using our Unity-based [46] UI that enables people to remotely teleoperate our robots to calibrate, collect training data, and run tests. For these experiments, teleoperators are tasked with i) repeatedly assembling and disassembling a kit of 5 bottled mouthwashes, or ii) sweeping piles of small Go pieces into a target goal zone marked with green tape, and autonomous resets using a scripted scrambling motion. Table 5 shows test performance of the models, and the amount of training data collected from 13 human operators in terms of transitions (i.e., pick and place actions for kit assembly, pushes for sweeping). We also show qualitative results of our robots in action for both tasks in our supplementary video. Further quantitative experiments were infeasible at time of submission due to COVID-19. Additional experiment details in the Appendix.

我们用真实的UR5e机器人验证关于包装和扫荡任务(在图1k和1m中显示)的Transporter Network。 对于装箱组装,机器人使用吸管式抓取器和工业Photoneo PhoXi相机(用于高分辨率和准确的IR深度). 对于扫荡,机器人使用刷端效应器和低成本的消费级Azure Kinect(用于RGB深度). 尽管COVID-19封锁阻止了物理访问,但我们仍然能够使用基于Unity的实验[46]UI进行真正的实验,使人们能够远程操作我们的机器人校准,收集训练数据,并进行测试. 对于这些实验uangeperators的任务有:一) 反复组装和拆卸5个瓶装的洗口袋,或二) 将小Go块的堆积物打入标有绿色胶带的目标球门区,并使用脚本拼接动作进行自动重置. 表5显示这些模型的测试性能,以及从13个人类操作人员收集的过渡性培训数据数量(即选取和放置装具装配动作,推推扫). 我们还在我们的补充录像中展示了我们机器人在行动上对这两项任务的质量结果。 由于COVID-19的缘故,在提交之时,进一步的数量实验是不可行的。 附录中的其他实验细节.

5 Conclusion

5 结论

In this work, we presented the Transporter Network, a simple model architecture that infers spatial displacements, which can parameterize robot actions from visual input. It makes no assumptions of objectness, exploits spatial symmetries, and is orders of magnitude more sample efficient in learning vision-based manipulation tasks than end-to-end alternatives: from stacking a pyramid of blocks, to assembling kits with unseen objects, to pushing piles of small objects with closed-loop feedback. In terms of its current limitations: it is sensitive to camera-robot calibration, and it remains unclear how to integrate torque/force actions with spatial action spaces. Overall, we are excited about this direction and plan to extend it to real-time high-rate control, and as well as tasks involving tool use.

在这部作品中,我们介绍了Transporter Network(Transporter Network),这是一个简单的模型架构,可以推断空间位移,它可以从视觉输入中参数化机器人动作. 它不作物体的假设,利用空间对称性,是量级比起端到端的替代方法,在学习基于视觉的操纵任务时更有效率的样本:从堆叠一个金字塔的块,到用隐形物体组装包,再到用闭路反馈推压小物体堆积. 就其目前的局限性而言:它对相机-机器人校准很敏感,仍然不清楚如何将矩/力行动与空间行动空间结合起来。 总体而言,我们对这个方向感到兴奋,并计划将其扩展到实时高率控制,以及涉及工具使用的任务.

Acknowledgments

Special thanks to Ken Goldberg, Razvan Surdulescu, Daniel Seita, Ayzaan Wahid, Vincent Vanhoucke, Anelia Angelova, for helpful feedback on writing, Sean Snyder, Jonathan Vela, Larry Bisares, Michael Villanueva, Brandon Hurd for operations and hardware support, Robert Baruch for software infrastructure, Jared Braun for UI contributions, Erwin Coumans for PyBullet advice, Laura Graesser for video narration.

承认

特别感谢肯·戈德伯格,拉兹万·苏尔杜列斯库,丹尼尔·塞塔,艾赞·瓦希德,文森特·万胡克,安妮莉娅·安杰洛娃对写作的有益反馈,肖恩·斯奈德,乔纳森·维拉,拉里·比萨雷斯,迈克尔·比利亚努耶娃,布兰登·胡尔德对操作和硬件支持,罗伯特·巴鲁赫对软件基础设施的支持,贾里德·布劳恩对UI的贡献,埃尔温·库曼对PyBullet的建议,劳拉·格雷瑟对视频的叙述.

References

参考文献

[1] S. Levine, C. Finn, T. Darrell, and P. Abbeel. End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research (JMLR) , 2016.

[2] D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, et al. Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation. Conference on Robot Learning (CoRL) , 2018.

[3] L. Manuelli, W. Gao, P. Florence, and R. Tedrake. kpam: Keypoint affordances for category-level robotic manipulation. International Symposium on Robotics Research (ISRR) , 2019.

[4] T. D. Kulkarni, A. Gupta, C. Ionescu, S. Borgeaud, M. Reynolds, A. Zisserman, and V. Mnih. Unsupervised learning of object keypoints for perception and control. NeurIPS , 2019.

[5] A. Nagabandi, K. Konolige, S. Levine, and V. Kumar. Deep dynamics models for learning dexterous manipulation. Conference on Robot Learning (CoRL) , 2020.

[6] X. Liu, R. Jonschkowski, A. Angelova, and K. Konolige. Keypose: Multi-view 3d labeling and keypoint estimation for transparent objects. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020.

[7] O. Mees, N. Abdo, M. Mazuran, and W. Burgard. Metric learning for generalizing spatial relations to new objects. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2017.

[8] P. Jund, A. Eitel, N. Abdo, and W. Burgard. Optimization beyond the convolution: Generalizing spatial relations with end-to-end metric learning. IEEE International Conference on Robotics and Automation (ICRA) , 2018.

[9] P. R. Florence, L. Manuelli, and R. Tedrake. Dense object nets: Learning dense visual object descriptors by and for robotic manipulation. Conference on Robot Learning (CoRL) , 2018.

[10] P. Florence, L. Manuelli, and R. Tedrake. Self-supervised correspondence in visuomotor policy learning. IEEE Robotics and Automation Letters (RA-L) , 2019.

[11] P. Sundaresan, J. Grannen, B. Thananjeyan, A. Balakrishna, M. Laskey, K. Stone, J. E. Gonzalez, and K. Goldberg. Learning rope manipulation policies using dense object descriptors trained on synthetic depth data. IEEE International Conference on Robotics and Automation (ICRA) , 2020.

[12] H. Suh and R. Tedrake. The surprising effectiveness of linear models for visual foresight in object pile manipulation. Workshop on Algorithmic Foundations of Robotics (WAFR) , 2020.

[13] R. Kondor and S. Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. International Conference on Machine Learning (ICML) , 2018.

[14] T. Cohen and M. Welling. Group equivariant convolutional networks. International Conference on Machine Learning (ICML) , 2016.

[15] R. Platt, C. Kohler, and M. Gualtieri. Deictic image mapping: An abstraction for learning pose invariant manipulation policies. AAAI Conference on Artificial Intelligence , 2019.

[16] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym. arXiv preprint arXiv:1606.01540 , 2016.

[17] Y. Yoon, G. N. DeSouza, and A. C. Kak. Real-time tracking and pose estimation for industrial objects using geometric features. IEEE International Conference on Robotics and Automation (ICRA) , 2003.

[18] M. Zhu, K. G. Derpanis, Y. Yang, S. Brahmbhatt, M. Zhang, C. Phillips, M. Lecce, and K. Daniilidis. Single image 3d object detection and pose estimation for grasping. ICRA , 2014.

[19] A. Zeng, K.-T. Yu, S. Song, D. Suo, E. Walker, A. Rodriguez, and J. Xiao. Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge. IEEE International Conference on Robotics and Automation (ICRA) , 2017.

[20] X. Deng, Y. Xiang, A. Mousavian, C. Eppner, T. Bretl, and D. Fox. Self-supervised 6d object pose estimation for robot manipulation. IEEE International Conference on Robotics and Automation (ICRA) , 2020.

[21] H. Wang, S. Sridhar, J. Huang, J. Valentin, S. Song, and L. J. Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. Computer Vision and Pattern Recognition (CVPR) , 2019.

[22] E. Frazzoli, M. A. Dahleh, and E. Feron. Maneuver-based motion planning for nonlinear systems with symmetries. IEEE Transactions on Robotics (T-RO) , 2005.

[23] M. Gualtieri, A. ten Pas, and R. Platt. Pick and place without geometric object models. IEEE International Conference on Robotics and Automation (ICRA) , 2018.

[24] K. Zakka, A. Zeng, J. Lee, and S. Song. Form2fit: Learning shape priors for generalizable assembly from disassembly. IEEE International Conference on Robotics and Automation (ICRA) , 2020.

[25] A. Hundt, B. Killeen, N. Greene, H. Wu, H. Kwon, C. Paxton, and G. D. Hager. “good robot!”: Efficient reinforcement learning for multi-step visual tasks with sim to real transfer. IEEE Robotics and Automation Letters (RA-L) , 2020.

[26] L. Berscheid, P. Meißner, and T. Kröger. Self-supervised learning for precise pick-and-place without object model. IEEE Robotics and Automation Letters (RA-L) , 2020.

[27] Y. Wu, W. Yan, T. Kurutach, L. Pinto, and P. Abbeel. Learning to manipulate deformable objects without demonstrations. Robotics: Science and Systems (RSS) , 2020.

[28] C. Devin, P. Rowghanian, C. Vigorito, W. Richards, and K. Rohanimanesh. Self-supervised goal-conditioned pick and place. arXiv preprint arXiv:2008.11466 , 2020.

[29] M. Khansari, D. Kappler, J. Luo, J. Bingham, and M. Kalakrishnan. Action image representation: Learning scalable deep grasping policies with zero real world data. IEEE International Conference on Robotics and Automation (ICRA) , 2020.

[30] S. Song, A. Zeng, J. Lee, and T. Funkhouser. Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations. IEEE Robotics and Automation Letters (RA-L) , 2020.

[31] A. Zeng, S. Song, K.-T. Yu, E. Donlon, F. R. Hogan, M. Bauza, D. Ma, O. Taylor, M. Liu, E. Romo, et al. Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. International Conference on Robotics and Automation (ICRA) , 2018.

[32] D. Morrison, P. Corke, and J. Leitner. Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach. Robotics: Science and Systems (RSS) , 2018.

[33] A. Zeng, S. Song, S. Welker, J. Lee, A. Rodriguez, and T. Funkhouser. Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2018.

[34] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2015.

[35] A. Zeng. Learning Visual Affordances for Robotic Manipulation . PhD thesis, Princeton University.

[36] B. Li, W. Wu, Q. Wang, F. Zhang, J. Xing, and J. Yan. Siamrpn++: Evolution of siamese visual tracking with very deep networks. Conference on Computer Vision and Pattern Recognition , 2019.

[37] L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr. Fully-convolutional siamese networks for object tracking. European Conference on Computer Vision (ECCV) , 2016.

[38] R. Tao, E. Gavves, and A. W. Smeulders. Siamese instance search for tracking. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.

[39] W. Luo, Y. Li, R. Urtasun, and R. Zemel. Understanding the effective receptive field in deep convolutional neural networks. Advances in Neural Information Processing Systems (NeurIPS) , 2016.

[40] D. Seita, P. Florence, J. Tompson, E. Coumans, V. Sindhwani, K. Goldberg, and A. Zeng. Learning to rearrange deformable cables, fabrics, and bags with goal-conditioned transporter networks. arXiv preprint arXiv:2012.03385 , 2020.

[41] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.

[42] F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. International Conference on Learning Representations , 2016.

[43] V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. International Conference on Machine Learning (ICML) , 2010.

[44] E. Coumans and Y. Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. GitHub Repository , 2016.

[45] C. M. Bishop. Mixture density networks. Aston University , 1994.

[46] Unity Technologies. Unity game engine. Version: 2018.4.0f1. URL https://unity.com .

[47] T. Schmidt, R. Newcombe, and D. Fox. Self-supervised visual descriptor learning for dense correspondence. IEEE Robotics and Automation Letters (RA-L) , 2016.

[48] R. Goroshin, M. F. Mathieu, and Y. LeCun. Learning to linearize under uncertainty. Advances in Neural Information Processing Systems (NeurIPS) , 2015.

[49] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2015.

[50] R. Rahmatizadeh, P. Abolghasemi, L. Bölöni, and S. Levine. Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration. ICRA , 2018.

[51] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. European Conference on Computer Vision (ECCV) , 2020.

[52] V. Sitzmann, J. N. Martel, A. W. Bergman, D. B. Lindell, and G. Wetzstein. Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems (NeurIPS) , 2020.

[53] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR) , 2015.

6 Appendix

6 附录

The appendix consists of: descriptions of tasks and evaluation metrics in the Ravens simulation framework, additional experimental results, analysis, ablation studies, and real system details.

附录包括:对雷文斯求助于模拟框架中的任务和评价度量衡的描述,额外的实验结果,分析,衰竭研究,以及真实的系统细节.

6.1 Simulation Experimentation, Real World Experimentation, and COVID-19

6.1 仿真实验、真实世界实验与 COVID-19

As introduced in the main paper, we execute experiments in simulation with Ravens as a means to provide a consistent environment for comparisons to baselines and ablations. Models evaluated in simulation are trained only on simulated data. Likewise models evaluated in the real-world are trained only on real data. In simulation, we avoid assumptions that cannot transfer to a real setup: observation data contains only 640x480 RGB-D images and camera parameters; actions are end effector poses (transposed into joint positions with inverse kinematics). The only exception to this is our GT-State baseline, which learns MLPs that take ground truth state information (e.g., object poses, bounding boxes, and color) as input – shown to be less sample efficient than Transporter Networks for many tasks. A limitation of Ravens is that the rendering software stack may not fully reflect the noise characteristics often found in real data: inaccuracies in camera calibration (e.g., intrinsics and extrinsics), noise in commodity depth sensors. Hence, we also test our method on a real system (details in Sec. 6.14) using teleoperated demonstration data, and present qualitative results in our supplementary video. Extensive quantitative experiments on real robots was difficult due to limited physical access during COVID-19.

正如主要论文中介绍的那样,我们与雷文斯进行模拟实验,作为提供与基线和堆积进行比较的一致环境的一种手段。 在模拟中评价的模型只接受模拟数据方面的培训。 同样,在现实世界中评价的模型也只接受关于真实数据的培训。 在模拟中,我们避免了无法转移到一个真实设置的假设:观测数据只包含640x480 RGB-D图像和相机参数;动作是端效应器所构成的(转换成带有反向动因子的联合位置). 唯一的例外是我们的GT状态基线,它学习了将地面真实状态信息(如对象姿势、边框和颜色)作为输入的MLP — — 显示其样本效率低于许多任务的运输网络。 Ravens的一个局限性是,渲染软件堆栈可能不能充分反映真实数据中经常发现的噪音特征:相机校准中的不准确(如内在和外在),商品深度传感器中的噪音. 因此,我们还利用远程操作演示数据,在真实系统中测试我们的方法(详情见第6.14节),并在我们的补充视频中介绍质量结果。 由于COVID-19期间的物理接触有限,对真正的机器人进行广泛的定量实验是困难的.

6.2 Ravens Simulation Framework

6.2 Ravens 仿真框架

In our simulation framework, Ravens, each task comes with a scripted oracle that provides expert demonstrations by randomly sampling the distribution of successful picking and placing actions – samples of which are made available to the learner. Ravens also has several attributes that make it useful for studying different areas that are beyond the scope of this work, including: 1) active learning, since algorithms can query an oracle during specific moments of uncertainty to improve learning efficiency, 2) reinforcement learning, since we provide reward functions that provide partial credit for all tasks (used only for evaluation in this work), and 3) active perception, since camera parameters (e.g., extrinsics) are defined by an action (static in our experiments, but could be dynamic), which provides an opportunity to study algorithms that improve learning through active control of the camera position and orientation.

在我们的模拟框架中,雷文斯,每个任务都有一个脚本甲骨文,通过随机抽样的方式提供专家演示成功采摘和放置动作的分布,这些样本提供给学习者。 Ravens还具有几个属性,使得它有利于研究超出本工作范围的不同领域,包括:1) 主动学习,因为算法可以在不确定的特定时刻查询一个甲骨文以提高学习效率,2) 强化学习,因为我们提供了奖励功能,为所有任务提供部分功劳(仅用于本工作的评价),3) 主动感知,因为相机参数(如外观学)是由一个动作定义的(在我们实验中是静态的,但可以是动态的),它提供了一个研究算法的机会,通过主动控制相机位置和方向来改进学习.

During both training and testing, all objects (including target zones) are randomly positioned and oriented in the workspace. To succeed in the case where only a single demonstration is provided, the learner needs to be invariant to unseen configurations of objects. Information about the multi-modality of a task or its sequential permutations is only made available from the distribution of demonstrations. For example, moving disks in Towers of Hanoi may have only one correct sequence of states, but the distribution of how each disk can be picked or placed is learned from multiple demonstrations. Or, when palletizing homogeneous boxes, the arrangement of boxes across each layer on the pallet must be transposed, and boxes should be stacked stably on top other boxes already on the stack to avoid toppling. In our experiments, Transporter Networks are trained from scratch, without any pre-training from vision datasets. Exploring multi-task training for more efficient generalization to new tasks is interesting future work.

在训练和测试期间,所有物体(包括目标区)在工作区随机定位和定向. 在只提供单一演示的情况下,学习者需要不熟悉物体的无形配置才能取得成功。 关于一项任务的多式性或其顺序排列的资料只能通过分发示范材料获得。 例如,河内塔的移动磁盘可能只有一个正确的状态序列,但每个磁也可如何选择或放置的分布是从多个演示中得知的. 或者,当托盘化同质箱时,托盘上每一层的箱的排列必须进行换乘,箱的上部应当用已经堆放在堆栈上的其他箱的刺痕堆叠,以避免翻转. 在我们的实验中,运输网从零开始训练,没有从视觉数据集进行任何预训. 探索多任务培训,以更高效地概括到新任务上,是未来有趣的工作.

Figure 6: Tasks: (row-major order) block-insertion, place-red-in-green, towers-of-hanoi, align-box-corner, stack-block-pyramid, palletizing-boxes, assembling-kits, packing-boxes, manipulating-rope, sweeping-piles. Goal states (not provided to learners) are shown in top left corner of each image.

图6:任务:(主要顺序):块状插入、位置红绿、高塔、对齐箱角、堆积块-金字塔、托盘箱、组装箱、包装箱、操纵rope、扫皮。 目标状态(没有提供给学习者)在每个图像的左上角显示。

6.3 Ravens-10 Benchmark Task Details

6.3 Ravens-10 基准任务细节

A set of 10 tasks (illustrated in Fig. 6) comprise our Ravens-10 benchmark. In each of these tasks, the agent acts with motion primitives parameterized by a sequence of two SE(2) end effector poses. Note that although the two motion-primitive-defining poses are in SE(2), manipulation behaviors that are in 3D can still be achieved (such as stacking blocks on top of each other) due to motion primitives that involve out-of-plane movement.

一套10项任务(在图6中标注)包括我们的Ravens-10基准。 在其中的每一项任务中,该剂都以两个SE(2)端效应器的序列参数化的运动原始物进行作用. 注意,虽然SE(2)中包含两个动原定型的姿势 grandparents,但由于涉及飞机外运动的动原体,3D中的操纵行为(如堆积块相互上方)仍然可以实现.

block-insertion: picking up an L-shaped red block and placing it into an L-shaped fixture with a corresponding hole. The block is 8cm in diameter while the hole is 9cm in diameter. The block and fixture configurations are randomly sampled in the workspace – rejected if in collision with each other, or if partially out of bounds in the workspace.

块插入器:捡起一个L形的红色块,放入一个L形的固定器,并有一个相应的孔. 块直径8cm,孔直径9cm. 区块和固定配置在工作空间中随机采样——如果相互碰撞,或者在工作空间中部分脱离界限,则被拒绝.

place-red-in-green: picking up red blocks and placing them into green bowls amidst other visually distracting objects (e.g., randomly positioned blocks and bowls of different colors). There can be multiple red blocks and/or multiple green bowls, with a random chance for more bowls than blocks. This task requires handling multiple candidate picking and placing locations. Since bowls are twice as large as the blocks, this task does not require precision. Red blocks and green bowls are first added to the workspace, then 10 objects (blocks or bowls of other colors) are randomly spawned in collision-free configurations.

** 放置-红绿**: 捡起红色块,放在其他可视分辨物体(如随机定位的块和不同颜色的碗)之间的绿色碗中。 可能存在多个红色块和/或多个绿色碗,随机机会比块多一些碗. 这项工作需要处理多个候选人选取和放置地点的问题。 由于碗是块块的两倍,因此这一任务不需要精确. 红块和绿碗首先加入工作空间,然后10个物体(其他颜色的块或碗)在无碰撞配置中随机产卵.

towers-of-hanoi: sequentially picking up disks and placing them into pegs such that all 3 disks initialized on one peg are moved to another, and that only smaller disks can be on top of larger ones. Completing this task requires sequencing 7 pick-and-places of the disks in a specific order. During evaluation, no information about which disk should be picked next is provided – the approach needs to infer this from contextual visual cues e.g., the positions or absence (due to occlusions) of other disks. This tests the capacity of the learner to infer the sequential semantics of Markovian vision-based manipulation tasks. Fig. 7 illustrates this task, its ground truth sequence, picking action distribution, and placing action distributions (unimodal for translation-only placing, or spiral shaped for the SE(2) placing). The base of the 3 pegs are randomly configured in the workspace, and disks are initialized on the peg on the lightest side of the base.

towers-of-hanoi: 依次拾起磁盘并放入插座,这样一个插座上初始化的3个磁盘全部移到另一个,只有较小的磁盘可以放在更大的磁盘上. 完成这项任务需要按特定顺序排列7个磁盘的拣选位置. 在评价期间,没有提供关于下一步应选哪个磁盘的信息 -- -- 这种方法需要从背景视觉提示推断,例如其他磁盘的位置或不存在(由于被占用)等。 这测试了学习者推断马尔科维安视觉操纵任务的顺序语义的能力. 图7说明了这项任务、其地面真实序列、选择动作分布和放置动作分布(仅翻译放置的单一模式,或SE(2)放置的螺旋形状)。 3页的基部在工作空间中随机配置,磁盘在基部最轻侧的基部的基部上初始化.

Figure 7: In Towers of Hanoi (a), the task is to sequentially pick-and-place 3 disks from the first peg to the third peg without placing a larger disk over a smaller disk (c). In each step, the distribution of successful picks is ring-shaped (b), and the distribution of successful pick-conditioned placements (d) is unimodal with translation-only placing, or spiral-shaped with SE(2) placing (projected onto a ring for visualization, where the ring traverses different rotations). The nature of this task evaluates the capacity of Transporter Networks to recover non-trivial spatial action distributions, and to sequence actions by leveraging contextual visual cues.

图7:在河内塔(a)中,任务是按顺序将3个磁盘从第一个连接点到第三个连接点,而不将更大的磁盘放在一个较小的磁盘(c)上。 在每个步骤中,成功选手的分布为环形(b),成功选手条件投放(d)的分布为单模式,仅限翻译投放,或螺旋形状为SE(2)投放(预计在环上可视化,其中环绕不同旋转). 这项任务的性质评价了Transporter Network恢复非三角空间动作分布的能力,以及利用上下文视觉提示来排列动作顺序的能力。

align-box-corner: picking up a randomly sized box and repositioning it on the tabletop to align one of its corners to a green L-shaped marker labeled on the tabletop. This task requires precision, and the ability to generalize to new box sizes. Aligning any of the box’s corners to the green marker yields task success. The marker and box are randomly configured in the workspace such that the box does not overlap the marker.

对齐箱角器:捡起一个随机大小的盒子,并将其重新放置在桌面上,使其一个角落与表顶上标注的绿色L形标记对齐. 这项任务需要精准度,并且能够概括到新的框大小. 将盒子的任何角落与绿色标记对齐后, 任务就会成功 。 标记和框在工作空间中随机配置,这样框就不会与标记重叠.

stack-block-pyramid: sequentially picking up 6 blocks (each with a different color) and stacking them into a pyramid of 3-2-1. The bottom 3 blocks can only be purple, green, or blue blocks. The middle 2 blocks can only be orange or yellow blocks. The top block can only be red. A successful pyramid is invariant to the permutation of blocks within each row, but not between rows. The blocks and base of the pyramid are randomly spawned in collision-free configurations.

stack-block-pyramid: 依次拾起6块(每块颜色不同),堆成3-2-1的金字塔. 底层3块只能是紫色,绿色,或蓝色的块. 中间的2块只能是橙色或黄色的块. 顶部块只能是红色的. 一个成功的金字塔对每行内的区块的轮廓变化无常,但不在行间. 金字塔的区块和底部在无碰撞配置中随机产卵.

palletizing-boxes: picking up fixed-sized boxes and stacking them on top of a pallet. To mimic real stability constraints, the arrangement of boxes should be transposed across each layer of the pallet. There are a fixed number of boxes – each one is spawned into the workspace prior to each pick-and-place action.

pallitizing-boxes: 拿起固定大小的盒子,堆放在托盘上方. 为了模仿真正的稳定性限制,箱的排列应该跨托盘的每层换乘. 有固定数量的盒子——每个盒子在每次采摘和放置动作前都产入工作空间.

Figure 8: Kitting objects used in training (left 14) and testing (right 6).

图8:用于训练(左14)和试验(右6)的击打物体。

assembling-kits: picking and placing different shaped objects onto corresponding locations on a board (visually marked with corresponding silhouettes of the objects). For each episode, 5 objects are randomly selected with replacement from a set of 20: 14 used during training (left), and 6 held out for testing (right). This task requires precise placing and generalizing to new objects by learning the concept of ”how things fit together” [24]. The kit and 5 objects are randomly spawned in collision-free configurations.

** 组装-基点**: 选取不同形状的物体并将其放置在棋盘上相应位置(可视地标上物体的相应圆筒)。 每集中,有5个物体被随机选取,替换的一组为20个:14个在训练期间使用(左),6个被扣进行测试(右). 这项任务要求通过学习“事物如何相适应”的概念,准确定位和概括到新的目标[24]。 包和5个物体在无碰撞配置中随机产卵.

packing-boxes: picking and placing randomly sized boxes tightly into a randomly sized container. This task is the hardest one in the benchmark, which requires not only relying on visual cues to tightly fit objects together, but also implicitly learning to order the objects in ways that maximize container coverage. During initialization, the container size is first randomly sampled, then randomly partitioned using a binary search tree (with minimum size of 5cm along each dimension). Each partition then parameterizes the size of each box, individually spawned into the workspace with a collision-free configuration.

** 包装箱**: 将随机大小的箱紧紧地选入随机大小的容器。 这项任务是基准中最困难的任务,它不仅需要依靠视觉提示来紧密地将物体组合在一起,而且还需要隐含地学习如何以最大程度的容器覆盖来排列物体. 在初始化期间,容器大小首先随机取样,然后使用二进制搜索树(每个尺寸的最小尺寸为5cm)进行随机分割. 每个分区然后将每个框的大小参数化,单独产入工作空间,并具有无碰撞配置.

manipulating-rope: manipulating a deformable rope such that it connects the two endpoints of an incomplete 3-sided square (colored in green). The cable is initialized as a randomly formed pile in the workspace, which needs to be untangled then aligned with the square. This task requires sequencing actions with closed-loop visual feedback to incrementally adjust the rope. The rope is implemented as a set of 20 articulated spherical beads, randomly dropped onto the workspace. The 3-sided square is randomly configured in the workspace.

** 操纵-rope**: 操纵可变形的绳子,使其连接不完全的3边方块的两个端点(颜色为绿色). 电缆作为工作空间中随机形成的堆积物初始化,需要解缠然后与方块对齐. 这项任务需要用闭路视反馈来排序动作,以逐渐调整绳子. 绳子作为一组20个明晰的球形珠子执行,随机投放到工作空间. 3面方块在工作空间中随机配置.

sweeping-piles: pushing piles of small objects (randomly initialized) into a desired target goal zone on the tabletop marked with green boundaries. This sequential task (inspired by [12]) requires sequencing pushes with closed-loop visual feedback and handling piles of small objects. In this task, the suction end effector is swapped with a spatula. The small objects and the target goal zone are randomly initialized in the workspace.

扫射-皮层:将堆积的小物体(随机初始化)推入标有绿色边界的台顶上的理想目标球区. 这个顺序任务(由[12]启发)需要用闭路视反馈和处理小物体堆积的顺序推. 在此任务中,吸管末端效应器与溅射器互换. 小物体和目标目标区在工作空间中随机初始化.

6.4 Additional Tasks

6.4 额外任务

In addition to the Ravens-10 tasks, we used two other tasks in the paper for analysis. The first was to provide a simplified test environment for testing the difference between stochastic and deterministic demonstrations (Sec. 4), and interpolation/extrapolation experiments with a small enough amount of degrees of freedom such that they could be plotted (Sec. 6.10). The second task was to test the SE(3) variant of Transporter Networks.

除了Ravens-10任务外,我们还利用了论文中的另外两项任务进行分析. 第一项是提供一个简化的试验环境,以测试具有弹性和决定性的演示(第4节)与内插/外插实验之间的差别,其自由度应小到足以进行绘图(第6.10节)。 第二个任务是测试Transporter Network的SE(3)变体.

2DoF simplified block-insertion: This task is identical to the block-insertion task, except: (i) the block is always in the same staring configuration, and (ii) the fixture varies only in translation. The environment initialization has 2 degrees of freedom total.

2DoF 简化块插入: 此任务与block-插入任务相同,但: (一)该块总是处于相同的凝视配置中, (二)固定图只在翻译上有所不同. 环境初始化总自由度为2度.

6DoF block insertion: This task is identical to the block-insertion task, except that the fixture varies in 6 degrees of freedom, requiring SE(3) placing. In addition to the table-plane degrees of freedom, the fixture also varies in height ($z$) as well as the two rotational degrees of freedom out of the plane, which may be represented by roll and pitch. The block varies only in SE(2). The environment initialization has 9 degrees of freedom total.

6 DoF 块插入: 这项任务与block-插入任务相同,只是固定在自由度的6度不同,需要SE(3)放置. 除了箴言自由的平面度以外,固定度在高度($z$)以及自由出飞机的两个旋转度也有所不同,它们可能由roll和pitch代表. 该区块仅在SE(2)中有所不同. 环境初始化总自由度为9度.

6.5 Evaluation Metrics

6.5 评测指标

Performance on each task in simulation is evaluated in one of two ways:

模拟中每项任务的业绩以两种方式之一加以评价:

Pose: object translation and rotation to target pose is less than a threshold $\tau=1$cm and $\omega=15^{\circ}$ respectively. If completing a task requires sequencing multiple actions, then each successful action (where an object is moved to its correct pose) returns a partial reward $r=\frac{1}{\textrm{\# of actions}}$. Total rewards for completing a task always sums to 1. Object symmetries are accounted for. Tasks: block-insertion, towers-of-hanoi, place-red-in-green, align-box-corner, stack-block-pyramid, assembling-kits.

• Pose:* 对象翻译和旋转到目标姿势分别低于$\tau=1$厘米和$\omega=15^{\circ}$的阈值. 如果完成一个任务需要排列多个动作的顺序,那么每个成功的动作(如果一个物体被移动到正确的位置)都会返回部分奖励$r=\frac{1}{\textrm{\# of actions}}$ . 完成一项任务的总回报总和为1。 对象对称性进行衡算. 任务:区块插入,哈诺伊塔,位置红绿,对齐箱角,堆块金字塔,组装金字塔.

Zone: ratio of object(s) in the target zone. We discretize each object’s 3D bounding box into 2cm3 voxels. Total reward is the fraction of total voxels in the target zone $r=\frac{\textrm{\# of voxels in target zone}}{\textrm{total \# of voxels}}$. Tasks: palletizing-boxes, packing-boxes, manipulating-cables, sweeping-piles. Achieving total reward $r=1$ on palletizing-boxes and packing-boxes requires tightly fitting all objects into the target zone (i.e., it is exponentially more difficult to achieve $r=0.9$ than it is to achieve the first $r=0.5$).

*区:*目标区物体比例. 我们把每个物体的 3D 边框压缩成 2cm3 伏克斯 。 奖励总额是目标区$r=\frac{\textrm{\# of voxels in target zone}}{\textrm{total \# of voxels}}$中全部葡萄酒的比例。 任务:托盘化-箱式,包装-箱式,操纵-箱式,扫码式. 在托盘式箱和包装箱上实现$r=1$的总奖励,需要严格地将所有物体装入目标区(即实现$r=0.9$比实现第一个$r=0.5$更加困难)。

6.6 Data Augmentation

6.6 数据增强

As is common with deep learning methods, we find that data augmentation can substantially benefit generalization performance for Transporter Nets. To control the experimental comparisons with baselines in this paper, all methods used an identical data augmentation distribution. Specifically, a random SE(2) transform, $T^{aug}_{\mathcal{W}}$, relative to the world frame $\mathcal{W}$ was sampled, and all observations (whether the images, or the object poses for the GT-State methods) were adjusted as if the data was observed from the augmentation frame $aug$ rather than the world frame. This augmentation makes the assumption that the absolute position of objects on the table plane does not matter for the tasks – this was indeed the case for all tasks shown. Note that this distribution provides augmentation along the 3 degrees of freedom of SE(2), but we do not apply other augmentations such as imagining different relative configurations of objects – this would not be a feasible assumption for certain tasks that involve multiple objects.

与深层学习方法一样,我们发现,数据扩增可以大大有利于运输商网的通用性能。 为了控制与本文中的基线的试验性比较,所有方法都采用了相同的数据增强分布。 具体来说,相对于世界框架$\mathcal{W}$的随机SE(2)变换($T^{aug}_{\mathcal{W}}$)被采样,所有观测结果(无论是图像,还是GT状态方法所显示的对象)都被调整,仿佛数据是从增强框架$aug$而不是世界框架观察到的. 这种增强可以假设,物体在平面上的绝对位置对于任务并不重要——所有显示的任务确实都是如此. 请注意,这种分布沿着SE(2)的3度自由提供了增强,但我们没有应用其他增强,比如想象对象的不同相对配置——对于涉及多个对象的某些任务来说,这不是一个可行的假设.

6.7 Additional Baseline Details

6.7 额外基线细节

Form2Fit [24] is a recent method for pick-and-place (also similar to Devin et al. [28]), using a matching network that computes dense visual descriptors [9, 47] to associate picking actions to placing actions. We use an author-provided implementation of Form2Fit with 43-layer ResNet backbones of similar parameter count for a fair comparison.

Form2Fit [24]是最近一种采摘和置放的方法(也类似于Devin等人[28]),使用匹配的网络来计算稠密的视觉描述符[9,47],将采摘动作与放置动作联系起来. 我们使用作者提供的Form2Fit与43层的ResNet主干线进行类似参数计数,进行公平比较.

ConvMLP is a model architecture first proposed in [1] for learning control policies from pixel input – commonly used in end-to-end literature. While typically this architecture is used for real-time continuous control and was originally used with a joint torque action space, we tried it here for our motion-primitive-based setting with a spatial action space. The input images are the same as that of Transporter Networks and Form2Fit: top-down reconstructed views from RGB-D data. The architecture consists of a series of three convolutional layers, followed by spatial soft(arg)max [48] and a multi-layer perceptron to regress a 6-dimensional vector of the two SE(2) end-effector poses. The architecture used was almost exactly as in [1], but with the following changes: (i) we used channel depths of 64, 64, 64, rather than the 64, 32, 16 used in the original paper, (ii) we used MLP layer widths of 128, 128, (iii) we used the MLP to regress mixture densities [45] rather than direct regression in order to handle multi-modality, and (iv) we added depth image input to the network, by replicating depth along 3 channels, processing it with one separate 7x7 convolutional layer, and then fusing with RGB after the first layer via concatenation. The addition of depth image input into the model was to be more directly comparable with the other image-based options, which also used depth. As in [1], we use ImageNet-pretrained weights from GoogLeNet [49] for the first RGB convolutional layer, which we find to aid in stable training and generalization. While this network is 10x shallower (3 layers) than the network used for Form2Fit and Transporter Nets (43 layers), we tried to train the same 43-layer backbone network used in Transporter Networks with a spatial soft(arg)max and MLP, but found that the shallower 3-layer network achieved better generalization at least for our tested hyperparameters. Since many tasks involved multi-modal policies, it was necessary to train the network in a way that handles this multi-modality. Mixture Density Networks (MDN) [45] have previously been used for end-to-end manipulation control, for example in [50]. As in [45], we trained these with a negative log likelihood loss. We used a mixture of 26 Gaussians, with a training temperature of 2.5. To match translation and rotational units on a common scale, we equalized 1 $m$ translation with $0.1$ radians. We also found improved generalization by taking the features $\mathbf{z}$ extracted by the spatial soft(arg)max, and concatenating $\mathbf{z}\leftarrow[\mathbf{z},\ \sin(\mathbf{z}),\ \cos(\mathbf{z})]$ as input to the MLP – this was inspired by [51, 52]. We experimented with dropout in the MLP to add regularization, but found this to decrease generalization; the noise from the convolutional feature extraction was enough to regularize the MLP layers. The models were trained with the Adam optimizer for 20,000 steps, batch size 4, with a learning rate of $2e^{-4}$.

ConvMLP是最早在[1]中为从像素输入学习控制政策而提出的模型架构 — 常用于端到端文献. 虽然这个架构通常用于实时连续控制,最初是使用联合扭矩动作空间,但我们在这里尝试了它,作为我们基于运动的具有空间动作空间的设定. 输入图像与Transporter Networks和Form2Fit:从RGB-D数据自上而下重建视图相同. 该架构由三层交替构成,其次为空间软(arg)max[48]和多层受体,以递归两个SE(2)端效应器的6维向量. 使用的建筑几乎与[1]相同,但有以下变化: (一)我们使用了64,64,64的通道深度,而不是原纸中使用的64,32,16的通道深度, (二)我们使用了128,128的MLP层宽, (三)我们使用MLP来递减混合物密度[45]而不是直接回归,以便处理多模式性, (四)我们通过沿着3个通道复制深度,用一个单独的7x7的卷积层进行处理,然后通过凝聚在第一层后与RGB产生裂纹. 在模型中添加深度图像输入,将与其他基于图像的选项更直接的可比性,后者也使用了深度. 与[1]一样,我们使用GoogleNet的ImageNet预加的重量[49]作为第一个RGB卷积层,我们发现这帮助了稳定的培训和概括. 虽然这个网络比Form2Fit和Transporter Nets(43层)使用的网络要浅10x(3层),但我们尝试用空间软(arg)max和MLP来训练运输网络中所使用的相同的43层骨干网络,但发现较浅的3层网络至少对我们测试的超参数实现了更好的概括. 由于许多任务涉及多式联运政策,因此有必要以处理这种多式联运的方式对网络进行培训。 混合密度网络(MDN)[45]此前曾用于端到端的操纵控制,例如[50]. 和[45]一样,我们用负对数概率损失来训练他们。 我们使用了26高斯人混合体,训练温度为2.5。 为了在通用尺度上匹配翻译和旋转单位,我们将1 $m$翻译与$0.1$弧度相等. 我们还通过将空间软(arg)max所提取的$\mathbf{z}$特性作为MLP的输入而发现了改进的概括性——这是受[51,52]启发的. 我们试验了MLP中的退学现象来增加正规化,但发现这降低了通俗化;从革命特征提取出来的噪音足以使MLP层规范化. 这些模型与亚当优化器一起接受了两万步,批量尺寸4的训练,学习率为$2e^{-4}$.

GT-State MLP is a model architecture for learning control policies directly from ground truth state (i.e., object poses and 3D bounding boxes) as input, similar to the baseline used in [10]. In the real world, this would be analogous to assuming perfect object pose estimation was provided by an external vision system. The model shares an identical MLP to the ConvMLP model described above, except the input $\mathbf{z}$ to the model was all degrees of freedom of the environment (object poses, bounding boxes, and in the case of the place-red-in-green task we also added the RGB color of each object as input). The bounding boxes are used for the 3 tasks (palletizing-boxes, align-box-corner, and packing-boxes) that involve boxes that vary over their length, width, and height, so the bounding boxes exactly parameterize these shapes. Some environments have a variable number of objects, and to handle this we padded the input vector with zeros to match the largest cardinality seen in the environment sampling. We used dropout in the MLP layers with a dropout rate of $p=0.1$, which we found to improve generalization. The models were trained with the Adam optimizer for 40,000 steps, batch size 128, with a learning rate of $2e^{-4}$.

GT-State MLP是直接从地面真核状态(即对象姿势和3D边框)学习控制政策的模型架构,作为输入,类似于[10]中使用的基准. 在现实世界中,这类似于假设一个外部视觉系统提供了完美的物体姿势估计。 该模型与上述ConvMLP模型共享一个相同的MLP,但该模型的输入$\mathbf{z}$是环境的所有自由度(对象外形,边框,在位置红绿任务的情况下,我们还添加了每个对象的RGB颜色作为输入). 边框用于3项任务(palletization-box,对齐箱-corner,以及包装箱),这些任务涉及的箱的长度,宽度和高度各不相同,所以边框精确地参数化了这些形状. 一些环境有可变的物体数量,为了处理这个问题,我们把输入矢量加零,以匹配环境采样中看到的最大的核心. 我们使用在 MLP 层的辍学率为 $p=0.1$, 我们发现这可以改善通俗化。 这些模型与亚当优化器一起接受了4万步,批号128的训练,学习率为$2e^{-4}$.

GT-State MLP 2-step is composed of two MLP networks, each of which are almost identical to GT-State MLP. The first regresses only the first SE(2) pose, and the result of this is added to the observation vector of the second network, which regresses the second SE(2) pose. We used this model to experiment with whether the benefit of Transporter Networks was due to their sequential model cascade, where the second model is conditioned on the first (i.e., pick-conditioned placing). We experimented with training the second MLP model on noisy regressions from the first model – but perhaps because our models were sufficiently regularized with $p=0.1$ dropout, we found this to not provide measurable advantages over simpler teacher forcing, in which the second model was only trained on the conditioning from the training data.

*GT-State MLP 2-step *由两个MLP网络组成,每个网络几乎与GT-State MLP相同. 第一种退缩只代表第一种SE(2)姿势,结果被添加到第二种网络的观测矢量中,而第二种SE(2)姿势则退缩. 我们利用这个模型来试验Transporter Network的好处是否是由于它们的顺序模型级联,其中第二个模型以第一个模型为条件(即选择条件放置). 我们试验了第二个MLP模型从第一个模型中进行噪音回归的培训 — — 但也许因为我们的模型已经与$p=0.1$ 辍学现象充分规范化,我们发现这并不能提供比简单的教师强迫化更显著的优势,其中第二个模型只从培训数据中培训了条件。

GT-State MLP 3-step SE(3) is almost identical to the 2-step model, except a third model is conditioned on the outputs of both the first and second model, and the additional out-of-plane degrees of freedom (height, roll, pitch) are regressed. We experimented with also using just one GT-State MLP for the SE(3) action space task, but found similar performance. Additionally, this model took in as input not only the SE(2) degrees of freedoms of objects, but their full SE(3) poses, represented with roll, pitch, yaw for the rotational degrees of freedom. We experimented with other forms of rotational observation, such as observing three 3D points on the object, which does not have 2$\pi$ wrapping issues, but for the 6DoF task tested we found similar performance to the roll, pitch, yaw observation.

*GT-State MLP 3-step SE(3)*与2-step模式几乎完全相同,但第三种模式都以第一种模式和第二种模式的输出为条件,额外的飞机外自由度(高度,滚动,投球)被递减. 我们还试验了仅使用一个GT状态的MLP来完成SE(3)动作空间任务,但发现了类似的性能. 此外,这种模式不仅将物体的SE(2)自由度作为输入,而且还将物体的全部SE(3)作为自由度的旋转度的表示。 我们试验了其他形式的旋转观测,比如在物体上观察三个3D点,它没有2个$\pi$的包装问题,但是对于6DoF任务测试,我们发现与滚,投, ya观测相似的性能.

6.8 Additional Analysis of Sample Efficiency and Baseline Comparisons

6.8 样本效率与基线对比的补充分析

Here we add additional analysis of the results in Tab. 2 and Tab. 3, which show the sample efficiency of baselines trained from stochastic demonstrations for each task, and evaluated on unseen test settings.

在此,我们补充对 Tab. 2 和 Tab. 3 中的结果进行的分析,这些分析显示了从对每项任务的分层演示中培训的基线的样本效率,并在看不见的测试设置上进行评估。

The results suggest that Transporter Networks accelerate end-to-end learning on the wide variety of tasks tested. We believe this is due to the inductive bias of the model, which i) incorporates a high-level prior (that manipulation involves rearranging 3D space), and ii) uses the spatial structure of the visual input to exploit symmetries in the data to better learn useful correlations.

结果表明,Transporter Network加快了对所测试的各种任务的端到端学习. 我们认为这是由于模型的诱导偏差,它一)包含一个高级前置(操纵涉及重排3D空间),二)利用视觉输入的空间结构来利用数据中的对称性来更好地学习有用的关联.

Among the two image-based baselines, Form2Fit performed better than ConvMLP on most tasks – all except the sweeping-piles task, which may be especially hard for the descriptor-based Form2Fit model to work well on the pile of small items, since no item in the pile is significantly different than the others, but rather the policy should be a function of their collective distribution. While for some tasks, like the place-red-in-green task, the Form2Fit model does comparably well to Transporter Networks, it struggles on tasks that require either more precision, or multi-step tasks. The Tab. 3 results for Form2Fit are also especially interesting – although the model can 1-shot generalize with 100% success on the simplified block insertion task, it starts to struggle fitting additional stochastic demonstrations. The ConvMLP model for the most part sees a monotonic increase in performance with additional demonstrations, but often does not generalize as precisely as other models in this small-data regime. In all scenarios tested, Transporter Networks outperformed both image-based baselines.

在两个基于图像的基线中,Form2Fit在大多数任务上的表现优于ConvMLP — — 所有任务都例外,因为基于描述的Form2Fit模型在堆积小项目上特别难以运作,因为堆积中没有任何项目与其他项目有显著区别,而政策应该是它们集体分布的功能. 虽然对于一些任务来说,比如位置红绿任务,Form2Fit模型与Transporter Networks相比做得相当好,但它在需要更精确,或多步骤的任务上挣扎. 表2的标签3结果 Fit也特别有趣 — — 尽管该模型可以在简化的块插入任务上以100%的成功率进行1射一般化,但它开始挣扎于适应额外的扭曲演示。 ConvMLP模型大多看到单调性能随着附加演示而增加,但往往不像这个小数据制度中的其他模型那样精确的概括. 在所有经过测试的情景中,Transporter Network的性能都超过了基于图像的基线.

For the state-based baselines, these experiments also show that Transporter Networks often require less data than MLP policies trained from ground truth state (e.g., list of object poses, bounding boxes, and color). Of course, it is possible to have a policy that consumes the ground truth state of the scene and performs perfectly on all tasks – indeed, the scripted expert policies do exactly this. For learning generalizable policies from demonstration, however, our experiments suggest that directly reasoning over the spatial configuration of visual cues may instead provide grounds for discovering more generalizable patterns for manipulation. The simplest example of this may be that it’s easier to learn, from few demonstrations, a policy that is ”pick a red pixel” and ”place in a green pixel”, rather than to regress these values in a continuous space, from a vector of continuous values. Other additional cues that may be easier to learn directly from images include: aligning edges and corners when packing boxes and assembling kits, or matching gripper shapes to object surfaces for grasping.

对于基于状态的基线,这些实验还表明,Transporter Network往往需要的数据少于从地面真实状态(如物体外观列表,边框,以及颜色)训练出来的MLP政策. 当然,可以制定一套政策,消耗现场的实际情况,并完美地完成所有任务 — — 事实上,有脚本的专家政策就是这样做的。 然而,对于从演示中学习一般政策而言,我们的实验表明,直接对视觉提示的空间配置进行推理,可能反而为发现更一般的操纵模式提供依据。 最简单的例子可能是,从很少的演示中可以弄清一个“选择一个红色像素”和“放在一个绿色像素中”的政策,而不是从一个持续值的矢量中在连续空间中倒退这些值。 其他可能比较容易直接从图像中学习的附加提示包括:在包装箱和组装包时将边缘和角对齐,或将抓取器形状与物体表面相匹配,以便抓取.

In addition to the difference of using either images or ground-truth state, it is also important to keep in mind that the GT-State MLP model fundamentally differs in a couple other ways as well, including that its action space is continuous rather than discrete, and it is trained with a multi-modal regression loss rather than a classification loss. It would be interesting to consider other model architectures that consume ground-truth state information and better model intra-object relationships and variable-dimensionality input, such as graph neural networks (GNNs), and also consider models that naturally include some translation equivariance, which MLPs do not. Given the ubiquitity of MLPs in continuous regression tasks, however, and its proximity to the ConvMLP baseline, we consider our chosen MLP model to be a highly relevant baseline.

除了使用图像或地面真实状态的不同之外,还必须铭记,GT-State MLP模型在几个其他方面也有着根本的不同,包括它的动作空间是连续的而不是离散的,它受到的训练是多模式回归损失而不是分类损失。 考虑使用地面真实状态信息的其他模型架构,以及更好的模型内对象关系和可变维度输入,如图神经网络(GNNs),以及考虑自然包括一些翻译等效的模型,MLPs没有. 然而,鉴于在持续回归任务中,多边劳工方案无处不在,而且接近ConvMLP基线,我们认为我们选定的多边劳工方案模型是一个高度相关的基线。

One additional note is that for the sweeping-piles task, we observe that a Transporter Network which is constrained to choose a sequence of two translation-only poses, where the rotation is determined as being orthogonal to the line between these two poses, works better than allowing the model to choose a rotation at each pose. This may be due to the specifics of the sweeping motion-primitive used for this task, but it also may indicate that the rigid-transformation bias in the model has its limits. The pile of objects does not get perfectly rigidly transformed, but rather has more complex dynamics. We plan to investigate this further in future work.

另一个说明是,对于扫地工任务,我们观察到一个运输网,它不得不选择两个只翻译的姿势的顺序,在那里,旋转被确定为正对这两个姿势之间的线条,它比让模型在每个姿势上选择一个旋转更好. 这可能是由于用于这一任务的粗略运动原型的具体特点,但也可能表明模型中的刚性转换偏差有其极限. 堆积的物体不会得到完全的刚性转变,而是具有更复杂的动态. 我们计划在今后的工作中进一步调查这一问题。

6.9 SE(3) Transporter Networks: Architecture Discussion and 6DoF Task Analysis

6.9 SE(3) Transporter Networks：架构讨论与 6DoF 任务分析

Network Architecture Considerations. We experimented with multiple versions of extending Transporter Networks into 6-degree-of-freedom, SE(3) action spaces. The hybrid discrete/continuous model we have presented in the main paper provides a number of nice features, and demonstrates the flexibility of the transport operation. It would also be interesting to try pure-discrete classification models, but fully sampling over the entire action space becomes burdensome with higher degrees of freedom. In the first SE(2) step, convolution can be used to efficiently imagine all overlays for the image-width and image-height dimensions, requiring sampling only over the in-plane rotations. However, the remaining 3 degrees of freedom in SE(3) are all orthogonal to the in-plane image dimensions. While a discrete version could be done with rotating 3D voxel grids, this might present memory and sparsity challenges. Instead, our presented choice of using continuous regression to directly regress the remaining degrees of freedom is an attractive one – essentially building on the same architecture as the SE(2) version with little modification.

** 网络结构考虑。 ** 我们试验了多个版本的 扩展Transporter Network 到6度自由,SE(3)行动空间。 我们在主要文件中介绍的混合离散/连续模型提供了一些良好的特点,并显示了运输作业的灵活性。 尝试纯分明的分类模型也是很有趣的,但对整个动作空间的完全采样随着自由度的提高而变得繁琐. 在第一个SE(2)步骤中,可以用来有效想象图像宽度和图像高度维度的所有覆盖,只需要在飞机内旋转上取样. 然而,SE(3)中剩下的3度自由度与飞机中图像维度完全正交. 虽然可以旋转3D voxel网格来完成离散版本,但这可能会带来内存和痉挛的挑战. 相反,我们提出的选择是使用持续回归来直接递减剩下的自由度,这是一个很有吸引力的选择 — — 基本上建立在与SE(2)版本相同的架构上,很少修改。

Here we provide additional details of our specific evaluated SE(3) approach with results shown in Tab. 4. As discussed in Section 3.1, all SE(2) degrees of freedom are resolved identically to the SE(2)-only models. An additional model then handles the remaining SE(3) degrees of freedom: roll, pitch, and $z$. The additional model is almost identical to the SE(2) pick-conditioned placing model, but is trained to do three independent regressions rather than one classification. In this way, it still benefits from the imagined overlay of the transport operation as an attention mechanism. To adjust the architecture for this purpose, we first increased the output dimensionality to give $d=24$ channels, rather than the $d=3$ channels used for cross-correlation in the the classification models. Additionally, rather than cross-correlating all 24 channels at once and reducing to an $\mathbb{R}^{H\times W\times k}$ tensor, for image height $H$, width $W$, and in-plane rotation discretizations $k$, we cross-correlate three distinct subsets of 8 channels to give an $\mathbb{R}^{H\times W\times k\times 3}$ tensor, which retains sufficient degrees of freedom to regress 3 independent values. These 3 values in this tensor can be directly trained to do regression, but we find precision improvements if we add a small amount of non-linearity. The small non-linearity mapping we use, $f(\cdot)$ in Sec. 3.1, is three independent MLP networks, which each take in one value and regress one value, and are each 3-layer, 32-width MLP networks with ReLu activations. This is similar to 1x1 convolutions on the spatial feature maps, but with non-linearity. We also find increased generalization if, rather than applying the regression loss only to the correct 1-hot label in $\mathbb{R}^{H\times W\times k}$, we also apply regression loss to a small window in that region. We used a window of 7x7 pixels, and +/- 1 discretization in $k$.

在此,我们进一步详细介绍我们具体评估的SE(3)方法,结果见Tab。 4. 如第3.1节所述,所有SE(2)自由度的解决与SE(2)唯一的模式相同。 然后,另一个模型处理剩下的SE(3)自由度:滚、投和$z$。 附加模型与SE(2)的选取条件放置模型几乎完全相同,但经过训练可以进行三个独立的回归而不是一个分类. 这样,它仍然得益于想象中的运输业务的叠加作为关注机制. 为了为此调整架构,我们首先增加了输出维度以给出$d=24$频道,而不是分类模型中用于交叉对映的$d=3$频道. 此外,我们不一次交叉连接所有24个信道,并缩小为$\mathbb{R}^{H\times W\times k}$ 百分位数,为图像高度$H$,宽$W$,以及飞机内旋转盘式$k$,而是将8个信道的三个不同的子集进行交叉连接,以给出$\mathbb{R}^{H\times W\times k\times 3}$百分位数,这保留了足够的自由度来递减3个独立值. 这个变压器中的这3个值可以直接被训练来进行回归,但是如果我们添加少量的非线性,我们就会找到精确的改进. 我们使用的小型非线性映射($f(\cdot)$ in Sec. 3.1)是三个独立的MLP网络,它们各自取一个值并递归一个值,是每个3层,32width的MLP网络带有ReLu激活. 这与空间特征图上的1x1演化类似,但具有非线性. 我们还发现,如果我们不把回归损失仅仅应用到$\mathbb{R}^{H\times W\times k}$中正确的1-热标签上,而是将回归损失应用到该区域的一个小窗口上,那么,我们就会更加普遍。 我们在$k$中使用了7x7像素的窗口,和+/-1盘式.

We tried variations to the above described approach, many of which worked reasonably well. One option, for example, is to use only one pick-conditioned placing network, which produces an $\mathbb{R}^{H\times W\times k\times 4}$ tensor, of which for the last tensor dimension, 1 channel is used for classification loss and the others are used for regression. We found, however, both the classification loss and the regression loss to benefit from using separate networks–this is potentially due to differences in the training dynamics of the classification and regression losses. We also tried various different methods of extracting the 3 regression channels from the spatial feature maps, including using spatial soft(arg)max from the overlaid crops followed by MLPs, and using concatenation rather than cross-correlation before the MLPs. Each of these worked reasonably well.

我们试图改变上述方法,其中许多方法运作良好。 例如,一个选项是只使用一个拣选条件的放置网络,这个网络产生$\mathbb{R}^{H\times W\times k\times 4}$ 变压器,其中最后一个变压器维,一个通道用于分类损失,其他用于回归. 然而,我们发现,分类损失和回归损失都因使用单独的网络而受益,这可能是由于分类和回归损失的培训动态不同所致。 我们还尝试了从空间特征图中提取3个回归通道的各种不同方法,包括使用从覆盖的作物中提取的空间软(arg)max,然后是MLPs,并在MLPs之前使用聚合而不是交叉对映. 每一项工作都相当顺利。

Additional 6DoF Task Performance Analysis. Here we expand on analysis of the results presented in Tab. 4. One important observation is that it was not clear, prior to these experiments, that the model would be able to successfully generalize due to its multi-step approach. In particular, the SE(2) placing step occurs before control of the remaining degrees of freedom. While this might be more challenging for different objects such as long and thin ones, we find that for our tested environments, we are able to get above-90% generalization on this challenging task given 1,000 training examples. Introspection into the accuracy of the different degrees of freedom of this model provides interesting comparisons as well. Specifically, for $n=1,000$ training examples, the mean absolute errors (MAE) for roll/pitch regressions were each in the range of $3$ to $4$ degrees, whereas the MAE for in-image-plane rotation (yaw) were in the range of $10$ degrees, and most often contributed to unsuccessful deployments in test environments. This suggests that future work building on this model could investigate an iterative approach in which the initial degrees of freedom would once again be optimized, potentially iteratively $N$ times, and potentially with increasingly fine resolution discretization. Additionally, it should be noted that in our tested model the discrete step occurs before the regression step, and so in a sense has a more challenging task, since there may not be good alignment of the remaining degrees of freedom. Additionally, we only used SE(2) augmentation as discussed in Sec. 6.6, since the presence of gravity does not make all roll/pitch-augmented worlds equal, and so the remaining SE(3) degrees of freedom do not benefit from the same level of augmentation. We do find however that even in the $n=1$ demonstrations case, with 38 % success, the SE(3) model behaves as if it uses one azimuth angle, consistent with a single roll/pitch being rotated around the $z$ axis, but this is actually a manifold of different roll/pitch values. This shows the benefit of SE(2) augmentation even for the remaining SE(3) degrees of freedom.

** 附加6DoF任务业绩分析。 ** 在此,我们详细介绍对Tab中介绍的结果的分析。 4. 一项重要意见是,在进行这些试验之前,尚不清楚该模型能否由于其多步骤方法而成功概括。 特别是,SE(2)在控制剩余自由程度之前就采取了步骤。 虽然这对长而薄的物体等不同物体来说可能更具挑战性,但我们认为,对我们经过测试的环境来说,我们能够从1 000个培训实例中了解这项具有挑战性的任务。 对这个模型不同自由度的准确性的回顾也提供了有趣的比较. 具体地说,对于$n=1,000$培训实例,卷/皮奇回归的平均绝对误差(MAE)在$3$到$4$度之间,而用于飞机内旋转(yaw)的MAE在$10$度之间,往往导致测试环境中部署失败。 这表明,今后在这种模式基础上开展的工作可以探讨一种迭代方法,即自由的初始程度将再次得到优化,有可能重复$N$次,并有可能随着分辨率的日益微小的磁盘化。 此外,应当指出,在我们测试的模式中,离散步骤发生在回归步骤之前,因此在某种程度上任务更具有挑战性,因为剩下的自由程度可能无法很好地调整。 此外,我们只使用了第6.6节中讨论的SE(2)增强,因为重力的存在并不能使所有卷/皮奇增强的世界均匀,因此剩余的SE(3)自由度不能从相同的增强水平中受益。 然而,我们确实发现,即便在$n=1$演示案中,成功率也达到了38%,SE(3)模型的表现就像使用了一个方位角,与围绕$z$轴旋转的单卷/pitch一致,但这实际上是不同卷/pitch值的倍数. 这表明SE(2)增强的好处,即使是剩余的SE(3)自由度。

Also as shown in Tab. 4, our tested GT-State MLP model significantly struggled with the 6DoF SE(3) task, much more so than it did with the 3DoF SE(2) task. Investigation into the cause of the challenge shows that this is primarily due to the additional degrees of freedom in the observation and action spaces. In particular we can see this in the tested 3-step model (described in Sec. 6.8), which often struggles to precisely regress the $x$ and $y$ translational degrees of freedom even with $n=1,000$ demonstrations, which was not a problem in the simpler task. Additionally, if we take this task’s data, but reduce the observation and action spaces to only be SE(2), then with $n=1,000$ demonstrations the model is once again able to precisely regress the $x$ and $y$ translations. We hypothesize that the mix of the stochastic demonstrations and the full 6-DoF placing is challenging for the model to learn a generalizable mapping with this amount of data.

同样如Tb. 4所示,我们测试过的GT状态MLP模型与6DoF SE(3)任务发生了很大困难,远比3DoF SE(2)任务更困难. 对挑战原因的调查表明,这主要是因为观察和行动空间中的自由程度增加。 特别是,我们可以从试验的三步模型(第6.8节中描述)中看到这一点,这种模型往往努力使$x$和$y$自由的翻译程度准确地倒退,即使有$n=1,000$的演示,这在简单的任务中不是一个问题。 此外,如果我们接受这项任务的数据,但将观测和行动空间缩小到只有SE(2),那么,随着$n=1,000$的演示,该模型再次能够准确地倒退$x$和$y$的翻译。 我们假设, 混杂的图解 和完全的6-DoF 设置 是一个挑战 模型学习 通用绘图 与这个数量的数据。

6.10 Interpolation and Extrapolation

6.10 插值与外推

Figure 9: Depictions of the generalization ability of different models on the simplified translation-only block-insertion task. Each episode is visualized as a mark on the workspace (representing the pose of the fixture) – color-coded by task success (blue for success, red for failure). Environment configurations in the training set are shown in gray, and the convex hull of the training samples are drawn with a dotted line. Testing episodes inside the convex hell represent interpolation; those outside represent extrapolation. Plot inspired by [10].

图9:不同模型在简化的只翻译块插入任务上的通缩能力偏差. 每集可视化为工作空间上的标记(代表固定器的姿势)——以任务成功(蓝色为成功,红色为失败)为颜色编码. 训练集中的环境配置以灰色显示,训练样本的对流船体用点线绘制. 对流地狱内部的测试事件代表内插;那些外插事件代表外插. 由[10]启发的图.

To gain a better qualitative understanding of the generalization abilities of Transporter Networks and the tested baseline models, we provide Fig. 9. Similar to the analysis in [10], this shows the spatial distribution of training configurations and successful/unsuccessful testing configurations–in this case, for translation-only block insertion with stochastic demonstrations. For each method we visualize its ability to interpolate and extrapolate from its training samples to handle new unseen poses of the target fixture.

为了从质量上更好地了解Transporter Network的通用能力和经过测试的基准模型,我们提供了图。 9. 与[10]中的分析类似,这显示了培训配置和成功/不成功的测试配置的空间分布,在此情况下,仅用于以有条理的演示插入翻译区块。 对于每一种方法,我们可视化它从训练样本中插入和推断的能力,以处理目标固定装置的新的看不见的外观。

As can be seen from these generalization plots, on this task Transporter Networks is capable of extrapolating to new testing episodes with just one demonstration. The dense deep feature transport operation enables the optimal placement of the block in the fixture to look similar under different picking and placing actions (exploiting translational equivariance of the task). This generalizes to new object configurations better than Conv MLP, which seems to only successfully interpolate around the majority of training episodes (often collapsing towards one mode of the training data).

从这些概括图中可以看出,在这个任务上,Transporter Network能够仅用一个演示推断到新的测试事件. 密集的深层特性运输操作使区块在固定位置上的最佳位置能够在不同的采摘和放置动作下看起来类似(探索任务在翻译上的均匀性). 这概括到比Conv MLP更好的新对象配置,它似乎只在大多数训练节目周围成功插入(往往向一种训练数据模式倒塌).

Form2Fit is also capable of extrapolating from a single example, since both its picking and placing networks (also modeled by FCNs) are translationally equivariant – such that the block insertion task amounts to detecting the block and fixture under different translations. Once the number of demonstrations increase, however, the matching network is forced to learn pixel-wise descriptors that index multiple possible picking actions to multiple possible placing actions. In the setting with stochastic demonstrations, the positive-pair samples available for training the matching network are sparse and limited only to the pick-and-place action pairs from demonstrations – making it difficult to geometrically regularize the descriptors uniformly over the pixels on the surface of the block and the fixture. Minor mis-matches during testing can also lead to a slight misalignment of the block, which leads to poor placement and task failure. This suggests that Form2Fit’s matching network requires substantial training data to successful converge to a meaningful solution. The distribution of successful episodes with Form2Fit are also uniformly distributed over the workspace, which further confirms that its failures come largely from mismatching pick-and-place actions, rather than its inability to spatially interpolate or extrapolate.

表单2 Fit还能够从单一的例子中推断出来,因为它的选取和放置网络(也由FCNs建模)在翻译上是等效的 — — 这样块插入任务就等于在不同翻译下检测块和固定. 然而,一旦演示次数增加,匹配网络就被迫学习像素式描述符,将多种可能的选取动作索引为多种可能的动作。 在有花样演示的环境下,可供培训匹配网络的正派样本很少,仅限于演示的挑点和位置动作对 — — 因此很难在区块表面和固定的像素上统一进行几何规范化描述。 测试过程中的细小错配也会导致区块稍有错配,导致放置不良和任务失败. 这表明表2 Fit的匹配网络需要大量的培训数据才能成功与有意义的解决方案汇合。 使用Form2Fit的成功剧集的分布也统一分布在工作区,这进一步证实其失败主要来自于不匹配的拣选和位置动作,而不是其无法在空间上进行插值或推算.

In the low data regime (with 10 demonstrations), GT-State MLP seems to overtrain to correlations that do not necessarily enable task success, e.g., that picking locations are conditioned on the location of the target fixture (see distribution of failed pick locations and fixture locations). By dividing GT-State MLP into two steps – one for picking and one for placing, we see less issues with incorrect correlations. However, the inability to extrapolate too far beyond the distribution of training data still remains (e.g., see failure testing configurations of fixture for GT-State MLP 2-Step with 100 demonstrations, which also caused an incorrect pick point prediction).

在低数据制度中(有10个演示),GT状态的MLP似乎过度了与不一定能够成功完成任务的关联性,例如,选择地点以目标固定位置为条件(见失败选择位置和固定位置的分布). 通过将GT状态的 MLP 分为两个步骤 — — 一个用于选择,一个用于放置,我们看到的问题较少与不正确的关联。 然而,除了分发培训数据外,无法推断过多的情况仍然存在(例如,看到GT-State MLP 2-Step的故障测试配置有100个演示,这也造成了不正确的采点预测)。

6.11 Ablation Studies

6.11 消融实验

Tab. 6 also reports the performance of various ablations of Transporter Networks:

标签。 6还报告了各种运输网的运行情况:

No-Transport uses two FCNs: one to predict a picking heatmap, another to predict placing heatmaps (one per discretized rotation). The transport model is not used here, hence there is no pick-conditioned placing. For simpler tasks, such as place-red-in-green, this could suffice if the picking model predicts higher values on red pixels, and the placing model predicts higher values on green pixels. Or, for block-insertion, if the picking model detects the same location on the block, and the placing model detects the same corresponding location and orientation in the fixture. However, it is much more challenging to execute tasks that require additional visual reasoning, such as aligning unseen box corners. For palletizing and packing boxes, the learned policies are generally capable of placing a few of the boxes into the target zones, but fail to align them to each other tightly to achieve higher completion scores above 55%.

** 不运输** 使用两个FCN:一个预测采暖图,另一个预测放置热图(每盘旋转一次)。 运输型号在此不使用,因此没有搭配条件。 对于更简单的任务,如位置红绿,如果采样模型在红色像素上预测较高的值,而放置模型在绿色像素上预测较高的值,就足够了. 或者,对于块插入,如果采样模型在块上检测到相同的位置,放置模型在固定图中检测到相同的相应位置和方向. 然而,执行需要额外视觉推理的任务,例如调整看不见的方框角,则更具挑战性。 对于托盘化和包装箱,所学政策一般能够将一些箱放入目标区,但未能紧密地相互配合,以达到高于55%的更高完成分数.

Per-Pixel-CE-Loss is a variant of the Transporter Network, where the cross-entropy loss over the entire image (representing a single probability distribution) is replaced with per-pixel cross-entropy, where each pixel represents its own probability distribution. During training, for every demonstration pick-and-place: the picking and placing pixels unraveled from the data are chosen as positive samples for the models respectively, while a fixed number of 100 random negative pixels are sampled from everywhere else. In general, our experiments suggest that the model i) takes more training iterations to converge, and ii) requires more hyper-parameter tuning to balance the ratio of positives to negatives to regularize the loss over time.

Per-Pixel-CE-Loss是运算网的一个变种,整个图像的交叉内损(代表单一概率分布)被换成每像素交叉内损,每个像素代表自己的概率分布. 在培训期间,每个演示选取和放置地点:从数据中分解的采取和放置像素分别被选为模型的阳性样本,而固定数量100个随机的负像素则从其他任何地方取样. 一般来说,我们的实验表明,模型一)需要更多的训练迭代来汇合,二)需要更多的超参数调制,以平衡正负之比,使损失随时间而规范化.

	block-insertion	place-red-in-green	towers-of-hanoi	align-box-corner	stack-block-pyramid
Method	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
Per-Pixel-CE-Loss	89.0	88.0	91.0	90.0	87.5	100	100	100	29.6	57.0	57.9	64.6	46.0	65.0	73.0	77.0	23.0	28.0	33.7	36.0
No-Transport	5.0	8.0	6.0	8.0	81.5	95.7	100	100	2.1	3.6	4.7	3.3	9.0	7.0	11.0	15.0	8.0	13.2	16.0	25.8
	palletizing-boxes	assembling-kits	packing-boxes	manipulating-rope	sweeping-piles
	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
Per-Pixel-CE-Loss	40.3	41.5	39.1	42.7	12.8	48.0	50.0	52.6	44.8	58.1	58.9	68.1	4.9	29.7	28.8	38.9	30.3	29.9	37.5	39.6
No-Transport	44.3	50.4	53.6	54.1	0.6	4.0	1.8	2.8	11.3	40.1	42.3	53.5	33.5	35.3	38.8	40.0	13.7	18.8	28.8	38.1

Table 6: Ablative comparisons. Task success rate (mean %) vs. # of demonstration episodes (1, 10, 100, or 1000) used in training.

	块插入	红绿色	哈诺伊塔	对齐框角	叠块金字塔
方法	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
每像素- CE- 损失	89.0	88.0	91.0	90.0	87.5	100	100	100	29.6	57.0	57.9	64.6	46.0	65.0	73.0	77.0	23.0	28.0	33.7	36.0
无运输	5.0	8.0	6.0	8.0	81.5	95.7	100	100	2.1	3.6	4.7	3.3	9.0	7.0	11.0	15.0	8.0	13.2	16.0	25.8
	托盘化框	组装( K)	包装箱	操纵绳	扫描页
	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000	1	10	100	1000
每像素- CE- 损失	40.3	41.5	39.1	42.7	12.8	48.0	50.0	52.6	44.8	58.1	58.9	68.1	4.9	29.7	28.8	38.9	30.3	29.9	37.5	39.6
都	44.3	50.4	53.6	54.1	0.6	4.0	1.8	2.8	11.3	40.1	42.3	53.5	33.5	35.3	38.8	40.0	13.7	18.8	28.8	38.1

表6:** 结构性比较。 ** 任务成功率(平均%)与培训中使用的演示事件(1、10、100或1000)的#。

6.12 Training Convergence

6.12 训练收敛

We train Transporter Networks with Adam [53], using fixed learning rates of $10^{-5}$. Our models are trained from scratch (i.e., random initialization) in Tensorflow. On the right, we show an example convergence plot over training from 100 demonstrations of manipulating rope. Compared with image-based baselines, Transporter Networks generally converge faster – within a few thousand training iterations, i.e., one or two hours of wall-clock time.

我们用 Adam [53] 训练Transporter Network,使用$10^{-5}$的固定学习率。 我们的模型是从零开始训练的(即随机初始化)在Tensorflow. 在右边,我们展示了一个从100个操纵绳子的演示中 得到训练的趋同图案。 与基于图像的基线相比,Transporter Network一般会更快的汇合——在几千个训练迭代内,即一两个小时的墙点时间.

6.13 Visualizing Transporter Network Predictions

6.13 可视化 Transporter Network 预测

Figure 10: Visualizing Transporter Network Predictions. For the kit assembly task (a), Transporter Network picking predictions show 3 dominant modes on 3 different objects (b) which are not necessarily centered on the object (which may reflect biases learned from the distribution of training data). If a pick is off-center e.g., on the circle block, (c, top), the Transporter Network correctly makes placing predictions along the edges of the target placement location – which is visualized as a ring on $\mathcal{Q}_{\textrm{place}}$ maxed over rotations. If a pick is on the corner of the unseen E-shaped object (c, bottom), then two dominant modes lie on the correct corresponding locations of their target placements – but also with false positive modes from geometric symmetries in the letter E. We also show modes of predicted placing values over rotations for pick-and-place locations centered on the object and fixture, which appear uniform for circle, 4 modes for the cross shape, 5 modes for star shape, 2 modes for the unseen E-shaped block (dominant one being correct). We also visualize how the predicted placing values over rotations change from uniform, to unimodal, when shifting one pixel to the right from the center of the circle.

图10:**可视化Transporter Network预测。 ** 就装箱组装任务(a)而言,Transporter Network采集预测显示3个不同对象(b)上的3个主导模式,这些对象不一定以对象为中心(这可能反映从培训数据分发中吸取的偏差)。 如果一个采样是离中心点的,例如,在圆形块上,(c,顶部),Transporter Network正确地将预测放在目标放置位置的边缘上,这个位置在旋转时最大可视化为$\mathcal{Q}_{\textrm{place}}$上的环. 如果一个采样位于未见的E形对象(c,底部)的角落,那么两个主导模式就位于其目标位置的正确对应位置上 — 但也带有字母E中几何对称的假阳性模式. 我们还显示以对象和固定位置为中心的选择和位置的预测值在旋转之上放置的模式,这些模式对圆形、十字形、恒星形状、看不见的E形块的两种模式(主要为正确模式)。 我们还可以直观地看到,在将一个像素从圆心向右移动时,预测的数值比旋转值从统一到单模式。

Fig. 10 visualizes Transporter Network dense pixel-wise predictions of $\mathcal{Q}_{\textrm{pick}}$ and $\mathcal{Q}_{\textrm{place}}$ for the kit assembly task (trained with 1,000 demonstrations), and how the predicted modes change with respect to the locations of objects, their shapes, and conditioned picks. These visualizations show that due to rotation and translation equivariance, our method can quickly learn modes that reflect symmetries in the data.

图10可视化了运算器网络对成套装配任务的$\mathcal{Q}_{\textrm{pick}}$和$\mathcal{Q}_{\textrm{place}}$的密集像素预测(经过了1,000次演示培训),以及预测模式在物体位置、形状和条件选择方面是如何变化的。 这些可视化显示,由于旋转和翻译等效,我们的方法可以快速学习在数据中反映对称性的模式.

These visualizations also show an interesting failure mode, in which predicted placements of the unseen E-shaped object returns several 180 degree flipped false positives (see Fig. 10c). This is because the learned features from this task rely strongly on local geometry. Since the E shape is a new unseen object, a flipped E looks similar enough to return a false positive signal there (though not enough to be the top mode).

这些可视化还显示出一种有趣的故障模式,其中对未见的E形物体的预测放置返回了数个180度翻转的假阳性(见图10c). 这是因为从这一任务中学到的特征强烈依赖于局部几何. 由于E形状是一个新的隐形对象,一个翻转的E看起来足够相似,可以返回那里一个虚假的正信号(虽然不足以成为顶级模式).

6.14 Real-World Experiments Details

6.14 真实世界实验细节

To demonstrate the real-world performance of Transporter Networks, we apply our approach to two challenging real-world tasks: assembling kits of mouthwash bottles with pick and place, and sweeping piles of small Go pieces with pushing. These tasks are shown in Fig. 11.

为了展示Transporter Network的实事求是的表现,我们将我们的方法应用于两个具有挑战性的实事求是的任务:把装满洗口瓶的包装起来,放好地方,用推力把小块块打碎。 这些任务如图11所示。

For kit assembly, we sequentially pick 5 small mouthwash bottles from a “bin” of identically shaped items and place each into a “kit” with 5 corresponding indentations for placement. This is a challenging task due to the multi-modality and ambiguity of the pick location (many identical items in the bin to pick from), as well as the tight millimeter tolerances of the destination kit. It is designed to mimic a production robotic task of placing items into a blister pack for consumer packaging. Kit assembly and disassembly are treated as separate tasks – both use the same Transporter Network architecture with autonomous switching between the two tasks at inference time. As shown in Table 5, our model achieves 98.9% on kit assembly, trained with 8,141 pick and place actions recorded from human demonstrations. Picking success and disassembly success are both above 99%.

就装箱而言,我们依次从一个形状相同的物品的“容器”中选取5个小口水瓶,并将每个瓶装入一个“基特”,加上5个相应的缩进。 这是一项具有挑战性的任务,因为采摘地点具有多种模式和模糊性(许多相同的物品在纸箱中取取出),而且目的地包的耐用度很强。 它旨在模仿一种生产机器人任务,即将物品装入一个用于消费包装的泡泡包。 Kit 组装和拆卸被作为单独的任务处理 — 两者在推论时间都使用相同的Transporter Network架构,在两个任务之间进行自主切换. 如表5所示,我们的模型在装配工具方面达到98.9%,培训了8 141名选手,并记录了人类演示中的行动。 选取成功和拆解成功率均高于99%.

Figure 11: Real-world tasks. Left: assembling kits of mouthwash bottles, right: pushing piles of small Go pieces.

图11:** 真正的世界任务**。 左:* 组装洗口瓶,* 右:* 推堆小块。

For sweeping, we sequentially push piles of small Go pieces into a target goal zone, marked with green tape. Upon completion of the task, the robot autonomously resets with a scripted scrambling motion. This task is challenging as it tests the capacity of the model to sequence actions using closed-loop visual feedback, and to do it efficiently. As shown in Table 5, our model achieves 98.3% on this task, trained with 6,759 pushing actions recorded from human demonstrations. The average number of sweeps needed to complete the task is 20 $\pm$ 2.5 sweeps, with the fastest run being 16 sweeps and the slowest 25. For reference, the average for human teleoperation is 18.1 sweeps.

为了扫射,我们依次将一堆小戈块推入目标球门区,并标注绿色胶带. 任务完成后,机器人以脚本拼音动作自动重排. 这项任务具有挑战性,因为它测试了模型利用闭路视觉反馈对动作进行排序的能力,并高效完成. 如表5所示,我们的模式在这项工作中实现了98.3%,经过了6 759次从人类演示中记录的推动行动的培训。 完成任务所需的平均扫码为20$\pm$2.5次扫码,最快运行为16次扫码,最慢为25次. 供参考,人类远程操作的平均值为18.1扫码.

To train our system for both tasks, we use human demonstration data captured by a remote teleoperation interface. The sub-optimal, biased and noisy distribution of human demonstrations presents a further challenge for robotic policy learning. In addition, 13 teleoperators of varying skill and performance level provide training data, which further increases the variance of ground-truth actions that we train from and subsequently increases the difficulty of the learning task.

为了训练我们的系统完成这两项任务,我们使用远程远程操作接口所捕获的人类演示数据. 对机器人政策学习而言,人类示威的分布不尽人意、有偏见和吵闹,这是又一个挑战。 此外,13名不同技能和业绩水平的远程操作员提供了培训数据,这进一步增加了我们培训的地面真实行动的差异,从而增加了学习任务的难度。

Hardware Setup

** 硬件设置**

Figure 12: Hardware setup. a) UR5e Robot, b) Piab suction cup, c) Photoneo camera, d) Linux PC, and e) air compressor and vacuum generator.

图12:硬件安装 a)UR5e机器人,b)Piab吸积杯,c)光电相机,d)Linux PC,以及e)空气压缩机和真空发生器.

The hardware setup for kit assembly is shown in Fig 12. Our experiments make use of 2 Universal Robots UR5 workstations (one for kit assembly and one for sweeping), each with an industrial Linux PC, vacuum system, Piab suction cup or brush end-effector, and depth camera (statically mounted above the workstation). To test the robustness of our method to diverse data sources, we use high quality depth camera observations from a Photoneo PhoXi Model M for kit assembly, and those from low cost consumer-grade Azure Kinect for sweeping. The Photoneo camera supplies depth (with 0.1mm rated depth precision) and a greyscale IR image, both at $1032\times 772$ resolution. The Kinect camera supplies depth as well as a color RGB image at $1280\times 720$ resolution.

成套装配的硬件设置见图12。 )和深度摄像头(固定安装在工作站上方), 为了测试我们的方法对不同数据源的强性,我们使用来自Photo PhoXi Model M的高质量深度摄像头来进行装配,以及来自低成本消费级的Azure Kinect来进行扫描。 Photoneo相机提供深度(含0.1mm额定深度精度)和一个灰度的IR图像,两者均在$1032\times 772$分辨率. Kinect相机提供深度以及$1280\times 720$分辨率的彩色RGB图像.

To calibrate the camera within the robot coordinate frame we use a two step procedure. Firstly, the intrinsics of the camera are calibrated by capturing multiple images of a large flat QR-Code panel in varying orientations, and we use OpenCV to calculate the camera intrinsics. Secondly, to calibrate extrinsics we attach QR code tags to the UR5 wrist joint and we capture multiple images of the robot in random end-effector poses. These images are then used to solve for the position of the robot base as well as the offset of the QR Tag to its respective joint with a stochastic optimization procedure.

为了校准机器人坐标框架内的相机,我们采用两个步骤的程序. 首先,通过向它们提供不同方向的大型平面QR-Code面板的多个图像来校准相机的内在内容,我们使用OpenCV来计算相机内在内容. 其次,为了校准外部,我们把QR代码标记附加到UR5腕关节上,我们捕捉机器人在随机端效应器姿势中的多个图像. 这些图像随后用于解析机器人基座的位置,以及将QR Tag与各自关节的抵消,并采用Stochtic优化程序.

A Linux system embedded with the robot gathers robot and camera data and forwards it to the teleoperator over the internet. Any commands passed to the robots are queued up to be processed one-by one. All data is logged in real-time via network to a cloud storage database for model training and evaluation.

与机器人嵌入的Linux系统收集机器人和相机数据,并通过互联网转发给电信操作员. 任何传递给机器人的命令都要排队处理. 所有数据都通过网络实时登录到云存储数据库,用于模型培训和评价.

Data Collection

** 数据收集**

Figure 13: Data Collection UI. Left: Unity-based UI. Middle: The operator selects the area where the object is to be picked up. Right: The operator can adjust the placement of the object and rotate it. The depth image around the pick-up point is overlaid and helps the operator align the placement.

图13:数据集 UI. *左:*以统一为基础的UI. 中: 运算符选择取对象的区域. 权利: 操作员可以调整对象的位置,旋转. 取点周围的深度图像被覆盖,帮助操作员对齐放置.

We designed a 2D User-Interface (UI) scripted in Unity [46] in order to control the robot workstations remotely. An overview of this UI is shown in Fig 13. Features of this UI include: configuration of the robot workcell, movement and actuation of the suction end-effector, configuration of parameterized action trajectories, calibration of the robot and control of the robot using the same action parameterization used to train our model.

我们在Unity设计了2D用户界面(UI)脚本[46],以远程控制机器人工作站. 这种UI的概况见图13。 这个UI的特征包括:机器人工作细胞的配置,吸积末端效应器的移动和激活,参数化动作轨迹的配置,机器人的校准,以及使用用于训练我们模型的相同动作参数化来控制机器人.

A standard teleoperation session is as follows. The operator views a 3D rendering of the workstation depth data. The synthetic camera view can be rotated and translated using mouse movements as desired by the operator. The operator determines with mouse clicks which object to pick by selecting the intended pick-up point on the rendered depth geometry. A region of the point cloud around the pick location is then “attached” to the mouse cursor, where it is rendered transparent and overlaid on the existing depth (illustrated in Fig 13). The operator is able to translate this geometry with movements of the mouse and the single rotational degree of freedom is controlled by the mouse wheel. Once the operator moves the object into the desired location, an additional mouse click will select the 3DOF place location. This interface allows the operator to visualize the intended 3D configuration of the pick-and-place operation before the action is executed. The discrete teleoperator pick-and-place locations are then translated into dense robot trajectories using a Cartesian controller, checked for collisions and other safety constraints, and subsequently executed on-robot. The user then repeats this process until the task is complete.

标准远程操作会话如下。 操作员查看工作站深度数据的三维渲染。 合成相机视图可以按操作员的要求,使用鼠标移动进行旋转和翻译. 操作员通过鼠标点击来决定哪个对象可以选择在渲染的深度几何中选择预定的拾取点。 然后,在鼠标光标上“附着”采摘位置周围的点云区域,使其具有透明度,并覆盖在现有深度上(图13)。 操作员能够用鼠标的移动来翻译这个几何,单旋转自由度由鼠标轮控制. 当操作员将对象移动到想要的位置时,再点击一个鼠标会选择3DOF的位置. 此接口允许操作员在动作执行前可视化预想的 pick-and-place 操作的3D配置. 离散的远程操作器选址位置然后使用笛卡尔控制器转换成密集的机器人轨迹,检查碰撞和其他安全约束,然后在机器人上执行. 用户然后重复这个过程,直到任务完成.

Figure 14: Autonomous Mode UI. User inputs the task to execute (by unique task code), and presses the play button to begin autonomous execution.

图14:** 自主模式UI** 用户输入要执行的任务(通过独特的任务代码),并按下播放按钮开始自动执行.

Note that we use the same data pipeline and motion primitives when executing our Transporter Networks model as is used during human teleoperation; the human UI interaction is replaced with our inference results, and additional logic is added to switch between assembly and disassembly tasks. Autonomous mode is executing within the standard UI by first inputting the task to execute (by unique task code identifier), and pressing start. The UI for this (and visualization of inference results) is shown in Fig 14.

请注意,我们在执行我们的Transporter Network模型时,使用与人类电信操作中使用的相同的数据管道和运动原始;人类UI交互被我们的推论结果所取代,并增加了额外的逻辑来切换组装和拆解任务. 自主模式正在标准UI范围内执行,首先输入要执行的任务(通过独特的任务代码标识符),并按下启动. 图14显示了这方面的UI(以及推断结果的可视化).

Using the above system we collected approximately 8,141 pick-and-place actions from 13 human operators in order to train the kit assembly model, an additional 2,300 actions for validation of model performance and hyper parameter tuning, and 6,759 pushing actions to train the sweeping model.

利用上述系统,我们从13个人类操作者那里收集了大约8,141个采摘和放置动作,以便训练装箱组装模型,另外还有2,300个用于验证模型性能和超参数调试的动作,还有6,759个推动动作来训练扫瞄模型.

Success Detection

** 检测成功**

For the on-robot evaluation in Section 4.2, we manually label robot actions for success or failure. However this process is time consuming and expensive. Additionally, our Transformer Network does not estimate when a multi-action kitting episode terminates (either successfully or unsuccessfully). So in order to perform fully-autonomous pick-and-place without manual success labelling, an automatic task-level success signal was implemented. We trained a ResNet50 classification model - pretrained on imagenet classification - to perform 3-way classification in order to detect full, empty or partial kits. We trained the classifier on 4848 training samples and evaluated on 128 validation set samples from a unique camera angle. Mean classification accuracy on the validation set was 96.9%. Upon completion of a pick-and-place attempt, classifier inference on the RGB image produces a success metric, which marks the successful completion of a Kitting (or emptying the kit) episode. This allows the fully autonomous mode of the system to alternate between tasks (kitting and emptying) upon completion.

对于第4.2节中的机器人操作评价,我们手动给机器人动作贴上成功或失败的标签. 然而,这一过程既费时又昂贵。 此外,我们的Transformer网络并不估计多动作套接字插曲何时终止(成功或不成功)。 因此,为了完全自主地进行采摘和选址而不进行人工成功标记,采用了自动任务级别成功信号. 我们培训了一个ResNet50分类模型 -- -- 对图像网分类进行预先培训 -- -- 以进行3条路的分类,以便检测全包、空包或部分包。 我们在4848个培训样本上训练了分类员, 并从独特的摄像机角度对128个验证样本进行了评估。 鉴定组的平均分类精度为96.9%. 在完成一个选档和选档尝试后,对RGB图像的分类推论产生了一个成功的度量标准,这标志着一个Kitting(或空包)插曲的成功完成. 这使得系统完全自主的模式在完成后可以在任务之间交替(坐稳和空闲).