Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World

域随机化：把深度神经网络从仿真迁移到真实世界

Josh Tobin Affiliation:  OpenAI and UC Berkeley EECS, josh@openai.com    Rachel Fong Affiliation:  OpenAI, {rfong, aray, jonas, woj}@openai.com    Alex Ray Affiliation:  OpenAI, {rfong, aray, jonas, woj}@openai.com    Jonas Schneider Affiliation:  OpenAI, {rfong, aray, jonas, woj}@openai.com    Wojciech Zaremba Affiliation:  OpenAI, {rfong, aray, jonas, woj}@openai.com    Pieter Abbeel Affiliation:  OpenAI, UC Berkeley EECS & ICSI, pieter@openai.com

Abstract

摘要

Bridging the ‘reality gap’ that separates simulated robotics from experiments on hardware could accelerate robotic research through improved data availability. This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator. With enough variability in the simulator, the real world may appear to the model as just another variation. We focus on the task of object localization, which is a stepping stone to general robotic manipulation skills. We find that it is possible to train a real-world object detector that is accurate to $1.5\text{\,}\mathrm{cm}$ and robust to distractors and partial occlusions using only data from a simulator with non-realistic random textures. To demonstrate the capabilities of our detectors, we show they can be used to perform grasping in a cluttered environment. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images (without pre-training on real images) to the real world for the purpose of robotic control.

弥合仿真机器人与真机实验之间的「现实鸿沟」，能让数据更好拿，从而加快机器人研究。本文探讨域随机化：一种在仿真图像上训练、并把渲染随机化后迁移到真实图像的简单做法。只要仿真里变化够大，真实世界在模型看来就可能只是又一种变体。我们聚焦物体定位——通往通用操作技能的垫脚石。结果表明，仅用带非写实随机纹理的仿真数据，就能训出在真机上约 $1.5\,\mathrm{cm}$ 精度、并能扛干扰物与部分遮挡的检测器。为展示能力，我们用它在杂乱场景里完成抓取。据我们所知，这是首次把仅在仿真 RGB 图像上训练（未在真实图像上预训练）的深度网络成功迁到真机、并用于机器人控制。

I INTRODUCTION

I 引言

Performing robotic learning in a physics simulator could accelerate the impact of machine learning on robotics by allowing faster, more scalable, and lower-cost data collection than is possible with physical robots. Learning in simulation is especially promising for building on recent results using deep reinforcement learning to achieve human-level performance on tasks like Atari [27] and robotic control [21, 38]. Deep reinforcement learning employs random exploration, which can be dangerous on physical hardware. It often requires hundreds of thousands or millions of samples [27], which could take thousands of hours to collect, making it impractical for many applications. Ideally, we could learn policies that encode complex behaviors entirely in simulation and successfully run those policies on physical robots with minimal additional training.

在物理仿真里做机器人学习，可比真机更快、更可扩展、成本更低地收集数据，从而加速机器学习对机器人的影响。仿真学习尤其适合承接近期深度强化学习的结果——例如在 Atari 上达到人类水平（[27]）以及机器人控制（[21, 38]）。深度强化学习依赖随机探索，在真机上可能危险；往往需要数十万乃至数百万样本（[27]），采集可耗上千小时，许多应用吃不消。理想情形是：在仿真里学完复杂行为策略，再以极少额外训练就能在真机上跑通。

Unfortunately, discrepancies between physics simulators and the real world make transferring behaviors from simulation challenging. System identification, the process of tuning the parameters of the simulation to match the behavior of the physical system, is time-consuming and error-prone. Even with strong system identification, the real world has unmodeled physical effects like nonrigidity, gear backlash, wear-and-tear, and fluid dynamics that are not captured by current physics simulators. Furthermore, low-fidelity simulated sensors like image renderers are often unable to reproduce the richness and noise produced by their real-world counterparts. These differences, known collectively as the reality gap, form the barrier to using simulated data on real robots.

可惜仿真与真实世界的差距让迁移很难。系统辨识——把仿真参数调到贴合物理系统——耗时且易错。即便辨识做得好，真实世界仍有未建模物理效应，如非刚性、齿轮间隙、磨损与流体，当前物理引擎抓不住。再者，图像渲染一类低保真仿真传感器，往往复现不出真机传感器的丰富与噪声。这些差异合称现实鸿沟，挡住了把仿真数据用到真机上的路。

Fig. 1: Illustration of our approach. An object detector is trained on hundreds of thousands of low-fidelity rendered images with random camera positions, lighting conditions, object positions, and non-realistic textures. At test time, the same detector is used in the real world with no additional training.

图 1：方法示意。检测器在数十万张低保真渲染图上训练，随机化相机位姿、光照、物体位置与非写实纹理；测试时同一检测器零额外训练直接用于真机。

This paper explores domain randomization, a simple but promising method for addressing the reality gap. Instead of training a model on a single simulated environment, we randomize the simulator to expose the model to a wide range of environments at training time. The purpose of this work is to test the following hypothesis: if the variability in simulation is significant enough, models trained in simulation will generalize to the real world with no additional training.

本文探讨域随机化——一种简单却有希望弥合现实鸿沟的方法。我们不在单一仿真环境里训练，而是随机化仿真器，让模型在训练时见到大量环境变体。本文旨在检验如下假设：若仿真中的变化足够大，在仿真里训出的模型就能零额外训练泛化到真实世界。

Though in principle domain randomization could be applied to any component of the reality gap, we focus on the challenge of transferring from low-fidelity simulated camera images. Robotic control from camera pixels is attractive due to the low cost of cameras and the rich data they provide, but challenging because it involves processing high-dimensional input data. Recent work has shown that supervised learning with deep neural networks is a powerful tool for learning generalizable representations from high-dimensional inputs [20], but deep learning relies on a large amount of labeled data. Labeled data is difficult to obtain in the real world for precise robotic manipulation behaviors, but it is easy to generate in a physics simulator.

原则上域随机化可作用于现实鸿沟的任一环节；我们聚焦从低保真仿真相机图像做迁移。用相机像素做机器人控制，成本低、信息丰富，但输入维度高。近期工作表明，用深度网络做监督学习能从高维输入学到可泛化表征（[20]），却依赖大量标注。精确操作行为的真机标注难拿，仿真里却很容易生成。

We focus on the task of training a neural network to detect the location of an object. Object localization from pixels is a well-studied problem in robotics, and state-of-the-art methods employ complex, hand-engineered image processing pipelines (e.g., [6], [5], [44]). This work is a first step toward the goal of using deep learning to improve the accuracy of object detection pipelines. Moreover, we see sim-to-real transfer for object localization as a stepping stone to transferring general-purpose manipulation behaviors.

我们的任务是训练网络检测物体位置。像素级物体定位在机器人学里研究已久，现有顶尖方法多用复杂、手工设计的图像处理流水线（如 [6]、[5]、[44]）。本文是用深度学习改进检测流水线精度的第一步；我们也把物体定位的仿真到真机迁移，看作迁到通用操作行为的垫脚石。

We find that for a range of geometric objects, we are able to train a detector that is accurate to around $1.5\text{\,}\mathrm{cm}$ in the real world using only simulated data rendered with simple, algorithmically generated textures. Although previous work demonstrated the ability to perform robotic control using a neural network pretrained on ImageNet and fine-tuned on randomized rendered pixels [37], this paper provides the first demonstration that domain randomization can be useful for robotic tasks requiring precision. We also provide an ablation study of the impact of different choices of randomization and training method on the success of transfer. We find that with a sufficient number of textures, pre-training the object detector using real images is unnecessary. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images to the real world for the purpose of robotic control.

对多种几何物体，我们仅用算法生成的简单纹理渲染的仿真数据，就能训出真机上约 $1.5\,\mathrm{cm}$ 精度的检测器。此前虽有工作表明：在 ImageNet 上预训练、再在随机化渲染像素上微调的网络可用于机器人控制（[37]），本文首次展示域随机化也能服务需要精度的机器人任务。我们还做了消融，分析不同随机化与训练选择对迁移成败的影响。发现纹理数量足够时，不必用真实图像预训练检测器。据我们所知，这是首次把仅在仿真 RGB 上训练的深度网络成功迁到真机并用于控制。

II RELATED WORK

II 相关工作

II-A Object detection and pose estimation for robotics

II-A 机器人中的物体检测与位姿估计

Object detection and pose estimation for robotics is a well-studied problem in the literature (see, e.g., [4], [5], [6], [10], [44], [50], [54]). Recent approaches typically involve offline construction or learning of a 3D model of objects in the scene (e.g., a full 3D mesh model [44] or a 3D metric feature representation [5]). At test time, features from the test data (e.g., Scale-Invariant Feature Transform [SIFT] features [12] or color co-occurrence histograms [10]) are matched with the 3D models (or features from the 3D models). For example, a black-box nonlinear optimization algorithm can be used to minimize the re-projection error of the SIFT points from the object model and the 2D points in the test image [4]. Most successful approaches rely on using multiple camera frames [6] or depth information [44]. There has also been some success with only monocular camera images [4].

机器人物体检测与位姿估计文献丰富（如 [4]、[5]、[6]、[10]、[44]、[50]、[54]）。近期方法通常先离线构建或学习场景中物体的三维模型（例如完整三维网格 [44]，或三维度量特征 [5]）。测试时，把测试数据中的特征（如 SIFT [12]，或颜色共现直方图 [10]）与三维模型（或其特征）匹配。例如，可用黑盒非线性优化最小化物体模型 SIFT 点与测试图二维点的重投影误差 [4]。多数成功方法依赖多相机帧 [6] 或深度 [44]；也有仅用单目成功的例子 [4]。

Compared to our method, traditional approaches require less extensive training and take advantage of richer sensory data, allowing them to detect the full 3D pose of objects (position and orientation) without any assumptions about the location or size of the surface on which the objects are placed. However, our approach avoids the challenging problem of 3D reconstruction, and employs a simple, easy to implement deep learning-based pipeline that may scale better to more challenging problems.

相对我们的方法，传统路线训练负担更轻，且常利用更丰富的传感，能在不对支撑面位置或尺寸做假设的情况下估计完整三维位姿（位置加朝向）。我们的方法则避开三维重建难题，采用简单易实现的深度学习流水线，或更易扩展到更难问题。

II-B Domain adaptation

II-B 域自适应

The computer vision community has devoted significant study to the problem of adapting vision-based models trained in a source domain to a previously unseen target domain (see, e.g., [9], [14], [15], [19], [23], [25], [51]). A variety of approaches have been proposed, including re-training the model in the target domain (e.g., [52]), adapting the weights of the model based on the statistics of the source and target domains (e.g., [22]), learning invariant features between domains (e.g., [47]), and learning a mapping from the target domain to the source domain (e.g., [43]). Researchers in the reinforcement learning community have also studied the problem of domain adaptation by learning invariant feature representations [13], adapting pretrained networks [35], and other methods. See [13] for a more complete treatment of domain adaptation in the reinforcement learning literature.

计算机视觉界对「源域训好的视觉模型如何迁到未见目标域」做了大量研究（如 [9]、[14]、[15]、[19]、[23]、[25]、[51]）。做法包括：在目标域重训（如 [52]）、按源/目标统计改权重（如 [22]）、学跨域不变特征（如 [47]）、学目标域到源域的映射（如 [43]）等。强化学习社区也研究过域自适应：学不变特征 [13]、适配预训练网络 [35] 等。更完整综述见 [13]。

In this paper we study the possibility of transfer from simulation to the real world without performing domain adaptation.

本文研究的是不做域自适应时，仿真到真机迁移的可能性。

II-C Bridging the reality gap

II-C 弥合现实鸿沟

Previous work on leveraging simulated data for physical robotic experiments explored several strategies for bridging the reality gap.

先前把仿真数据用于真机实验的工作，探索过多种弥合现实鸿沟的策略。

One approach is to make the simulator closely match the physical reality by performing system identification and using high-quality rendering. Though using realistic RGB rendering alone has had limited success for transferring to real robotic tasks [16], incorporating realistic simulation of depth information can allow models trained on rendered images to transfer reasonably well to the real world [32]. Combining data from high-quality simulators with other approaches like fine-tuning can also reduce the number of labeled samples required in the real world [34].

一条路是把仿真做得尽量贴近物理现实：做系统辨识，并用高质量渲染。仅靠写实 RGB 渲染，迁到真机任务的成功率有限 [16]；若再纳入写实深度仿真，渲染图上训的模型迁到真机可更像样 [32]。高质量仿真再配合微调等手段，也能减少真机标注量 [34]。

Unlike these approaches, ours allows the use of low-quality renderers optimized for speed and not carefully matched to real-world textures, lighting, and scene configurations.

与这些不同，我们的方法允许使用为速度优化、未仔细对齐真实纹理/光照/场景配置的低质量渲染器。

Other work explores using domain adaptation techniques to bridge the reality gap. It is often faster to fine-tune a controller learned in simulation than to learn from scratch in the real world [7, 18]. In [11], the authors use a variational autoencoder trained on simulated data to encode trajectories of motor outputs corresponding to a desired behavior type (e.g., reaching, grasping) as a low-dimensional latent code. A policy is learned on real data mapping features to distributions over latent codes. The learned policy overcomes the reality gap by choosing latent codes that correspond to the desired physical behavior via exploration.

另一些工作用域自适应弥合鸿沟。在仿真里学好的控制器，微调往往比真机从零学更快 [7, 18]。在 [11] 中，作者用仿真数据训变分自编码器，把对应某类期望行为（如伸手、抓取）的电机输出轨迹编成低维潜码；再在真机数据上学习从特征到潜码分布的策略。该策略通过探索选择对应期望物理行为的潜码，从而跨越现实鸿沟。

Domain adaptation has also been applied to robotic vision. Rusu et al. [36] explore using the progressive network architecture to adapt a model that is pre-trained on simulated pixels, and find it has better sample efficiency than fine-tuning or training in the real-world alone. In [46], the authors explore learning a correspondence between domains that allows the real images to be mapped into a space understood by the model. While both of the preceding approaches require reward functions or labeled data, which can be difficult to obtain in the real world, Mitash and collaborators [26] explore pretraining an object detector using realistic rendered images with randomized lighting from 3D models to bootstrap an automated learning learning process that does not require manually labeling data and uses only around 500 real-world samples.

域自适应也被用于机器人视觉。Rusu 等人 [36] 用渐进网络适配在仿真像素上预训练的模型，发现样本效率优于微调或仅在真机训练。在 [46] 中，作者学习跨域对应，把真实图像映到模型能理解的空间。前述两类多需奖励或标注，真机上往往难拿；Mitash 等人 [26] 则用带随机光照的写实渲染（来自三维模型）预训练检测器，再启动几乎不需人工标注、仅约 500 张真机样本的自动学习流程。

A related idea, iterative learning control, employs real-world data to improve the dynamics model used to determine the optimal control behavior, rather than using real-world data to improve the controller directly. Iterative learning control starts with a dynamics model, applies the corresponding control behavior on the real system, and then closes the loop by using the resulting data to improve the dynamics model. Iterative learning control has been applied to a variety of robotic control problems, from model car control (e.g., [1] and [8]) to surgical robotics (e.g., [48]).

相关思路是迭代学习控制：用真机数据改进用于求最优控制的动力学模型，而不是直接改控制器。流程是：从动力学模型出发，在真系统上施加相应控制，再用所得数据回写改进模型。迭代学习控制已用于多种机器人问题，从模型车控制（如 [1]、[8]）到手术机器人（如 [48]）。

Domain adaptation and iterative learning control are important tools for addressing the reality gap, but in contrast to these approaches, ours requires no additional training on real-world data. Our method can also be combined easily with most domain adaptation techniques.

域自适应与迭代学习控制都是应对现实鸿沟的重要工具；与之相对，我们的方法不要求在真机数据上再训练。它也容易与多数域自适应技术组合。

Several authors have previously explored the idea of using domain randomization to bridge the reality gap.

已有若干工作用域随机化弥合现实鸿沟。

In the context of physics adaptation, Mordatch and collaborators [28] show that training a policy on an ensemble of dynamics models can make the controller robust to modeling error and improve transfer to a real robot. Similarly, in [2], the authors train a policy to pivot a tool held in the robot’s gripper in a simulator with randomized friction and action delays, and find that it works in the real world and is robust to errors in estimation of the system parameters.

在物理适配方面，Mordatch 等人 [28] 表明：在动力学模型集成上训练策略，能让控制器对建模误差更鲁棒，并改善迁到真机的表现。类似地，[2] 在随机化摩擦与动作延迟的仿真里训练用夹爪枢转工具的策略，发现真机可用，且对系统参数估计误差更稳。

Rather than relying on controller robustness, Yu et al. [53] use a model trained on varied physics to perform system identification using online trajectory data, but their approach is not shown to succeed in the real world. Rajeswaran et al. [33] explore different training strategies for learning from an ensemble of models, including adversarial training and adapting the ensemble distribution using data from the target domain, but also do not demonstrate successful real-world transfer.

Yu 等人 [53] 不靠控制器鲁棒性，而是用在多样物理上训好的模型、借助在线轨迹做系统辨识，但未展示真机成功。Rajeswaran 等人 [33] 探索了从模型集成学习的多种训练策略（含对抗训练、用目标域数据适配集成分布），同样未给出成功的真机迁移。

Researchers in computer vision have used 3D models as a tool to improve performance on real images since the earliest days of the field (e.g., [30], [24]). More recently, 3D models have been used to augment training data to aid transferring deep neural networks between datasets and prevent over-fitting on small datasets for tasks like viewpoint estimation [40] and object detection [42], [29]. Recent work has explored using only synthetic data for training 2D object detectors (i.e., predicting a bounding box for objects in the scene). In [31], the authors find that by pretraining a network on ImageNet and fine-tuning on synthetic data created from 3D models, better detection performance on the PASCAL dataset can be achieved than training with only a few labeled examples from the real dataset.

计算机视觉从很早起就用三维模型提升真图表现（如 [30]、[24]）。更近则把三维模型当增广，帮助跨数据集迁移深度网络、并在小数据上防过拟合，任务包括视点估计 [40] 与物体检测 [42]、[29]。也有工作仅用合成数据训二维检测器（预测包围框）。在 [31] 中，作者发现：先在 ImageNet 预训练，再在由三维模型生成的合成数据上微调，可比仅用少量真实标注在 PASCAL 上取得更好检测性能。

In contrast to our work, most object detection results in computer vision use realistic textures, but do not create coherent 3D scenes. Instead, objects are rendered against a solid background or a randomly chosen photograph. As a result, our approach allows our models to understand the 3D spatial information necessary for rich interactions with the physical world.

与我们不同，多数视觉检测结果用写实纹理，却不构建连贯的三维场景；物体常被渲在纯色背景或随机照片上。我们的做法让模型理解与物理世界丰富交互所需的三维空间信息。

Sadeghi and Levine’s work [37] is the most similar to our own. The authors demonstrate that a policy mapping images to controls learned in a simulator with varied 3D scenes and textures can be applied successfully to real-world quadrotor flight. However, their experiments – collision avoidance in hallways and open spaces – do not demonstrate the ability to deal with high-precision tasks. Our approach also does not rely on precise camera information or calibration, instead randomizing the position, orientation, and field of view of the camera in the simulator. Whereas their approach chooses textures from a dataset of around $200$ pre-generated materials, most of which are realistic, our approach is the first to use only non-realistic textures created by a simple random generation process, which allows us to train on hundreds of thousands (or more) of unique texturizations of the scene.

与我们最接近的是 Sadeghi 与 Levine [37]。他们表明：在多样三维场景与纹理的仿真里学到的「图像到控制」策略，可成功用于真机四旋翼飞行。但其实验——走廊与空旷处的避障——未展示高精度任务能力。我们的方法也不依赖精确相机信息或标定，而是在仿真中随机化相机位置、朝向与视场。他们从约 $200$ 种预生成材质中选纹理（多数偏写实）；我们首次仅用简单随机过程生成的非写实纹理，从而能在数十万（乃至更多）种独特场景纹理化上训练。

III METHOD

III 方法

Given some objects of interest $\{s_{i}\}_{i}$, our goal is to train an object detector $d(I_{0})$ that maps a single monocular camera frame $I_{0}$ to the Cartesian coordinates $\{(x_{i},y_{i},z_{i})\}_{i}$ of each object. In addition to the objects of interest, our scenes sometimes contain distractor objects that must be ignored by the network. Our approach is to train a deep neural network in simulation using domain randomization. The remainder of this section describes the specific domain randomization and neural network training methodology we use.

给定感兴趣物体集合 $\{s_{i}\}_{i}$，目标是训练检测器 $d(I_{0})$：把单张单目相机帧 $I_{0}$ 映到各物体笛卡尔坐标 $\{(x_{i},y_{i},z_{i})\}_{i}$。场景中有时还有干扰物，网络须忽略。我们的做法是用域随机化在仿真中训练深度网络。本节其余部分说明所用的域随机化与网络训练细节。

III-A Domain randomization

III-A 域随机化

The purpose of domain randomization is to provide enough simulated variability at training time such that at test time the model is able to generalize to real-world data. We randomize the following aspects of the domain for each sample used during training:

域随机化的目的是在训练时提供足够仿真变化，使测试时能泛化到真机数据。每个训练样本我们随机化以下方面：

• • Number and shape of distractor objects on the table
• • Position and texture of all objects on the table
• • Textures of the table, floor, skybox, and robot
• • Position, orientation, and field of view of the camera
• • Number of lights in the scene
• • Position, orientation, and specular characteristics of the lights
• • Type and amount of random noise added to images

• 桌上干扰物的数量与形状
• 桌上所有物体的位置与纹理
• 桌子、地面、天空盒与机器人的纹理
• 相机的位置、朝向与视场
• 场景中灯光数量
• 灯光的位置、朝向与高光特性
• 加到图像上的随机噪声类型与强度

Since we use a single monocular camera image from an uncalibrated camera to estimate object positions, we fix the height of the table in simulation, effectively creating a 2D pose estimation task. Random textures are chosen among the following:

由于我们用未标定单目相机的单帧估计物体位置，仿真中固定桌面高度，实质上做成二维位姿估计任务。随机纹理从下列中选取：

(a) A random RGB value
(b) A gradient between two random RGB values
(c) A checker pattern between two random RGB values

(a) 随机 RGB 值
(b) 两个随机 RGB 之间的渐变
(c) 两个随机 RGB 之间的棋盘格

The textures of all objects are chosen uniformly at random – the detector does not have access to the color of the object(s) of interest at training time, only their size and shape. We render images using the MuJoCo Physics Engine’s [45] built-in renderer. This renderer is not intended to be photo-realistic, and physically plausible choices of textures and lighting are not needed.

所有物体纹理均匀随机——训练时检测器看不到感兴趣物体的颜色，只知其尺寸与形状。图像用 MuJoCo 物理引擎 [45] 内置渲染器生成。该渲染器本不以照片级真实为目标，也不需要物理上合理的纹理与光照选择。

Between $0$ and $10$ distractor objects are added to the table in each scene. Distractor objects on the floor or in the background are unnecessary, despite some clutter (e.g., cables) on the floor in our real images.

每张场景桌上加入 $0$ 到 $10$ 个干扰物。地面或背景干扰物并非必需，尽管我们的真机图像地面上确有线缆等杂物。

Our method avoids calibration and precise placement of the camera in the real world by randomizing characteristics of the cameras used to render images in training. We manually place a camera in the simulated scene that approximately matches the viewpoint and field of view of the real camera. Each training sample places the camera randomly within a $(10\times 5\times 10)$ cm box around this initial point. The viewing angle of the camera is calculated analytically to point at a fixed point on the table, and then offset by up to $0.1$ radians in each direction. The field of view is also scaled by up to $5\text{\,}\mathrm{\%}$ from the starting point.

方法通过在训练渲染时随机化相机特性，避免真机上的标定与精确摆放。我们手工在仿真中放置一台相机，使其视点与视场大致匹配真机；每个训练样本再在该初始点附近 $(10\times 5\times 10)$ cm 盒内随机放置相机。相机朝向解析地指向桌上定点，再在各方向最多偏移 $0.1$ 弧度；视场相对起点最多缩放 $5\%$。

III-B Model architecture and training

III-B 模型结构与训练

Fig. 2: The model architecture used in our experiments. Each vertical bar corresponds to a layer of the model. ReLU nonlinearities are used throughout, and max pooling occurs between each of the groupings of convolutional layers. The input is an image from an external webcam downsized to $(224\times 224)$ and the output of the network predicts the $(x,y,z)$ coordinates of object(s) of interest.

图 2：实验所用模型结构。每根竖条对应一层；全程用 ReLU，各组卷积之间有最大池化。输入为外置摄像头图像，缩到 $(224\times 224)$；网络输出预测感兴趣物体的 $(x,y,z)$ 坐标。

We parametrize our object detector with a deep convolutional neural network. In particular, we use a modified version the VGG-16 architecture [39] shown in Figure 2. We chose this architecture because it performs well on a variety of computer vision tasks, and because it has a wide availability of pretrained weights. We use the standard VGG convolutional layers, but use smaller fully connected layers of sizes $256$ and $64$ and do not use dropout. For the majority of our experiments, we use weights obtained by pretraining on ImageNet to initialize the convolutional layers, which we hypothesized would be essential to achieving transfer. In practice, we found that using random weight initialization works as well in most cases.

我们用深度卷积网络参数化检测器，具体是图 2 所示、改过的 VGG-16 [39]。选择该结构是因其在多种视觉任务上表现好，且易找到预训练权重。卷积层沿用标准 VGG，全连接改为较小的 $256$ 与 $64$，且不用 dropout。多数实验用 ImageNet 预训练权重初始化卷积层——我们原以为这对迁移必不可少；实践中发现随机初始化在多数情形下同样管用。

We train the detector through stochastic gradient descent on the $L_{2}$ loss between the object positions estimated by the network and the true object positions using the Adam optimizer [17]. We found that using a learning rate of around $1\mathrm{e}{-4}$ (as opposed to the standard $1\mathrm{e}{-3}$ for Adam) improved convergence and helped avoid a common local optimum, mapping all objects to the center of the table.

训练用网络估计位置与真值之间的 $L_{2}$ 损失，经 Adam [17] 随机梯度下降。学习率约 $1\mathrm{e}{-4}$（而非 Adam 常用的 $1\mathrm{e}{-3}$）有助于收敛，并避开「把所有物体都映到桌心」这一常见局部最优。

IV EXPERIMENTS

IV 实验

IV-A Experimental Setup

IV-A 实验设置

Fig. 3: The geometric objects used in our experiments.

图 3：实验所用几何物体。

We evaluated our approach by training object detectors for each of eight geometric objects. We constructed mesh representations for each object to render in the simulator. Each training sample consists of (a) a rendered image of the object and one or more distractors (also from among the geometric object set) on a simulated tabletop and (b) a label corresponding to the Cartesian coordinates of the center of mass of the object in the world frame.

我们为八种几何物体各训检测器。为每种物体建网格以便仿真渲染。每个训练样本包含：(a) 物体与一个或多个干扰物（同属几何物体集合）放在仿真桌面上的渲染图；(b) 物体质心在世界坐标系下的笛卡尔坐标标签。

For each experiment, we performed a small hyperparameter search, evaluating combinations of two learning rates ($1\mathrm{e}{-4}$ and $2\mathrm{e}{-4}$) and three batch sizes ($25$, $50$, and $100$). We report the performance of the best network.

每组实验做小规模超参搜索：两种学习率（$1\mathrm{e}{-4}$、$2\mathrm{e}{-4}$）与三种批大小（$25$、$50$、$100$）。报告最佳网络表现。

The goals of our experiments are:

实验目标是：

(a) Evaluate the localization accuracy of our trained detectors in the real world, including in the presence of distractor objects and partial occlusions
(b) Assess which elements of our approach are most critical for achieving transfer from simulation to the real world
(c) Determine whether the learned detectors are accurate enough to perform robotic manipulation tasks

(a) 评估训好的检测器在真机上的定位精度，含干扰物与部分遮挡情形
(b) 判断方法中哪些因素对仿真到真机迁移最关键
(c) 判断学到的检测器是否准到足以支撑机器人操作任务

Detection error for various objects, cm
Evaluation type	Object only	Distractors	Occlusions
Cone	$1.3\pm 1.1$[^1]	$1.5\pm 1.0$	$1.4\pm 0.6$
Cube	$1.3\pm 0.6$	$1.8\pm 1.2$	$1.4\pm 0.6^{\text{1}}$
Cylinder	$1.1\pm 0.9^{\text{1}}$	$1.9\pm 2.8$	$1.9\pm 2.9$
Hexagonal Prism	$0.7\pm 0.5$	$0.6\pm 0.3^{\text{1}}$	$1.0\pm 1.0^{\text{1}}$
Pyramid	$0.9\pm 0.3^{\text{1}}$	$1.0\pm 0.5^{\text{1}}$	$1.1\pm 0.7^{\text{1}}$
Rectangular Prism	$1.3\pm 0.7$	$1.2\pm 0.4^{\text{1}}$	$0.9\pm 0.6$
Tetrahedron	$0.8\pm 0.4^{\text{1}}$	$1.0\pm 0.4^{\text{1}}$	$3.2\pm 5.8$
Triangular Prism	$0.9\pm 0.4^{\text{1}}$	$0.9\pm 0.4^{\text{1}}$	$1.9\pm 2.2$

TABLE I:

各类物体检测误差，厘米
评估类型	仅物体	有干扰物	有遮挡
圆锥	$1.3\pm 1.1$[^1]	$1.5\pm 1.0$	$1.4\pm 0.6$
立方体	$1.3\pm 0.6$	$1.8\pm 1.2$	$1.4\pm 0.6^{\text{1}}$
圆柱	$1.1\pm 0.9^{\text{1}}$	$1.9\pm 2.8$	$1.9\pm 2.9$
六棱柱	$0.7\pm 0.5$	$0.6\pm 0.3^{\text{1}}$	$1.0\pm 1.0^{\text{1}}$
四棱锥	$0.9\pm 0.3^{\text{1}}$	$1.0\pm 0.5^{\text{1}}$	$1.1\pm 0.7^{\text{1}}$
长方体	$1.3\pm 0.7$	$1.2\pm 0.4^{\text{1}}$	$0.9\pm 0.6$
四面体	$0.8\pm 0.4^{\text{1}}$	$1.0\pm 0.4^{\text{1}}$	$3.2\pm 5.8$
三棱柱	$0.9\pm 0.4^{\text{1}}$	$0.9\pm 0.4^{\text{1}}$	$1.9\pm 2.2$

表 I：

IV-B Localization accuracy

IV-B 定位精度

To evaluate the accuracy of learned detectors in the real world, we captured $480$ webcam images of one or more geometric objects on a table at a distance of $70\text{\,}\mathrm{cm}105\text{\,}\mathrm{cm}$ from the camera. The camera position remains constant across all images. We did not control for lighting conditions or the rest of the scene around the table (e.g., all images contain part of the robot and tape and wires on the floor). We measured ground truth positions for a single object per image by aligning the object on a grid on the tabletop. Each of the eight geometric objects has 60 labeled images in the dataset: $20$ with the object alone on the table, $20$ in which one or more distractor objects are present on the table, and $20$ in which the object is also partially occluded by another object.

为评估真机精度，我们采集 $480$ 张摄像头图像：桌上一个或多个几何物体，相机距离约 $70\,\mathrm{cm}$–$105\,\mathrm{cm}$。各图相机位姿固定；未控制光照或桌周场景（例如图中都有部分机器人，以及地面胶带与线缆）。单物体真值位置用桌面网格对齐测得。八种几何物体各有 60 张标注图：$20$ 张仅物体、$20$ 张有一个或多个干扰物、$20$ 张物体被部分遮挡。

Table I summarizes the performance of our models on the test set. Our object detectors are able to localize objects to within $1.5\text{\,}\mathrm{cm}$ (on average) in the real world and perform well in the presence of clutter and partial occlusions. Though the accuracy of our trained detectors is promising, note that they are still over-fitting[^2] the simulated training data, where error is $0.3\text{\,}\mathrm{cm}0.5\text{\,}\mathrm{cm}$. Even with over-fitting, the accuracy is comparable at a similar distance to the translation error in traditional techniques for pose estimation in clutter from a single monocular camera frame [5] that use higher-resolution images.

表 I 汇总测试集表现。检测器在真机上平均能把物体定到 $1.5\,\mathrm{cm}$ 内，且在杂乱与部分遮挡下仍可用。尽管精度可观，模型仍对仿真训练数据过拟合[^2]——仿真误差约 $0.3\,\mathrm{cm}$–$0.5\,\mathrm{cm}$。即便过拟合，在相近距离下其精度已可比肩传统单目杂乱场景位姿估计方法的平移误差 [5]（后者还用更高分辨率图像）。

IV-C Ablation study

IV-C 消融实验

To evaluate the importance of different factors of our training methodology, we assessed the sensitivity of the algorithm to the following:

为评估训练方法各因素的重要性，我们考察算法对以下项的敏感度：

• • Number of training images
• • Number of unique textures seen in training
• • Use of random noise in pre-processing
• • Presence of distractors in training
• • Randomization of camera position in training
• • Use of pre-trained weights in the detection model

• 训练图像数量
• 训练中见过的独特纹理数
• 预处理是否加随机噪声
• 训练时是否含干扰物
• 训练时是否随机化相机位置
• 检测模型是否用预训练权重

We found that the method is at least somewhat sensitive to all of the factors except the use of random noise.

我们发现：除随机噪声外，方法对上述因素都至少有一定敏感。

Fig. 4: Sensitivity of test error on real images to the number of simulated training examples used. Each training example corresponds to a single labeled example of an object on the table with between 0 and 10 distractor objects. Lighting and all textures are randomized between iterations.

图 4：真机测试误差对仿真训练样本数的敏感度。每个训练样本对应一张桌上物体（含 0–10 个干扰物）的标注例；光照与全部纹理在迭代间随机化。

Figure 4 shows the sensitivity to the number of training samples used for pre-trained models and models trained from scratch. Using a pre-trained model, we are able to achieve relatively accurate real-world detection performance with as few as $5,000$ training samples, but performance improves up to around $50,000$ samples.

图 4 给出预训练模型与从零训练模型对训练样本数的敏感度。用预训练模型时，少至 $5{,}000$ 样本即可达到相对准确的真机检测，性能约到 $50{,}000$ 样本仍在提升。

Figure 4 also compares to the performance of a model trained from scratch (i.e., without using pre-trained ImageNet weights). Our hypothesis that pre-training would be essential to generalizing to the real world proved to be false. With a large amount of training data, random weight initialization can achieve nearly the same performance in transferring to the real world as does pre-trained weight initialization. The best detectors for a given object were often those initialized with random weights. However, using a pre-trained model can significantly improve performance when less training data is used.

图 4 也对比了从零训练（不用 ImageNet 预训练权重）的表现。我们原以为预训练对泛化到真机必不可少，结果被证伪。数据量大时，随机初始化迁到真机的表现几乎不输预训练初始化；某物体的最佳检测器往往正是随机初始化的。不过在数据较少时，预训练能显著提升表现。

Figure 5 shows the sensitivity to the number of unique texturizations of the scene when trained on a fixed number ($10,000$) of training examples. We found that performance degrades significantly when fewer than $1,000$ textures are used, indicating that for our experiments, using a large number of random textures (in addition to random distractors and object positions) is necessary to achieving transfer. Note that when $1,000$ random textures are used in training, the performance using $10,000$ images is comparable to that of using only $1,000$ images, indicating that in the low data regime, texture randomization is more important than randomization of object positions.[^3]

图 5 给出在固定训练例数（$10{,}000$）下，对独特场景纹理化数量的敏感度。少于 $1{,}000$ 种纹理时性能明显变差，说明在我们的实验里，大量随机纹理（外加随机干扰物与物体位置）对实现迁移很必要。注意：训练用 $1{,}000$ 种随机纹理时，$10{,}000$ 张图的表现与仅用 $1{,}000$ 张相近——说明在低数据区间，纹理随机化比物体位置随机化更重要。[^3]

Fig. 5: Sensitivity to amount of texture randomization. In each case, the detector was trained using $10,000$ random object positions and combinations of distractors, but only the given number of unique texturizations and lighting conditions were used.

图 5：对纹理随机化程度的敏感度。各组均用 $10{,}000$ 组随机物体位置与干扰物组合训练，但独特纹理化与光照条件数量不同。

Table II examines the performance of the algorithm when random noise, distractors, and camera randomization are removed in training. Incorporating distractors during training appears to be critical to resilience to distractors in the real world. Randomizing the position of the camera also consistently provides a slight accuracy boost, but reasonably high accuracy is achievable without it. Adding noise during pretraining appears to have a negligible effect. In practice, we found that adding a small amount of random noise to images at training time improves convergence and makes training less susceptible to local minima.

表 II 考察去掉随机噪声、干扰物与相机随机化时的表现。训练中加入干扰物，对真机抗干扰似乎至关重要。随机化相机位置也持续带来小幅精度提升，但没有它仍可达到相当高的精度。预训练阶段加噪声影响可忽略。实践中我们发现：训练时给图像加少量随机噪声有助于收敛，并降低陷入局部最优的风险。

Average detection error on geometric shapes by method, cm[^4]
Evaluation	Real images
type	Object only	Distractors	Occlusions
Full method	$\bf{1.3\pm 0.6}$	$\bf{1.8\pm 1.7}$	$\bf{2.4\pm 3.0}$
No noise added	$1.4\pm 0.7$	$1.9\pm 2.0$	$\bf{2.4\pm 2.8}$
No camera randomization	$2.0\pm 2.1$	$2.4\pm 2.3$	$2.9\pm 3.5$
No distractors in training	$1.5\pm 0.6$	$7.2\pm 4.5$	$7.4\pm 5.3$

TABLE II:

按方法汇总的几何形状平均检测误差，厘米[^4]
评估	真机图像
类型	仅物体	有干扰物	有遮挡
完整方法	$\bf{1.3\pm 0.6}$	$\bf{1.8\pm 1.7}$	$\bf{2.4\pm 3.0}$
不加噪声	$1.4\pm 0.7$	$1.9\pm 2.0$	$\bf{2.4\pm 2.8}$
无相机随机化	$2.0\pm 2.1$	$2.4\pm 2.3$	$2.9\pm 3.5$
训练无干扰物	$1.5\pm 0.6$	$7.2\pm 4.5$	$7.4\pm 5.3$

表 II：

IV-D Robotics experiments

IV-D 机器人实验

To demonstrate the potential of this technique for transferring robotic behaviors learned in simulation to the real world, we evaluated the use of our object detection networks for localizing an object in clutter and performing a prescribed grasp. For two of our most consistently accurate detectors, we evaluated the ability to pick up the detected object in 20 increasingly cluttered scenes using the positions estimated by the detector and off-the-shelf motion planning software [41]. To test the robustness of our method to discrepancies in object distributions between training and test time, some of our test images contain distractors placed at orientations not seen during training (e.g., a hexagonal prism placed on its side).

为展示该技术把仿真学到的机器人行为迁到真机的潜力，我们评估用检测网络在杂乱中定位物体并执行指定抓取。对两个 一贯最准的检测器，我们在 20 个逐步更杂乱的场景里，用检测器估计的位置与现成运动规划软件 [41] 尝试抓取。为测试训练与测试物体分布差异下的鲁棒性，部分测试图含训练未见朝向的干扰物（如侧放的六棱柱）。

We deployed the pipeline on a Fetch robot [49], and found it was able to successfully detect and pick up the target object in 38 out of 40 trials, including in highly cluttered scenes with significant occlusion of the target object. Note that the trained detectors have no prior information about the color of the target object, only its shape and size, and are able to detect objects placed closely to other objects of the same color.

流水线部署在 Fetch 机器人 [49] 上：40 次试验中有 38 次成功检测并抓起目标，包括目标被严重遮挡的高度杂乱场景。注意：训好的检测器事先不知道目标颜色，只知其形状与尺寸，且能检测紧挨同色物体放置的目标。

To test the performance of our object detectors on real-world objects with non-uniform textures, we trained an object detector to localize a can of Spam from the YCB Dataset [3]. At training time, the can was present on the table along with geometric object distractors. At test time, instead of using geometric object distractors, we placed other food items from the YCB set on the table. The detector was able to ignore the previously unseen distractors and pick up the target in 9 of 10 trials.

为测试非均匀纹理真机物体上的表现，我们训了检测器定位 YCB 数据集 [3] 中的 Spam 罐头。训练时桌上是罐头加几何干扰物；测试时干扰物换成 YCB 其他食品。检测器能忽略未见过的干扰物，并在 10 次试验中 9 次成功抓起目标。

Figure 6 shows examples of the robot grasping trials. For videos, please visit the web page associated with this paper.[^5]

图 6 给出抓取试验示例。视频见论文配套网页。[^5]

Fig. 6: Two representative executions of grasping objects using vision learned in simulation only. The object detector network estimates the positions of the object of interest, and then a motion planner plans a simple sequence of motions to grasp the object at that location.

图 6：仅用仿真视觉完成抓取的两次代表性执行。检测网络估计感兴趣物体位置，运动规划器再规划一串简单动作在该处抓取。

V CONCLUSION

V 结论

We demonstrated that an object detector trained only in simulation can achieve high enough accuracy in the real world to perform grasping in clutter. Future work will explore how to make this technique reliable and effective enough to perform tasks that require contact-rich manipulation or higher precision.

我们展示：仅在仿真中训练的物体检测器，能在真机上达到足以在杂乱中抓取的精度。未来工作将探索如何让该技术足够可靠有效，以支撑接触丰富或更高精度的操作。

Future directions that could improve the accuracy of object detectors trained using domain randomization include:

有望用域随机化进一步提升检测精度的方向包括：

• • Using higher resolution camera frames
• • Optimizing model architecture choice
• • Introducing additional forms of texture, lighting, and rendering randomization to the simulation and training on more data
• • Incorporating multiple camera viewpoints, stereo vision, or depth information
• • Combining domain randomization with domain adaptation

• 使用更高分辨率相机帧
• 优化模型结构选择
• 在仿真中引入更多纹理、光照与渲染随机化，并加大训练数据
• 纳入多相机视点、立体视觉或深度信息
• 把域随机化与域自适应结合

Domain randomization is a promising research direction toward bridging the reality gap for robotic behaviors learned in simulation. Deep reinforcement learning may allow more complex policies to be learned in simulation through large-scale exploration and optimization, and domain randomization could be an important tool for making such policies useful on real robots.

域随机化是弥合「仿真所学机器人行为」与真机之间鸿沟的有希望方向。深度强化学习或许可通过大规模探索与优化在仿真中学到更复杂策略；域随机化有望成为让这类策略在真机上可用的重要工具。

References

参考文献

[1] Pieter Abbeel, Morgan Quigley, and Andrew Y Ng. Using inaccurate models in reinforcement learning. In Proceedings of the 23rd international conference on Machine learning , pages 1–8. ACM, 2006.

[2] Rika Antonova, Silvia Cruciani, Christian Smith, and Danica Kragic. Reinforcement learning for pivoting task. arXiv preprint arXiv:1703.00472 , 2017.

[3] Berk Calli, Arjun Singh, Aaron Walsman, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In Advanced Robotics (ICAR), 2015 International Conference on , pages 510–517. IEEE, 2015.

[4] Alvaro Collet, Dmitry Berenson, Siddhartha S Srinivasa, and Dave Ferguson. Object recognition and full pose registration from a single image for robotic manipulation. In Robotics and Automation, 2009. ICRA’09. IEEE International Conference on , pages 48–55. IEEE, 2009.

[5] Alvaro Collet, Manuel Martinez, and Siddhartha S Srinivasa. The moped framework: Object recognition and pose estimation for manipulation. The International Journal of Robotics Research , 30(10):1284–1306, 2011.

[6] Alvaro Collet and Siddhartha S Srinivasa. Efficient multi-view object recognition and full pose estimation. In Robotics and Automation (ICRA), 2010 IEEE International Conference on , pages 2050–2055. IEEE, 2010.

[7] Mark Cutler and Jonathan P How. Efficient reinforcement learning for robots using informative simulated priors. In Robotics and Automation (ICRA), 2015 IEEE International Conference on , pages 2605–2612. IEEE, 2015.

[8] Mark Cutler, Thomas J Walsh, and Jonathan P How. Reinforcement learning with multi-fidelity simulators. In Robotics and Automation (ICRA), 2014 IEEE International Conference on , pages 3888–3895. IEEE, 2014.

[9] Lixin Duan, Dong Xu, and Ivor Tsang. Learning with augmented features for heterogeneous domain adaptation. arXiv preprint arXiv:1206.4660 , 2012.

[10] Staffan Ekvall, Danica Kragic, and Frank Hoffmann. Object recognition and pose estimation using color cooccurrence histograms and geometric modeling. Image and Vision Computing , 23(11):943–955, 2005.

[11] Ali Ghadirzadeh, Atsuto Maki, Danica Kragic, and Mårten Björkman. Deep predictive policy training using reinforcement learning. arXiv preprint arXiv:1703.00727 , 2017.

[12] Iryna Gordon and David G Lowe. What and where: 3d object recognition with accurate pose. In Toward category-level object recognition , pages 67–82. Springer, 2006.

[13] Abhishek Gupta, Coline Devin, YuXuan Liu, Pieter Abbeel, and Sergey Levine. Learning invariant feature spaces to transfer skills with reinforcement learning. ICLR 2017, to appear , 2017.

[14] Judy Hoffman, Sergio Guadarrama, Eric Tzeng, Ronghang Hu, Jeff Donahue, Ross Girshick, Trevor Darrell, and Kate Saenko. Lsda: Large scale detection through adaptation. In Neural Information Processing Symposium (NIPS) , 2014.

[15] Judy Hoffman, Erik Rodner, Jeff Donahue, Trevor Darrell, and Kate Saenko. Efficient learning of domain-invariant image representations. arXiv preprint arXiv:1301.3224 , 2013.

[16] Stephen James and Edward Johns. 3d simulation for robot arm control with deep q-learning. arXiv preprint arXiv:1609.03759 , 2016.

[17] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

[18] J Zico Kolter and Andrew Y Ng. Learning omnidirectional path following using dimensionality reduction. In Robotics: Science and Systems , 2007.

[19] Brian Kulis, Kate Saenko, and Trevor Darrell. What you saw is not what you get: Domain adaptation using asymmetric kernel transforms. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on , pages 1785–1792. IEEE, 2011.

[20] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature , 521(7553):436–444, 2015.

[21] Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Research , 17(39):1–40, 2016.

[22] Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normalization for practical domain adaptation. arXiv preprint arXiv:1603.04779 , 2016.

[23] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. Learning transferable features with deep adaptation networks. In ICML , pages 97–105, 2015.

[24] David G Lowe. Three-dimensional object recognition from single two-dimensional images. Artificial intelligence , 31(3):355–395, 1987.

[25] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430 , 2009.

[26] Chaitanya Mitash, Kostas E Bekris, and Abdeslam Boularias. A self-supervised learning system for object detection using physics simulation and multi-view pose estimation. arXiv preprint arXiv:1703.03347 , 2017.

[27] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature , 518(7540):529–533, 2015.

[28] Igor Mordatch, Kendall Lowrey, and Emanuel Todorov. Ensemble-cio: Full-body dynamic motion planning that transfers to physical humanoids. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on , pages 5307–5314. IEEE, 2015.

[29] Yair Movshovitz-Attias, Takeo Kanade, and Yaser Sheikh. How useful is photo-realistic rendering for visual learning? In Computer Vision–ECCV 2016 Workshops , pages 202–217. Springer, 2016.

[30] Ramakant Nevatia and Thomas O Binford. Description and recognition of curved objects. Artificial Intelligence , 8(1):77–98, 1977.

[31] Xingchao Peng, Baochen Sun, Karim Ali, and Kate Saenko. Learning deep object detectors from 3d models. In Proceedings of the IEEE International Conference on Computer Vision , pages 1278–1286, 2015.

[32] Benjamin Planche, Ziyan Wu, Kai Ma, Shanhui Sun, Stefan Kluckner, Terrence Chen, Andreas Hutter, Sergey Zakharov, Harald Kosch, and Jan Ernst. Depthsynth: Real-time realistic synthetic data generation from cad models for 2.5 d recognition. arXiv preprint arXiv:1702.08558 , 2017.

[33] Aravind Rajeswaran, Sarvjeet Ghotra, Sergey Levine, and Balaraman Ravindran. Epopt: Learning robust neural network policies using model ensembles. arXiv preprint arXiv:1610.01283 , 2016.

[34] Stephan R Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. Playing for data: Ground truth from computer games. In European Conference on Computer Vision , pages 102–118. Springer, 2016.

[35] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671 , 2016.

[36] Andrei A Rusu, Matej Vecerik, Thomas Rothörl, Nicolas Heess, Razvan Pascanu, and Raia Hadsell. Sim-to-real robot learning from pixels with progressive nets. arXiv preprint arXiv:1610.04286 , 2016.

[37] Fereshteh Sadeghi and Sergey Levine. (cad) 2 RL: Real single-image flight without a single real image. arXiv preprint arXiv:1611.04201 , 2016.

[38] John Schulman, Sergey Levine, Pieter Abbeel, Michael I Jordan, and Philipp Moritz. Trust region policy optimization. In ICML , pages 1889–1897, 2015.

[39] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

[40] Hao Su, Charles R Qi, Yangyan Li, and Leonidas J Guibas. Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views. In Proceedings of the IEEE International Conference on Computer Vision , pages 2686–2694, 2015.

[41] Ioan A Sucan and Sachin Chitta. Moveit! http://moveit.ros.org .

[42] Baochen Sun and Kate Saenko. From virtual to reality: Fast adaptation of virtual object detectors to real domains. In BMVC , volume 1, page 3, 2014.

[43] Yaniv Taigman, Adam Polyak, and Lior Wolf. Unsupervised cross-domain image generation. arXiv preprint arXiv:1611.02200 , 2016.

[44] Jie Tang, Stephen Miller, Arjun Singh, and Pieter Abbeel. A textured object recognition pipeline for color and depth image data. In Robotics and Automation (ICRA), 2012 IEEE International Conference on , pages 3467–3474. IEEE, 2012.

[45] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on , pages 5026–5033. IEEE, 2012.

[46] Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Pieter Abbeel, Sergey Levine, Kate Saenko, and Trevor Darrell. Adapting deep visuomotor representations with weak pairwise constraints. In Workshop on the Algorithmic Foundations of Robotics (WAFR) , 2016.

[47] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 , 2014.

[48] Jur Van Den Berg, Stephen Miller, Daniel Duckworth, Humphrey Hu, Andrew Wan, Xiao-Yu Fu, Ken Goldberg, and Pieter Abbeel. Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations. In Robotics and Automation (ICRA), 2010 IEEE International Conference on , pages 2074–2081. IEEE, 2010.

[49] Melonee Wise, Michael Ferguson, Derek King, Eric Diehr, and David Dymesich. Fetch and freight: Standard platforms for service robot applications. In Workshop on Autonomous Mobile Service Robots , 2016.

[50] Patrick Wunsch and Gerd Hirzinger. Real-time visual tracking of 3d objects with dynamic handling of occlusion. In Robotics and Automation, 1997. Proceedings., 1997 IEEE International Conference on , volume 4, pages 2868–2873. IEEE, 1997.

[51] Jun Yang, Rong Yan, and Alexander G Hauptmann. Cross-domain video concept detection using adaptive svms. In Proceedings of the 15th ACM international conference on Multimedia , pages 188–197. ACM, 2007.

[52] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Advances in neural information processing systems , pages 3320–3328, 2014.

[53] Wenhao Yu, C Karen Liu, and Greg Turk. Preparing for the unknown: Learning a universal policy with online system identification. arXiv preprint arXiv:1702.02453 , 2017.

[54] Stefan Zickler and Manuela M Veloso. Detection and localization of multiple objects. In Humanoid Robots, 2006 6th IEEE-RAS International Conference on , pages 20–25. IEEE, 2006.

APPENDIX

附录

V-A Randomly generated samples from our method

V-A 本方法随机生成的样本

Figure 7 displays a selection of the images used during training for the object detectors detailed in the paper.

图 7 展示文中检测器训练阶段所用图像的选例。

Fig. 7: A selection of randomly textured scenes used in the training phase of our method

图 7：训练阶段所用随机纹理场景选例