3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Computer Science Department, University of Freiburg, Freiburg, Germany
cicek@cs.uni-freiburg.de
BIOSS Centre for Biological Signalling Studies, Freiburg, Germany
University Hospital Freiburg, Renal Division, Faculty of Medicine, University of Freiburg, Freiburg, Germany
Department of Psychiatry and Psychotherapy, University Medical Center Freiburg, Freiburg, Germany
Google DeepMind, London, UK
Abstract
摘要
This paper introduces a network for volumetric segmentation that learns from sparsely annotated volumetric images. We outline two attractive use cases of this method: (1) In a semi-automated setup, the user annotates some slices in the volume to be segmented. The network learns from these sparse annotations and provides a dense 3D segmentation. (2) In a fully-automated setup, we assume that a representative, sparsely annotated training set exists. Trained on this data set, the network densely segments new volumetric images. The proposed network extends the previous u-net architecture from Ronneberger et al. by replacing all 2D operations with their 3D counterparts. The implementation performs on-the-fly elastic deformations for efficient data augmentation during training. It is trained end-to-end from scratch, i.e., no pre-trained network is required. We test the performance of the proposed method on a complex, highly variable 3D structure, the Xenopus kidney, and achieve good results for both use cases.
本文提出一种用于体积分割的网络,它从稀疏标注的体积图像中学习。我们概述了该方法的两种有吸引力的用法:(1)在半自动设定中,用户在待分割的体积里标注若干切片。网络从这些稀疏标注中学习,并给出稠密的三维分割。(2)在全自动设定中,我们假定已有一套有代表性、只做了稀疏标注的训练集。在该数据集上训练后,网络再对新的体积图像做稠密分割。所提出的网络扩展了 Ronneberger 等人先前的 u-net 结构,把所有二维运算替换成对应的三维运算。实现在训练过程中即时做弹性形变,以便高效地做数据增强。网络从零端到端训练,即不需要预训练网络。我们在一种复杂且高度可变的三维结构——非洲爪蟾(Xenopus)肾脏——上测试了该方法,两种用法都取得了良好结果。
Keywords: Convolutional neural networks · 3D · Biomedical volumetric image segmentation · Xenopus kidney · Semi-automated · Fully-automated · Sparse annotation
关键词: 卷积神经网络 · 三维 · 生物医学体积图像分割 · 非洲爪蟾肾脏 · 半自动 · 全自动 · 稀疏标注
1 Introduction
1 引言
Volumetric data is abundant in biomedical data analysis. Annotation of such data with segmentation labels causes difficulties, since only 2D slices can be shown on a computer screen. Thus, annotation of large volumes in a slice-by-slice manner is very tedious. It is inefficient, too, since neighboring slices show almost the same information. Especially for learning based approaches that require a significant amount of annotated data, full annotation of 3D volumes is not an effective way to create large and rich training data sets that would generalize well.
体积数据在生物医学数据分析中大量存在。给这类数据加上分割标注会遇到困难,因为计算机屏幕上只能显示二维切片。因此,按切片逐张标注大型体积非常乏味。这也没有效率,因为相邻切片几乎显示同一信息。对于需要大量标注数据的学习方法而言,把三维体积全部标注一遍,并不是构建规模大、内容丰富、且能良好泛化的训练集的有效办法。
© Springer International Publishing AG 2016
S. Ourselin et al. (Eds.): MICCAI 2016, Part II, LNCS 9901, pp. 424–432, 2016.
DOI: 10.1007/978-3-319-46723-8_49
Fig. 1. Application scenarios for volumetric segmentation with the 3D u-net. (a) Semi-automated segmentation: the user annotates some slices of each volume to be segmented. The network predicts the dense segmentation. (b) Fully-automated segmentation: the network is trained with annotated slices from a representative training set and can be run on non-annotated volumes.
图 1. 用 3D u-net 做体积分割的应用场景。(a) 半自动分割:用户在每本待分割的体积上标注若干切片。网络预测稠密分割。(b) 全自动分割:网络用一套有代表性的训练集中的已标注切片来训练,然后可以在未标注的体积上运行。
In this paper, we suggest a deep network that learns to generate dense volumetric segmentations, but only requires some annotated 2D slices for training. This network can be used in two different ways as depicted in Fig. 1: the first application case just aims on densification of a sparsely annotated data set; the second learns from multiple sparsely annotated data sets to generalize to new data. Both cases are highly relevant.
在本文中,我们提出一种深度网络,它学习生成稠密的体积分割,但训练时只需要若干已标注的二维切片。该网络可以按图 1 所示的两种方式使用:第一种用法只是要把稀疏标注的数据集补成稠密标注;第二种则从多个稀疏标注的数据集中学习,并泛化到新数据。两种情形都很有实际意义。
The network is based on the previous u-net architecture, which consists of a contracting encoder part to analyze the whole image and a successive expanding decoder part to produce a full-resolution segmentation [11]. While the u-net is an entirely 2D architecture, the network proposed in this paper takes 3D volumes as input and processes them with corresponding 3D operations, in particular, 3D convolutions, 3D max pooling, and 3D up-convolutional layers. Moreover, we avoid bottlenecks in the network architecture [13] and use batch normalization [4] for faster convergence.
该网络基于先前的 u-net 结构:收缩的编码器部分用来分析整幅图像,随后扩张的解码器部分用来产生全分辨率分割 [11]。u-net 是完全二维的结构,而本文提出的网络以三维体积为输入,并用相应的三维运算来处理,特别是三维卷积、三维最大池化和三维上卷积层。此外,我们避免网络结构中的瓶颈 [13],并使用批归一化 [4] 以加快收敛。
In many biomedical applications, only very few images are required to train a network that generalizes reasonably well. This is because each image already comprises repetitive structures with corresponding variation. In volumetric images, this effect is further pronounced, such that we can train a network on just two volumetric images in order to generalize to a third one. A weighted loss function and special data augmentation enable us to train the network with only few manually annotated slices, i.e., from sparsely annotated training data.
在许多生物医学应用中,只需很少的图像就可以训练出一个泛化能力尚可的网络。这是因为每张图像本身就已经包含重复出现、同时又有相应变化的结构。在体积图像中,这一效应更加明显,因此我们可以只用两本体积图像训练网络,并泛化到第三本。加权损失函数和专门的数据增强,使我们能够只用少量手工标注切片,也就是从稀疏标注的训练数据来训练网络。
We show the successful application of the proposed method on difficult confocal microscopic data set of the Xenopus kidney. During its development, the Xenopus kidney forms a complex structure [7] which limits the applicability of pre-defined parametric models. First we provide qualitative results to demonstrate the quality of the densification from few annotated slices. These results are supported by quantitative evaluations. We also provide experiments which shows the effect of the number of annotated slices on the performance of our network. The Caffe [5] based network implementation is provided as OpenSource.
我们在困难的非洲爪蟾(Xenopus)肾脏共聚焦显微数据集上展示了所提方法的成功应用。发育过程中,非洲爪蟾肾脏会形成复杂结构 [7],这限制了预定义参数模型的适用范围。我们首先给出定性结果,说明由少量标注切片补成稠密标注的质量。这些结果得到定量评价的支持。我们还提供了实验,说明标注切片数量对网络性能的影响。基于 Caffe [5] 的网络实现以开源形式提供。
Challenging biomedical 2D images can be segmented with an accuracy close to human performance by CNNs today [3,11,12]. Due to this success, several attempts have been made to apply 3D CNNs on biomedical volumetric data. Milletari et al. [9] present a CNN combined with a Hough voting approach for 3D segmentation. However, their method is not end-to-end and only works for compact blob-like structures. The approach of Kleesiek et al. [6] is one of few end-to-end 3D CNN approaches for 3D segmentation. However, their network is not deep and has only one max-pooling after the first convolutions; therefore, it is unable to analyze structures at multiple scales.
如今,卷积神经网络已经能以接近人类的精度分割具有挑战性的二维生物医学图像 [3,11,12]。受这一成功推动,已有若干工作尝试把三维卷积神经网络用到生物医学体积数据上。Milletari 等人 [9] 提出一种结合霍夫投票的卷积神经网络,用于三维分割。不过他们的方法不是端到端的,并且只适用于紧凑的团块状结构。Kleesiek 等人 [6] 的方法是少数用于三维分割的端到端三维卷积神经网络之一。不过他们的网络不深,并且只在最初的卷积之后做一次最大池化,因此无法在多个尺度上分析结构。
Our work is based on the 2D u-net [11] which won several international segmentation and tracking competitions in 2015. The architecture and the data augmentation of the u-net allows learning models with very good generalization performance from only few annotated samples. It exploits the fact that properly applied rigid transformations and slight elastic deformations still yield biologically plausible images. Up-convolutional architectures like the fully convolutional networks for semantic segmentation [8] and the u-net are still not wide spread and we know of only one attempt to generalize such an architecture to 3D [14]. In this work by Tran et al., the architecture is applied to videos and full annotation is available for training. The highlight of the present paper is that it can be trained from scratch on sparsely annotated volumes and can work on arbitrarily large volumes due to its seamless tiling strategy.
我们的工作基于二维 u-net [11],该网络在 2015 年赢得了多项国际分割与跟踪竞赛。u-net 的结构和数据增强,使得只需少量标注样本就能学到泛化性能很好的模型。它利用了这样一个事实:适当施加的刚性变换和轻微弹性形变,仍然会得到在生物学上说得通的图像。像用于语义分割的全卷积网络 [8] 以及 u-net 这类上卷积结构当时仍不普及,据我们所知,只有一次把这类结构推广到三维的尝试 [14]。在 Tran 等人的那项工作中,该结构被用于视频,并且训练时有完整标注。本文的要点是:它可以在稀疏标注的体积上从零训练,并且凭借无缝拼接策略,能够处理任意大的体积。
2 Network Architecture
2 网络结构
图 2. 3D u-net 结构。蓝色方块表示特征图。每个特征图上方标出通道数。
Figure 2 illustrates the network architecture. Like the standard u-net, it has an analysis and a synthesis path each with four resolution steps. In the analysis path, each layer contains two convolutions each followed by a rectified linear unit (ReLu), and then a max pooling with strides of two in each dimension. In the synthesis path, each layer consists of an upconvolution of by strides of two in each dimension, followed by two convolutions each followed by a ReLu. Shortcut connections from layers of equal resolution in the analysis path provide the essential high-resolution features to the synthesis path. In the last layer a convolution reduces the number of output channels to the number of labels which is 3 in our case. The architecture has 19069955 parameters in total. Like suggested in [13] we avoid bottlenecks by doubling the number of channels already before max pooling. We also adopt this scheme in the synthesis path.
图 2 展示了网络结构。与标准 u-net 一样,它有一条分析路径和一条合成路径,各包含四个分辨率台阶。在分析路径中,每一层包含两次 卷积,每次卷积后接一个修正线性单元(ReLu),然后是在每个方向上步长为 2 的 最大池化。在合成路径中,每一层包含一次在每个方向上步长为 2 的 上卷积,随后是两次 卷积,每次卷积后接一个 ReLu。分析路径中同等分辨率层的捷径连接,向合成路径提供必要的高分辨率特征。在最后一层, 卷积把输出通道数减到标签数,在我们的情形中为 3。该结构总共有 19,069,955 个参数。按 [13] 的建议,我们在最大池化之前就把通道数加倍,以避免瓶颈。合成路径也采用同一做法。
The input to the network is a voxel tile of the image with 3 channels. Our output in the final layer is voxels in , , and directions respectively. With a voxel size of , the approximate receptive field becomes for each voxel in the predicted segmentation. Thus, each output voxel has access to enough context to learn efficiently.
网络的输入是图像中一块带 3 个通道、大小为 体素的图块。最后一层的输出在 、 和 方向上分别为 体素。按体素尺寸 计算,预测分割中每个体素的近似感受野为 。因此,每个输出体素都能获得足够的上下文,从而有效学习。
We also introduce batch normalization (“BN”) before each ReLU. In [4], each batch is normalized during training with its mean and standard deviation and global statistics are updated using these values. This is followed by a layer to learn scale and bias explicitly. At test time, normalization is done via these computed global statistics and the learned scale and bias. However, we have a batch size of one and few samples. In such applications, using the current statistics also at test time works the best.
我们还在每个 ReLU 之前引入批归一化(“BN”)。在 [4] 中,训练时每个批次用自身的均值和标准差做归一化,并用这些值更新全局统计量。随后有一层显式学习缩放和偏置。测试时,用这些已计算的全局统计量以及学到的缩放和偏置来做归一化。不过,我们的批量大小为 1,且样本很少。在这类应用中,测试时也使用当前统计量效果最好。
The important part of the architecture, which allows us to train on sparse annotations, is the weighted softmax loss function. Setting the weights of unlabeled pixels to zero makes it possible to learn from only the labelled ones and, hence, to generalize to the whole volume.
该结构中使我们能够在稀疏标注上训练的重要部分,是加权 softmax 损失函数。把未标注像素的权重设为零,就只从已标注的像素学习,从而推广到整个体积。
3 Implementation Details
3 实现细节
3.1 Data
3.1 数据
We have three samples of Xenopus kidney embryos at Nieuwkoop-Faber stage 36–37 [10]. One of them is shown in Fig. 1 (left). 3D Data have been recorded in four tiles with three channels at a voxel size of using a Zeiss LSM 510 DUO inverted confocal microscope equipped with a Plan-Apochromat 40x/1.3 oil immersion objective lens. We stitched the tiles to large volumes using XuvTools [1]. The first channel shows Tomato-Lectin coupled to Fluorescein at 488 nm excitation wavelength. The second channel shows DAPI stained cell nuclei at 405 nm excitation. The third channel shows Beta-Catenin using a secondary antibody labelled with Cy3 at 564 nm excitation marking the cell membranes. We manually annotated some orthogonal , , and slices in each volume using Slicer3D [2]. The annotation positions were selected according to good data representation i.e. annotation slices were sampled as uniformly as possible in all 3 dimensions. Different structures were given the labels 0: “inside the tubule”; 1: “tubule”; 2: “background”, and 3: “unlabeled”. All voxels in the unlabelled slices also get the label 3 (“unlabeled”). We ran all our experiments on down-sampled versions of the original resolution by factor of two in each dimension. Therefore, the data sizes used in the experiments are , and in dimensions for our sample 1, 2, and 3, respectively. The number of manually annotated slices in orthogonal (, , ) slices are (7, 5, 21), (6, 7, 12), and (4, 5, 10) for sample 1, 2, and 3, respectively.
我们有三份处于 Nieuwkoop-Faber 36–37 期 [10] 的非洲爪蟾肾脏胚胎样本。其中一份显示在图 1(左)。三维数据用配备 Plan-Apochromat 40x/1.3 油浸物镜的 Zeiss LSM 510 DUO 倒置共聚焦显微镜采集,按四个图块、三个通道记录,体素尺寸为 。我们用 XuvTools [1] 把这些图块拼接成大型体积。第一通道显示与荧光素偶联的番茄凝集素,激发波长为 488 nm。第二通道显示 DAPI 染色的细胞核,激发波长为 405 nm。第三通道显示用 Cy3 标记的二抗所标记的 β-连环蛋白,激发波长为 564 nm,用以标出细胞膜。我们在每个体积中用 Slicer3D [2] 手工标注了若干正交的 、 和 切片。标注位置按良好的数据代表性来选取,也就是说,标注切片在三个维度上尽可能均匀取样。不同结构被赋予标签 0:「小管内部」;1:「小管」;2:「背景」;以及 3:「未标注」。未标注切片中的全部体素也得到标签 3(「未标注」)。我们的全部实验都在把原分辨率每个方向下采样二分之一后的版本上进行。因此,实验中使用的数据尺寸在 方向上,对样本 1、2、3 分别为 、 和 。正交方向(, , )上手工标注的切片数,对样本 1、2、3 分别为 (7, 5, 21)、(6, 7, 12) 和 (4, 5, 10)。
3.2 Training
3.2 训练
Besides rotation, scaling and gray value augmentation, we apply a smooth dense deformation field on both data and ground truth labels. For this, we sample random vectors from a normal distribution with standard deviation of 4 in a grid with a spacing of 32 voxels in each direction and then apply a B-spline interpolation. The network output and the ground truth labels are compared using softmax with weighted cross-entropy loss, where we reduce weights for the frequently seen background and increase weights for the inner tubule to reach a balanced influence of tubule and background voxels on the loss. Voxels with label 3 (“unlabled”) do not contribute to the loss computation, i.e. have a weight of 0. We use the stochastic gradient descent solver of the Caffe [5] framework for network training. To enable training of big 3D networks we used the memory efficient cuDNN convolution layer implementation. Data augmentation is done on-the-fly, which results in as many different images as training iterations. We ran 70000 training iterations on an NVIDIA TitanX GPU, which took approximately 3 days.
除了旋转、缩放和灰度增强,我们还对数据和真值标签施加平滑的稠密形变场。具体做法是:在每个方向间隔 32 个体素的网格上,从标准差为 4 的正态分布中抽样随机向量,再做 B 样条插值。网络输出与真值标签用带加权交叉熵损失的 softmax 来比较;我们降低常见背景的权重,提高小管内部的权重,使小管与背景体素对损失的影响更为均衡。标签为 3(“unlabled”)的体素不参与损失计算,即权重为 0。我们使用 Caffe [5] 框架的随机梯度下降求解器来训练网络。为了能够训练大型三维网络,我们使用了节省显存的 cuDNN 卷积层实现。数据增强在训练过程中即时进行,因此有多少次训练迭代,就有多少张不同的图像。我们在一块 NVIDIA TitanX GPU 上运行了 70000 次训练迭代,大约花了 3 天。
4 Experiments
4 实验
4.1 Semi-Automated Segmentation
4.1 半自动分割
For semi-automated segmentation, we assume that the user needs a full segmentation of a small number of volumetric images, and does not have prior segmentations. The proposed network allows the user to annotate a few slices from each volume and let the network create the dense volumetric segmentation.
对于半自动分割,我们假定用户需要对少量体积图像做完整分割,并且事先没有分割结果。所提出的网络允许用户在每个体积上标注少数切片,然后由网络生成稠密的体积分割。
图 3. (a) 我们第 3 份非洲爪蟾(Xenopus)肾脏的共聚焦记录。(b) 所提出的带批归一化的 3D u-net 得到的稠密分割。
For a qualitative assessment, we trained the network on all three sparsely annotated samples. Figure 3 shows the segmentation results for our 3rd sample. The network can find the whole 3D volume segmentation from a few annotated slices and saves experts from full volume annotation.
为做定性评估,我们在全部三份稀疏标注样本上训练了网络。图 3 给出了第 3 份样本的分割结果。网络能够从少数标注切片得到整本三维体积的分割,从而使专家不必做整本体积标注。
To assess the quantitative performance in the semi-automated setup, we uniformly partitioned the set of all 77 manually annotated slices from all 3 samples into three subsets and did a 3-fold cross validation both with and without batch normalization. To this end, we removed the test slices and let them stay unlabeled. This simulates an application where the user provides an even sparser annotation. To measure the gain from using the full 3D context, we compare the results to a pure 2D implementation, that treats all labelled slices as independent images. We present the results of our experiment in Table 1. Intersection over Union (IoU) is used as accuracy measure to compare dropped out ground truth slices to the predicted 3D volume. The IoU is defined as true positives/(true positives + false negatives + false positives). The results show that our approach is able to already generalize from only very few annotated slices leading to a very accurate 3D segmentation with little annotation effort.
为评估半自动设定中的定量性能,我们把来自全部 3 份样本的全部 77 张手工标注切片均匀分成三个子集,并分别在使用和不使用批归一化的情形下做三分交叉验证。为此,我们去掉测试切片,让它们保持未标注。这模拟了用户提供更稀疏标注的应用情形。为衡量使用完整三维上下文所带来的增益,我们把结果与纯二维实现相比较,后者把所有已标注切片当作互不相关的图像来处理。实验结果列在表 1 中。我们用交并比(IoU)作为精度度量,把被留出的真值切片与预测的三维体积相比较。IoU 定义为真正例 /(真正例 + 假反例 + 假正例)。结果表明,我们的方法已经能够从很少的标注切片中泛化,从而以较小的标注工作量得到非常准确的三维分割。
We also analyzed the effect of the number of annotated slices on the network performance. To this end, we simulated a one sample semi-automated segmentation. We started using 1 annotated slice in each orthogonal direction and increased the number of annotated slices gradually. We report the high performance gain of our network for each sample (S1, S2, and S3) with every few additional ground truth (“GT”) slices in Table 2. The results are taken from networks which were trained for 10 h with batch normalization. For testing we used the slices not used in any setup of this experiment.
我们还分析了标注切片数量对网络性能的影响。为此,我们模拟了单样本半自动分割。我们从每个正交方向使用 1 张标注切片开始,再逐步增加标注切片数。表 2 报告了每份样本(S1、S2 和 S3)在每增加少量真值(“GT”)切片时,网络所获得的显著性能提升。这些结果来自带批归一化、训练了 10 小时的网络。测试时,我们使用了本实验任一设定都未用过的切片。
| Test slices | 3D w/o BN | 3D with BN | 2D with BN |
|---|---|---|---|
| Subset 1 | 0.822 | 0.855 | 0.785 |
| Subset 2 | 0.857 | 0.871 | 0.820 |
| Subset 3 | 0.846 | 0.863 | 0.782 |
| Average | 0.842 | 0.863 | 0.796 |
Table 1. Cross validation results for semi-automated segmentation (IoU)
| Test slices | 3D w/o BN | 3D with BN | 2D with BN |
|---|---|---|---|
| Subset 1 | 0.822 | 0.855 | 0.785 |
| Subset 2 | 0.857 | 0.871 | 0.820 |
| Subset 3 | 0.846 | 0.863 | 0.782 |
| Average | 0.842 | 0.863 | 0.796 |
表 1. 半自动分割的交叉验证结果(IoU)
| GT slices | GT voxels | IoU S1 | IoU S2 | IoU S3 |
|---|---|---|---|---|
| 1,1,1 | 2.5% | 0.331 | 0.483 | 0.475 |
| 2,2,1 | 3.3% | 0.676 | 0.579 | 0.738 |
| 3,3,2 | 5.7% | 0.761 | 0.808 | 0.835 |
| 5,5,3 | 8.9% | 0.856 | 0.849 | 0.872 |
Table 2. Effect of # of slices for semi-automated segmentation (IoU)
| GT slices | GT voxels | IoU S1 | IoU S2 | IoU S3 |
|---|---|---|---|---|
| 1,1,1 | 2.5% | 0.331 | 0.483 | 0.475 |
| 2,2,1 | 3.3% | 0.676 | 0.579 | 0.738 |
| 3,3,2 | 5.7% | 0.761 | 0.808 | 0.835 |
| 5,5,3 | 8.9% | 0.856 | 0.849 | 0.872 |
表 2. 切片数量对半自动分割的影响(IoU)
| Test volume | 3D w/o BN | 3D with BN | 2D with BN |
|---|---|---|---|
| 1 | 0.655 | 0.761 | 0.619 |
| 2 | 0.734 | 0.798 | 0.698 |
| 3 | 0.779 | 0.554 | 0.325 |
| Average | 0.723 | 0.704 | 0.547 |
Table 3. Cross validation results for fully-automated segmentation (IoU)
| Test volume | 3D w/o BN | 3D with BN | 2D with BN |
|---|---|---|---|
| 1 | 0.655 | 0.761 | 0.619 |
| 2 | 0.734 | 0.798 | 0.698 |
| 3 | 0.779 | 0.554 | 0.325 |
| Average | 0.723 | 0.704 | 0.547 |
表 3. 全自动分割的交叉验证结果(IoU)
4.2 Fully-Automated Segmentation
4.2 全自动分割
The fully-automated segmentation setup assumes that the user wants to segment a large number of images recorded in a comparable setting. We further assume that a representative training data set can be assembled.
全自动分割设定假定用户想要分割在可比条件下采集的大量图像。我们进一步假定可以组建一套有代表性的训练数据集。
To estimate the performance in this setup we trained on two (partially annotated) kidney volumes and used the trained network to segment the third volume. We report the result on all 3 possible combinations of training and test volumes. Table 3 summarizes the IoU as in the previous section over all annotated 2D slices of the left out volume. In this experiment BN also improves the result, except for the third setting, where it was counterproductive. We think that the large differences in the data sets are responsible for this effect. The typical use case for the fully-automated segmentation will work on much larger sample sizes, where the same number of sparse labels could be easily distributed over much more data sets to obtain a more representative training data set.
为估计这一设定中的性能,我们在两本(部分标注的)肾脏体积上训练,再用训练好的网络去分割第三本体积。我们报告全部 3 种训练与测试体积组合上的结果。表 3 按上一节同样的方式,汇总了被留出体积中全部已标注二维切片上的 IoU。在这一实验中,批归一化也改进了结果,但第三种设定除外,那时它起了反作用。我们认为,各数据集之间的较大差异造成了这一效应。全自动分割的典型用法会面对大得多的样本量,同样数量的稀疏标签可以很容易地分散到更多数据集上,从而得到更有代表性的训练集。
5 Conclusion
5 结论
We have introduced an end-to-end learning method that semi-automatically and fully-automatically segments a 3D volume from a sparse annotation. It offers an accurate segmentation for the highly variable structures of the Xenopus kidney. We achieve an average IoU of 0.863 in 3-fold cross validation experiments for the semi-automated setup. In a fully-automated setup we demonstrate the performance gain of the 3D architecture to an equivalent 2D implementation. The network is trained from scratch, and it is not optimized in any way for this application. We expect that it will be applicable to many other biomedical volumetric segmentation tasks. Its implementation is provided as OpenSource.
我们提出了一种端到端学习方法,能够从稀疏标注出发,以半自动和全自动的方式分割三维体积。它为非洲爪蟾(Xenopus)肾脏这种高度可变的结构提供了准确分割。在半自动设定的三分交叉验证实验中,我们取得了 0.863 的平均 IoU。在全自动设定中,我们展示了三维结构相对于同等二维实现的性能增益。该网络从零训练,并且没有针对这一应用做任何专门优化。我们预期它将适用于许多其他生物医学体积分割任务。其实现以开源形式提供。
Acknowledgments. We thank the DFG (EXC 294 and CRC-1140 KIDGEM Project Z02 and B07) for supporting this work. Ahmed Abdulkadir acknowledges funding by the grant KF3223201LW3 of the ZIM (Zentrales Innovationsprogramm Mittelstand). Soeren S. Lienkamp acknowledges funding from DFG (Emmy Noether-Programm). We also thank Elitsa Goykovka for the useful annotations and Alena Sammarco for the excellent technical assistance in imaging.
致谢。 感谢 DFG(EXC 294 以及 CRC-1140 KIDGEM 项目 Z02 和 B07)对本研究的支持。Ahmed Abdulkadir 感谢 ZIM(Zentrales Innovationsprogramm Mittelstand)基金 KF3223201LW3 的资助。Soeren S. Lienkamp 感谢 DFG(Emmy Noether-Programm)的资助。我们还感谢 Elitsa Goykovka 提供了有用的标注,以及 Alena Sammarco 在成像中给予的出色技术协助。
References
参考文献
- Emmenlauer, M., Ronneberger, O., Ponti, A., Schwarb, P., Griffa, A., Filippi, A., Nitschke, R., Driever, W., Burkhardt, H.: Xuvtools: free, fast and reliable stitching of large 3D datasets. J. Microscopy 233(1), 42–60 (2009)
- Fedorov, A., Beichel, R., Kalpathy-Cramer, J., Finet, J., Fillion-Robin, J.C., Pujol, S., Bauer, C., Jennings, D., Fennessy, F., Sonka, M., et al.: 3D slicer as an image computing platform for the quantitative imaging network. J. Magn. Reson. Imaging 30(9), 1323–1341 (2012)
- Hariharan, B., Arbeláez, P., Girshick, R., Malik, J.: Hypercolumns for object segmentation and fine-grained localization. In: Proceeding CVPR, pp. 447–456 (2015)
- Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. CoRR abs/1502.03167 (2015)
- Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: Proceeding ACMMM, pp. 675–678 (2014)
- Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-Hein, K., Bendszus, M., Biller, A.: Deep mri brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage (2016)
- Lienkamp, S., Ganner, A., Boehlke, C., Schmidt, T., Arnold, S.J., Schäfer, T., Romaker, D., Schuler, J., Hoff, S., Powelske, C., Eifler, A., Krönig, C., Bullerkotte, A., Nitschke, R., Kuehn, E.W., Kim, E., Burkhardt, H., Brox, T., Ronneberger, O., Gloy, J., Walz, G.: Inversin relays frizzled-8 signals to promote proximal pronephros development. PNAS 107(47), 20388–20393 (2010)
- Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceeding CVPR, pp. 3431–3440 (2015)
- Milletari, F., Ahmadi, S., Kroll, C., Plate, A., Rozanski, V.E., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Bötzel, K., Navab, N.: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound. CoRR abs/1601.07014 (2016)
- Nieuwkoop, P., Faber, J.: Normal Table of Xenopus laevis (Daudin). Garland, New York (1994)
- Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24574-4_28
- Seyedhosseini, M., Sajjadi, M., Tasdizen, T.: Image segmentation with cascaded hierarchical models and logistic disjunctive normal networks. In: Proceeding ICCV, pp. 2168–2175 (2013)
- Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. CoRR abs/1512.00567 (2015)
- Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., Paluri, M.: Deep end2end voxel2voxel prediction. CoRR abs/1511.06681 (2015)


