Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Automatic Semantic Segmentation of Brain Gliomas from MRI Images Using a Deep Cascaded Neural Network

Automatic Semantic Segmentation of Brain Gliomas from MRI Images Using a Deep Cascaded Neural... Hindawi Journal of Healthcare Engineering Volume 2018, Article ID 4940593, 14 pages https://doi.org/10.1155/2018/4940593 Research Article Automatic Semantic Segmentation of Brain Gliomas from MRI Images Using a Deep Cascaded Neural Network 1,2 1 2,3 1 1 Shaoguo Cui , Lei Mao, Jingfeng Jiang , Chang Liu, and Shuyu Xiong College of Computer Science and Engineering, Chongqing University of Technology, Chongqing 400054, China Medical Physics Department, University of Wisconsin, Madison, WI 53705, USA Biomedical Engineering Department, Michigan Technological University, Houghton, MI 49931, USA Correspondence should be addressed to Shaoguo Cui; cuishaoguo2002@163.com Received 30 November 2017; Accepted 11 February 2018; Published 19 March 2018 Academic Editor: Weide Chang Copyright © 2018 Shaoguo Cui et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Brain tumors can appear anywhere in the brain and have vastly different sizes and morphology. Additionally, these tumors are often diffused and poorly contrasted. Consequently, the segmentation of brain tumor and intratumor subregions using magnetic resonance imaging (MRI) data with minimal human interventions remains a challenging task. In this paper, we present a novel fully automatic segmentation method from MRI data containing in vivo brain gliomas. This approach can not only localize the entire tumor region but can also accurately segment the intratumor structure. The proposed work was based on a cascaded deep learning convolutional neural network consisting of two subnetworks: (1) a tumor localization network (TLN) and (2) an intratumor classification network (ITCN). The TLN, a fully convolutional network (FCN) in conjunction with the transfer learning technology, was used to first process MRI data. The goal of the first subnetwork was to define the tumor region from an MRI slice. Then, the ITCN was used to label the defined tumor region into multiple subregions. Particularly, ITCN exploited a convolutional neural network (CNN) with deeper architecture and smaller kernel. The proposed approach was validated on multimodal brain tumor segmentation (BRATS 2015) datasets, which contain 220 high-grade glioma (HGG) and 54 low-grade glioma (LGG) cases. Dice similarity coefficient (DSC), positive predictive value (PPV), and sensitivity were used as evaluation metrics. Our experimental results indicated that our method could obtain the promising segmentation results and had a faster segmentation speed. More specifically, the proposed method obtained comparable and overall better DSC values (0.89, 0.77, and 0.80) on the combined (HGG + LGG) testing set, as compared to other methods reported in the literature. Additionally, the proposed approach was able to complete a segmentation task at a rate of 1.54 seconds per slice. determine (1) the presence of a tumor, (2) the inclusion of 1. Introduction peritumoral edema, and (3) the spread into other locations Although brain cancers are less prevalent, they are very such as the CNS [3]. lethal. Among them, gliomas are the most common brain Compared to CT, MRI or contrast-enhanced MRI tumors. They can be graded into low-grade gliomas (LGG) becomes the imaging modality of choice for diagnosis and and high-grade gliomas (HGG), with the latter being more treatment planning in the brain because of its sensitivity aggressive and infiltrative than the former [1]. A glioma is and superior image contrast in soft tissues. However, the highly invasive because it tends to aggressively grow and multiplicity and complexity of the brain tumors under MRI could quickly invade the central nervous system (CNS). often make tumor recognition and segmentation difficult According to US National Cancer Institute, approximately for radiologists and other clinicians [4]. Consequently, auto- matic segmentation of heterogeneous tumors can greatly 18,000 Americans are diagnosed with a glioma every year; many of them die within 14 months [2]. In clinical practice, impact the clinical medicine by freeing physicians from the medical imaging, mainly computed tomography (CT) and burden of the manual depiction of tumors. Furthermore, if magnetic resonance imaging (MRI), has been used to computer algorithms can provide robust and quantitative 2 Journal of Healthcare Engineering of the CNN in the ImageNet Large-Scale Visual Recognition measurements of tumor depiction, these automated mea- surements will greatly aid in the clinical management of Challenge (ILSVRC) [23]. Recent deep learning methods for brain tumors. automatic brain tumor segmentation are summarized below In the past few decades, significant research efforts in the in Table 2. computer vision and image processing community have been However, the above-mentioned CNN methods were all devoted to developing computer-aided systems that can be based on the patch-wise method in which (medical) images used for automated tumor characterization/classification were often divided into patches during the training and test- [5–21]. Although some systems were tested and showed good ing. The advantage of this method was that it could take performance, the fully automatic detection and subsequent advantage of the existing classification model of the natural diagnosis of brain tumors have not been massively used in image and solve the problem of the class label imbalance in the clinical settings, thereby indicating that some major MRI images. Despite its popularity, operating on image developments are still needed [21]. patches was computationally time-consuming. Recalling, Based on MRI data, our primary goal of this paper was to given a typical image size (e.g., 256 × 256), a large number propose a new fast and accurate computer system that could of patches (65535) were required as inputs for prediction. first localize complete tumor region and then segment the Furthermore, this method was not end-to-end and per- more detailed intratumor structure. Our computer system formed the segmentation task by independently classifying contained two major steps. First, by leveraging an FCN the central pixel of a patch, which will result in some errors [22], a tumor location map was first obtained. In the second and need postprocessing. Thus, the expensive computation and postprocessing become the bottleneck of its real-time step, a deep learning ensemble of the CNN was used to clas- sify the tumor region into four subregions: (1) necrosis, (2) clinic application. edema, (3) nonenhancing tumor, and (4) enhancing tumor. Recently, Shelhamer et al. [22] presented a novel FCN for In this study, the performance of the proposed algorithm semantic segmentation of natural scene images. This model was assessed in a public database containing 274 cases of can be trained in an end-to-end manner (also known as pixel-wise). Their results showed that the FCN outperformed in vivo gliomas. The paper is structured as follows: Section 2 presents the the previous methods for semantic segmentation of a natural related works in the automated brain cancer segmentation. scene image in performance and speed. Inspired by the work Particularly, attention was given to computer systems based in [22], we proposed a hybrid approach by constructing a on machine learning. The proposed two-step (cascaded) deep cascaded neural network. Our main contribution of this work is to propose a hybrid neural network is described in Section 3. The emphases are on the design methodology and training methods for the cascaded neural network for the purpose of segmentation of performance assessment. In Section 4, results of our numer- brain tumors including segmentation of intratumor subre- ical experiments are summarized followed by some closing gions, from MRI data. This model consists of one FCN and remarks in Section 5. one CNN. This combination enables us to perform pixel semantic predictions by taking advantage of both a pixel- wise method and a patch-wise method. Formally, in this cas- 2. Relevant Work and Our Contributions caded neural network, an FCN was first used to localize the In recent years, many methods have been proposed to tumor region from an MRI slice and then a CNN with deeper automatically segment brain tumors based on MRI data. architecture and smaller kernels was used to classify brain tumor into multiple subregions. This approach can not only These methods can be largely divided into two categories: (1) hand-crafted feature and classifier methods based on obtain the better segmentation accuracy but can also speed traditional machine learning such as support vector the prediction efficiency. machine (SVM) and random forests (RF) [5–13] and (2) fully automatic methods based on deep learning using the 3. Methods CNN [14–21]. Methods in the first category use manually extracted fea- 3.1. Construction of the Deep Cascaded Neural Network. The tures, and these features are input to classifiers. In other starting point of the proposed system is in vivo MRI data words, once these hand-crafted features are solely deter- consisting of four different sequences (FLAIR, T1, T1c, and mined by human operators, classifiers “weigh” them during T2), and the endpoint becomes a characterized tumor (see the training but cannot modify these features in any way. Figure 1). In the output image, a brain tumor is classified into One significant concern of hand-crafted features stems from four different zones: necrosis, edema, nonenhancing tumor, the fact that these features could have significant inter- and and enhancing tumor. intrauser variability. A brief summary of these methods can More specifically, the architecture of the proposed system be found in Table 1. includes an FCN followed by a CNN which accompanies In contrast, methods in the second category can self-learn small convolution kernels (see Figure 1). So the segmentation the feature representations adapted to a specific task from task based on this cascaded network can be divided into two training data. Recently, deep learning neural networks, espe- major steps. In the first step, the pixel-wise FCN was used to cially CNNs, are rapidly gaining their popularity in the com- quickly localize the tumor by marking the tumor region. puter vision community. This trend has certainly been Then, the patch-wise CNN was used to further categorize accelerated after the recent record-shattering performance the above-identified tumor region into different subregions Journal of Healthcare Engineering 3 Table 1: A summary of brain tumor segmentation methods based on traditional machine learning. Only methods using MRI data were included in this table. Number Publication Database Summary of method Performance 20 cases of in vivo A hybrid method combining an 1 Corso et al. [5] brain tumors; affinity-based segmentation method 0.62–0.69 (Jaccard) T1, T1-C, T2, FLAIR with a generative model Synthetic data from A cellular automata method combining 0.72 (DICE complete 2 Hamamci et al. [6] Utah + in vivo data a probability framework tumor) from Harvard BrainWeb data + in vivo A novel saliency model for lesion 83%~95% brain tumors; 3 Mehmood et al. [7] localization and an N-cut graph segmentation (classification T1, T1-weighted, model for classification accuracy) T2, T2-weighted MICCAI-BRATS 2013 Hand-crafted features + a support 0.86 (DICE complete 4 Havaei et al. [8] dataset vector machine tumor) MICCAI-BRATS 2013 Automated wavelet-based features + a 0.88 (DICE complete 5 Usman and Rajpoot [9] dataset random forest classifier tumor) Combine a random forest model with a MICCAI-BRATS 2013 0.88 (DICE complete 6 Tustison et al. [10] framework of regularized probabilistic dataset tumor) segmentation GT: 0.89 AC: 0.84 40 multichannel Decision forests using context-aware NE: 0.70 E: 0.72 7 Zikic et al. [11] MR images, spatial features for automatic segmentation including DTI of high-grade gliomas (10/30 tests) Using appearance- and context-based MICCAI-BRATS 0.83 (DICE complete 8 Pinto et al. [12] features to feed an extremely randomized 2013 dataset tumor) forest GT: 0.84 NE: 0.70 AC: 0.84 E: 0.72 10 multispectral Combines support vector machine 9 Bauer et al. [13] patient datasets classification with conditional random fields (Intrapatient regularized) Table 2: A summary of brain tumor segmentation methods based on deep-learning neural networks. Only methods using MRI data were included in this table. Performance (DICE) Number Publication Database Summary of method Complete Core Enh MICCAI-BRATS 1 Urban et al. [14] 3D CNN with 3D convolutional kernels 0.87 0.77 0.73 2013 dataset MICCAI-BRATS Apply a CNN in a sliding-window 2 Zikic et al. [15] 0.84 0.74 0.69 2013 dataset fashion in the 3D space MICCAI-BRATS A CNN with two pathways of both local 3 Davy et al. [16] 0.85 0.74 0.68 2013 dataset and global information MICCAI-BRATS Structured prediction was used 4 Dvorak and Menze [17] 0.83 0.75 0.77 2013 dataset together with a CNN MICCAI-BRATS 5 Pereira et al. [18] A CNN with small 3 × 3 kernels 0.88 0.83 0.77 2013 dataset A cascade neural network architecture MICCAI-BRATS in which “the output of a basic CNN is treated 6 Havaei et al. [19] 0.88 0.79 0.73 2013 dataset as an additional source of information for a subsequent CNN” An ensemble of 2D convolutional neural MICCAI-BRATS 7 Lyksborg et al. [20] networks +doing a volumetric segmentation 0.80 0.64 0.59 2014 dataset by three steps MICCAI-BRATS Using 3D CNN, two-scale extracted 8 Kamnitsas et al. [21] 0.85 0.67 0.63 2015 dataset feature, 3D dense CRF as postprocessing 4 Journal of Healthcare Engineering Input Output Tumor localization network (TLN) Intratumor classification network (ITCN) Output of TLN Tumor candidates Figure 1: An illustrative overview of the proposed deep cascaded convolutional neural network for a fast and accurate tumor segmentation. representing different pathologies. This system design was mitigating the loss of local image features and (2) combining motivated and justified as follows. First, the FCN can take local information obtained from intermediate layers (i.e., max pooling 4 and max pooling 3, resp.) with the global a whole image as the input and localization of a complete tumor only requires one-pass of the forward propagation. information in these deep layers (i.e., after 7 convolution Thus, it can remarkably improve the segmentation effi- layers). All relevant parameters used in the subnet TLN are ciency. Second, this combination of FCN and CNN can alle- shown in Table 3 below. viate the pixel sample class imbalance problem which is serious in MRI images. Thus, it can capture better segmen- 3.1.2. A Description of ITCN. The proposed ITCN includes tation details. Third, the intratumor characterization in the two convolutional layer groups (3 layers each), two max second step will only need to be applied to the tumor pooling layers, and three fully connected layers. Recall that regions localized in the first step instead of the entire image, the TLN yields a binary tumor map for a given MRI image thereby significantly reducing forward computing time. and the ITCN (see Figure 3) further classifies the identified Hereafter, the FCN and the CNN are referred as to tumor tumor into 4 different subregions. Formally, for each location localization network (TLN) and intratumor classification i, j within the identified tumor map, 4 patches (size of network (ITCN), respectively. 33 × 33) centered on the i, j location were extracted from the original 4 input channels (FLAIR, T1, T1c, and T2) and 3.1.1. A Description of TLN. We modified the FCN-8s archi- subsequently used as the input to the ITCN. More details of tecture [22] to model our TLN. The input channels (RGB) in this ITCN subnet are listed in Table 4. the original FCN-8s were changed to 4 channels in order to In the ITCN, as inspired by the work of Simonyan and account for 4 different MRI modalities. And the 21 output Zisserman [24], multiple convolutional layers with small ker- channels in the original FCN-8s were changed to 2, corre- nels (3 × 3 pixels) were used. An alternative approach would sponding to either the tumor region or the nontumor region. be an architecture with fewer layers and larger kernels. The- As shown in Figure 2, after the operations of the convolution oretically, two cascaded convolutional layers with two 3 × 3 and pooling, the feature map became smaller in size (see kernels have similar effects on the receptive fields, as com- Table 3). To obtain a higher resolution of the final features, pared to one convolutional layer with a 5 × 5 kernel. But the input images (size 240 × 240) were padded to 438 × 438 two cascaded layers with two 3 × 3 kernels result in more using zero padding [22]. Additionally, the deconvolution complex nonlinearities and fewer weights. Fewer weights was applied so that the size of output image matched with lead to a less computing cost and can also alleviate the pos- that of the input image. It is worth noting that multiple con- sibility of overfitting. It is generally understood that, with volutional kernels were used in each convolutional layer for a the increase of the CNN’s depth, a CNN can gain higher better feature extraction (e.g., edges, curves, and corner). representation capacity. As shown in Figure 3, in each of We observed that a significant amount of low-level the two pooling layers, a 3 × 3 overlapping subwindow with feature details such as location and edge could be lost after a stride of 2 was applied to the feature maps for reducing convolution striding and pooling. However, these lost fea- feature dimension and integrating higher-level features. tures were valuable for semantic segmentation. Thus, two The detailed hyperparameters of the ITCN can be found skip connections [22] were introduced for two purposes: (1) in Table 4 below. Journal of Healthcare Engineering 5 Input Output Figure 2: An illustration of the architecture of the TLN subnet for pixel-wise prediction. Table 3: Parameters used in the subnet TLN. In each convolutional layer, the feature maps had been padded by 1 prior to the convolution so that all intermediate feature maps do not change their sizes before and after the convolution. Number Layer name Filter size Stride Number of Filters Output ∗ ∗ ∗ 1 Conv 1_1 + ReLU 3 3 1 64 438 438 64 ∗ ∗ ∗ 2 Conv 1_2 + ReLU 3 3 1 64 438 438 64 ∗ ∗ ∗ 3 Max pooling 1 222 — 219 219 64 ∗ ∗ ∗ 4 Conv 2_1 + ReLU 3 3 1 128 219 219 128 ∗ ∗ ∗ 5 Conv 2_2 + ReLU 3 3 1 128 219 219 128 ∗ ∗ ∗ 6 Max pooling 2 222 — 110 110 128 ∗ ∗ ∗ 7 Conv 3_1 + ReLU 3 3 1 256 110 110 256 ∗ ∗ ∗ 8 Conv 3_2 + ReLU 3 3 1 256 110 110 256 ∗ ∗ ∗ 9 Conv 3_3 + ReLU 3 3 1 256 110 110 256 ∗ ∗ ∗ 10 Max pooling 3 222 — 55 55 256 ∗ ∗ ∗ 11 Conv 4_1 + ReLU 3 3 1 512 55 55 512 ∗ ∗ ∗ 12 Conv 4_2 + ReLU 3 3 1 512 55 55 512 ∗ ∗ ∗ 13 Conv 4_3 + ReLU 3 3 1 512 55 55 512 ∗ ∗ ∗ 14 Max pooling 4 222 — 28 28 512 ∗ ∗ ∗ 15 Conv 5_1 + ReLU 3 3 1 512 28 28 512 ∗ ∗ ∗ 16 Conv 5_2 + ReLU 3 3 1 512 28 28 512 ∗ ∗ ∗ 17 Conv 5_3 + ReLU 3 3 1 512 28 28 512 ∗ ∗ ∗ 18 Max pooling 5 222 — 14 14 512 ∗ ∗ ∗ 19 Conv 6 + ReLU 7 7 1 4096 8 8 4096 ∗ ∗ ∗ 20 Conv 7 + ReLU 1 1 1 4096 8 8 4096 3.2. Implementation. All numerical experiments were con- 3.2.1. Preprocessing. As recommended by the literature [25], ducted using a Dell workstation equipped with dual Intel MRI data were preprocessed before the proposed cascaded E5-2603 CPUs and a middle-end GPU graphic card (GeForce neural network was applied. Basically, the N4ITK method GTX 1080, NVIDIA, CA, USA). The operation system of the was first used to correct the distortion of MRI data, followed workstation is Ubuntu (version 14.04). The proposed cas- by data normalization. caded neural network has been implemented using Python Given an image X, x i, j is the intensity correspond- (version 2.7) under the framework of Caffe, an open-source ing to the jth column at the ith row of X i, j =1,2, … , deep learning platform (http://caffe.berkeleyvision.org/). 240 . The data intensity normalization procedure is briefly Some essential details are discussed below. described below: Conv 1 group Max pooling 1 Conv 2 group Max pooling 2 Conv 3 group Max pooling 3 Conv 4 group Max pooling 4 Fusing Fusing Conv 5 group Deconv Deconv Max pooling 5 Conv 6 - 7 Deconv 6 Journal of Healthcare Engineering Fully connected Fully connected Max Max pooling 1 pooling 2 33 × 33 × 4 Conv 1_1 Conv 1_2 Conv 1_3 Conv 1_1 Conv 1_2 Conv 1_3 Fully connected Figure 3: An illustration of the second subnet ITCN for the intratumoral classification. The classification was done in a patch-to-patch fashion. Table 4: A list of parameters used in the proposed subnet ITCN. In each convolutional layer, the feature maps had been padded by 1 prior to the convolution so that the convolution do not change the size of the resultant feature map. Number Layer name Filter size Stride Number of filters FC units Output ∗ ∗ ∗ 1 Conv 1_1 + LReLU 331 64 — 33 33 64 ∗ ∗ ∗ 2 Conv 1_2 + LReLU 331 64 — 33 33 64 ∗ ∗ ∗ 3 Conv 1_3 + LReLU 331 64 — 33 33 64 ∗ ∗ ∗ 4 Max pooling 1 332 —— 16 16 64 ∗ ∗ ∗ 5 Conv 2_1 + LReLU 3 3 1 128 — 16 16 128 ∗ ∗ ∗ 6 Conv 2_2 + LReLU 3 3 1 128 — 16 16 128 ∗ ∗ ∗ 7 Conv 2_3 + LReLU 3 3 1 128 — 16 16 128 ∗ ∗ ∗ 8 Max pooling 2 332 —— 8 8 128 9 FC1 + dropout —— — 8192 256 10 FC2 + dropout —— — 256 128 11 FC3 + softmax —— — 128 4 (1) Removed the top 1% and bottom 1% from each slice We randomly selected 3 different cases from the FLAIR data- of the MRI data. set. As shown in Figure 4 below, it is easy to find that the above-mentioned data normalization can improve the com- (2) For each slice of MRI data X, a normalized image X parability of different slices. was obtained. In the scaled image X , each intensity 3.2.2. Convolution Operation. Each feature map Z shown in value x i, j can be obtained as follows: Figures 1, 2, and 3 was associated with one convolution kernel. Z was computed as follows: xi, j − X x i, j = , 1 Z = b + 〠 W ∗ X , 2 r r r=1 where x i, j is the gray value of pixel i, j prior to the where k is the number of input channels, b is a bias term, X is normalization and X and X are the mean and standard devi- an image from the rth input channel, and W is the weight ation of the unscaled image X, respectively. ∗ associated with the rth channel. In (2), denotes a convolu- The above-mentioned preprocessing method was used to tion operator. process each modality MRI data including FLAIR, T1, T1c, and T2. Particularly, the FLAIR images were generated using 3.2.3. Nonlinear Activation Function. In our study, the TLN fluid-attenuated inversion recovery protocol and useful in used rectified linear unit (ReLU) function [23] to perform terms of differentiating the brain tumor from its normal nonlinear transformations. This selection was because ReLU background. Figure 4 presents some FLAIR slices before could achieve better results as compared to the classical and after using the proposed image intensity normalization. sigmoid and hyperbolic tangent functions. The use of ReLU … Journal of Healthcare Engineering 7 (a) (b) Figure 4: Randomly selected examples of FLAIR slices before (a) and after (b) the above-mentioned intensity normalization. was also able to accelerate the training [26]. Mathematically, arbitrary prediction for the ith pixel, the predition loss can the ReLU function is defined below: be defined as fz = max 0, z 3 C L θ = −〠 Y log Y , 6 ij ij In the ITCN, the leaky rectifier linear unit (LReLU) [27] j=1 was used. This was because imposing zeros (see (3)) could negatively affect the calculation of gradients. During the ′ where Y , Y, and C are a one-hot vector, the predicted prob- training of this neural network, zero gradients will signifi- ability distribution, and the number of classes, respectively. cantly slow down the adjustments of weights. The LReLU In the TLN, predictions were made for each pixel of function reads the input image so that the loss function can be written as follows: fz = max 0, z + α min 0, z , 4 S C where α is the leakiness parameter [18]. ′ ′ L θ = − 〠〠 Y log Y , 7 ij ij To address the multiclassification problem, a well- i=1 j=1 known softmax function was used to transform the neural network outputs to probability distributions. Softmax is where C =2 and S is the pixel number of the input image. In defined as follows: every training, only one input image was used (the size of minibatch was 1). Now referring to the ITCN, the loss function was calcu- Y = soft max Z = , 5 i i lated in conjunction with the concept of mini-batch. Thus, the loss function has the following form, where Z is the output from the ith neuron and Y is the prob- i i M C ability of input pixel corresponding to the ith class. In the ″ ′ L θ = − 〠〠 Y log Y , 8 TLN, i =1 or 2 because the TLN was to perform a binary clas- ij ij i=1 j=1 sification in the first step. In the ITCN, i =1,2, 3,4 since the ITCN was to classify the MRI data into four classes. where C =4 and M is the size of minibatch. Of note, in this 3.2.4. Loss Function. Given a set of weights of the proposed study, M = 256. neural network θ, a categorical cross-entropy loss function To achieve better generation ability and avoid overfitting, was used to compute the loss of ground truth and pre- L2 regularization terms were also added to (7) and (8). Thus, dicted probability distribution. Mathematically, under an the final forms of the loss functions are 8 Journal of Healthcare Engineering S C Q the dropout regularization [31] and the dropout ratio was 1 λ 2 ′ ′ ′ L θ = − 〠〠 Y log Y + 〠 θ , 9 set to 0.5 in all fully connected layers. Weight decay was ij ij k S S i=1 j=1 k=1 set as 0.005. M C Q 3.3. Datasets and Evaluation Metrics. In order to train and 1 λ 2 ″ ′ ′ L θ = − 〠〠 Y log Y + 〠 θ , 10 ij ij k evaluate the proposed system, numerical experiments were M M i=1 j=1 k=1 carried out using in vivo human patient data provided by the BRATS 2015 database [32]. The BRATS 2015 database where λ is a regularization constant and Q is the number of contains 220 HGG and 54 LGG. Experimental data have model parameter. been labeled, and five labels were used: normal brain tissues (noncancerous zone), necrosis, edema, nonenhancing tumor, 3.2.5. Optimization Method. Equations (9) and (10) were and enhancing tumor. These pixel-wise delineations were minimized using the minibatch stochastic gradient descent considered the ground truth in this study. Each case contains (SGD) algorithm. To avoid numerical oscillations and four sequences of MRI data, namely, T1, T1c, T2, and FLAIR. accelerate convergence, the momentum method [23] was The dimension of each MRI modality is 155 × 240 × 240 (slice used. This process can be described as iterations from number × length × width). All MRI data were spatially regis- (11) to (13). tered and stored as signed 16-bit integers. But only positive g = ∇ L θ , 11 t−1 t−1 values were used. The tenfold crossvalidation method [33] was used to m = μ ∗ m − η g , t t−1 t t evaluate the proposed system. More specifically, the 274 cases were divided into a training set (240 cases) and a θ = θ + m t t−1 t testing set (34 cases). The 240 training cases were equally divided into 10 subsets in which 9 subsets were used as In (11), (12), and (13), the subscript t is the iteration the training and 1 subset was used as the validation. In ′ ″ number and θ corresponds to θ in (9) or θ in (10). L θ t−1 the training phase of the TLN subnet, all subregions within is the loss function when a parameter set θ is used. g , m , t−1 t a tumor were merged into one tumor region. Thus, in the and μ are the gradient, momentum, and momentum coeffi- binary ground truth, zero represents the noncancerous tis- cient, respectively. We set μ =0 99 and μ =0 9 in the TLN sues while one represents cancerous regions. In the train- and ITCN, respectively. Here, η is the learning rate. ing phase of the ITCN subnet, we randomly selected To suppress the SGD noise and guarantee conver- 4,700,000 image patches (33 × 33) from the training set, gence, the learning rate η attenuates linearly from the ini- which correspond to 1,175,000 patches for each label (4 tial learning rate η to the final learning rate η as the 0 τ different classes). iteration progresses: The quantitative evaluations were conducted for 3 differ- ent tumor regions: complete tumor region (including all four η =1 − γ η + γη , 14 t 0 τ tumor subregions), core tumor region (including all tumor structures except edema), and enhancing tumor region (only γ = , including the enhanced tumor structure). For each type of regions, we compute DSC [34], PPV, and sensitivity [35] as where τ is the total iteration number. In this study, we set quantitative evaluation metrics. η = η /100. τ 0 DSC measures the overlap between the ground truth and the automatic segmentation. It is defined as 3.2.6. Training Details. The initial and final learning rates of the TLN model were set to 1e−8 and 1e−10, respectively. P ∩ T 1 1 DSC = , 16 The total iteration τ =2e6, and the momentum coefficient P + T /2 1 1 was 0.99. In the ITCN subnet, the initial and final learning rates were set to 1e−3 and 1e−5, respectively. In the ITCN where P and T represent the positive values of the model 1 1 subnet, the total iteration τ =2e6 and the momentum coeffi- prediction and the ground truth, respectively. cient μ =0 9. PPV is the proportion of the true positive in all segmen- During the training of the TLN subnet, we used the trans- tation tumor points. It is defined as fer learning technique [28, 29]. The initial weights were P ∩ T obtained from a pretrained model that was trained using 1 1 PPV = 17 ImageNet in [24]. But initial weights of the 4th input channel P were initialized using the average of the original 3 input Sensitivity is the proportion of the detected tumor points channel (RGB) weights. And the final two output channels in all ground truth tumor points. It is defined as were initialized with the Xavier method [30]. Then, fine- tuning of the TLN was performed by the optimization pro- P ∩ T 1 1 Sensitivity = 18 cess described above ((11), (12), and (13)) using the MRI training data. However, the training of the ITCN subnet was started from scratch and the weights were initialized The proposed system was compared with some other with the Xavier method [30]. To avoid overfitting, we used published methods. Those methods all have been validated Journal of Healthcare Engineering 9 (a) (b) (c) (d) (e) (f) Figure 5: Representative examples of computer segmentation results of four brain tumors. (a–d) The original FLAIR, T1, T1c, and T2 slices, respectively. (e) The ground truth overlaid with the FLAIR image. (f) Segmentation results overlaid with the FLAIR image. (e, f) Red, green, yellow, and blue colors denote necrosis, edema, nonenhancing tumor, and enhancing tumor, respectively. on the BRATS 2015 dataset. A one-step segmentation 4. Results method based on the FCN-8s was also implemented for the purpose of comparison. The FCN-8s can segment the input 4.1. Qualitative Observations. Overall, we found that the pro- MRI images into 5 classes in a single step. posed system can accurately delineate gliomas. Visual 10 Journal of Healthcare Engineering (a) (b) (c) Figure 6: Two slices of computer segmentation result in a testing case: (a–c) the ground truth, results of tumor localization using the TLN subnet, and the intratumor segmentation results using the ITCN subnet, respectively. (a, c) Red, green, yellow, and blue colors denote necrosis, edema, nonenhancing tumor, and enhancing tumor, respectively. inspections were conducted for testing data to validate the classification results by the FCN-8s (the second column). segmentation results of our proposed method. Figure 5 Furthermore, boundaries of various subregions obtained by shows four selected examples. It can be observed that our the FCN-8s were overly smoothed and, perhaps, inaccurate. method can effectively localize and segment brain tumors But our method using the ITCN had better boundaries of with vastly different shapes and sizes. Visually, the computer the enhancing and nonenhancing regions. segmentation is comparable to the ground truth. Also, the proposed system led to good details around 4.2. Evaluation and Comparison. The quantitative compari- boundaries. Figure 6 presents two representative examples sons with other methods in terms of DSC are summarized of this observation. Since these brain tumors are complex, in Tables 5 and 6. All experiments were conducted on the Figure 6 shows some good showcase examples. During the BRATS 2015 dataset. The results of Table 5 were obtained process, we found that the TLN subnet was able to effectively by using the combined testing set of HGG and LGG, whereas identify nearly all the tumor pixels. Subsequently, the ITCN results shown in Table 6 only used HGG data. subnet efficiently classified the tumor region into four subre- Obviously, the proposed cascaded neural network obtains gions. Our method could largely detect the complete tumor the comparable and better DSC value on all tumor regions. and classify it to different tumor subregions from multimod- Based on the combined testing dataset (see Table 5), our ality MRI images though there were a few misclassifications. method obtained better comprehensive performance values This is not surprising because, pathologically, the brain gli- (0.89, 0.77, and 0.80) as compared to other methods. oma tumors invade their surrounding tissues rather than dis- Although the method proposed by Kamnitsas et al. [21] yields placing them. Hence, the appearance of cancerous tissues a slightly higher DSC value in the complete tumor, they and their surrounding (normal) tissues could be fairly similar obtained lower DSC values in core tumor and enhancing under MRI. tumor. Actually, in their work, a 3D CNN and the structure We also found that, as compared to the FCN-8s with one- prediction technology were adopted (i.e., conditional random step segmentation, the proposed system could segment het- field). Thus, it is computationally time-consuming and needs erogeneous gliomas with a better boundary detail. The results extra postprocessing. Furthermore, the method proposed by of the proposed method and FCN-8s are compared in Dong et al. [36] yielded a slightly higher DSC value in core Figure 7. Five different typical slices representing signifi- tumor and Yi et al. [37] yielded the same DSC value in enhanc- cantly different tumor shapes and sizes are shown in this fig- ing tumor. ure. It is easy to see that the results obtained from the As can be seen in Table 6, based on the HGG testing data- proposed method (the third column) are more similar to set, our method obtained the highest DSC values in the com- the ground truth (the first column), as compared to the plete tumor and enhancing tumor categories. Although the Journal of Healthcare Engineering 11 (a) (b) (c) Figure 7: Examples of segmentation results from five typical slices comparing the FCN-8s (b) and the proposed method (c). (a) The ground truth. In this figure, red, green, yellow, and blue colors denote necrosis, edema, nonenhancing tumor, and enhancing tumor, respectively. Recently, we found that Pereira et al. [39] also proposed a method proposed by Dong et al. [36] yielded a higher DSC value in the core tumor cases, it obtained a lower DSC value hierarchical brain tumor segmentation approach from MRI HGG images. The difference between their method and our in the complete tumor category. 12 Journal of Healthcare Engineering Table 5: A summary of DSC quantitative comparison on BRATS 2015 combined dataset (HGG and LGG). DSC Method Dataset Grade Complete Core Enh BRATS 2015 Challenge Combined 0.78 0.65 0.75 Pereira et al. [38] BRATS 2015 Training Combined 0.87 0.73 0.68 Havaei et al. [19] BRATS 2015 Challenge Combined 0.79 0.58 0.69 BRATS 2015 Challenge Combined 0.85 0.67 0.63 Kamnitsas et al. [21] BRATS 2015 Training Combined 0.90 0.76 0.73 Dong et al. [36] BRATS 2015 Training Combined 0.86 0.86 0.65 Yi et al. [37] BRATS 2015 Training Combined 0.89 0.76 0.80 FCN-8s BRATS 2015 Training Combined 0.84 0.71 0.63 Proposed BRATS 2015 Training Combined 0.89 0.77 0.80 Table 6: A summary of DSC quantitative comparison on BRATS 2015 HGG dataset. DSC Method Dataset Grade Complete Core Enh Pereira et al. [38] BRATS 2015 Training HGG 0.87 0.75 0.75 Havaei et al. [19] BRATS 2015 Challenge HGG —— — Kamnitsas et al. [21] BRATS 2015 Training HGG —— — Dong et al. [36] BRATS 2015 Training HGG 0.88 0.87 0.81 Yi et al. [37] BRATS 2015 Training HGG 0.89 0.79 0.80 FCN-8s BRATS 2015 Training HGG 0.88 0.76 0.71 Proposed BRATS 2015 Training HGG 0.90 0.81 0.81 Table 7: A comparison of our proposed method with hierarchical brain tumor segmentation [39] on DSC, PPV, and sensitivity metrics. DSC PPV Sensitivity Method Complete Core Enh Complete Core Enh Complete Core Enh Pereira et al. [39] 0.85 0.76 0.74 0.80 0.78 0.74 0.92 0.79 0.78 Proposed 0.90 0.81 0.81 0.91 0.77 0.87 0.87 0.84 0.76 Additionally, the segmentation speed for testing data was Table 8: Comparisons of segmentation time among six different methods. The estimation of time for the proposed method was also documented (see Table 8). Computational performance based on the acceleration of GPU. of the first four methods was obtained through respective publications [18, 19, 21, 36]. The proposed method is efficient Method Time as compared to other methods. It only takes averagely 1.54 Pereira et al. [18] 8 s–24 min seconds in order to segment a slice and only runs slightly Havaei et al. [19] 8 min slower than the FCN-8s (0.98 seconds). This is understand- able because the proposed method needs two-stage segmen- Kamnitsas et al. [21] 30 s tation while the FCN-8s only needs a forward computation. Dong et al. [36] 2-3 s However, the FCN-8s yields less accurate and overly smooth FCN-8s 0.98 s boundary maps. Of note, adopting the FCN for image seman- Proposed 1.54 s tic segmentation is faster than the traditional method based on patch-wise [22, 36]; despite computational efficiency, tests reported in the literature were done using slightly different method is that they adopted the FCN in both first and second computing platforms. steps. We compared the results of our method with their method (see Table 7). Our proposed approach obtained the better DSC values (0.90, 0.81, and 0.81) in all tumor regions. 5. Discussions and Conclusions Furthermore, the proposed method also yielded higher PPV values in the complete and enhancing tumor categories and In this work, a cascaded neural network was designed, imple- a higher sensitivity in the core tumor category. Of note, Per- mented, and tested. The proposed system consists of two eira et al. [39] trained and tested on the BRATS 2013 dataset steps. In the first step, the TLN subnet was used to localize but we on the BRATS 2015 dataset. the brain tumor. Then, the ITCN subnet was applied to the Journal of Healthcare Engineering 13 identified tumor regions to further classify the tumor into in neuro-oncology: the avenue to a cure for malignant gli- oma,” CA: A Cancer Journal for Clinicians, vol. 60, no. 3, four subregions. We also adopted the advanced technologies pp. 166–193, 2010. to train and optimize the proposed cascaded neural network. [4] G. Tabatabai, R. Stupp, M. J. van den Bent et al., “Molecular Numerical experiments were conducted on 274 patient diagnostics of gliomas: the clinical perspective,” Acta Neuro- in vivo data sets from the BRATS 2015. DSC, PPV, and sen- pathologica, vol. 120, no. 5, pp. 585–592, 2010. sitivity were used as metrics for segmentation accuracy. [5] J. J. Corso, E. Sharon, S. Dube, S. El-Saden, U. Sinha, and Based on quantitative and qualitative evaluations, we A. Yuille, “Efficient multilevel brain tumor segmentation with found that the proposed approach was able to accurately integrated Bayesian model classification,” IEEE Transactions localize and segment complex brain tumors. We stipulate on Medical Imaging, vol. 27, no. 5, pp. 629–640, 2008. that there are two reasons. First, the ITCN subnet only [6] A. Hamamci, N. Kucuk, K. Karaman, K. Engin, and G. Unal, represents and subsequently classifies the intratumoral “Tumor-cut: segmentation of brain tumors on contrast region whereas other methods need to represent and clas- enhanced MR images for radiosurgery applications,” IEEE sify all heterogeneous brain tissues. Second, intratumor Transactions on Medical Imaging, vol. 31, no. 3, pp. 790–804, subregions are usually very small proportions of the entire image. Other neural networks (e.g., FCN-8s) may suffer [7] I. Mehmood, N. Ejaz, M. Sajjad, and S. W. Baik, “Prioritization from the imbalance of different pixel labels. In the TLN of brain MRI volumes using medical image perception model subnet, our proposed method merged different tumor sub- and tumor region segmentation,” Computers in Biology and regions into a whole tumor. Thus, the imbalance can be Medicine, vol. 43, no. 10, pp. 1471–1483, 2013. somewhat mitigated. In the ITCN subnet, we adopted the [8] M. Havaei, H. Larochelle, P. Poulin, and P.-M. Jodoin, same quantity image patches of each class to train and “Within-brain classification for brain tumor segmentation,” optimize the model. In the future, deep learning neural International Journal of Computer Assisted Radiology and Sur- gery, vol. 11, no. 5, pp. 777–788, 2016. networks could be expanded to include histological data and other data to further improve clinical management [9] K. Usman and K. Rajpoot, “Brain tumor classification from multi-modality MRI using wavelets and machine learning,” of brain cancers [40]. Pattern Analysis and Applications, vol. 20, no. 3, pp. 871– Furthermore, the proposed cascaded neural network can, 881, 2017. on average, complete a segmentation task within 1.54 sec- [10] N. J. Tustison, K. L. Shrinidhi, M. Wintermark et al., “Optimal onds. The proposed TLN subset only requires a forward symmetric multimodal templates and concatenated random computation for localizing the whole tumor region in the first forests for supervised brain tumor segmentation (simplified) step. Then, the ITCN subnet only needs to classify tumor with ANTsR,” Neuroinformatics, vol. 13, no. 2, pp. 209–225, candidate pixels into different class subregions within a much-reduced region located by the TLN, thereby improving [11] D. Zikic, B. Glocker, E. Konukoglu et al., “Decision forests for the computing efficiency. tissue-specific segmentation of high-grade gliomas in multi- channel MR,” in Medical Image Computing and Computer- Conflicts of Interest Assisted Intervention – MICCAI 2012. MICCAI 2012, vol 7512, Lecture Notes in Computer Science, N. Ayache, H. The authors declare that they have no conflicts of interest. Delingette, P. Golland, and K. Mori, Eds., Springer, Berlin, Heidelberg, 2012. Acknowledgments [12] A. Pinto, S. Pereira, H. Correia, J. Oliveira, D. M. Rasteiro, and C. A. Silva, “Brain tumour segmentation based on extremely This research is funded by Chongqing Science and Technology randomized forest with high-level features,” in 2015 37th Commission (Grant no. cstc2016jcyjA0383) and Humanity Annual International Conference of the IEEE Engineering in and Social Science Key Project of Chongqing Municipal Medicine and Biology Society (EMBC), Milan, Italy, August Education Commission (Grant no. 16SKGH133). This research is also in part supported by Scientific and Techno- [13] S. Bauer, L.-P. Nolte, and M. Reyes, “Fully automatic segmen- logical Research Program of Chongqing Municipal Education tation of brain tumor images using support vector machine Commission (Grant no. KJ1709210) and Graduate Innova- classification in combination with hierarchical conditional tion Fund of Chongqing University of Technology (Grant random field regularization,” in Medical Image Computing no. YCX2016230). and Computer-Assisted Intervention – MICCAI 2011. MIC- CAI 2011, vol 6893, Lecture Notes in Computer Science, G. Fichtinger, A. Martel, and T. Peters, Eds., Springer, Berlin, References Heidelberg, 2011. [1] D. N. Louis, H. Ohgaki, O. D. Wiestler et al., “The 2007 WHO [14] G. Urban, M. Bendszus, F. Hamprecht, and J. Kleesiek, “Multi- classification of tumours of the central nervous system,” Acta modal brain tumor segmentation using deep convolutional Neuropathologica, vol. 114, no. 2, pp. 97–109, 2007. neural networks,” in MICCAI BraTS (Brain Tumor Segmenta- [2] S. Bauer, R. Wiest, L. P. Nolte, and M. Reyes, “A survey of tion) Challenge. Proceedings, Winning Contribution, pp. 31– MRI-based medical image analysis for brain tumor studies,” 35, Boston, MA, USA, 2014. Physics in Medicine & Biology, vol. 58, no. 13, pp. R97–R129, [15] D. Zikic, Y. Ioannou, M. Brown, and A. Criminisi, “Segmenta- tion of brain tumor tissues with convolutional neural net- [3] E. G. Van Meir, C. G. Hadjipanayis, A. D. Norden, H. K. works,” in Proceedings MICCAI-BRATS, pp. 36–39, Boston, Shu, P. Y. Wen, and J. J. Olson, “Exciting new advances MA, USA, 2014. 14 Journal of Healthcare Engineering [32] B. H. Menze, A. Jakab, S. Bauer et al., “The multimodal brain [16] A. Davy, M. Havaei, D. Warder-Farley et al., “Brain tumor seg- mentation with deep neural networks,” in Proceedings MIC- tumor image segmentation benchmark (BRATS),” IEEE CAI-BRATS, Boston, MA, USA, 2014. Transactions on Medical Imaging, vol. 34, no. 10, pp. 1993– 2024, 2015. [17] P. Dvorak and B. Menze, “Structured prediction with convolu- tional neural networks for multimodal brain tumor segmenta- [33] Y. Bengio, “Practical recommendations for gradient-based tion,” in Proceedings MICCAI-BRATS, , pp. 13–24, Munich, training of deep architectures,” in Neural Networks: Tricks of Germany, 2015. the Trade, vol 7700pp. 437–478, Springer, Berlin, Heidelberg. [18] S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumor [34] L. R. Dice, “Measures of the amount of ecologic association segmentation using convolutional neural networks in MRI between species,” Ecology, vol. 26, no. 3, pp. 297–302, 1945. images,” IEEE Transactions on Medical Imaging, vol. 35, [35] K.-L. Tseng, Y.-L. Lin, W. Hsu, and C.-Y. Huang, “Joint no. 5, pp. 1240–1251, 2016. sequence learning and cross-modality convolution for 3D bio- [19] M. Havaei, A. Davy, D. Warde-Farley et al., “Brain tumor medical segmentation,” in 2017 IEEE Conference on Computer segmentation with deep neural networks,” Medical Image Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, Analysis, vol. 35, pp. 18–31, 2017. July 2017. [20] M. Lyksborg, O. Puonti, M. Agn, and R. Larsen, “An ensemble [36] H. Dong, G. Yang, F. Liu, Y. Mo, and Y. Guo, “Automatic of 2D convolutional neural networks for tumor segmentation,” brain tumor detection and segmentation using U-net based in Scandinavian Conference on Image Analysis. SCIA 2015, vol fully convolutional networks,” in Medical Image Understand- 9127, Lecture Notes in Computer Science, Springer, Cham, ing and Analysis. MIUA 2017. Communications in Computer and Information Science, vol 723, M. Valdés Hernández and [21] K. Kamnitsas, C. Ledig, V. F. J. Newcombe et al., “Efficient V. González-Castro, Eds., Springer, Cham, 2017. multi-scale 3D CNN with fully connected CRF for accurate [37] D. Yi, M. Zhou, Z. Chen, and O. Gevaert, “3-D convolutional brain lesion segmentation,” Medical Image Analysis, vol. 36, neural networks for glioblastoma segmentation,” 2016, pp. 61–78, 2017. http://arxiv.org/abs/1611.04534. [22] E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional [38] S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Deep convolu- networks for semantic segmentation,” IEEE Transactions on tional neural networks for the segmentation of gliomas in Pattern Analysis and Machine Intelligence, vol. 39, no. 4, multi-sequence MRI,” in Brainlesion: Glioma, Multiple Sclero- pp. 640–651, 2017. sis, Stroke and Traumatic Brain Injuries. BrainLes, A. Crimi, B. [23] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet clas- Menze, O. Maier, M. Reyes, and H. Handels, Eds., vol. 9556 of sification with deep convolutional neural networks,” in Lecture Notes in Computer Science, Springer, Cham, 2015. Advances in Neural Information Processing Systems, Carson, [39] S. Pereira, A. Oliveira, V. Alves, and C. A. Silva, “On hierarchi- NV, USA, 2012. cal brain tumor segmentation in MRI using fully convolutional [24] K. Simonyan and A. Zisserman, “Very deep convolutional net- neural networks: a preliminary study,” in 2017 IEEE 5th Portu- works for large-scale image recognition,” 2014, http://arxiv guese Meeting on Bioengineering (ENBENG), Coimbra, Portu- .org/abs/1409.1556. gal, February 2017. [25] B. B. Avants, N. Tustison, and G. Song, “Advanced normaliza- [40] R. F. Barajas Jr, J. G. Hodgson, J. S. Chang et al., “Glioblastoma tion tools (ANTS),” Insight Journal, vol. 2, pp. 1–35, 2009. multiforme regional genetic and cellular expression patterns: influence on anatomic and physiologic MR imaging,” Radiol- [26] K. Jarrett, K. Kavukcuoglu, and Y. LeCun, “What is the best ogy, vol. 254, no. 2, pp. 564–576, 2010. multi-stage architecture for object recognition?,” in 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, September-Octeober 2009. [27] A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinear- ities improve neural network acoustic models,” in Proceedings of the 30th International Conference on Ma- chine Learning, Atlanta, GA, USA, 2013. [28] H.-C. Shin, H. R. Roth, M. Gao et al., “Deep convolutional neural networks for computer-aided detection: CNN architec- tures, dataset characteristics and transfer learning,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1285– 1298, 2016. [29] N. Tajbakhsh, J. Y. Shin, S. R. Gurudu et al., “Convolutional neural networks for medical image analysis: full training or fine tuning?,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1299–1312, 2016. [30] X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the Thirteenth International Conference on Artificial Intelli- gence and Statistics, Sardinia, Italy, 2010. [31] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Healthcare Engineering Hindawi Publishing Corporation

Automatic Semantic Segmentation of Brain Gliomas from MRI Images Using a Deep Cascaded Neural Network

Loading next page...
 
/lp/hindawi-publishing-corporation/automatic-semantic-segmentation-of-brain-gliomas-from-mri-images-using-DOT74vtRCc

References (42)

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2018 Shaoguo Cui et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
2040-2295
eISSN
2040-2309
DOI
10.1155/2018/4940593
Publisher site
See Article on Publisher Site

Abstract

Hindawi Journal of Healthcare Engineering Volume 2018, Article ID 4940593, 14 pages https://doi.org/10.1155/2018/4940593 Research Article Automatic Semantic Segmentation of Brain Gliomas from MRI Images Using a Deep Cascaded Neural Network 1,2 1 2,3 1 1 Shaoguo Cui , Lei Mao, Jingfeng Jiang , Chang Liu, and Shuyu Xiong College of Computer Science and Engineering, Chongqing University of Technology, Chongqing 400054, China Medical Physics Department, University of Wisconsin, Madison, WI 53705, USA Biomedical Engineering Department, Michigan Technological University, Houghton, MI 49931, USA Correspondence should be addressed to Shaoguo Cui; cuishaoguo2002@163.com Received 30 November 2017; Accepted 11 February 2018; Published 19 March 2018 Academic Editor: Weide Chang Copyright © 2018 Shaoguo Cui et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Brain tumors can appear anywhere in the brain and have vastly different sizes and morphology. Additionally, these tumors are often diffused and poorly contrasted. Consequently, the segmentation of brain tumor and intratumor subregions using magnetic resonance imaging (MRI) data with minimal human interventions remains a challenging task. In this paper, we present a novel fully automatic segmentation method from MRI data containing in vivo brain gliomas. This approach can not only localize the entire tumor region but can also accurately segment the intratumor structure. The proposed work was based on a cascaded deep learning convolutional neural network consisting of two subnetworks: (1) a tumor localization network (TLN) and (2) an intratumor classification network (ITCN). The TLN, a fully convolutional network (FCN) in conjunction with the transfer learning technology, was used to first process MRI data. The goal of the first subnetwork was to define the tumor region from an MRI slice. Then, the ITCN was used to label the defined tumor region into multiple subregions. Particularly, ITCN exploited a convolutional neural network (CNN) with deeper architecture and smaller kernel. The proposed approach was validated on multimodal brain tumor segmentation (BRATS 2015) datasets, which contain 220 high-grade glioma (HGG) and 54 low-grade glioma (LGG) cases. Dice similarity coefficient (DSC), positive predictive value (PPV), and sensitivity were used as evaluation metrics. Our experimental results indicated that our method could obtain the promising segmentation results and had a faster segmentation speed. More specifically, the proposed method obtained comparable and overall better DSC values (0.89, 0.77, and 0.80) on the combined (HGG + LGG) testing set, as compared to other methods reported in the literature. Additionally, the proposed approach was able to complete a segmentation task at a rate of 1.54 seconds per slice. determine (1) the presence of a tumor, (2) the inclusion of 1. Introduction peritumoral edema, and (3) the spread into other locations Although brain cancers are less prevalent, they are very such as the CNS [3]. lethal. Among them, gliomas are the most common brain Compared to CT, MRI or contrast-enhanced MRI tumors. They can be graded into low-grade gliomas (LGG) becomes the imaging modality of choice for diagnosis and and high-grade gliomas (HGG), with the latter being more treatment planning in the brain because of its sensitivity aggressive and infiltrative than the former [1]. A glioma is and superior image contrast in soft tissues. However, the highly invasive because it tends to aggressively grow and multiplicity and complexity of the brain tumors under MRI could quickly invade the central nervous system (CNS). often make tumor recognition and segmentation difficult According to US National Cancer Institute, approximately for radiologists and other clinicians [4]. Consequently, auto- matic segmentation of heterogeneous tumors can greatly 18,000 Americans are diagnosed with a glioma every year; many of them die within 14 months [2]. In clinical practice, impact the clinical medicine by freeing physicians from the medical imaging, mainly computed tomography (CT) and burden of the manual depiction of tumors. Furthermore, if magnetic resonance imaging (MRI), has been used to computer algorithms can provide robust and quantitative 2 Journal of Healthcare Engineering of the CNN in the ImageNet Large-Scale Visual Recognition measurements of tumor depiction, these automated mea- surements will greatly aid in the clinical management of Challenge (ILSVRC) [23]. Recent deep learning methods for brain tumors. automatic brain tumor segmentation are summarized below In the past few decades, significant research efforts in the in Table 2. computer vision and image processing community have been However, the above-mentioned CNN methods were all devoted to developing computer-aided systems that can be based on the patch-wise method in which (medical) images used for automated tumor characterization/classification were often divided into patches during the training and test- [5–21]. Although some systems were tested and showed good ing. The advantage of this method was that it could take performance, the fully automatic detection and subsequent advantage of the existing classification model of the natural diagnosis of brain tumors have not been massively used in image and solve the problem of the class label imbalance in the clinical settings, thereby indicating that some major MRI images. Despite its popularity, operating on image developments are still needed [21]. patches was computationally time-consuming. Recalling, Based on MRI data, our primary goal of this paper was to given a typical image size (e.g., 256 × 256), a large number propose a new fast and accurate computer system that could of patches (65535) were required as inputs for prediction. first localize complete tumor region and then segment the Furthermore, this method was not end-to-end and per- more detailed intratumor structure. Our computer system formed the segmentation task by independently classifying contained two major steps. First, by leveraging an FCN the central pixel of a patch, which will result in some errors [22], a tumor location map was first obtained. In the second and need postprocessing. Thus, the expensive computation and postprocessing become the bottleneck of its real-time step, a deep learning ensemble of the CNN was used to clas- sify the tumor region into four subregions: (1) necrosis, (2) clinic application. edema, (3) nonenhancing tumor, and (4) enhancing tumor. Recently, Shelhamer et al. [22] presented a novel FCN for In this study, the performance of the proposed algorithm semantic segmentation of natural scene images. This model was assessed in a public database containing 274 cases of can be trained in an end-to-end manner (also known as pixel-wise). Their results showed that the FCN outperformed in vivo gliomas. The paper is structured as follows: Section 2 presents the the previous methods for semantic segmentation of a natural related works in the automated brain cancer segmentation. scene image in performance and speed. Inspired by the work Particularly, attention was given to computer systems based in [22], we proposed a hybrid approach by constructing a on machine learning. The proposed two-step (cascaded) deep cascaded neural network. Our main contribution of this work is to propose a hybrid neural network is described in Section 3. The emphases are on the design methodology and training methods for the cascaded neural network for the purpose of segmentation of performance assessment. In Section 4, results of our numer- brain tumors including segmentation of intratumor subre- ical experiments are summarized followed by some closing gions, from MRI data. This model consists of one FCN and remarks in Section 5. one CNN. This combination enables us to perform pixel semantic predictions by taking advantage of both a pixel- wise method and a patch-wise method. Formally, in this cas- 2. Relevant Work and Our Contributions caded neural network, an FCN was first used to localize the In recent years, many methods have been proposed to tumor region from an MRI slice and then a CNN with deeper automatically segment brain tumors based on MRI data. architecture and smaller kernels was used to classify brain tumor into multiple subregions. This approach can not only These methods can be largely divided into two categories: (1) hand-crafted feature and classifier methods based on obtain the better segmentation accuracy but can also speed traditional machine learning such as support vector the prediction efficiency. machine (SVM) and random forests (RF) [5–13] and (2) fully automatic methods based on deep learning using the 3. Methods CNN [14–21]. Methods in the first category use manually extracted fea- 3.1. Construction of the Deep Cascaded Neural Network. The tures, and these features are input to classifiers. In other starting point of the proposed system is in vivo MRI data words, once these hand-crafted features are solely deter- consisting of four different sequences (FLAIR, T1, T1c, and mined by human operators, classifiers “weigh” them during T2), and the endpoint becomes a characterized tumor (see the training but cannot modify these features in any way. Figure 1). In the output image, a brain tumor is classified into One significant concern of hand-crafted features stems from four different zones: necrosis, edema, nonenhancing tumor, the fact that these features could have significant inter- and and enhancing tumor. intrauser variability. A brief summary of these methods can More specifically, the architecture of the proposed system be found in Table 1. includes an FCN followed by a CNN which accompanies In contrast, methods in the second category can self-learn small convolution kernels (see Figure 1). So the segmentation the feature representations adapted to a specific task from task based on this cascaded network can be divided into two training data. Recently, deep learning neural networks, espe- major steps. In the first step, the pixel-wise FCN was used to cially CNNs, are rapidly gaining their popularity in the com- quickly localize the tumor by marking the tumor region. puter vision community. This trend has certainly been Then, the patch-wise CNN was used to further categorize accelerated after the recent record-shattering performance the above-identified tumor region into different subregions Journal of Healthcare Engineering 3 Table 1: A summary of brain tumor segmentation methods based on traditional machine learning. Only methods using MRI data were included in this table. Number Publication Database Summary of method Performance 20 cases of in vivo A hybrid method combining an 1 Corso et al. [5] brain tumors; affinity-based segmentation method 0.62–0.69 (Jaccard) T1, T1-C, T2, FLAIR with a generative model Synthetic data from A cellular automata method combining 0.72 (DICE complete 2 Hamamci et al. [6] Utah + in vivo data a probability framework tumor) from Harvard BrainWeb data + in vivo A novel saliency model for lesion 83%~95% brain tumors; 3 Mehmood et al. [7] localization and an N-cut graph segmentation (classification T1, T1-weighted, model for classification accuracy) T2, T2-weighted MICCAI-BRATS 2013 Hand-crafted features + a support 0.86 (DICE complete 4 Havaei et al. [8] dataset vector machine tumor) MICCAI-BRATS 2013 Automated wavelet-based features + a 0.88 (DICE complete 5 Usman and Rajpoot [9] dataset random forest classifier tumor) Combine a random forest model with a MICCAI-BRATS 2013 0.88 (DICE complete 6 Tustison et al. [10] framework of regularized probabilistic dataset tumor) segmentation GT: 0.89 AC: 0.84 40 multichannel Decision forests using context-aware NE: 0.70 E: 0.72 7 Zikic et al. [11] MR images, spatial features for automatic segmentation including DTI of high-grade gliomas (10/30 tests) Using appearance- and context-based MICCAI-BRATS 0.83 (DICE complete 8 Pinto et al. [12] features to feed an extremely randomized 2013 dataset tumor) forest GT: 0.84 NE: 0.70 AC: 0.84 E: 0.72 10 multispectral Combines support vector machine 9 Bauer et al. [13] patient datasets classification with conditional random fields (Intrapatient regularized) Table 2: A summary of brain tumor segmentation methods based on deep-learning neural networks. Only methods using MRI data were included in this table. Performance (DICE) Number Publication Database Summary of method Complete Core Enh MICCAI-BRATS 1 Urban et al. [14] 3D CNN with 3D convolutional kernels 0.87 0.77 0.73 2013 dataset MICCAI-BRATS Apply a CNN in a sliding-window 2 Zikic et al. [15] 0.84 0.74 0.69 2013 dataset fashion in the 3D space MICCAI-BRATS A CNN with two pathways of both local 3 Davy et al. [16] 0.85 0.74 0.68 2013 dataset and global information MICCAI-BRATS Structured prediction was used 4 Dvorak and Menze [17] 0.83 0.75 0.77 2013 dataset together with a CNN MICCAI-BRATS 5 Pereira et al. [18] A CNN with small 3 × 3 kernels 0.88 0.83 0.77 2013 dataset A cascade neural network architecture MICCAI-BRATS in which “the output of a basic CNN is treated 6 Havaei et al. [19] 0.88 0.79 0.73 2013 dataset as an additional source of information for a subsequent CNN” An ensemble of 2D convolutional neural MICCAI-BRATS 7 Lyksborg et al. [20] networks +doing a volumetric segmentation 0.80 0.64 0.59 2014 dataset by three steps MICCAI-BRATS Using 3D CNN, two-scale extracted 8 Kamnitsas et al. [21] 0.85 0.67 0.63 2015 dataset feature, 3D dense CRF as postprocessing 4 Journal of Healthcare Engineering Input Output Tumor localization network (TLN) Intratumor classification network (ITCN) Output of TLN Tumor candidates Figure 1: An illustrative overview of the proposed deep cascaded convolutional neural network for a fast and accurate tumor segmentation. representing different pathologies. This system design was mitigating the loss of local image features and (2) combining motivated and justified as follows. First, the FCN can take local information obtained from intermediate layers (i.e., max pooling 4 and max pooling 3, resp.) with the global a whole image as the input and localization of a complete tumor only requires one-pass of the forward propagation. information in these deep layers (i.e., after 7 convolution Thus, it can remarkably improve the segmentation effi- layers). All relevant parameters used in the subnet TLN are ciency. Second, this combination of FCN and CNN can alle- shown in Table 3 below. viate the pixel sample class imbalance problem which is serious in MRI images. Thus, it can capture better segmen- 3.1.2. A Description of ITCN. The proposed ITCN includes tation details. Third, the intratumor characterization in the two convolutional layer groups (3 layers each), two max second step will only need to be applied to the tumor pooling layers, and three fully connected layers. Recall that regions localized in the first step instead of the entire image, the TLN yields a binary tumor map for a given MRI image thereby significantly reducing forward computing time. and the ITCN (see Figure 3) further classifies the identified Hereafter, the FCN and the CNN are referred as to tumor tumor into 4 different subregions. Formally, for each location localization network (TLN) and intratumor classification i, j within the identified tumor map, 4 patches (size of network (ITCN), respectively. 33 × 33) centered on the i, j location were extracted from the original 4 input channels (FLAIR, T1, T1c, and T2) and 3.1.1. A Description of TLN. We modified the FCN-8s archi- subsequently used as the input to the ITCN. More details of tecture [22] to model our TLN. The input channels (RGB) in this ITCN subnet are listed in Table 4. the original FCN-8s were changed to 4 channels in order to In the ITCN, as inspired by the work of Simonyan and account for 4 different MRI modalities. And the 21 output Zisserman [24], multiple convolutional layers with small ker- channels in the original FCN-8s were changed to 2, corre- nels (3 × 3 pixels) were used. An alternative approach would sponding to either the tumor region or the nontumor region. be an architecture with fewer layers and larger kernels. The- As shown in Figure 2, after the operations of the convolution oretically, two cascaded convolutional layers with two 3 × 3 and pooling, the feature map became smaller in size (see kernels have similar effects on the receptive fields, as com- Table 3). To obtain a higher resolution of the final features, pared to one convolutional layer with a 5 × 5 kernel. But the input images (size 240 × 240) were padded to 438 × 438 two cascaded layers with two 3 × 3 kernels result in more using zero padding [22]. Additionally, the deconvolution complex nonlinearities and fewer weights. Fewer weights was applied so that the size of output image matched with lead to a less computing cost and can also alleviate the pos- that of the input image. It is worth noting that multiple con- sibility of overfitting. It is generally understood that, with volutional kernels were used in each convolutional layer for a the increase of the CNN’s depth, a CNN can gain higher better feature extraction (e.g., edges, curves, and corner). representation capacity. As shown in Figure 3, in each of We observed that a significant amount of low-level the two pooling layers, a 3 × 3 overlapping subwindow with feature details such as location and edge could be lost after a stride of 2 was applied to the feature maps for reducing convolution striding and pooling. However, these lost fea- feature dimension and integrating higher-level features. tures were valuable for semantic segmentation. Thus, two The detailed hyperparameters of the ITCN can be found skip connections [22] were introduced for two purposes: (1) in Table 4 below. Journal of Healthcare Engineering 5 Input Output Figure 2: An illustration of the architecture of the TLN subnet for pixel-wise prediction. Table 3: Parameters used in the subnet TLN. In each convolutional layer, the feature maps had been padded by 1 prior to the convolution so that all intermediate feature maps do not change their sizes before and after the convolution. Number Layer name Filter size Stride Number of Filters Output ∗ ∗ ∗ 1 Conv 1_1 + ReLU 3 3 1 64 438 438 64 ∗ ∗ ∗ 2 Conv 1_2 + ReLU 3 3 1 64 438 438 64 ∗ ∗ ∗ 3 Max pooling 1 222 — 219 219 64 ∗ ∗ ∗ 4 Conv 2_1 + ReLU 3 3 1 128 219 219 128 ∗ ∗ ∗ 5 Conv 2_2 + ReLU 3 3 1 128 219 219 128 ∗ ∗ ∗ 6 Max pooling 2 222 — 110 110 128 ∗ ∗ ∗ 7 Conv 3_1 + ReLU 3 3 1 256 110 110 256 ∗ ∗ ∗ 8 Conv 3_2 + ReLU 3 3 1 256 110 110 256 ∗ ∗ ∗ 9 Conv 3_3 + ReLU 3 3 1 256 110 110 256 ∗ ∗ ∗ 10 Max pooling 3 222 — 55 55 256 ∗ ∗ ∗ 11 Conv 4_1 + ReLU 3 3 1 512 55 55 512 ∗ ∗ ∗ 12 Conv 4_2 + ReLU 3 3 1 512 55 55 512 ∗ ∗ ∗ 13 Conv 4_3 + ReLU 3 3 1 512 55 55 512 ∗ ∗ ∗ 14 Max pooling 4 222 — 28 28 512 ∗ ∗ ∗ 15 Conv 5_1 + ReLU 3 3 1 512 28 28 512 ∗ ∗ ∗ 16 Conv 5_2 + ReLU 3 3 1 512 28 28 512 ∗ ∗ ∗ 17 Conv 5_3 + ReLU 3 3 1 512 28 28 512 ∗ ∗ ∗ 18 Max pooling 5 222 — 14 14 512 ∗ ∗ ∗ 19 Conv 6 + ReLU 7 7 1 4096 8 8 4096 ∗ ∗ ∗ 20 Conv 7 + ReLU 1 1 1 4096 8 8 4096 3.2. Implementation. All numerical experiments were con- 3.2.1. Preprocessing. As recommended by the literature [25], ducted using a Dell workstation equipped with dual Intel MRI data were preprocessed before the proposed cascaded E5-2603 CPUs and a middle-end GPU graphic card (GeForce neural network was applied. Basically, the N4ITK method GTX 1080, NVIDIA, CA, USA). The operation system of the was first used to correct the distortion of MRI data, followed workstation is Ubuntu (version 14.04). The proposed cas- by data normalization. caded neural network has been implemented using Python Given an image X, x i, j is the intensity correspond- (version 2.7) under the framework of Caffe, an open-source ing to the jth column at the ith row of X i, j =1,2, … , deep learning platform (http://caffe.berkeleyvision.org/). 240 . The data intensity normalization procedure is briefly Some essential details are discussed below. described below: Conv 1 group Max pooling 1 Conv 2 group Max pooling 2 Conv 3 group Max pooling 3 Conv 4 group Max pooling 4 Fusing Fusing Conv 5 group Deconv Deconv Max pooling 5 Conv 6 - 7 Deconv 6 Journal of Healthcare Engineering Fully connected Fully connected Max Max pooling 1 pooling 2 33 × 33 × 4 Conv 1_1 Conv 1_2 Conv 1_3 Conv 1_1 Conv 1_2 Conv 1_3 Fully connected Figure 3: An illustration of the second subnet ITCN for the intratumoral classification. The classification was done in a patch-to-patch fashion. Table 4: A list of parameters used in the proposed subnet ITCN. In each convolutional layer, the feature maps had been padded by 1 prior to the convolution so that the convolution do not change the size of the resultant feature map. Number Layer name Filter size Stride Number of filters FC units Output ∗ ∗ ∗ 1 Conv 1_1 + LReLU 331 64 — 33 33 64 ∗ ∗ ∗ 2 Conv 1_2 + LReLU 331 64 — 33 33 64 ∗ ∗ ∗ 3 Conv 1_3 + LReLU 331 64 — 33 33 64 ∗ ∗ ∗ 4 Max pooling 1 332 —— 16 16 64 ∗ ∗ ∗ 5 Conv 2_1 + LReLU 3 3 1 128 — 16 16 128 ∗ ∗ ∗ 6 Conv 2_2 + LReLU 3 3 1 128 — 16 16 128 ∗ ∗ ∗ 7 Conv 2_3 + LReLU 3 3 1 128 — 16 16 128 ∗ ∗ ∗ 8 Max pooling 2 332 —— 8 8 128 9 FC1 + dropout —— — 8192 256 10 FC2 + dropout —— — 256 128 11 FC3 + softmax —— — 128 4 (1) Removed the top 1% and bottom 1% from each slice We randomly selected 3 different cases from the FLAIR data- of the MRI data. set. As shown in Figure 4 below, it is easy to find that the above-mentioned data normalization can improve the com- (2) For each slice of MRI data X, a normalized image X parability of different slices. was obtained. In the scaled image X , each intensity 3.2.2. Convolution Operation. Each feature map Z shown in value x i, j can be obtained as follows: Figures 1, 2, and 3 was associated with one convolution kernel. Z was computed as follows: xi, j − X x i, j = , 1 Z = b + 〠 W ∗ X , 2 r r r=1 where x i, j is the gray value of pixel i, j prior to the where k is the number of input channels, b is a bias term, X is normalization and X and X are the mean and standard devi- an image from the rth input channel, and W is the weight ation of the unscaled image X, respectively. ∗ associated with the rth channel. In (2), denotes a convolu- The above-mentioned preprocessing method was used to tion operator. process each modality MRI data including FLAIR, T1, T1c, and T2. Particularly, the FLAIR images were generated using 3.2.3. Nonlinear Activation Function. In our study, the TLN fluid-attenuated inversion recovery protocol and useful in used rectified linear unit (ReLU) function [23] to perform terms of differentiating the brain tumor from its normal nonlinear transformations. This selection was because ReLU background. Figure 4 presents some FLAIR slices before could achieve better results as compared to the classical and after using the proposed image intensity normalization. sigmoid and hyperbolic tangent functions. The use of ReLU … Journal of Healthcare Engineering 7 (a) (b) Figure 4: Randomly selected examples of FLAIR slices before (a) and after (b) the above-mentioned intensity normalization. was also able to accelerate the training [26]. Mathematically, arbitrary prediction for the ith pixel, the predition loss can the ReLU function is defined below: be defined as fz = max 0, z 3 C L θ = −〠 Y log Y , 6 ij ij In the ITCN, the leaky rectifier linear unit (LReLU) [27] j=1 was used. This was because imposing zeros (see (3)) could negatively affect the calculation of gradients. During the ′ where Y , Y, and C are a one-hot vector, the predicted prob- training of this neural network, zero gradients will signifi- ability distribution, and the number of classes, respectively. cantly slow down the adjustments of weights. The LReLU In the TLN, predictions were made for each pixel of function reads the input image so that the loss function can be written as follows: fz = max 0, z + α min 0, z , 4 S C where α is the leakiness parameter [18]. ′ ′ L θ = − 〠〠 Y log Y , 7 ij ij To address the multiclassification problem, a well- i=1 j=1 known softmax function was used to transform the neural network outputs to probability distributions. Softmax is where C =2 and S is the pixel number of the input image. In defined as follows: every training, only one input image was used (the size of minibatch was 1). Now referring to the ITCN, the loss function was calcu- Y = soft max Z = , 5 i i lated in conjunction with the concept of mini-batch. Thus, the loss function has the following form, where Z is the output from the ith neuron and Y is the prob- i i M C ability of input pixel corresponding to the ith class. In the ″ ′ L θ = − 〠〠 Y log Y , 8 TLN, i =1 or 2 because the TLN was to perform a binary clas- ij ij i=1 j=1 sification in the first step. In the ITCN, i =1,2, 3,4 since the ITCN was to classify the MRI data into four classes. where C =4 and M is the size of minibatch. Of note, in this 3.2.4. Loss Function. Given a set of weights of the proposed study, M = 256. neural network θ, a categorical cross-entropy loss function To achieve better generation ability and avoid overfitting, was used to compute the loss of ground truth and pre- L2 regularization terms were also added to (7) and (8). Thus, dicted probability distribution. Mathematically, under an the final forms of the loss functions are 8 Journal of Healthcare Engineering S C Q the dropout regularization [31] and the dropout ratio was 1 λ 2 ′ ′ ′ L θ = − 〠〠 Y log Y + 〠 θ , 9 set to 0.5 in all fully connected layers. Weight decay was ij ij k S S i=1 j=1 k=1 set as 0.005. M C Q 3.3. Datasets and Evaluation Metrics. In order to train and 1 λ 2 ″ ′ ′ L θ = − 〠〠 Y log Y + 〠 θ , 10 ij ij k evaluate the proposed system, numerical experiments were M M i=1 j=1 k=1 carried out using in vivo human patient data provided by the BRATS 2015 database [32]. The BRATS 2015 database where λ is a regularization constant and Q is the number of contains 220 HGG and 54 LGG. Experimental data have model parameter. been labeled, and five labels were used: normal brain tissues (noncancerous zone), necrosis, edema, nonenhancing tumor, 3.2.5. Optimization Method. Equations (9) and (10) were and enhancing tumor. These pixel-wise delineations were minimized using the minibatch stochastic gradient descent considered the ground truth in this study. Each case contains (SGD) algorithm. To avoid numerical oscillations and four sequences of MRI data, namely, T1, T1c, T2, and FLAIR. accelerate convergence, the momentum method [23] was The dimension of each MRI modality is 155 × 240 × 240 (slice used. This process can be described as iterations from number × length × width). All MRI data were spatially regis- (11) to (13). tered and stored as signed 16-bit integers. But only positive g = ∇ L θ , 11 t−1 t−1 values were used. The tenfold crossvalidation method [33] was used to m = μ ∗ m − η g , t t−1 t t evaluate the proposed system. More specifically, the 274 cases were divided into a training set (240 cases) and a θ = θ + m t t−1 t testing set (34 cases). The 240 training cases were equally divided into 10 subsets in which 9 subsets were used as In (11), (12), and (13), the subscript t is the iteration the training and 1 subset was used as the validation. In ′ ″ number and θ corresponds to θ in (9) or θ in (10). L θ t−1 the training phase of the TLN subnet, all subregions within is the loss function when a parameter set θ is used. g , m , t−1 t a tumor were merged into one tumor region. Thus, in the and μ are the gradient, momentum, and momentum coeffi- binary ground truth, zero represents the noncancerous tis- cient, respectively. We set μ =0 99 and μ =0 9 in the TLN sues while one represents cancerous regions. In the train- and ITCN, respectively. Here, η is the learning rate. ing phase of the ITCN subnet, we randomly selected To suppress the SGD noise and guarantee conver- 4,700,000 image patches (33 × 33) from the training set, gence, the learning rate η attenuates linearly from the ini- which correspond to 1,175,000 patches for each label (4 tial learning rate η to the final learning rate η as the 0 τ different classes). iteration progresses: The quantitative evaluations were conducted for 3 differ- ent tumor regions: complete tumor region (including all four η =1 − γ η + γη , 14 t 0 τ tumor subregions), core tumor region (including all tumor structures except edema), and enhancing tumor region (only γ = , including the enhanced tumor structure). For each type of regions, we compute DSC [34], PPV, and sensitivity [35] as where τ is the total iteration number. In this study, we set quantitative evaluation metrics. η = η /100. τ 0 DSC measures the overlap between the ground truth and the automatic segmentation. It is defined as 3.2.6. Training Details. The initial and final learning rates of the TLN model were set to 1e−8 and 1e−10, respectively. P ∩ T 1 1 DSC = , 16 The total iteration τ =2e6, and the momentum coefficient P + T /2 1 1 was 0.99. In the ITCN subnet, the initial and final learning rates were set to 1e−3 and 1e−5, respectively. In the ITCN where P and T represent the positive values of the model 1 1 subnet, the total iteration τ =2e6 and the momentum coeffi- prediction and the ground truth, respectively. cient μ =0 9. PPV is the proportion of the true positive in all segmen- During the training of the TLN subnet, we used the trans- tation tumor points. It is defined as fer learning technique [28, 29]. The initial weights were P ∩ T obtained from a pretrained model that was trained using 1 1 PPV = 17 ImageNet in [24]. But initial weights of the 4th input channel P were initialized using the average of the original 3 input Sensitivity is the proportion of the detected tumor points channel (RGB) weights. And the final two output channels in all ground truth tumor points. It is defined as were initialized with the Xavier method [30]. Then, fine- tuning of the TLN was performed by the optimization pro- P ∩ T 1 1 Sensitivity = 18 cess described above ((11), (12), and (13)) using the MRI training data. However, the training of the ITCN subnet was started from scratch and the weights were initialized The proposed system was compared with some other with the Xavier method [30]. To avoid overfitting, we used published methods. Those methods all have been validated Journal of Healthcare Engineering 9 (a) (b) (c) (d) (e) (f) Figure 5: Representative examples of computer segmentation results of four brain tumors. (a–d) The original FLAIR, T1, T1c, and T2 slices, respectively. (e) The ground truth overlaid with the FLAIR image. (f) Segmentation results overlaid with the FLAIR image. (e, f) Red, green, yellow, and blue colors denote necrosis, edema, nonenhancing tumor, and enhancing tumor, respectively. on the BRATS 2015 dataset. A one-step segmentation 4. Results method based on the FCN-8s was also implemented for the purpose of comparison. The FCN-8s can segment the input 4.1. Qualitative Observations. Overall, we found that the pro- MRI images into 5 classes in a single step. posed system can accurately delineate gliomas. Visual 10 Journal of Healthcare Engineering (a) (b) (c) Figure 6: Two slices of computer segmentation result in a testing case: (a–c) the ground truth, results of tumor localization using the TLN subnet, and the intratumor segmentation results using the ITCN subnet, respectively. (a, c) Red, green, yellow, and blue colors denote necrosis, edema, nonenhancing tumor, and enhancing tumor, respectively. inspections were conducted for testing data to validate the classification results by the FCN-8s (the second column). segmentation results of our proposed method. Figure 5 Furthermore, boundaries of various subregions obtained by shows four selected examples. It can be observed that our the FCN-8s were overly smoothed and, perhaps, inaccurate. method can effectively localize and segment brain tumors But our method using the ITCN had better boundaries of with vastly different shapes and sizes. Visually, the computer the enhancing and nonenhancing regions. segmentation is comparable to the ground truth. Also, the proposed system led to good details around 4.2. Evaluation and Comparison. The quantitative compari- boundaries. Figure 6 presents two representative examples sons with other methods in terms of DSC are summarized of this observation. Since these brain tumors are complex, in Tables 5 and 6. All experiments were conducted on the Figure 6 shows some good showcase examples. During the BRATS 2015 dataset. The results of Table 5 were obtained process, we found that the TLN subnet was able to effectively by using the combined testing set of HGG and LGG, whereas identify nearly all the tumor pixels. Subsequently, the ITCN results shown in Table 6 only used HGG data. subnet efficiently classified the tumor region into four subre- Obviously, the proposed cascaded neural network obtains gions. Our method could largely detect the complete tumor the comparable and better DSC value on all tumor regions. and classify it to different tumor subregions from multimod- Based on the combined testing dataset (see Table 5), our ality MRI images though there were a few misclassifications. method obtained better comprehensive performance values This is not surprising because, pathologically, the brain gli- (0.89, 0.77, and 0.80) as compared to other methods. oma tumors invade their surrounding tissues rather than dis- Although the method proposed by Kamnitsas et al. [21] yields placing them. Hence, the appearance of cancerous tissues a slightly higher DSC value in the complete tumor, they and their surrounding (normal) tissues could be fairly similar obtained lower DSC values in core tumor and enhancing under MRI. tumor. Actually, in their work, a 3D CNN and the structure We also found that, as compared to the FCN-8s with one- prediction technology were adopted (i.e., conditional random step segmentation, the proposed system could segment het- field). Thus, it is computationally time-consuming and needs erogeneous gliomas with a better boundary detail. The results extra postprocessing. Furthermore, the method proposed by of the proposed method and FCN-8s are compared in Dong et al. [36] yielded a slightly higher DSC value in core Figure 7. Five different typical slices representing signifi- tumor and Yi et al. [37] yielded the same DSC value in enhanc- cantly different tumor shapes and sizes are shown in this fig- ing tumor. ure. It is easy to see that the results obtained from the As can be seen in Table 6, based on the HGG testing data- proposed method (the third column) are more similar to set, our method obtained the highest DSC values in the com- the ground truth (the first column), as compared to the plete tumor and enhancing tumor categories. Although the Journal of Healthcare Engineering 11 (a) (b) (c) Figure 7: Examples of segmentation results from five typical slices comparing the FCN-8s (b) and the proposed method (c). (a) The ground truth. In this figure, red, green, yellow, and blue colors denote necrosis, edema, nonenhancing tumor, and enhancing tumor, respectively. Recently, we found that Pereira et al. [39] also proposed a method proposed by Dong et al. [36] yielded a higher DSC value in the core tumor cases, it obtained a lower DSC value hierarchical brain tumor segmentation approach from MRI HGG images. The difference between their method and our in the complete tumor category. 12 Journal of Healthcare Engineering Table 5: A summary of DSC quantitative comparison on BRATS 2015 combined dataset (HGG and LGG). DSC Method Dataset Grade Complete Core Enh BRATS 2015 Challenge Combined 0.78 0.65 0.75 Pereira et al. [38] BRATS 2015 Training Combined 0.87 0.73 0.68 Havaei et al. [19] BRATS 2015 Challenge Combined 0.79 0.58 0.69 BRATS 2015 Challenge Combined 0.85 0.67 0.63 Kamnitsas et al. [21] BRATS 2015 Training Combined 0.90 0.76 0.73 Dong et al. [36] BRATS 2015 Training Combined 0.86 0.86 0.65 Yi et al. [37] BRATS 2015 Training Combined 0.89 0.76 0.80 FCN-8s BRATS 2015 Training Combined 0.84 0.71 0.63 Proposed BRATS 2015 Training Combined 0.89 0.77 0.80 Table 6: A summary of DSC quantitative comparison on BRATS 2015 HGG dataset. DSC Method Dataset Grade Complete Core Enh Pereira et al. [38] BRATS 2015 Training HGG 0.87 0.75 0.75 Havaei et al. [19] BRATS 2015 Challenge HGG —— — Kamnitsas et al. [21] BRATS 2015 Training HGG —— — Dong et al. [36] BRATS 2015 Training HGG 0.88 0.87 0.81 Yi et al. [37] BRATS 2015 Training HGG 0.89 0.79 0.80 FCN-8s BRATS 2015 Training HGG 0.88 0.76 0.71 Proposed BRATS 2015 Training HGG 0.90 0.81 0.81 Table 7: A comparison of our proposed method with hierarchical brain tumor segmentation [39] on DSC, PPV, and sensitivity metrics. DSC PPV Sensitivity Method Complete Core Enh Complete Core Enh Complete Core Enh Pereira et al. [39] 0.85 0.76 0.74 0.80 0.78 0.74 0.92 0.79 0.78 Proposed 0.90 0.81 0.81 0.91 0.77 0.87 0.87 0.84 0.76 Additionally, the segmentation speed for testing data was Table 8: Comparisons of segmentation time among six different methods. The estimation of time for the proposed method was also documented (see Table 8). Computational performance based on the acceleration of GPU. of the first four methods was obtained through respective publications [18, 19, 21, 36]. The proposed method is efficient Method Time as compared to other methods. It only takes averagely 1.54 Pereira et al. [18] 8 s–24 min seconds in order to segment a slice and only runs slightly Havaei et al. [19] 8 min slower than the FCN-8s (0.98 seconds). This is understand- able because the proposed method needs two-stage segmen- Kamnitsas et al. [21] 30 s tation while the FCN-8s only needs a forward computation. Dong et al. [36] 2-3 s However, the FCN-8s yields less accurate and overly smooth FCN-8s 0.98 s boundary maps. Of note, adopting the FCN for image seman- Proposed 1.54 s tic segmentation is faster than the traditional method based on patch-wise [22, 36]; despite computational efficiency, tests reported in the literature were done using slightly different method is that they adopted the FCN in both first and second computing platforms. steps. We compared the results of our method with their method (see Table 7). Our proposed approach obtained the better DSC values (0.90, 0.81, and 0.81) in all tumor regions. 5. Discussions and Conclusions Furthermore, the proposed method also yielded higher PPV values in the complete and enhancing tumor categories and In this work, a cascaded neural network was designed, imple- a higher sensitivity in the core tumor category. Of note, Per- mented, and tested. The proposed system consists of two eira et al. [39] trained and tested on the BRATS 2013 dataset steps. In the first step, the TLN subnet was used to localize but we on the BRATS 2015 dataset. the brain tumor. Then, the ITCN subnet was applied to the Journal of Healthcare Engineering 13 identified tumor regions to further classify the tumor into in neuro-oncology: the avenue to a cure for malignant gli- oma,” CA: A Cancer Journal for Clinicians, vol. 60, no. 3, four subregions. We also adopted the advanced technologies pp. 166–193, 2010. to train and optimize the proposed cascaded neural network. [4] G. Tabatabai, R. Stupp, M. J. van den Bent et al., “Molecular Numerical experiments were conducted on 274 patient diagnostics of gliomas: the clinical perspective,” Acta Neuro- in vivo data sets from the BRATS 2015. DSC, PPV, and sen- pathologica, vol. 120, no. 5, pp. 585–592, 2010. sitivity were used as metrics for segmentation accuracy. [5] J. J. Corso, E. Sharon, S. Dube, S. El-Saden, U. Sinha, and Based on quantitative and qualitative evaluations, we A. Yuille, “Efficient multilevel brain tumor segmentation with found that the proposed approach was able to accurately integrated Bayesian model classification,” IEEE Transactions localize and segment complex brain tumors. We stipulate on Medical Imaging, vol. 27, no. 5, pp. 629–640, 2008. that there are two reasons. First, the ITCN subnet only [6] A. Hamamci, N. Kucuk, K. Karaman, K. Engin, and G. Unal, represents and subsequently classifies the intratumoral “Tumor-cut: segmentation of brain tumors on contrast region whereas other methods need to represent and clas- enhanced MR images for radiosurgery applications,” IEEE sify all heterogeneous brain tissues. Second, intratumor Transactions on Medical Imaging, vol. 31, no. 3, pp. 790–804, subregions are usually very small proportions of the entire image. Other neural networks (e.g., FCN-8s) may suffer [7] I. Mehmood, N. Ejaz, M. Sajjad, and S. W. Baik, “Prioritization from the imbalance of different pixel labels. In the TLN of brain MRI volumes using medical image perception model subnet, our proposed method merged different tumor sub- and tumor region segmentation,” Computers in Biology and regions into a whole tumor. Thus, the imbalance can be Medicine, vol. 43, no. 10, pp. 1471–1483, 2013. somewhat mitigated. In the ITCN subnet, we adopted the [8] M. Havaei, H. Larochelle, P. Poulin, and P.-M. Jodoin, same quantity image patches of each class to train and “Within-brain classification for brain tumor segmentation,” optimize the model. In the future, deep learning neural International Journal of Computer Assisted Radiology and Sur- gery, vol. 11, no. 5, pp. 777–788, 2016. networks could be expanded to include histological data and other data to further improve clinical management [9] K. Usman and K. Rajpoot, “Brain tumor classification from multi-modality MRI using wavelets and machine learning,” of brain cancers [40]. Pattern Analysis and Applications, vol. 20, no. 3, pp. 871– Furthermore, the proposed cascaded neural network can, 881, 2017. on average, complete a segmentation task within 1.54 sec- [10] N. J. Tustison, K. L. Shrinidhi, M. Wintermark et al., “Optimal onds. The proposed TLN subset only requires a forward symmetric multimodal templates and concatenated random computation for localizing the whole tumor region in the first forests for supervised brain tumor segmentation (simplified) step. Then, the ITCN subnet only needs to classify tumor with ANTsR,” Neuroinformatics, vol. 13, no. 2, pp. 209–225, candidate pixels into different class subregions within a much-reduced region located by the TLN, thereby improving [11] D. Zikic, B. Glocker, E. Konukoglu et al., “Decision forests for the computing efficiency. tissue-specific segmentation of high-grade gliomas in multi- channel MR,” in Medical Image Computing and Computer- Conflicts of Interest Assisted Intervention – MICCAI 2012. MICCAI 2012, vol 7512, Lecture Notes in Computer Science, N. Ayache, H. The authors declare that they have no conflicts of interest. Delingette, P. Golland, and K. Mori, Eds., Springer, Berlin, Heidelberg, 2012. Acknowledgments [12] A. Pinto, S. Pereira, H. Correia, J. Oliveira, D. M. Rasteiro, and C. A. Silva, “Brain tumour segmentation based on extremely This research is funded by Chongqing Science and Technology randomized forest with high-level features,” in 2015 37th Commission (Grant no. cstc2016jcyjA0383) and Humanity Annual International Conference of the IEEE Engineering in and Social Science Key Project of Chongqing Municipal Medicine and Biology Society (EMBC), Milan, Italy, August Education Commission (Grant no. 16SKGH133). This research is also in part supported by Scientific and Techno- [13] S. Bauer, L.-P. Nolte, and M. Reyes, “Fully automatic segmen- logical Research Program of Chongqing Municipal Education tation of brain tumor images using support vector machine Commission (Grant no. KJ1709210) and Graduate Innova- classification in combination with hierarchical conditional tion Fund of Chongqing University of Technology (Grant random field regularization,” in Medical Image Computing no. YCX2016230). and Computer-Assisted Intervention – MICCAI 2011. MIC- CAI 2011, vol 6893, Lecture Notes in Computer Science, G. Fichtinger, A. Martel, and T. Peters, Eds., Springer, Berlin, References Heidelberg, 2011. [1] D. N. Louis, H. Ohgaki, O. D. Wiestler et al., “The 2007 WHO [14] G. Urban, M. Bendszus, F. Hamprecht, and J. Kleesiek, “Multi- classification of tumours of the central nervous system,” Acta modal brain tumor segmentation using deep convolutional Neuropathologica, vol. 114, no. 2, pp. 97–109, 2007. neural networks,” in MICCAI BraTS (Brain Tumor Segmenta- [2] S. Bauer, R. Wiest, L. P. Nolte, and M. Reyes, “A survey of tion) Challenge. Proceedings, Winning Contribution, pp. 31– MRI-based medical image analysis for brain tumor studies,” 35, Boston, MA, USA, 2014. Physics in Medicine & Biology, vol. 58, no. 13, pp. R97–R129, [15] D. Zikic, Y. Ioannou, M. Brown, and A. Criminisi, “Segmenta- tion of brain tumor tissues with convolutional neural net- [3] E. G. Van Meir, C. G. Hadjipanayis, A. D. Norden, H. K. works,” in Proceedings MICCAI-BRATS, pp. 36–39, Boston, Shu, P. Y. Wen, and J. J. Olson, “Exciting new advances MA, USA, 2014. 14 Journal of Healthcare Engineering [32] B. H. Menze, A. Jakab, S. Bauer et al., “The multimodal brain [16] A. Davy, M. Havaei, D. Warder-Farley et al., “Brain tumor seg- mentation with deep neural networks,” in Proceedings MIC- tumor image segmentation benchmark (BRATS),” IEEE CAI-BRATS, Boston, MA, USA, 2014. Transactions on Medical Imaging, vol. 34, no. 10, pp. 1993– 2024, 2015. [17] P. Dvorak and B. Menze, “Structured prediction with convolu- tional neural networks for multimodal brain tumor segmenta- [33] Y. Bengio, “Practical recommendations for gradient-based tion,” in Proceedings MICCAI-BRATS, , pp. 13–24, Munich, training of deep architectures,” in Neural Networks: Tricks of Germany, 2015. the Trade, vol 7700pp. 437–478, Springer, Berlin, Heidelberg. [18] S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumor [34] L. R. Dice, “Measures of the amount of ecologic association segmentation using convolutional neural networks in MRI between species,” Ecology, vol. 26, no. 3, pp. 297–302, 1945. images,” IEEE Transactions on Medical Imaging, vol. 35, [35] K.-L. Tseng, Y.-L. Lin, W. Hsu, and C.-Y. Huang, “Joint no. 5, pp. 1240–1251, 2016. sequence learning and cross-modality convolution for 3D bio- [19] M. Havaei, A. Davy, D. Warde-Farley et al., “Brain tumor medical segmentation,” in 2017 IEEE Conference on Computer segmentation with deep neural networks,” Medical Image Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, Analysis, vol. 35, pp. 18–31, 2017. July 2017. [20] M. Lyksborg, O. Puonti, M. Agn, and R. Larsen, “An ensemble [36] H. Dong, G. Yang, F. Liu, Y. Mo, and Y. Guo, “Automatic of 2D convolutional neural networks for tumor segmentation,” brain tumor detection and segmentation using U-net based in Scandinavian Conference on Image Analysis. SCIA 2015, vol fully convolutional networks,” in Medical Image Understand- 9127, Lecture Notes in Computer Science, Springer, Cham, ing and Analysis. MIUA 2017. Communications in Computer and Information Science, vol 723, M. Valdés Hernández and [21] K. Kamnitsas, C. Ledig, V. F. J. Newcombe et al., “Efficient V. González-Castro, Eds., Springer, Cham, 2017. multi-scale 3D CNN with fully connected CRF for accurate [37] D. Yi, M. Zhou, Z. Chen, and O. Gevaert, “3-D convolutional brain lesion segmentation,” Medical Image Analysis, vol. 36, neural networks for glioblastoma segmentation,” 2016, pp. 61–78, 2017. http://arxiv.org/abs/1611.04534. [22] E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional [38] S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Deep convolu- networks for semantic segmentation,” IEEE Transactions on tional neural networks for the segmentation of gliomas in Pattern Analysis and Machine Intelligence, vol. 39, no. 4, multi-sequence MRI,” in Brainlesion: Glioma, Multiple Sclero- pp. 640–651, 2017. sis, Stroke and Traumatic Brain Injuries. BrainLes, A. Crimi, B. [23] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet clas- Menze, O. Maier, M. Reyes, and H. Handels, Eds., vol. 9556 of sification with deep convolutional neural networks,” in Lecture Notes in Computer Science, Springer, Cham, 2015. Advances in Neural Information Processing Systems, Carson, [39] S. Pereira, A. Oliveira, V. Alves, and C. A. Silva, “On hierarchi- NV, USA, 2012. cal brain tumor segmentation in MRI using fully convolutional [24] K. Simonyan and A. Zisserman, “Very deep convolutional net- neural networks: a preliminary study,” in 2017 IEEE 5th Portu- works for large-scale image recognition,” 2014, http://arxiv guese Meeting on Bioengineering (ENBENG), Coimbra, Portu- .org/abs/1409.1556. gal, February 2017. [25] B. B. Avants, N. Tustison, and G. Song, “Advanced normaliza- [40] R. F. Barajas Jr, J. G. Hodgson, J. S. Chang et al., “Glioblastoma tion tools (ANTS),” Insight Journal, vol. 2, pp. 1–35, 2009. multiforme regional genetic and cellular expression patterns: influence on anatomic and physiologic MR imaging,” Radiol- [26] K. Jarrett, K. Kavukcuoglu, and Y. LeCun, “What is the best ogy, vol. 254, no. 2, pp. 564–576, 2010. multi-stage architecture for object recognition?,” in 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, September-Octeober 2009. [27] A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinear- ities improve neural network acoustic models,” in Proceedings of the 30th International Conference on Ma- chine Learning, Atlanta, GA, USA, 2013. [28] H.-C. Shin, H. R. Roth, M. Gao et al., “Deep convolutional neural networks for computer-aided detection: CNN architec- tures, dataset characteristics and transfer learning,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1285– 1298, 2016. [29] N. Tajbakhsh, J. Y. Shin, S. R. Gurudu et al., “Convolutional neural networks for medical image analysis: full training or fine tuning?,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1299–1312, 2016. [30] X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proceedings of the Thirteenth International Conference on Artificial Intelli- gence and Statistics, Sardinia, Italy, 2010. [31] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

Journal

Journal of Healthcare EngineeringHindawi Publishing Corporation

Published: Mar 19, 2018

There are no references for this article.