Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Multiple sclerosis segmentation method in magnetic resonance imaging using fuzzy connectedness, binarization, mathematical morphology, and 3D reconstruction

Multiple sclerosis segmentation method in magnetic resonance imaging using fuzzy connectedness,... Introduction Magnetic resonance imaging (MRI) is the most used medical modality for diagnosis and monitoring of multiple sclerosis (MS). A segmentation process is an important task to quantify lesion and its progression. However, manual segmen- tation of 3D images is tedious, time-consuming, and often not reproducible. The state of the art presents results with room for improvements. Consequently, a semiautomatic segmentation process is proposed and described in this study. Methods The method consists on a 3D segmentation semiautomatic process for MS lesions in MRI. It initiates by firstly carrying out a preprocessing stage; thus, contrast adjustment is applied to enhance sclerosis regions from other brain information. Secondly, a feature extraction block based on fuzzy connectedness is performed so as to isolate sclerosis lesions from other brain regions. Finally, 3D brain reconstruction is executed along with sclerosis to provide a useful 3D information. Results The robustness of this approach is demonstrated by high correlation between the results and their corresponding gold standard. The results were also obtained by computing parameters of accuracy of image segmentation, as well as overlap Dice. The proposed method reached true positive of 75.61%, false positive of 16.37%, and DICE of 78.23%. Conclusion The high correlation between specialist and proposed approach outcome, a better monitoring of the disease, is provided; the specialist can understand the patient’s symptoms, thereby increasing the patient’s quality of life. . . . . Keywords Magnetic resonance imaging (MRI) Multiple sclerosis (MS) Segmentation Fuzzy connectedness (FC) Reconstruction Gold standard (GS) Introduction causes focal lesions in CNS white matter (WM) (Roy et al. 2018; Tomas-Fernandez and Warfield 2015). Although the Multiple sclerosis (MS) is the major cause of non-traumatic cause of MS is still unknown, several studies suggest it may neurological disability in young adults in Europe and North be caused by combination among genetic, environmental, and America, affecting approximately 2.5 million people world- immunological factors (Tomas-Fernandez and Warfield wide. It is an autoimmune disease of the central nervous sys- 2015). tem (CNS), in which inflammatory demyelination of axons MS can present inflammatory and neurodegenerative com- ponents. The acute demyelination and inflammatory axonal transection may be responsible for the disease symptoms. * André Luiz Costa de Arruda Degeneration appears to be the main reason for the disability a.arruda@unifesp.br and progression in MS. Several putative therapeutic strategies for remyelination and neuroprotection are now transitioning Laboratory of Image and Signal Processing of the Institute of Science from the laboratory to early phase clinical trials (Bhargava and Technology of Federal University of São Paulo, UNIFESP, 330 et al. 2015). Talim St. room 108-Jardim Aeroporto, CEP, São José dos Nowadays, magnetic resonance imaging (MRI) is used in Campos, SP 12231-280, Brazil the diagnosis and monitoring of MS that is because the sensi- Department of Imaging Diagnosis of Federal University of São tivity of structural MRI shows WM lesions in time and space Paulo, UNIFESP, 800 Napoleão de Barros St. Vila Clementino, CEP, without contrast injection (Roy et al. 2018; Valverde et al. São Paulo, SP 04024-002, Brazil 292 Res. Biomed. Eng. (2020) 36:291–301 2017;Jainet al. 2015). These WM lesions are visible on a algorithm in an independent dataset and compared the re- MRI brain scan and appear hyperintense on T2-weighted or sults with data from three experienced raters. In this work, fluid-attenuated inversion recovery (FLAIR) images. they determine the MS lesion using two algorithms that Since MRI was introduced in the early 1980s to diagnose runs under the two statistical parametric mapping (SPM) and assess MS, this exam has become the primary imaging packages: Lesion growth algorithm based (LGA) on modality to monitor its natural evolution (Tomas-Fernandez SPM8, LGA on SPM12, and lesion predicting algorithm and Warfield 2015). Assessment of the disease implication (LPA) based on SPM12. In this evaluation, they found a using MRI for research and clinical trials requires quantifica- strong correlation between manual and automated segmen- tion of the volume on images. It has been shown that volume tation, obtaining DICE 0.6, 0.53, and 0.57 for LGA SPM8, and location of lesions by their segmentations are important LGA SPM12, and LPA SPM12, respectively. In Roura biomarkers of MS and can be used to detect its onset or track et al. (2015), an automated segmentation using T1w and its progression. However, lesions vary in size, shape, intensi- FLAIR images is explored. This approach consists of two ty, and location, which make their automatic and accurate steps: segmentation of the brain tissue according to gray segmentation challenging (Dimitrova 1977). matter, white matter, and cerebrospinal fluid in T1w im- Manual delineations are considered the gold standard (GS), ages, followed by the segmentation of the lesions as out- but segmenting lesions from 3D images is tedious, time-con- liers to the brain tissue in the FLAIR image. In the study of suming, and often not reproducible (Jain et al. 2015; Roy et al. Roura et al. (2015), the quantitative evaluation reached 2018). Besides that, errors can occur due to low lesion contrast DICE of 0.3, 0.33, and 0.43 in three distinct databases. and unclear boundaries caused by changing tissue properties These automatic targeting methods prevent user variability and partial volume effects (Tomas-Fernandez and Warfield and reduce time consumption, but the accuracy of these 2015). Furthermore, there is an inherent reliability challenge methods has not yet achieved their highest possible potential. associated with lesion segmentation. Images produced by any This feature is less pronounced in semiautomatic segmenta- imaging device are inherently fuzzy. This fuzziness comes tions that require a specialist to initialize. In the literature, there from several sources like spatial and temporal resolution lim- is still no viable standard tool for daily clinical practice. itations, blur and/or noise, and background intensity variation. Automatic detection of multiple sclerosis lesions is still a chal- In addition to these factors, different tissues, organs, and ana- lenging problem, living room for, in a primary step, focus on a tomic structures manifest heterogeneity of intensity values of reliable semiautomatic method. object regions in acquired images (Udupa and Saha 2003). In this paper, a semiautomatic segmentation based in This problem is accentuated in MRI, because this type of fuzzy connectedness (FC) was introduced as a robust exam does not have any uniform intensity scale (like comput- method for WM lesion segmentation on 3D FLAIR ed tomography); acquisition of images in different scanners MRI. The method is independent of scanner and acquisi- and with different contrast properties can add complexity to tion protocol and also does not require a huge large train- the segmentation(Royetal. 2018). ing image database of expert lesion segmentations. A semiautomatic or automatic MS segmentation method is Consequently, the objective of this study is to construct required to reduce the time of this task and intra-rater variabil- and validate a proposed segmentation method for a 3D ity uncertainty among segmentation made by different spe- segmentation, using as core operation fuzzy connected- cialists (Egger et al. 2017; Udupa et al. 1997). This will be ness with fixed weights to compute the affinity level be- especially important for large clinical trials, since many im- tween pixels. In addition, the evaluation was performed in ages need to be analyzed and processed. For clinical practice, a set of MRI scans to compute its accuracy; hence, results this segmentation method would allow the measurement of were compared quantitatively and qualitatively with GS lesion volumes, standardizing and quantifying MRI observa- made manually by specialists. The applied parameters of tion (Jain et al. 2015). accuracy were extracted from the proposed approach: a Significant works on MS segmentation have been pub- framework for evaluating image segmentation algorithms lished in recent years (Beaumont et al. 2016;Eggeretal. (Udupa et al. 2006) and overlap Dice (Dice 1945); thus, 2017; Roura et al. 2015). In a study by Beaumont et al. punctual comparison with the results obtained with the (2016) is presented an automatic segmentation from mul- literature was allowed. timodal graph cutting. The results of this segmentation are The project is running under a strong collaboration of com- good, but they vary because they depend on the input pa- plementary work teams from Federal University of São Paulo rameters of the algorithm, which are directly associated (UNIFESP), specifically, between the Medical Imaging with the total load of the lesion. The automation of this Processing Group of Institute of Science and Technology method is complicated because the initialization of the pa- (ICT-UNIFESP) coordinated by Prof. Dr. Matheus Cardoso rameters is very important to achieve satisfactory results. Moraes and the group of DDI-UNIFESP coordinated by Prof. Egger et al. (2017) evaluated the Schmidt et al. (2012) Dr. Nitamar Abdala. Res. Biomed. Eng. (2020) 36:291–301 293 Fig. 1 Overview methodology of the proposed semiautomatic segmentation Material and methods methodology (Udupa and Samarasekera 1996). It is followed by post-processing using binarization, and mathematical mor- In this study, a semiautomatic segmentation of MS areas was phology takes place to enhance the previously extracted infor- constructed and evaluated in a set of 32 MS lesions in different mation. In the final step, the 3D reconstruction of brain volume MRI exams made by scanners Philips Achieva 3TX and with the segmented MS is carried out, hence providing better Siemens Skyra 3TX. Each exam contains about 143 slices in visualization of the brain regions affected by MS. The block DICOM-format, 16-bits resolution, with positive values. The diagram shown in Fig. 1 resumes the segmentation process. exams were provided by the Department of Imaging Diagnosis of the São Paulo School of Medicine (DDI- Stage 01 ➔ preprocessing This stage is divided in two steps: UNIFESP) through XNAT platform: PACS Research brain isolation and contrast adjustment. Medical images are Management and storage of data and clinical images of re- characterized by a composition of small differences in signal search projects in DDI-UNIFESP. The ethics committee with intensities between different types of tissues, noise, manufac- the number 03830718.9.0000.5505 approved the study proto- ture, etc. Hence, differences, ambiguities, and uncertainties col to allow the medical images manipulation. are, by default, introduced during image formation. These The evaluation was performed by computing the mean and imprecisions could make difficult a thorough discrimination standard deviation of true positive (TP), false positive (FP), of the exact location and area of the ROI (Pednekar and false negative (FN) (Udupa et al. 2006), overlap (OR) Kakadiaris 2006). Because of these circumstances, an intensi- (Kupinski and Giger 1998), and overlap Dice (OD) (Dice ty level normalization process takes place by contrast en- 1945) was also calculated to compare the results obtained hancement, hence normalizing and minimizing possible dif- previous work. Experts manually made and revised the gold ference among scanners, providing an intensity levels normal- standards under the collaboration described above. ization concerning sclerosis’ intensities and surrounding. Three main stages are applied to describe the segmentation Since MS is better observed in T2 and FLAIR images as a methodology. The contrast adjustment is performed to intensify region with high-intensity pixels, the contrast of the image is the pixels of the region with MS during preprocessing.The adjusted to enhance the demyelinated regions. And to normal- feature extraction is firstly performed by acquiring information ize the input images, it is needed to preprocess the images with from the region of interest (ROI) in three dimensions by con- filters and contrast adjustments. This makes the method robust and independent of the sensor used. structing a connectivity map using fuzzy connectedness 294 Res. Biomed. Eng. (2020) 36:291–301 with values of the original image I (Fig. 3c). After that, we bo applied a spatial Gaussian filter obtaining I (Fig. 3d), the filt parameters of filtering were kernel of 9 × 9 pixels, and the mean of the local intensities is covered by the kernel and standard deviation (sigma) of 0.8. Next, a flat-field correction (Seibert et al. 1998), resulting in I followed by an edge flat sharpening obtaining I was serially performed to reduce sharp shading distortion (Fig. 3e and f ). Finally, the image histo- gram was adjusted with a narrowing at 0.2 and 0.7 thresholds resulting I (Fig. 3g). The used parameters’ values were found empirical, by first- ly visually evaluating the resultant value that mainly empha- sizes the ROI. The number of trying and spend time during this analytical/empirical process was not measured. Stage 02 ➔ Feature Extraction This stage combines opera- tions to acquire and polish the most of MS information. First, fuzzy connectedness methodology (Udupa and Samarasekera 1996; Udupa et al. 1997) is applied to increase discrimination between MS tissues and the rest of image. Secondly, a polishing and enhancement of the discriminated information are carried out by a binarization and mathematical morphology process. Step 01→ fuzzy connectedness is a semiautomatic segmen- tation method based on region growing. The process relies on a combination of criteria that takes into account homogeneity and intensity features from a selected region (Cardenas et al. 2013). The fuzzy connectedness process starts with a MS ROI selected and a seed defined (initialization). Then, a voxel of a Fig. 2 Preprocessing stage. a Original image, Io. b Brain isolated with ROI must be selected by user, as it is a semiautomatic method. contrast enhancement, Ia Second, the image homogeneity and intensity features are combined among a seed and its neighbors in a parameter Figure 2 displays the input and output of this stage with one called affinity (Udupa and Samarasekera 1996). Thirdly, by slice from an MRI exam utilized in this work. using the Dijkstra graph theory methodology, the connectivity map among the planted seed and each voxel of the image is Step 01 ➔ brain isolation At the beginning of this step, the finally constructed. Please, for more details about fuzzy con- original image I (Fig. 3a ) is the input. First, from I an image o o nectedness algorithm, refer to (Nyúl et al. 2002; Udupa and with contrast elongation I at the lower threshold of 0.15 and ac Samarasekera 1996; Wilcox and Hirshkowitz 2015). greater than 0.2 was created (Chang and Wu 1998). Second, During the initialization process, a pixel of I ROI should an image with an equalized histogram I was generated from he be selected (first seed). With the selected pixel, the mean and I . Consequently, we multiplied these two results, I by I , o ac he standard deviation of the local homogeneity (m and s )and 1 1 and applied Otsu binarization process, obtaining I (Otsu otsu intensity (m and s ) of the objects were calculated (Cardenas 2 2 1979). With I , an opening operation was carried out with otsu et al. 2013;Nyúlet al. 2002; Udupa and Samarasekera 1996; a sphere of radius 6 voxels to isolate the brain, which is the Wilcox and Hirshkowitz 2015). Specifically, for this ap- ROI, from other regions and information included in the im- proach, the ROI was acquired by choosing a slice containing age. The brain I (Fig. 3b) was located as the binarized brain a MS region and clicking in this ROI’s central voxel to acquire region with the largest volume. This step was carried out with the required intensity information in a window of 15 by 15 the purpose of improving the next step of contrast adjustment pixels over the slice’s ROI. Wechose a 15 by15windowsize, of the original image, prioritizing the pixels only pertinent to since it showed to be sufficient to acquire the information and the cerebral volume of the exam. is sufficiently small to not overcome ROI regions. Once this is done, the affinity of the first seed with its six Step 02➔ contrast adjustment Firstly, I and I were mul- o brain neighbors (north, south, east, west, back, and front) was cal- tiplied, isolating only brain information, resulting in the brain culated. The highest affinity values of this first interaction Res. Biomed. Eng. (2020) 36:291–301 295 Fig. 3 Results of each preprocessing calculation. a Original image, I . b Binarized isolated brain, I . c Isolated brain with original image values, I .d o brain bo I with Gaussian filtering, I . e Shade flat field correction, I . f Edge sharpening, I . g Preprocessing final image with histogram adjust, I bo filt flat sharp a were used as reference for a stop condition criterion, proposed where w and w are respectively the weights assigned to 1 2 for this application to avoid computing the complete image homogeneity and intensity with values of 0.3 and 0.7. Connectivity, decreasing computational cost. The established Values were calibrated and chosen by focus on the best seg- value was empirically defined as 70% of the highest affinity. mentation result during a parameter calibration stage. Hence, the region growing is performed until the connectivity Consequently, the connectivity (pixel pertinence level in drops to the mentioned value. the ROI) of the analyzed pixel μ (d) is updated as the mini- While the stopping condition is not reached, a growth loop mum between μ (c,d) and the connectivity of the current seed (GL) process is still running. This process provides the intrin- μ (c)from: sic fuzzy connectedness interactions (Cardenas et al. 2013; μ ðÞ d ¼ minðÞ μðÞ c; d ; μ ðÞ c ð4Þ k α k Nyúl et al. 2002; Udupa and Samarasekera 1996; Wilcox and Hirshkowitz 2015). Hence, first we compute the homoge- with Eq. 4, the affinity value is not able to increase, con- neity μ (c, d) and intensity μ (c, d) between the current seed ψ ϕ tinuing the same value of seed connectivity or decreasing (c) and the analyzed pixel (d), computed through: until reaching the stop condition. As mentioned above, this 2 is carried out for the 6 neighbors of the current seed. If the jj fcðÞ−fdðÞ −m −0:5 1 neighbor was already assigned previously as a seed, we μ ðÞ c; d ¼ e ð1Þ call it an ex-seed; hence, the calculations are not performed for that pixel, since, by being a seed, it was already classi- fied as part of the desired object. The neighbors that were 0:5ðÞ fcðÞþfdðÞ −m analyzed are added as new seeds in a queue of seeds and −0:5 μ ðÞ c; d ¼ e ð2Þ sorted according to connectivity. The seed that was the current one is removed from this queue and placed an in- in which f(c)and f(d) are respectively the values of the image dicator of ex-seed. The dynamic of the process considers at the position of the seed pixel and the neighbor pixel ana- the pixel that has the highest current connectivity as the lyzed. With these two parameters, GL computes the similarity new seed. level μ (c,d) between analyzed pixel and current seed, Upon reaching the stop condition, GL is terminated, and denominated affinity, and determined by: we have the 3D connectivity matrix (M ) with values between μ ðÞ c; d ¼ w  μ ðÞ c; d þ w  μ ðÞ c; d ð3Þ 0and 1 (Fig. 4 Mc) . The stop condition was designed and 1 2 α Ψ Φ calibrate to terminated as soon as the region growing process 296 Res. Biomed. Eng. (2020) 36:291–301 affected by MS, improving medical analysis, and enormously increase the success of clinical decisions. Accordingly, in an accurate and overall view, the specialists will be able to take advantage of special details regarding the MS volume and location, leading to a better understanding of the symptoms that the patient presents or will present. In addition, with the follow-up through examinations, it will be possible to observe the volumetric growth of the disease. In this step, we need the MS extracted O , (Fig. 5) which was described above and the complete brain isolated I brain (Fig. 5)computedin preprocessing stage. With DICOM voxel information, a interpolation process is performed, and the brain volume is rebuilt along with segmented MS, O (Paluszek and Thomas 2017). Fig. 4 Feature extraction stage. a Image after contrast enhancement. leaves the MS area. As mentioned above, it assures the extrac- tion of the lesion and enormously decreases computational cost. Step 02➔ post-processing In this step, M is binarized (O ) c b without threshold, since the stop condition makes M c, contains only MS information all values is transformed to 1 resulting in (Fig. 4 O ). Next, with the binary object, we perform mathematical morphology (Haralick and Sternberg 1987) closing operation in O to fill any small apertures caused by noise resulting in O (Fig. 4 O ). f f Figure 4 demonstrates a one slice process; nevertheless, theprocedureisoccurring in a3Dspace, sincefront and back voxels were considered during connectivity computation. (b) Connectivity matrix resulting from FC. (c) Binarized object. (d) Object after closing operation Stage 03 ➔ 3D reconstruction The brain 3D reconstruction is Fig. 5 Reconstruction stage. a Binarized brain. b Segmented MS. c Reconstructed brain with MS. performed to facilitate visualization of the brain volumes Res. Biomed. Eng. (2020) 36:291–301 297 Results complementary to TP. The high FN value means the voxel that the method did not register as belonging to the lesion. To evaluate the method outcomes, the proposed method was This can occur due to the low sensitivity in the method of applied in 32 MS lesions from FLAIR MRI exams of patients weights applied empirically or also by the value of the stop with MS performed on the Philips Achieva 3TX and Siemens condition to decrease or use the computational value. Skyra 3TX, containing about 143 slices by exam. The exams To compare with other results, we calculated the DICE were provided by DDI-UNIFESP XNAT platform: PACS which was 78.23%. Dice similarity coefficient values range Research Management and storage of data and clinical images from 0 to 1, where 0 corresponds to no overlap between two of research projects. The computational cost was based on a objects and 1 corresponds to perfect overlap. The false- computer with an Intel® Core™ i7-4790 K CPU @ 4.00GHz positive fraction and true-positive fraction were computed processor, 16GB RAM, Windows 10 64-bit and MATLAB for each lesion (Fig. 8) to indicate the percentage of voxels 2016a software from the Image and Signal Processing correctly or incorrectly classified as lesion by the method. Laboratory without any code optimization. The MRI exams The average segmentation time of all lesions was 2.97 s ± have been segmented and compared with GS made by experts 0.331 s, much faster than manual targets that can last for from DDI-UNIFESP. minutes. This manual segmentation was performed by the In Figs. 6 and 7, some results of the proposed method are specialist collaborator of this project. This result can be im- displayed. Figure 6 shows that the corresponding boundaries proved by code optimization and/or by using different pro- of segmented lesions by the proposed method can reach ROI gramming language such as C++ or Python. boundaries. However, some pixels with lower intensity (seen In this graph, we can see that in most of the lesions, the TP with gray color in figure), excluded by the proposed method was above average, and the FP was below average. The le- as part of the lesion, can, in fact, belong to the lesion. In Fig. 7, sions that do not obey this rule can be for reasons such as the high similarity between the images segmented by this possible imprecision of the specialist segmentation moment approach and their GS is noticeable. in regions that are not intense but are homogeneous, and as The statistical evaluation was carried out by applying the our method is mathematical, it is possible to carefully evaluate proposed methodology in the 32 MS lesion volumes. Next, the relevance of the pixel in ROI. And small lesions that have each segmented result was compared with its corresponding a reduced area for initialization of segmentation may not pres- gold standard (GS) made manually by an expert. The numerical ent adequate values of intensity and/or homogeneity, so that assessment of accuracy was obtained by computing corre- the method may not ideally initialize, compromising targets, sponding parameters of accuracy true positive (TP), false pos- neighboring pixels of the first seed. itive (FP), and false negative (FN) (Udupa et al. 2006), as well as overlap (Kupinski and Giger 1998) and overlap Dice (Dice 1945). The parameter of all 32 volumes is shown in Table 1, Discussion and the mean and standard deviation is shown in Table 2. As can be observed in Table 1, the results obtained have The use of MRI to diagnose demyelinating diseases makes it high accuracy with TP around 75% and an FP near 16%. FN is necessary to develop computational methods that assist the specialist (Roy et al. 2018; Storelli et al. 2016). However, despite efforts, lacking accurate methods and results makes the task of MS segmentation challenging. This research was carried out to evaluate the performance of fuzzy connectedness in the segmentation of MS in MRI, seeing that a non-manual segmentation method is required for this task. The developed method is working fast, saving the specialist’s time when performing MS segmentation. The ob- tained results, computed in 3D domain with challenge images, also indicate a relatively high correlation with manual seg- mentation from specialists, making the follow-up of the dis- ease less susceptible to subjective interpretation of the differ- ent specialists. The methodology presented in this work for the segmenta- tion of MS in the brain was divided into three stages. The first, preprocessing, the contrast of the image is adjusted in order to evidence the ROI. The second, feature extraction,uses fuzzy Fig. 6 A ROI of MS lesions (Io) and their corresponding boundaries of segmented lesions by the proposed method (result) connectedness to calculate the connectivity matrix for ROI, 298 Res. Biomed. Eng. (2020) 36:291–301 Fig. 7 Results obtained with the proposed method. In the left side the reconstruction of the brain with the MS lesion (in green the lesion contours of the GS are displayed, the contour of the segmentation with the segmented by specialist and in brown by the proposed method) proposed method is in the middle, and on the right is exposed the 3D binarization, and mathematical morphology. Finally, the brain computational resources are different. Nevertheless, compare with MS is reconstructed in three dimensions to obtain volu- and contrast corresponding outcome are important to help metric visualization of the regions affected by the disease. corroborate efficiency. The work produced by Jain and col- The high TP value and low FP value indicate that this leagues (Jain et al. 2015) compare 3 unsupervised classifica- segmentation method has a smaller variation comparing with tion methods of automatic segmentation based on stochastic the segmentations performed by a specialist or different spe- modeling of voxel intensity distribution Msmetrix, LST, and cialists (Udupa et al. 1997). Moreover, taking into account the Lesion-TOADS with Dice 0.69 ± 0.14, 0.71 ± 0.18, and 0.63 difficulties of the images because they are from different ± 0.17, respectively. Although it is tempting to propose auto- sources, the accuracy and robustness of the method are veri- matic segmentation, these methods of unsupervised classifica- fied, thus offering a new semiautomatic alternative to carry tion do not yet present high accuracy. In comparison, in our out the segmentation of one of most common demyelinating proposed approach, the proposed method reached DICE of diseases. 78.23 ± 8.51 with a semiautomatic segmentation method. Concerning previous work in the literature, a thorough According to the literature in this field, recent techniques pro- comparison of outcomes from different works is beyond the posed for MS lesion segmentation include supervised learning scope of this study, since the datasets, evaluation indexes, and methods such as decision random forests, ensemble methods, Res. Biomed. Eng. (2020) 36:291–301 299 Table 1 Assessment of accuracy Lesion number Lesion volume (mm ) TP (%) FN (%) FP (%) Time Dice (%) of the individual lesions, connected with corresponding 1 1635.95 42.87 57.12 0.5 3.28 59.81 value 2 813.85 78.45 21.54 4.18 2.8 85.92 3 1584.38 58.65 41.34 1.3 2.82 73.34 4 678.72 41.64 58.35 1.51 2.61 58.18 5 399.19 77.26 22.73 14.17 3.24 80.72 6 474.49 89.13 10.86 44.56 3.03 76.28 7 578.67 86.09 13.9 7.48 2.8 88.95 8 621.99 80.43 19.56 12.6 2.52 83.34 9 604.46 56.48 43.51 0.51 3.39 71.96 10 393.00 77.95 22.04 5.24 3.12 85.11 11 771.56 86.09 13.9 6.14 2.68 89.57 12 269.22 72.03 27.96 32.56 3.57 70.42 13 236.21 86.46 13.53 50.21 3.06 73.07 14 750.93 76.78 23.21 7.55 3.14 83.31 15 787.03 77.71 22.28 18.47 3.2 79.23 16 756.09 73.53 26.46 5.04 2.84 82.36 17 794.25 94.93 5.06 27.27 2.96 85.45 18 550.82 57.3 42.69 11.79 2.71 67.78 19 325.95 90.5 9.49 40.5 3.59 78.36 20 241.37 84.61 15.38 24.35 3.25 80.99 21 148.54 78.47 21.52 31.94 2.66 74.59 22 1202.72 73.75 26.24 12.17 2.99 79.34 23 456.95 66.59 33.4 14.89 2.2 73.39 24 298.10 55.7 44.29 5.19 3.52 69.24 25 204.24 87.87 12.12 44.94 2.59 75.49 26 19,302.05 50.32 49.67 7.52 3.34 63.76 27 429.10 83.65 16.34 4.56 3.11 88.89 28 685.94 76.63 23.36 15.11 2.84 79.94 29 615.80 87.81 12.18 27.06 2.67 81.74 30 470.36 97.48 2.51 27.8 3.17 86.55 31 396.09 73.68 26.31 0.00 2.73 84.85 32 962.38 98.95 1.04 16.92 2.76 91.68 non-local means, and k-nearest neighbors (Valverde et al. is very robust and fast. A not so high accuracy when small 2017). This type of classification has good results in the train- sclerosis regions are concerning may be seen as a limitation, ing dataset but contains disadvantages such as the construc- as the method may end up computing regions that do not tion of a considerably large training dataset that encompasses belong to the sclerosis that is because the mean and standard MS lesions of all possible forms, intensities, and heteroge- deviation of the region of interest may not be representative neous textures in WM (Jain et al. 2015). enough for small regions. Another possible reasons are the Because it is semiautomatic, this method requires the spe- empirical values found in the initiation and applied filters. cialist to identify the area with sclerosis and initialize the seg- Thus, inaccurately initializing the segmentation process may mentation. In lesions of medium and large region, the method limit the possible best outcome. Moreover, if a region without Table 2 Mean and standard TP (%) FP (%) FN (%) Overlap (%) DICE (%) deviation of the accuracy assessment of the proposed 75.61 ± 15.02 16.37 ± 14.54 24.37 ± 15.02 64.98 78.23 ± 8.51 approach 300 Res. Biomed. Eng. (2020) 36:291–301 Fig. 8 TP and FP obtained for each slice and its comparison with the average obtained MS is selected, through the affinity among voxels, it will be learning to find the best parameters for weights, filters, and wrongly segmented; hence, the correct choice of specialist is windows size to start the method. important. Acknowledgments Laboratory of Image and Signal Processing of the Institute of Science and Technology (LaPIS- ICT-UNIFESP) and Department of Diagnostic Imaging (DDI-UNIFESP-SP) for the provision of MRI with the XNAT system and Coordination for the Improvement of Conclusion Higher Education Personnel (CAPES) for financial support. . The methodology developed and applied in this study present- Compliance with ethical standards ed high TP and low FP values. The segmentation time, 2.97 s Conflict of interest The authors declare that they do not have conflict of ± 0.331 s, comparing with the manual, is much faster than interest. manual ones, in which can last for minutes; in addition, man- ual segmentation may become a hard and time-consuming Ethical approval University Ethics Committee approved the study (ap- task depending on the dataset size. Consequently, with this proval number 3.243.081) study, it was possible to observe the robustness of fuzzy con- Open Access This article is licensed under a Creative Commons nectedness in the segmentation of multiple sclerosis, using Attribution 4.0 International License, which permits use, sharing, adap- simple weights to calculate the affinity between voxels. The tation, distribution and reproduction in any medium or format, as long as main contributions of this work were (i) a preprocessing stage you give appropriate credit to the original author(s) and the source, pro- to enhance MS areas and volumes; (ii) the specific weight’s vide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included values to improve fuzzy connectedness in MS segmentation in the article's Creative Commons licence, unless indicated otherwise in a and evaluation; (iii) a set of mathematical morphology opera- credit line to the material. If material is not included in the article's tions to reconstruct a binary version of MS volume; and (iv) a Creative Commons licence and your intended use is not permitted by 3D reconstructed brain with MS regions segmented and statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this highlighted, so as to improve clinical analysis allowing the licence, visit http://creativecommons.org/licenses/by/4.0/. specialist to observe encephalon affected by the disease. In order to overcome the limitations mentioned at the end of the Discussion section, as well as increasing accuracy, fu- ture works will focus on increasing dataset to be able to asso- ciate deep and/or machine learning methods for this applica- References tion. Consequently, in future work, we will investigate the potential of Bhattacharyya affinity and dynamic weight func- Beaumont, J. et al. Multiple sclerosis lesion segmentation using an auto- tions for FC allied to CNN to make segmentation more reli- mated multimodal graph cut. HAL, 2016. able, overcoming initialization dependence, making it Bhargava P, Lang A, al-Louzi O, Carass A, Prince J, Calabresi PA, et al. Applying an open-source segmentation algorithm to different oct completely automatic, and improving accuracy (Roy et al. devices in multiple sclerosis patients and healthy controls: 2018;Valverde et al. 2017). We also can create a machine Res. Biomed. Eng. (2020) 36:291–301 301 implications for clinical trials. Mult Scler Int. 2015;2015. ISBN: Seibert JA, Boone JM, Lindfors KK. Flat-field correction technique for 2090-2654, ISSN: 20902662:1–10. https://doi.org/10.1155/2015/ digital detectors, Proc. SPIE 3336, Medical Imaging: Physics of 136295. Medical Imaging, 1998. Cardenas DAC, Moraes MC, Furuie SS. Segmentação do lúmen em Storelli L, Pagani E, Rocca MA, Horsfield MA, Gallo A, Bisecco A, et al. imagens de IOCT usando fuzzy connectedness e reconstrução A semiautomatic method for multiple sclerosis lesion segmentation binária morfológica. Rev Brasil Engenharia Biomed. 2013. ISSN: on Dual-Echo MR imaging: application in a multicenter context. Am 15173151;29(1):32–44. https://doi.org/10.4322/rbeb.2013.004 . J Neuroradiol. 2016. ISBN: 0195-6108, ISSN: 1936959X;37(11): Chang D, Wu W. Image contrast enhancement based on a histogram 2043–9. https://doi.org/10.3174/ajnr.A4874. transformation of local standard deviation. 1998;17(4):518–531. Tomas-Fernandez X, Warfield SK. A model of population and subject Dice LR. Measures of the amount of ecologic association between species (MOPS) intensities with application to multiple sclerosis lesion seg- author. Ecology. 1945;26(3):297–302. https://doi.org/10.2307/ mentation. IEEE Trans Med Imaging. 2015. ISBN: 0278-0062 VO - 34, ISSN: 1558254X;34(6):1349–61. https://doi.org/10.1109/TMI. Dimitrova E. Deep 3D convolutional encoder networks with shortcuts for 2015.2393853. multiscale feature integration applied to multiple sclerosis lesion Udupa JK, Saha PK. Fuzzy connectedness and image segmentation. Proc segmentation. Akusherstvo i Ginekologiya. 1977. ISBN: 1558-254, IEEE. 2003. ISBN: 0018-9219, ISSN: 00189219;91(10):1649–69. ISSN: 1558254X;16(6):452–7. https://doi.org/10.1109/TMI.2016. https://doi.org/10.1109/JPROC.2003.817883. Udupa JK, Samarasekera S. Fuzzy connectedness and object definition: Egger C, et al. NeuroImage: clinical MRI FLAIR lesion segmentation in theory, algorithms, and applications in image segmentation. multiple sclerosis: does automated segmentation hold up with man- Graphical Models Image Process. 1996;58(3):246–61. ual annotation? NeuroImage: Clin. 2017. ISSN: 2213-1582;13:264– Udupa JK, Wei L, Samarasekera S, Miki Y, van Buchem MA, Grossman 70. https://doi.org/10.1016/j.nicl.2016.11.020. RI. Multiple sclerosis lesion quantification using fuzzy- connected- Haralick, R. M.; Sternberg, S R Image analysis morphology 1987;4:532– ness principles. IEEE Trans Med Imaging. 1997. ISBN: 0278-0062, ISSN: 02780062;16(5):598–609. https://doi.org/10.1109/42. Jain S, et al. Automatic segmentation and volumetry of multiple sclerosis brain lesions from MR images. NeuroImage: Clin. 2015. ISBN: Udupa JK, LeBlanc VR, Zhuge Y, Imielinska C, Schmidt H, Currie LM, 1460-9568, ISSN: 22131582;8:367–75. https://doi.org/10.1016/j. et al. A framework for evaluating image segmentation algorithms. nicl.2015.05.003. Comput Med Imaging Graph. 2006. ISBN: 0895-6111, ISSN: Kupinski MA, Giger ML. Automated seeded lesion segmentation on 08956111;30(2):75–87. https://doi.org/10.1016/j.compmedimag. digital mammograms. IEEE Trans Med Imaging. 1998;17(4):510– 2005.12.001. Valverde S, Cabezas M, Roura E, González-Villà S, Pareto D, Vilanova Nyúl LG, Falcão AX, Udupa JK. Fuzzy-connected 3D image segmenta- JC, et al. Improving automated multiple sclerosis lesion segmenta- tion at interactive speeds. Graph Model. 2002. ISSN: tion with a cascaded 3D convolutional neural network approach. 15240703;64(5):259–81. https://doi.org/10.1016/S1077-3169(02) NeuroImage. 2017. ISBN: 1095-9572 (electronic) 1053-8119 00005-9. (linking), ISSN: 10959572;155:159–68. https://doi.org/10.1016/j. Otsu N. A threshold selection method from gray-level histograms. IEEE neuroimage.2017.04.034. Transact Syst Man Cybern. 1979;9(1):62–6. Wilcox T, Hirshkowitz A. Iterative relative fuzzy connectedness for mul- Paluszek, M.; Thomas, S. MATLAB graphics. In: MATLAB Machine tiple objects with multiple seeds. Comput Vis Image Underst. Learning, 2017. ISBN: 9781484222492. 2015;85(01):1–27. https://doi.org/10.1016/j.neuroimage.2013.08. Pednekar AS, Kakadiaris IA. Image segmentation based on fuzzy con- nectedness using dynamic weights. IEEE Trans Image Process. Schmidt et al., 2012. An automated tool for detection of FLAIR- 2006. ISBN: 1057-7149, ISSN: 10577149;15(6):1555–62. https:// hyperintense white-matter lesions in multiple sclerosis. doi.org/10.1109/TIP.2006.871165. NeuroImage. 2012;59:3774–83. Roura E, et al. A toolbox for multiple sclerosis lesion segmentation. Neuroradiology. 2015. ISBN: 0023401515522;57:1031–43. https://doi.org/10.1007/s00234-015-1552-2. Publisher’snote Springer Nature remains neutral with regard to jurisdic- Roy, S. et al. Multiple sclerosis lesion segmentation from brain MRI via tional claims in published maps and institutional affiliations. fully convolutional Neural Networks no 2013, 2018. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Research on Biomedical Engineering Springer Journals

Multiple sclerosis segmentation method in magnetic resonance imaging using fuzzy connectedness, binarization, mathematical morphology, and 3D reconstruction

Download PDF
11 pages

Loading next page...
 
/lp/springer-journals/multiple-sclerosis-segmentation-method-in-magnetic-resonance-imaging-0GqHz7xI3J
Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2020
ISSN
2446-4732
eISSN
2446-4740
DOI
10.1007/s42600-020-00070-y
Publisher site
See Article on Publisher Site

Abstract

Introduction Magnetic resonance imaging (MRI) is the most used medical modality for diagnosis and monitoring of multiple sclerosis (MS). A segmentation process is an important task to quantify lesion and its progression. However, manual segmen- tation of 3D images is tedious, time-consuming, and often not reproducible. The state of the art presents results with room for improvements. Consequently, a semiautomatic segmentation process is proposed and described in this study. Methods The method consists on a 3D segmentation semiautomatic process for MS lesions in MRI. It initiates by firstly carrying out a preprocessing stage; thus, contrast adjustment is applied to enhance sclerosis regions from other brain information. Secondly, a feature extraction block based on fuzzy connectedness is performed so as to isolate sclerosis lesions from other brain regions. Finally, 3D brain reconstruction is executed along with sclerosis to provide a useful 3D information. Results The robustness of this approach is demonstrated by high correlation between the results and their corresponding gold standard. The results were also obtained by computing parameters of accuracy of image segmentation, as well as overlap Dice. The proposed method reached true positive of 75.61%, false positive of 16.37%, and DICE of 78.23%. Conclusion The high correlation between specialist and proposed approach outcome, a better monitoring of the disease, is provided; the specialist can understand the patient’s symptoms, thereby increasing the patient’s quality of life. . . . . Keywords Magnetic resonance imaging (MRI) Multiple sclerosis (MS) Segmentation Fuzzy connectedness (FC) Reconstruction Gold standard (GS) Introduction causes focal lesions in CNS white matter (WM) (Roy et al. 2018; Tomas-Fernandez and Warfield 2015). Although the Multiple sclerosis (MS) is the major cause of non-traumatic cause of MS is still unknown, several studies suggest it may neurological disability in young adults in Europe and North be caused by combination among genetic, environmental, and America, affecting approximately 2.5 million people world- immunological factors (Tomas-Fernandez and Warfield wide. It is an autoimmune disease of the central nervous sys- 2015). tem (CNS), in which inflammatory demyelination of axons MS can present inflammatory and neurodegenerative com- ponents. The acute demyelination and inflammatory axonal transection may be responsible for the disease symptoms. * André Luiz Costa de Arruda Degeneration appears to be the main reason for the disability a.arruda@unifesp.br and progression in MS. Several putative therapeutic strategies for remyelination and neuroprotection are now transitioning Laboratory of Image and Signal Processing of the Institute of Science from the laboratory to early phase clinical trials (Bhargava and Technology of Federal University of São Paulo, UNIFESP, 330 et al. 2015). Talim St. room 108-Jardim Aeroporto, CEP, São José dos Nowadays, magnetic resonance imaging (MRI) is used in Campos, SP 12231-280, Brazil the diagnosis and monitoring of MS that is because the sensi- Department of Imaging Diagnosis of Federal University of São tivity of structural MRI shows WM lesions in time and space Paulo, UNIFESP, 800 Napoleão de Barros St. Vila Clementino, CEP, without contrast injection (Roy et al. 2018; Valverde et al. São Paulo, SP 04024-002, Brazil 292 Res. Biomed. Eng. (2020) 36:291–301 2017;Jainet al. 2015). These WM lesions are visible on a algorithm in an independent dataset and compared the re- MRI brain scan and appear hyperintense on T2-weighted or sults with data from three experienced raters. In this work, fluid-attenuated inversion recovery (FLAIR) images. they determine the MS lesion using two algorithms that Since MRI was introduced in the early 1980s to diagnose runs under the two statistical parametric mapping (SPM) and assess MS, this exam has become the primary imaging packages: Lesion growth algorithm based (LGA) on modality to monitor its natural evolution (Tomas-Fernandez SPM8, LGA on SPM12, and lesion predicting algorithm and Warfield 2015). Assessment of the disease implication (LPA) based on SPM12. In this evaluation, they found a using MRI for research and clinical trials requires quantifica- strong correlation between manual and automated segmen- tion of the volume on images. It has been shown that volume tation, obtaining DICE 0.6, 0.53, and 0.57 for LGA SPM8, and location of lesions by their segmentations are important LGA SPM12, and LPA SPM12, respectively. In Roura biomarkers of MS and can be used to detect its onset or track et al. (2015), an automated segmentation using T1w and its progression. However, lesions vary in size, shape, intensi- FLAIR images is explored. This approach consists of two ty, and location, which make their automatic and accurate steps: segmentation of the brain tissue according to gray segmentation challenging (Dimitrova 1977). matter, white matter, and cerebrospinal fluid in T1w im- Manual delineations are considered the gold standard (GS), ages, followed by the segmentation of the lesions as out- but segmenting lesions from 3D images is tedious, time-con- liers to the brain tissue in the FLAIR image. In the study of suming, and often not reproducible (Jain et al. 2015; Roy et al. Roura et al. (2015), the quantitative evaluation reached 2018). Besides that, errors can occur due to low lesion contrast DICE of 0.3, 0.33, and 0.43 in three distinct databases. and unclear boundaries caused by changing tissue properties These automatic targeting methods prevent user variability and partial volume effects (Tomas-Fernandez and Warfield and reduce time consumption, but the accuracy of these 2015). Furthermore, there is an inherent reliability challenge methods has not yet achieved their highest possible potential. associated with lesion segmentation. Images produced by any This feature is less pronounced in semiautomatic segmenta- imaging device are inherently fuzzy. This fuzziness comes tions that require a specialist to initialize. In the literature, there from several sources like spatial and temporal resolution lim- is still no viable standard tool for daily clinical practice. itations, blur and/or noise, and background intensity variation. Automatic detection of multiple sclerosis lesions is still a chal- In addition to these factors, different tissues, organs, and ana- lenging problem, living room for, in a primary step, focus on a tomic structures manifest heterogeneity of intensity values of reliable semiautomatic method. object regions in acquired images (Udupa and Saha 2003). In this paper, a semiautomatic segmentation based in This problem is accentuated in MRI, because this type of fuzzy connectedness (FC) was introduced as a robust exam does not have any uniform intensity scale (like comput- method for WM lesion segmentation on 3D FLAIR ed tomography); acquisition of images in different scanners MRI. The method is independent of scanner and acquisi- and with different contrast properties can add complexity to tion protocol and also does not require a huge large train- the segmentation(Royetal. 2018). ing image database of expert lesion segmentations. A semiautomatic or automatic MS segmentation method is Consequently, the objective of this study is to construct required to reduce the time of this task and intra-rater variabil- and validate a proposed segmentation method for a 3D ity uncertainty among segmentation made by different spe- segmentation, using as core operation fuzzy connected- cialists (Egger et al. 2017; Udupa et al. 1997). This will be ness with fixed weights to compute the affinity level be- especially important for large clinical trials, since many im- tween pixels. In addition, the evaluation was performed in ages need to be analyzed and processed. For clinical practice, a set of MRI scans to compute its accuracy; hence, results this segmentation method would allow the measurement of were compared quantitatively and qualitatively with GS lesion volumes, standardizing and quantifying MRI observa- made manually by specialists. The applied parameters of tion (Jain et al. 2015). accuracy were extracted from the proposed approach: a Significant works on MS segmentation have been pub- framework for evaluating image segmentation algorithms lished in recent years (Beaumont et al. 2016;Eggeretal. (Udupa et al. 2006) and overlap Dice (Dice 1945); thus, 2017; Roura et al. 2015). In a study by Beaumont et al. punctual comparison with the results obtained with the (2016) is presented an automatic segmentation from mul- literature was allowed. timodal graph cutting. The results of this segmentation are The project is running under a strong collaboration of com- good, but they vary because they depend on the input pa- plementary work teams from Federal University of São Paulo rameters of the algorithm, which are directly associated (UNIFESP), specifically, between the Medical Imaging with the total load of the lesion. The automation of this Processing Group of Institute of Science and Technology method is complicated because the initialization of the pa- (ICT-UNIFESP) coordinated by Prof. Dr. Matheus Cardoso rameters is very important to achieve satisfactory results. Moraes and the group of DDI-UNIFESP coordinated by Prof. Egger et al. (2017) evaluated the Schmidt et al. (2012) Dr. Nitamar Abdala. Res. Biomed. Eng. (2020) 36:291–301 293 Fig. 1 Overview methodology of the proposed semiautomatic segmentation Material and methods methodology (Udupa and Samarasekera 1996). It is followed by post-processing using binarization, and mathematical mor- In this study, a semiautomatic segmentation of MS areas was phology takes place to enhance the previously extracted infor- constructed and evaluated in a set of 32 MS lesions in different mation. In the final step, the 3D reconstruction of brain volume MRI exams made by scanners Philips Achieva 3TX and with the segmented MS is carried out, hence providing better Siemens Skyra 3TX. Each exam contains about 143 slices in visualization of the brain regions affected by MS. The block DICOM-format, 16-bits resolution, with positive values. The diagram shown in Fig. 1 resumes the segmentation process. exams were provided by the Department of Imaging Diagnosis of the São Paulo School of Medicine (DDI- Stage 01 ➔ preprocessing This stage is divided in two steps: UNIFESP) through XNAT platform: PACS Research brain isolation and contrast adjustment. Medical images are Management and storage of data and clinical images of re- characterized by a composition of small differences in signal search projects in DDI-UNIFESP. The ethics committee with intensities between different types of tissues, noise, manufac- the number 03830718.9.0000.5505 approved the study proto- ture, etc. Hence, differences, ambiguities, and uncertainties col to allow the medical images manipulation. are, by default, introduced during image formation. These The evaluation was performed by computing the mean and imprecisions could make difficult a thorough discrimination standard deviation of true positive (TP), false positive (FP), of the exact location and area of the ROI (Pednekar and false negative (FN) (Udupa et al. 2006), overlap (OR) Kakadiaris 2006). Because of these circumstances, an intensi- (Kupinski and Giger 1998), and overlap Dice (OD) (Dice ty level normalization process takes place by contrast en- 1945) was also calculated to compare the results obtained hancement, hence normalizing and minimizing possible dif- previous work. Experts manually made and revised the gold ference among scanners, providing an intensity levels normal- standards under the collaboration described above. ization concerning sclerosis’ intensities and surrounding. Three main stages are applied to describe the segmentation Since MS is better observed in T2 and FLAIR images as a methodology. The contrast adjustment is performed to intensify region with high-intensity pixels, the contrast of the image is the pixels of the region with MS during preprocessing.The adjusted to enhance the demyelinated regions. And to normal- feature extraction is firstly performed by acquiring information ize the input images, it is needed to preprocess the images with from the region of interest (ROI) in three dimensions by con- filters and contrast adjustments. This makes the method robust and independent of the sensor used. structing a connectivity map using fuzzy connectedness 294 Res. Biomed. Eng. (2020) 36:291–301 with values of the original image I (Fig. 3c). After that, we bo applied a spatial Gaussian filter obtaining I (Fig. 3d), the filt parameters of filtering were kernel of 9 × 9 pixels, and the mean of the local intensities is covered by the kernel and standard deviation (sigma) of 0.8. Next, a flat-field correction (Seibert et al. 1998), resulting in I followed by an edge flat sharpening obtaining I was serially performed to reduce sharp shading distortion (Fig. 3e and f ). Finally, the image histo- gram was adjusted with a narrowing at 0.2 and 0.7 thresholds resulting I (Fig. 3g). The used parameters’ values were found empirical, by first- ly visually evaluating the resultant value that mainly empha- sizes the ROI. The number of trying and spend time during this analytical/empirical process was not measured. Stage 02 ➔ Feature Extraction This stage combines opera- tions to acquire and polish the most of MS information. First, fuzzy connectedness methodology (Udupa and Samarasekera 1996; Udupa et al. 1997) is applied to increase discrimination between MS tissues and the rest of image. Secondly, a polishing and enhancement of the discriminated information are carried out by a binarization and mathematical morphology process. Step 01→ fuzzy connectedness is a semiautomatic segmen- tation method based on region growing. The process relies on a combination of criteria that takes into account homogeneity and intensity features from a selected region (Cardenas et al. 2013). The fuzzy connectedness process starts with a MS ROI selected and a seed defined (initialization). Then, a voxel of a Fig. 2 Preprocessing stage. a Original image, Io. b Brain isolated with ROI must be selected by user, as it is a semiautomatic method. contrast enhancement, Ia Second, the image homogeneity and intensity features are combined among a seed and its neighbors in a parameter Figure 2 displays the input and output of this stage with one called affinity (Udupa and Samarasekera 1996). Thirdly, by slice from an MRI exam utilized in this work. using the Dijkstra graph theory methodology, the connectivity map among the planted seed and each voxel of the image is Step 01 ➔ brain isolation At the beginning of this step, the finally constructed. Please, for more details about fuzzy con- original image I (Fig. 3a ) is the input. First, from I an image o o nectedness algorithm, refer to (Nyúl et al. 2002; Udupa and with contrast elongation I at the lower threshold of 0.15 and ac Samarasekera 1996; Wilcox and Hirshkowitz 2015). greater than 0.2 was created (Chang and Wu 1998). Second, During the initialization process, a pixel of I ROI should an image with an equalized histogram I was generated from he be selected (first seed). With the selected pixel, the mean and I . Consequently, we multiplied these two results, I by I , o ac he standard deviation of the local homogeneity (m and s )and 1 1 and applied Otsu binarization process, obtaining I (Otsu otsu intensity (m and s ) of the objects were calculated (Cardenas 2 2 1979). With I , an opening operation was carried out with otsu et al. 2013;Nyúlet al. 2002; Udupa and Samarasekera 1996; a sphere of radius 6 voxels to isolate the brain, which is the Wilcox and Hirshkowitz 2015). Specifically, for this ap- ROI, from other regions and information included in the im- proach, the ROI was acquired by choosing a slice containing age. The brain I (Fig. 3b) was located as the binarized brain a MS region and clicking in this ROI’s central voxel to acquire region with the largest volume. This step was carried out with the required intensity information in a window of 15 by 15 the purpose of improving the next step of contrast adjustment pixels over the slice’s ROI. Wechose a 15 by15windowsize, of the original image, prioritizing the pixels only pertinent to since it showed to be sufficient to acquire the information and the cerebral volume of the exam. is sufficiently small to not overcome ROI regions. Once this is done, the affinity of the first seed with its six Step 02➔ contrast adjustment Firstly, I and I were mul- o brain neighbors (north, south, east, west, back, and front) was cal- tiplied, isolating only brain information, resulting in the brain culated. The highest affinity values of this first interaction Res. Biomed. Eng. (2020) 36:291–301 295 Fig. 3 Results of each preprocessing calculation. a Original image, I . b Binarized isolated brain, I . c Isolated brain with original image values, I .d o brain bo I with Gaussian filtering, I . e Shade flat field correction, I . f Edge sharpening, I . g Preprocessing final image with histogram adjust, I bo filt flat sharp a were used as reference for a stop condition criterion, proposed where w and w are respectively the weights assigned to 1 2 for this application to avoid computing the complete image homogeneity and intensity with values of 0.3 and 0.7. Connectivity, decreasing computational cost. The established Values were calibrated and chosen by focus on the best seg- value was empirically defined as 70% of the highest affinity. mentation result during a parameter calibration stage. Hence, the region growing is performed until the connectivity Consequently, the connectivity (pixel pertinence level in drops to the mentioned value. the ROI) of the analyzed pixel μ (d) is updated as the mini- While the stopping condition is not reached, a growth loop mum between μ (c,d) and the connectivity of the current seed (GL) process is still running. This process provides the intrin- μ (c)from: sic fuzzy connectedness interactions (Cardenas et al. 2013; μ ðÞ d ¼ minðÞ μðÞ c; d ; μ ðÞ c ð4Þ k α k Nyúl et al. 2002; Udupa and Samarasekera 1996; Wilcox and Hirshkowitz 2015). Hence, first we compute the homoge- with Eq. 4, the affinity value is not able to increase, con- neity μ (c, d) and intensity μ (c, d) between the current seed ψ ϕ tinuing the same value of seed connectivity or decreasing (c) and the analyzed pixel (d), computed through: until reaching the stop condition. As mentioned above, this 2 is carried out for the 6 neighbors of the current seed. If the jj fcðÞ−fdðÞ −m −0:5 1 neighbor was already assigned previously as a seed, we μ ðÞ c; d ¼ e ð1Þ call it an ex-seed; hence, the calculations are not performed for that pixel, since, by being a seed, it was already classi- fied as part of the desired object. The neighbors that were 0:5ðÞ fcðÞþfdðÞ −m analyzed are added as new seeds in a queue of seeds and −0:5 μ ðÞ c; d ¼ e ð2Þ sorted according to connectivity. The seed that was the current one is removed from this queue and placed an in- in which f(c)and f(d) are respectively the values of the image dicator of ex-seed. The dynamic of the process considers at the position of the seed pixel and the neighbor pixel ana- the pixel that has the highest current connectivity as the lyzed. With these two parameters, GL computes the similarity new seed. level μ (c,d) between analyzed pixel and current seed, Upon reaching the stop condition, GL is terminated, and denominated affinity, and determined by: we have the 3D connectivity matrix (M ) with values between μ ðÞ c; d ¼ w  μ ðÞ c; d þ w  μ ðÞ c; d ð3Þ 0and 1 (Fig. 4 Mc) . The stop condition was designed and 1 2 α Ψ Φ calibrate to terminated as soon as the region growing process 296 Res. Biomed. Eng. (2020) 36:291–301 affected by MS, improving medical analysis, and enormously increase the success of clinical decisions. Accordingly, in an accurate and overall view, the specialists will be able to take advantage of special details regarding the MS volume and location, leading to a better understanding of the symptoms that the patient presents or will present. In addition, with the follow-up through examinations, it will be possible to observe the volumetric growth of the disease. In this step, we need the MS extracted O , (Fig. 5) which was described above and the complete brain isolated I brain (Fig. 5)computedin preprocessing stage. With DICOM voxel information, a interpolation process is performed, and the brain volume is rebuilt along with segmented MS, O (Paluszek and Thomas 2017). Fig. 4 Feature extraction stage. a Image after contrast enhancement. leaves the MS area. As mentioned above, it assures the extrac- tion of the lesion and enormously decreases computational cost. Step 02➔ post-processing In this step, M is binarized (O ) c b without threshold, since the stop condition makes M c, contains only MS information all values is transformed to 1 resulting in (Fig. 4 O ). Next, with the binary object, we perform mathematical morphology (Haralick and Sternberg 1987) closing operation in O to fill any small apertures caused by noise resulting in O (Fig. 4 O ). f f Figure 4 demonstrates a one slice process; nevertheless, theprocedureisoccurring in a3Dspace, sincefront and back voxels were considered during connectivity computation. (b) Connectivity matrix resulting from FC. (c) Binarized object. (d) Object after closing operation Stage 03 ➔ 3D reconstruction The brain 3D reconstruction is Fig. 5 Reconstruction stage. a Binarized brain. b Segmented MS. c Reconstructed brain with MS. performed to facilitate visualization of the brain volumes Res. Biomed. Eng. (2020) 36:291–301 297 Results complementary to TP. The high FN value means the voxel that the method did not register as belonging to the lesion. To evaluate the method outcomes, the proposed method was This can occur due to the low sensitivity in the method of applied in 32 MS lesions from FLAIR MRI exams of patients weights applied empirically or also by the value of the stop with MS performed on the Philips Achieva 3TX and Siemens condition to decrease or use the computational value. Skyra 3TX, containing about 143 slices by exam. The exams To compare with other results, we calculated the DICE were provided by DDI-UNIFESP XNAT platform: PACS which was 78.23%. Dice similarity coefficient values range Research Management and storage of data and clinical images from 0 to 1, where 0 corresponds to no overlap between two of research projects. The computational cost was based on a objects and 1 corresponds to perfect overlap. The false- computer with an Intel® Core™ i7-4790 K CPU @ 4.00GHz positive fraction and true-positive fraction were computed processor, 16GB RAM, Windows 10 64-bit and MATLAB for each lesion (Fig. 8) to indicate the percentage of voxels 2016a software from the Image and Signal Processing correctly or incorrectly classified as lesion by the method. Laboratory without any code optimization. The MRI exams The average segmentation time of all lesions was 2.97 s ± have been segmented and compared with GS made by experts 0.331 s, much faster than manual targets that can last for from DDI-UNIFESP. minutes. This manual segmentation was performed by the In Figs. 6 and 7, some results of the proposed method are specialist collaborator of this project. This result can be im- displayed. Figure 6 shows that the corresponding boundaries proved by code optimization and/or by using different pro- of segmented lesions by the proposed method can reach ROI gramming language such as C++ or Python. boundaries. However, some pixels with lower intensity (seen In this graph, we can see that in most of the lesions, the TP with gray color in figure), excluded by the proposed method was above average, and the FP was below average. The le- as part of the lesion, can, in fact, belong to the lesion. In Fig. 7, sions that do not obey this rule can be for reasons such as the high similarity between the images segmented by this possible imprecision of the specialist segmentation moment approach and their GS is noticeable. in regions that are not intense but are homogeneous, and as The statistical evaluation was carried out by applying the our method is mathematical, it is possible to carefully evaluate proposed methodology in the 32 MS lesion volumes. Next, the relevance of the pixel in ROI. And small lesions that have each segmented result was compared with its corresponding a reduced area for initialization of segmentation may not pres- gold standard (GS) made manually by an expert. The numerical ent adequate values of intensity and/or homogeneity, so that assessment of accuracy was obtained by computing corre- the method may not ideally initialize, compromising targets, sponding parameters of accuracy true positive (TP), false pos- neighboring pixels of the first seed. itive (FP), and false negative (FN) (Udupa et al. 2006), as well as overlap (Kupinski and Giger 1998) and overlap Dice (Dice 1945). The parameter of all 32 volumes is shown in Table 1, Discussion and the mean and standard deviation is shown in Table 2. As can be observed in Table 1, the results obtained have The use of MRI to diagnose demyelinating diseases makes it high accuracy with TP around 75% and an FP near 16%. FN is necessary to develop computational methods that assist the specialist (Roy et al. 2018; Storelli et al. 2016). However, despite efforts, lacking accurate methods and results makes the task of MS segmentation challenging. This research was carried out to evaluate the performance of fuzzy connectedness in the segmentation of MS in MRI, seeing that a non-manual segmentation method is required for this task. The developed method is working fast, saving the specialist’s time when performing MS segmentation. The ob- tained results, computed in 3D domain with challenge images, also indicate a relatively high correlation with manual seg- mentation from specialists, making the follow-up of the dis- ease less susceptible to subjective interpretation of the differ- ent specialists. The methodology presented in this work for the segmenta- tion of MS in the brain was divided into three stages. The first, preprocessing, the contrast of the image is adjusted in order to evidence the ROI. The second, feature extraction,uses fuzzy Fig. 6 A ROI of MS lesions (Io) and their corresponding boundaries of segmented lesions by the proposed method (result) connectedness to calculate the connectivity matrix for ROI, 298 Res. Biomed. Eng. (2020) 36:291–301 Fig. 7 Results obtained with the proposed method. In the left side the reconstruction of the brain with the MS lesion (in green the lesion contours of the GS are displayed, the contour of the segmentation with the segmented by specialist and in brown by the proposed method) proposed method is in the middle, and on the right is exposed the 3D binarization, and mathematical morphology. Finally, the brain computational resources are different. Nevertheless, compare with MS is reconstructed in three dimensions to obtain volu- and contrast corresponding outcome are important to help metric visualization of the regions affected by the disease. corroborate efficiency. The work produced by Jain and col- The high TP value and low FP value indicate that this leagues (Jain et al. 2015) compare 3 unsupervised classifica- segmentation method has a smaller variation comparing with tion methods of automatic segmentation based on stochastic the segmentations performed by a specialist or different spe- modeling of voxel intensity distribution Msmetrix, LST, and cialists (Udupa et al. 1997). Moreover, taking into account the Lesion-TOADS with Dice 0.69 ± 0.14, 0.71 ± 0.18, and 0.63 difficulties of the images because they are from different ± 0.17, respectively. Although it is tempting to propose auto- sources, the accuracy and robustness of the method are veri- matic segmentation, these methods of unsupervised classifica- fied, thus offering a new semiautomatic alternative to carry tion do not yet present high accuracy. In comparison, in our out the segmentation of one of most common demyelinating proposed approach, the proposed method reached DICE of diseases. 78.23 ± 8.51 with a semiautomatic segmentation method. Concerning previous work in the literature, a thorough According to the literature in this field, recent techniques pro- comparison of outcomes from different works is beyond the posed for MS lesion segmentation include supervised learning scope of this study, since the datasets, evaluation indexes, and methods such as decision random forests, ensemble methods, Res. Biomed. Eng. (2020) 36:291–301 299 Table 1 Assessment of accuracy Lesion number Lesion volume (mm ) TP (%) FN (%) FP (%) Time Dice (%) of the individual lesions, connected with corresponding 1 1635.95 42.87 57.12 0.5 3.28 59.81 value 2 813.85 78.45 21.54 4.18 2.8 85.92 3 1584.38 58.65 41.34 1.3 2.82 73.34 4 678.72 41.64 58.35 1.51 2.61 58.18 5 399.19 77.26 22.73 14.17 3.24 80.72 6 474.49 89.13 10.86 44.56 3.03 76.28 7 578.67 86.09 13.9 7.48 2.8 88.95 8 621.99 80.43 19.56 12.6 2.52 83.34 9 604.46 56.48 43.51 0.51 3.39 71.96 10 393.00 77.95 22.04 5.24 3.12 85.11 11 771.56 86.09 13.9 6.14 2.68 89.57 12 269.22 72.03 27.96 32.56 3.57 70.42 13 236.21 86.46 13.53 50.21 3.06 73.07 14 750.93 76.78 23.21 7.55 3.14 83.31 15 787.03 77.71 22.28 18.47 3.2 79.23 16 756.09 73.53 26.46 5.04 2.84 82.36 17 794.25 94.93 5.06 27.27 2.96 85.45 18 550.82 57.3 42.69 11.79 2.71 67.78 19 325.95 90.5 9.49 40.5 3.59 78.36 20 241.37 84.61 15.38 24.35 3.25 80.99 21 148.54 78.47 21.52 31.94 2.66 74.59 22 1202.72 73.75 26.24 12.17 2.99 79.34 23 456.95 66.59 33.4 14.89 2.2 73.39 24 298.10 55.7 44.29 5.19 3.52 69.24 25 204.24 87.87 12.12 44.94 2.59 75.49 26 19,302.05 50.32 49.67 7.52 3.34 63.76 27 429.10 83.65 16.34 4.56 3.11 88.89 28 685.94 76.63 23.36 15.11 2.84 79.94 29 615.80 87.81 12.18 27.06 2.67 81.74 30 470.36 97.48 2.51 27.8 3.17 86.55 31 396.09 73.68 26.31 0.00 2.73 84.85 32 962.38 98.95 1.04 16.92 2.76 91.68 non-local means, and k-nearest neighbors (Valverde et al. is very robust and fast. A not so high accuracy when small 2017). This type of classification has good results in the train- sclerosis regions are concerning may be seen as a limitation, ing dataset but contains disadvantages such as the construc- as the method may end up computing regions that do not tion of a considerably large training dataset that encompasses belong to the sclerosis that is because the mean and standard MS lesions of all possible forms, intensities, and heteroge- deviation of the region of interest may not be representative neous textures in WM (Jain et al. 2015). enough for small regions. Another possible reasons are the Because it is semiautomatic, this method requires the spe- empirical values found in the initiation and applied filters. cialist to identify the area with sclerosis and initialize the seg- Thus, inaccurately initializing the segmentation process may mentation. In lesions of medium and large region, the method limit the possible best outcome. Moreover, if a region without Table 2 Mean and standard TP (%) FP (%) FN (%) Overlap (%) DICE (%) deviation of the accuracy assessment of the proposed 75.61 ± 15.02 16.37 ± 14.54 24.37 ± 15.02 64.98 78.23 ± 8.51 approach 300 Res. Biomed. Eng. (2020) 36:291–301 Fig. 8 TP and FP obtained for each slice and its comparison with the average obtained MS is selected, through the affinity among voxels, it will be learning to find the best parameters for weights, filters, and wrongly segmented; hence, the correct choice of specialist is windows size to start the method. important. Acknowledgments Laboratory of Image and Signal Processing of the Institute of Science and Technology (LaPIS- ICT-UNIFESP) and Department of Diagnostic Imaging (DDI-UNIFESP-SP) for the provision of MRI with the XNAT system and Coordination for the Improvement of Conclusion Higher Education Personnel (CAPES) for financial support. . The methodology developed and applied in this study present- Compliance with ethical standards ed high TP and low FP values. The segmentation time, 2.97 s Conflict of interest The authors declare that they do not have conflict of ± 0.331 s, comparing with the manual, is much faster than interest. manual ones, in which can last for minutes; in addition, man- ual segmentation may become a hard and time-consuming Ethical approval University Ethics Committee approved the study (ap- task depending on the dataset size. Consequently, with this proval number 3.243.081) study, it was possible to observe the robustness of fuzzy con- Open Access This article is licensed under a Creative Commons nectedness in the segmentation of multiple sclerosis, using Attribution 4.0 International License, which permits use, sharing, adap- simple weights to calculate the affinity between voxels. The tation, distribution and reproduction in any medium or format, as long as main contributions of this work were (i) a preprocessing stage you give appropriate credit to the original author(s) and the source, pro- to enhance MS areas and volumes; (ii) the specific weight’s vide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included values to improve fuzzy connectedness in MS segmentation in the article's Creative Commons licence, unless indicated otherwise in a and evaluation; (iii) a set of mathematical morphology opera- credit line to the material. If material is not included in the article's tions to reconstruct a binary version of MS volume; and (iv) a Creative Commons licence and your intended use is not permitted by 3D reconstructed brain with MS regions segmented and statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this highlighted, so as to improve clinical analysis allowing the licence, visit http://creativecommons.org/licenses/by/4.0/. specialist to observe encephalon affected by the disease. In order to overcome the limitations mentioned at the end of the Discussion section, as well as increasing accuracy, fu- ture works will focus on increasing dataset to be able to asso- ciate deep and/or machine learning methods for this applica- References tion. Consequently, in future work, we will investigate the potential of Bhattacharyya affinity and dynamic weight func- Beaumont, J. et al. Multiple sclerosis lesion segmentation using an auto- tions for FC allied to CNN to make segmentation more reli- mated multimodal graph cut. HAL, 2016. able, overcoming initialization dependence, making it Bhargava P, Lang A, al-Louzi O, Carass A, Prince J, Calabresi PA, et al. Applying an open-source segmentation algorithm to different oct completely automatic, and improving accuracy (Roy et al. devices in multiple sclerosis patients and healthy controls: 2018;Valverde et al. 2017). We also can create a machine Res. Biomed. Eng. (2020) 36:291–301 301 implications for clinical trials. Mult Scler Int. 2015;2015. ISBN: Seibert JA, Boone JM, Lindfors KK. Flat-field correction technique for 2090-2654, ISSN: 20902662:1–10. https://doi.org/10.1155/2015/ digital detectors, Proc. SPIE 3336, Medical Imaging: Physics of 136295. Medical Imaging, 1998. Cardenas DAC, Moraes MC, Furuie SS. Segmentação do lúmen em Storelli L, Pagani E, Rocca MA, Horsfield MA, Gallo A, Bisecco A, et al. imagens de IOCT usando fuzzy connectedness e reconstrução A semiautomatic method for multiple sclerosis lesion segmentation binária morfológica. Rev Brasil Engenharia Biomed. 2013. ISSN: on Dual-Echo MR imaging: application in a multicenter context. Am 15173151;29(1):32–44. https://doi.org/10.4322/rbeb.2013.004 . J Neuroradiol. 2016. ISBN: 0195-6108, ISSN: 1936959X;37(11): Chang D, Wu W. Image contrast enhancement based on a histogram 2043–9. https://doi.org/10.3174/ajnr.A4874. transformation of local standard deviation. 1998;17(4):518–531. Tomas-Fernandez X, Warfield SK. A model of population and subject Dice LR. Measures of the amount of ecologic association between species (MOPS) intensities with application to multiple sclerosis lesion seg- author. Ecology. 1945;26(3):297–302. https://doi.org/10.2307/ mentation. IEEE Trans Med Imaging. 2015. ISBN: 0278-0062 VO - 34, ISSN: 1558254X;34(6):1349–61. https://doi.org/10.1109/TMI. Dimitrova E. Deep 3D convolutional encoder networks with shortcuts for 2015.2393853. multiscale feature integration applied to multiple sclerosis lesion Udupa JK, Saha PK. Fuzzy connectedness and image segmentation. Proc segmentation. Akusherstvo i Ginekologiya. 1977. ISBN: 1558-254, IEEE. 2003. ISBN: 0018-9219, ISSN: 00189219;91(10):1649–69. ISSN: 1558254X;16(6):452–7. https://doi.org/10.1109/TMI.2016. https://doi.org/10.1109/JPROC.2003.817883. Udupa JK, Samarasekera S. Fuzzy connectedness and object definition: Egger C, et al. NeuroImage: clinical MRI FLAIR lesion segmentation in theory, algorithms, and applications in image segmentation. multiple sclerosis: does automated segmentation hold up with man- Graphical Models Image Process. 1996;58(3):246–61. ual annotation? NeuroImage: Clin. 2017. ISSN: 2213-1582;13:264– Udupa JK, Wei L, Samarasekera S, Miki Y, van Buchem MA, Grossman 70. https://doi.org/10.1016/j.nicl.2016.11.020. RI. Multiple sclerosis lesion quantification using fuzzy- connected- Haralick, R. M.; Sternberg, S R Image analysis morphology 1987;4:532– ness principles. IEEE Trans Med Imaging. 1997. ISBN: 0278-0062, ISSN: 02780062;16(5):598–609. https://doi.org/10.1109/42. Jain S, et al. Automatic segmentation and volumetry of multiple sclerosis brain lesions from MR images. NeuroImage: Clin. 2015. ISBN: Udupa JK, LeBlanc VR, Zhuge Y, Imielinska C, Schmidt H, Currie LM, 1460-9568, ISSN: 22131582;8:367–75. https://doi.org/10.1016/j. et al. A framework for evaluating image segmentation algorithms. nicl.2015.05.003. Comput Med Imaging Graph. 2006. ISBN: 0895-6111, ISSN: Kupinski MA, Giger ML. Automated seeded lesion segmentation on 08956111;30(2):75–87. https://doi.org/10.1016/j.compmedimag. digital mammograms. IEEE Trans Med Imaging. 1998;17(4):510– 2005.12.001. Valverde S, Cabezas M, Roura E, González-Villà S, Pareto D, Vilanova Nyúl LG, Falcão AX, Udupa JK. Fuzzy-connected 3D image segmenta- JC, et al. Improving automated multiple sclerosis lesion segmenta- tion at interactive speeds. Graph Model. 2002. ISSN: tion with a cascaded 3D convolutional neural network approach. 15240703;64(5):259–81. https://doi.org/10.1016/S1077-3169(02) NeuroImage. 2017. ISBN: 1095-9572 (electronic) 1053-8119 00005-9. (linking), ISSN: 10959572;155:159–68. https://doi.org/10.1016/j. Otsu N. A threshold selection method from gray-level histograms. IEEE neuroimage.2017.04.034. Transact Syst Man Cybern. 1979;9(1):62–6. Wilcox T, Hirshkowitz A. Iterative relative fuzzy connectedness for mul- Paluszek, M.; Thomas, S. MATLAB graphics. In: MATLAB Machine tiple objects with multiple seeds. Comput Vis Image Underst. Learning, 2017. ISBN: 9781484222492. 2015;85(01):1–27. https://doi.org/10.1016/j.neuroimage.2013.08. Pednekar AS, Kakadiaris IA. Image segmentation based on fuzzy con- nectedness using dynamic weights. IEEE Trans Image Process. Schmidt et al., 2012. An automated tool for detection of FLAIR- 2006. ISBN: 1057-7149, ISSN: 10577149;15(6):1555–62. https:// hyperintense white-matter lesions in multiple sclerosis. doi.org/10.1109/TIP.2006.871165. NeuroImage. 2012;59:3774–83. Roura E, et al. A toolbox for multiple sclerosis lesion segmentation. Neuroradiology. 2015. ISBN: 0023401515522;57:1031–43. https://doi.org/10.1007/s00234-015-1552-2. Publisher’snote Springer Nature remains neutral with regard to jurisdic- Roy, S. et al. Multiple sclerosis lesion segmentation from brain MRI via tional claims in published maps and institutional affiliations. fully convolutional Neural Networks no 2013, 2018.

Journal

Research on Biomedical Engineering – Springer Journals

Published: Sep 1, 2020

There are no references for this article.