Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/springer-journals/quantitative-analysis-of-second-harmonic-generated-images-of-collagen-8d7uEojTfP
- Publisher
- Springer Journals
- Copyright
- Copyright © The Author(s) 2022
- ISSN
- 2446-4732
- eISSN
- 2446-4740
- DOI
- 10.1007/s42600-022-00250-y
- Publisher site
- See Article on Publisher Site
Abstract
Purpose The human body is a complex structure. Its strength is ensured by the collagen protein which exists under the form of fibers. The quantitative analysis of these fibers in biological tissues can be very interesting to establish a relation- ship between the microstructure and their functions. This analysis is usually performed using two-photon microscopy and second harmonic generated (SHG) images. Lately, more and more researchers focused on the use of SHG images since it is a non-invasive technique and allows the capture of collagen fibers only. Many image-processing techniques can be used to extract quantitative information from those images such as fiber orientations, dimensions, and density. Therefore, accurate measure extraction depends mainly on the used image processing methods and, thus, it is necessary to know what process- ing technique to use. Methods The main purpose of this article is to exhibit the most used techniques in collagen fiber quantitative analysis then categorize them according to the information to extract. A comparison of three most used methods in fiber orientation’s estimation is carried out. Result and conclusion Despite the considerable number of papers aiming to quantitatively analyze collagen fibers from SHG images, two main aspects were not deeply covered. First, the use of deep learning algorithms is still limited even for segmentation and denoizing applications. Second, most of the studies processed in this review focused on two-dimensional SHG images and did not take into consideration collagen fibers as a three-dimensional volume. Keywords Information extraction · Image processing · Segmentation · Orientation · Density · Fiber morphology Introduction fibrils at a 100-nm scale. The fundamental structural unit of these fibrils is a triple helix with a length of 300 nm and a diam- Collagen is the most abundant protein in the human body and eter of 1.5 nm (Lodish et al. 2000). For simplicity, the bundles in mammals, in general. This protein is what holds the body are also often called collagen fibers, and their typical dimensions together since it ensures the strength and elasticity of the body’s are in the order of 1 to a few tens of micrometers in diameter and connective tissues. It can be divided into different types. Around several hundreds of micrometers in length. The study of these 80 to 90% of the collagen in the human body are collagen types fibers, which are essential to the proper function of tissues, is I (skin, tendons, bones, ligaments), II (cartilages), and III (scar fundamental in understanding the etiology of pathologies, their tissue, muscles, vessel walls) (Lodish et al. 2000). It is mostly evolution, and in improving their clinical diagnosis and manage- found in the form of fibers, but it is important to recall that the ment. It is a multidisciplinary field involving mechanics, image structures which can be seen in microscopic images depends processing, chemistry, biology, etc. Relevant studies targeting on the imaging scale and what is observed: one can see bundles this aspect require both suitable imaging techniques and reliable composed of collagen fibrils at a 20-µm scale or single collagen image analysis methods. Early studies conducted on this protein led to the characteri- zation at the microscopic scale (around 30 µm) of many tissues * Zeineb Nejim composed essentially of collagen by the mean of histology. nejim.zeineb@gmail.com This technique consists in studying the microscopic structure Mines Saint-Etienne, Univ Lyon, Univ Jean Monnet, of biological tissues and the relations between individual ele- Etablissement Francais du Sang, INSERM, U 1059 ments (Lowe et al. 2015). It involves a chemically destructive Sainbiose, Centre CIS, F – 42023 Saint-Etienne, France Vol.:(0123456789) 1 3 Research on Biomedical Engineering process, which can only be performed on ex vivo samples, and to multi-photon laser by generating second harmonics across eventually a slicing step, which prevents from 3D observations. the spectral region between 400 and 500 nm (Theodossiou et al. The process can also have an impact on the microstructure of 2006). This property, called second harmonic generation (SHG), the considered specimen. For example, freezing of the tissue is is an asset to capture images of collagen fiber only, as this signal used for visualization purposes, which may destroy some of its is specific and can be separated from other signals. components. Its use remains, however, a standard for pathology The cited imaging modalities introduced some improve- diagnoses in clinics. Hence, the study of the microstructure evo- ment on how to capture sufficiently good images to extract lution of a biological system ex vivo under a mechanical load is information related to the structure and the function of col- impossible with histology. In order to deal with this issue, new lagen fibers. Studying the organization of the collagen fibers imaging techniques have been tested and proved their efficiency. is of interest in biomedical research since it allows diag- Among those imaging modalities, one can cite first scanning nosing fibrosis (Campagnola 2011; Strupler et al. 2007) or electron microscopy (SEM) (Prado et al. 2003; Orberg et al. analyzing their interaction with cancerous cells (Bredfeldt 1982). It allows obtaining images with a resolution of 1–20 nm, et al. 2014). Moreover, in Brown et al. (2003), the authors but it lacks in-depth signal: only the peripheral surface can be tried to quantify the dynamics of collagen modification in imaged accurately. As a remedy, other techniques would ide- tumors in vivo after pharmacologic intervention. Besides, ally ensure a precise quantification of the collagen fibers within other researchers focused on the quantification of fiber ori- the volume of the material as oriented structures in space in an entation in order to study structure-to-function relationships adapted scale (1–100 µm). For instance, X-ray computed tomog- such as in pressurized vessels (Ayyalasomayajula et al. 2019; raphy (XRCT) and X-ray micro-tomography are well suited for Cavinato et al. 2017; Schriefl et al. 2013) or waviness and quantifying collagen fiber architecture because they allow cap- density in order to identify the impact of sample aging (Sug- turing their microstructure through larger fields of view (up to ita and Matsumoto 2017; Wu et al. 2016). 1.7 mm × 1.7 mm) as compared to other microscopy techniques However, this type of quantification is complicated and (Bailly et al. 2018; Disney et al. 2017; Walton et al. 2015). In requires dedicated methods. To date, the conducted research in addition, it offers a resolution of 20–100 µm, though a com- this e fi ld succeeded in characterizing collagen structures only promise between resolution and field of view must be made. in 2D planes although those tissues are three dimensional. However, the addition of X-ray contrast agents may change the In the present review, we focus on the analysis of those behavior of the specimen components, restraining their use. SHG images of collagen fibers and the different techniques Optical coherence tomography (OCT) (Fujimoto et al. 2000) developed to extract information aimed at characterizing those has been used as an alternative (Babalola et al. 2014; Ugryu- fiber networks. We emphasize on the accurate quantification mova et al. 2009). Just like XRCT, OCT provides a resolution of several quantities such as the fiber orientation, waviness, of 1–15 µm, but it does not allow capturing individual compo- and dimensions in addition to the collagen density. To this nents of a specimen. This makes quantitative analysis hard to aim, the paper is organized as follow: the first section will achieve such as for aortic ostial lesions where it is not possible explain the physics behind SHG imaging. The second section to clear the blood at the entrance to neighboring arteries. It is will focus on the different image processing techniques used dependent on the considered biological tissue scattering and to quantify collagen fibers in SHG image. The third section absorption. Yet, it is possible to use optical clearing agents to will exhibit what has been done in the literature to extract reduce light scattering but it can have an impact on the tissue the collagen fiber geometric, composition, and morphological structure. Recently, a strong interest was shown toward fluo- information from SHG images. This review will end with a rescence microscopy, which requires the use of stains, but has comparison between of three common methods used to extract limited physico-chemical modification of biological tissues. For fiber orientation from SHG images of collagen networks. instance, confocal microscopy was often used (Wu et al. 2003; Stein et al. 2008) because of its resolution of around 160 nm and its capability to capture images through the specimen depth. Study design Later, the emergence of powerful lasers enabled multi-photon microscopy. This imaging technique, with (Chen et al. 2012; Articles that could answer questions relating to the quanti- Polzer et al. 2013; Yeh et al. 2002) or without polarizer (Ayy- tative analysis collagen fibers were carefully selected. Only alasomayajula et al. 2019; Cavinato et al. 2017) does not harm papers dealing with biological tissues and second harmonic the sample because it is less exposed to the laser. It offers a scale generation acquisition technique were taken into considera- for representation of the order of a micrometer and a resolu- tion. Papers dealing with polarized SHG and papers with tion up to 150–200 nm. Besides, it allows imaging deeper into only abstracts were excluded. The search was performed the sample and thus collecting more images in the depth (up to using some of the most popular digital repositories (Google 500 µm (Yamada et al. 2014)). Additionally, collagen fibers react Scholar, PubMed, Springer, Science Direct and Optica 1 3 Research on Biomedical Engineering publishing group) by using keywords “SHG,” “collagen,” the ground state and the excited state is smaller than the sum and “quantitative analysis.” of the energy of the two photons, the nonlinear process can The collected articles were divided into three categories with occur. In this case, the probability that a fluorescent molecule respect to the quantitative information extracted (the geometry absorbs simultaneously two infrared photons is a quadratic and morphology of collagen b fi ers and the composition of the function of the excitation radiance (So 2002). considered specimen in collagen). Then, these articles were cat- The possibility to take microscopic images in three egorized following the techniques used to improve SHG images dimensions (i.e., depth discrimination) is considered one of or to extract relevant information. The most frequent techniques the most interesting properties of two-photon microscopes. were selected to be cited and explained in this review. It originates from the almost absence of out-of-focus light resulted by the reflection. Eighty percent or more of the total florescence signal may be cramped in a region of 1 µm thick - SHG acquisition technique ness around the focal plane of a two-photon excitation (So 2002). Notable two-photon excitation occurs where the pho- Optical harmonics were first discovered in the 1960s when ton density is high. It corresponds to the focal volume of the the high-intensity pulsed lasers have been invented. Franken microscope, which can be as small as ~ 0.1 µm (Zipfel et al. et al. (1961) observed second harmonic generation (SHG) in 2003). The laser needs to scan the entire specimen in the crystalline quartz by using a Q-switched ruby laser. It became three dimensions to generate a 3D image, which may involve a very used method to characterize the second-order non- relatively long acquisition times according to the volume linear optical (NLO) response of emerging materials, espe- size and acquisition parameters. Finally, the use of two low- cially organic NLO materials. It led to an increase of its use energy photons limits the risk of photo-damage of the sam- in different fields such as biomedical research (Singer and Wu ple (Svoboda and Yasuda 2006). In addition, it maximizes 2013). SHG imaging in biology was reported by Freund et al. the probability of detecting photons per excitation event in (1986) when he tried to characterize the polarity of collagen the right spot and, thus, minimizes photo-bleaching (when fibers in a rat-tail tendon. Campagnola et al. (2002) reported the molecule loses its ability to fluoresce) and photo-toxicity. a more recent practical implementation in where the authors succeeded in imaging biological tissue at high resolution and fast acquisition rate. In order to collect SHG images of col- Method description lagen in biological tissues, a two-photon light microscopy has been developed. This imaging technique is based on the exci- We here provide a succinct description of the methods tation of a molecule to a virtual state by two photons which allowing extracting quantitative information from SHG are then converted into a single photon of same total energy images of collagen fibers. We first introduce some common at double frequency, without absorption or re-emission of pre-processing techniques. Then, we focus on the different photons. For two-photon excitation, photons in the infrared image transformations that may be used to analyze the SHG spectral range are used under highly intense laser illumination images. Finally, we highlight the methods useful to extract (for example, Ti:sapphire lasers). Infrared photons are chosen and select information from SHG images. Fig. 1 displays the because of their low energy. When the energy gap between methods that will be covered in this section. Fig. 1 Overview of SHG images manipulation techniques 1 3 Research on Biomedical Engineering CLAHE is a good technique to improve local contrast Pre‑processing and to enhance edges. Compared to other adaptive histo- gram equalization techniques, it limits the noise amplifica- In order to process an SHG image or stack and to extract as much accurate information as possible, it is important to remove tion. However, if the image is too noisy, a phenomenon of noise amplification may occur. The combination of CLAHE, the noise. SHG images of tissues present usually Poisson noise because of their poor signal-to-noise ratio (Bredfeldt et al. 2014). median filter, and edge sharpener (such as high-pass filters) can be successful to maintain the image high spatial fre- Common methods in pre-processing SHG images are introduced in the following and are summarized up in Table 1. quency content. Median filter Directional filters It is simple and widely used filter to reduce noise in images When there is a need to study oriented features in an image, directional filters can be used (Bamberger and Smith 1992). and to smooth them. Median filter (Huang et al. 1979) is a nonlinear smoothing filter. The value of each pixel in the They consist in a filter bank containing lines in different directions. They can be used to detect edges or to identify image is replaced by the median value of pixel’s intensity in a previously defined neighborhood of size m * m. objects orientations. Those filters have wedge-shaped pass- band spectral regions, and are therefore usually referred to Median filters work well for removing random salt and pep- per noises (Gonzalez and Woods 2018). However, this kind of as wedge or fan filters (Simoncelli and Farid 1996 ). When the orientation of the wedge is known, the determination of filters does not allow the suppression of Gaussian noise (Ohki et al. 1995) which can be dealt with through deconvolution. objects direction is straightforward. Wedge filters are an easy to implement and efficient tool They do not reduce the difference in brightness of images and, hence, preserve edges. However, when the signal-to-noise ratio to study oriented objects in images. However, in order to have a fine description of the image features, it is necessary of the image is small, or the neighborhood is too large, median filters tend to delete useful information and produce false noise to have thin wedges in as many directions as possible which will add more computational costs. edges. Gradient Contrast enhancement The gradient vector is a fundamental approach for find- The recognition of image features depends on the image contrast. However, the contrast can be distorted by the ing extrema of a continuous and smooth function in space (Hyvärinen et al. 2009). The gradient is defined as the partial imaging system because of poor illumination conditions. For this purpose, histogram equalization is widely used. A derivatives of a function with respect to all its components. For 2D images, the gradient is usually achieved by the con- well-acquired gray-scale image should cover black and white pixels. It is also better that the image’s shades are evenly volution of the image by a couple of filters based on the Sobel filter (Sobel 1968) or the Prewitt operator (Prewitt distributed (i.e., the image histogram is uniform). Many contrast transforms can be used for this purpose such as his- 1970). The fact that small displacements are taken into consid- togram equalization, adaptive histogram equalization, and contrast limited adaptive histogram equalization (CLAHE) eration to compute the gradient allows capturing as much details as possible in any direction. The gradient works fine (Mustafa and Abdul Kader 2018). Here, we will focus on the CLAHE algorithm, which is very used on SHG images with clear images without much noise. However, if the con- sidered image is noisy, the gradient will not bring any useful (Koch et al. 2014; Hu et al. 2012). CLAHE is a variant of adaptive histogram equalization information. (Pizer et al. 1987). It consists in computing histograms of distinct regions and using them to redistribute the pixel Frangi filter intensity values of the image. The difference between CLAHE and other adaptive histogram equalization algo- The Frangi filter (Frangi et al. 1998) was first developed to be a vessel enhancement filter. However, it was used to rithms is that it clips the histogram at a pre-defined value (i.e., if a histogram bin is higher than the clip limit, those detect both vessel-like and tube-like structures in images. Because of the collagen fiber morphology, which can be pixels are clipped and uniformly shared with other bins before proceeding to the histogram equalization). It oper- assimilated to tubes, the Frangi filter was used to extract the fibers from SHG images. ates on small regions of the image called tiles. To remove the artificial boundaries between the different tiles, bi-linear The Frangi filter is based on the computation of the image’s Hessian matrix. In the proposed framework, the interpolation is used. 1 3 Research on Biomedical Engineering derivative of an image corresponds to its convolution with Fast Fourier transform (FFT) derivatives of Gaussians. The second derivative of a Gauss- ian kernel shape allows to measure the contrast between The FFT is an efficient method to study the spatial frequency the region in and out of a range (− s, s), s being the stand- distribution of the pixels in an image. The Fourier transform ard deviation of the Gaussian. Through the analysis of the (FT) was initially used to characterize linear systems and to eigenvalues of the image’s Hessian matrix, it is possible identify their frequency components that make a continuous to extract the direction of the smallest vessel’s curvature waveform (Bergland 1969). Images are processed using the which corresponds to the main directions in which the local discrete Fourier transform (DFT). The DFT coefficients can be second-order structure of the image can be decomposed computed by the FFT. This transform is a computationally cheap (Frangi et al. 1998). The eigenvalue decomposition gives and fast algorithm originally introduced by Cooley et al. (1969). three orthonormal directions, which allow describing ves- Different approaches can be chosen to compute the FFT (Rader sels in images. 1968; Bluestein 1970; Bruun 1978; Rivard 1977). The 2D DFT The use of multiscale Frangi filter, through the analysis is a two-time process (Rivard 1977). It consists in combining of the eigenvalues of the Hessian matrix, makes it possible vertical and horizontal 1D DFT of an array into one 2D trans- to capture the smallest details of an image and, thus, avoid form that makes sense. First, a 1D DFT over horizontal lines of the application of different filters of different sizes. However, an image is performed. Then, a 1D DFT over vertical lines is the Frangi filter may not take into consideration any object applied on the result of the previous operation. in the image, which does not have a circular cross-section. The 2D FFT is an efficient operator to characterize an image and to capture the variation of its texture. However, Image transformations the space notion is lost when the transition from space to frequency is done. In fact, the 2D Fourier transform gives A strong interest was shown to signal decomposition because information of the global contents and changes in frequency of the uneven distribution of signal energy in the frequency without knowledge on the section of the image that corre- domain. It consists in dividing the signal spectrum into its sponds to it. Besides, Fourier transform may not work accu- sub-spectra, which are then treated individually (Akansu rately to reconstruct an image, which is highly non-smooth 2001). Signal decomposition was used for many applica- (Jaffar Iqbal Barbhuiya and Hemachandran 2013). tions such as compression and feature extraction. For image analysis, and particularly for studying the collagen fibers in Wavelet transform (WT) SHG images, several image decomposition methods have been used: the fast Fourier transform (FFT), the wavelet Wavelet methods have become a widely used tool in image transform (WT), the radon transform (RT), and the Hough processing during the last 20 years. This is due to their ability transform (HT). Table 2 sums up these methods. to analyze non-stationary structures and characterize local properties. An image is mapped to a phase space, which is Table 1 Pre-processing methods Methods Output Advantages Drawbacks Median filter Low-passed image -It preserves edges -It can delete useful information when the -It works well with random and salt and SNR is low pepper noise (Gonzalez and Woods -It does not work with Gaussian noise (Ohki 2018) et al. 1995) CLAHE Image with equalized histogram -It improves local contrast and enhance -It does not work properly on very fine edges details -It limits noise amplification compared to -It is complex and computationally expen- other histogram equalization techniques sive Directional filters Filtered image with respect to the -It is a good tool to study oriented objects -It needs to be fine tuned to capture all direction of the highest intensity -It is easy to implement details -It may be computationally expensive Gradient Double images representing the -It allows to capture small details -It is limited with noisy images gradient values at each pixel Frangi filter Images with only tube-like objects -It captures the smallest details (Frangi -It only takes into consideration objects with et al. 1998) circular cross-section (Frangi et al. 1998) 1 3 Research on Biomedical Engineering parameterized by a scale/size/resolution and a time/space Radon transform (RT) parameters. Wavelet transform is an alternative to the Fourier transform which characterizes the image in a time/space The radon transform is a mathematical transformation based on frequency space (Dahlke et al. 2008). It was first introduced by projections, which is the basis of computed tomography (CT). Grossmann and Morlet (1984) as an elegant multi-resolution We can also use it to detect edges. The radon transform con- signal processing tool thanks to its ability to naturally vary the sists in performing different projections of an image according time–frequency resolution (Akansu 2001). It is a mathematical to different angles. The resulting projection corresponds to the function of zero average used to divide a function into integral of the line integral (i.e., the sum of the pixel intensi- components at different scales. Each scale is computed using ties in every direction) (Deans 2007). In other terms, the RT a specific wavelet generated from an initial function named maps an image from Cartesian coordinates to polar ones. This mother wavelet by dilation and translation. The dilation allows transform can also be applied to 3D images. In this case, the carrying out a multi-scale analysis and enables to capture small integral is taken over planes. The RT data is usually referred to details. as sinograms. It is possible to perform a wavelet decomposition of an image The use of a FFT gives qualitative information about fiber (in 2D or even for higher dimensions) in order to compress the orientations. To deal with this issue, it is possible to apply an data or to obtain a vector of features that characterizes the data in RT on the result of the FFT. Since it is based on projections, it a basis of wavelet. It is helpful to capture the orientation changes gives quantitative information for each considered angle. It is in an image. For this matter, we need to perform a 2D discrete important to have a sufficient number of angles to get accurate wavelet transform (DWT). As for the 2D FFT, it can be gener- results in detecting and extracting the fiber orientations. ated using the horizontal and vertical 1D DWT. The main advantage of the wavelet transform is that it Hough transform (HT) provides a localization in both space and frequency domains. The wavelet transform allows capturing small and coarse The Hough transform (Hough 1962) was first introduced to detect details. Indeed, wavelet transforms over-perform traditional lines in images. This algorithm was then simplified by Duda and Fourier transforms in representing functions with sharp Hart (1972) and generalized to detect circles and curves. The peak discontinuities and in correctly decomposing and original HT algorithm assumes that every line in an image can reconstructing non-stationary, non-periodic, and finite be represented by a unique couple (slope, intercept). Duda and signals (Jaffar Iqbal Barbhuiya and Hemachandran 2013). Hart (1972) changed this representation by the couple (angle, It can also be used to detect discontinuities and irregularities distance), where the angle and the distance correspond to the in signals. However, this technique is computationally polar coordinates of a considered line in the image (the distance expensive for fine decomposition. The choice of the mother being the distance between the image origin and its projection on wavelet and the number of decompositions can highly the line). A matrix called accumulator is created where its axes influence the result. correspond to the parameters characterizing the line. Thus, for each pixel of the image, the accumulator is incremented for all possible lines passing through that pixel. The presence of an edge Table 2 Image transformation methods Methods Output Advantages Drawbacks FFT Complex representation of -It captures the variation of the image texture -It loses spatial information the image in the frequency -It does not work properly with highly non-smooth domain images (Jaffar Iqbal Barbhuiya and Hemachandran 2013) WT Decomposed image -It provides a localization in both space and -The result highly depends on the choice of the frequency domains mother wavelet -It detects discontinuities and irregularities -It is computationally expensive for fine decomposi- (Jaffar Iqbal Barbhuiya and Hemachandran tion 2013) -It captures small and coarse details (Jaffar Iqbal Barbhuiya and Hemachandran 2013) RT Projection data -It gives information with respect to the angle -It depends on the chosen number of angles (Deans of projection (Deans 2007) 2007) -It is computationally expensive for fine analysis HT Polar map of the image -It corrects properly the detected edges (Leav- -It works better on detected edges ers 1992) -It does not distinguish between objects if they are -It can be used to estimate object orientations aligned 1 3 Research on Biomedical Engineering corresponds to a high value position in the accumulator (Leavers regardless of the positions of objects. On the contrary, local 1992). A reconstruction of the initial image is possible by retriev- thresholding looks for a threshold in a neighborhood around ing the parameters corresponding to the peaks in the accumulator. any pixel of the image. The HT gives good results when applied on an image where The main advantages of thresholding techniques are their the edges were already detected. It works fine with noisy data. simplicity and their fast computation. This type of segmentation This method allows reconstructing an edge if it is discontinuous works well when the image’s histogram presents two or more when performing the edge detection algorithm. Therefore, the peaks. However, it is highly sensitive to the tackled problem and application of the HT needs a prior step to detect the edges. For is specic fi to the considered image. In addition, it takes only into linear objects, the HT is a good method to detect edge orienta- consideration the intensity of the pixel/voxel and not its spatial tion directly from the accumulator matrix. However, its effec- information, which makes this method highly sensitive to noise tiveness depends on the considered image: if two objects are (Yuheng and Hao 2017). In fact, small areas or isolated pixels aligned in an image, the HT will exhibit them as one. can be classified as independent regions even though they rep- resent noise or belong to another region. Besides, to segment Information selection and extraction SHG stacks where the pixels intensity decreases with depth, it is complicated to find a threshold that takes into consideration After the pre-processing of an image, the SHG image analysis that phenomenon. needs to extract as much valuable information as possible. For this matter, it is possible to extract this information through a Region‑based segmentation Unlike pixel-based segmenta- spatial characterization or a statistical one. For both types of tion which classifies a pixel-based on its intensity value with- characterizations, many methods can be used. Some of them are out taking into consideration the spatial context, region-based detailed hereafter and are summed up in Table 3. segmentation looks for pixels having similar features. Several techniques belong to this category such as region growing algo- Spatial information selection rithm (Adams and Bischof 1994; Mancas et al. 2006), split and merge algorithm (Damiand and Resch 2003; Chaudhuri and To analyze an image, it is important to consider the spatial dis- Agrawal 2010), and clustering (Thilagamani and Shanthi 2011). tribution of the pixel intensity. This is possible through several Our interest is paid to the region-growing algorithm since it has techniques: (i) segmentation, which transforms an SHG image been used in quantifying SHG images of collagen. First, the user into a binary image where only the collagen fibers are repre- selects initial seed points to be in a region. Then the algorithm sented; (ii) skeletonization, which determines the center line of checks iteratively if the adjacent pixels should be added to the the collagen fibers in the SHG images and thus, allows extract- region according to one or several of available criteria (gray ing geometrical information about the fibers. scale texture, intensity, color, etc.) (Yuheng and Hao 2017). Region-based segmentation allows partitioning the image Pixel‑based segmentation This type of segmentation aims into sub-regions. However, those methods depend on the to gather pixels corresponding to an object and mark them. choice of seed points and do not work properly on non- It is based on their intensity similarity and spatial prox- smoothly varying regions. Besides, a threshold is needed imity. The (automatic) thresholding segmentation is the as a criterion to construct the regions; thus, its choice is easiest method for image segmentation. Otsu thresholding important. Finally, it is a local technique with no global view algorithm (1979) is the most used one, especially on SHG on the image and it is sensitive to noise, which may lead to images, because of its simplicity in addition to the fact that it an over-segmentation. works particularly well when the considered image contains two classes (an object and the background). Its principle is Edge‑based segmentation An important feature carrying infor- to find the threshold that maximizes the interclass variance mation about objects is their borders, i.e., the discontinuities of a two-classes histogram. In addition to this method, sev- in the pixels’ intensity. To detect the gray level discontinui- eral other approaches exist to compute the threshold such ties, the most common approach is based on detecting edges, as entropy-based thresholding (Khattak et al. 2015; Luthon which represents a set of connected pixels forming a boundary et al. 2004), minimum error thresholding (Kittler and Illing- between two regions (Gonzalez and Woods 2018). There is a worth 1986), moment-preserving thresholding (Tsai 1985), gap between the pixel values of two adjacent regions. Those dis- fuzzy set thresholding (Tizhoosh 2005), etc. continuities can be either step edges or line edges. Step edges are Thresholding decomposes the image gray scale informa- characterized by the sudden change in the pixel intensity from tion with respect to gray level of targeted objects. There a region to another. Line edges correspond to a sudden change are two types of thresholding segmentation: global and of the pixel values followed by another sudden change to return local. The global threshold looks at the global picture: it to the initial value within a short distance (Senthilkumaran and divides the image into two regions (background and target) Rajesh 2009). However, in real images, it is impossible to find 1 3 Research on Biomedical Engineering those types of edges because of the smoothing introduced by the through their skeletons. It was developed to extract collagen optical systems or by the low-frequency components of images. b fi ers from SHG images in order to estimate their orientation One can find ramp edges instead of step edges and roof edges and geometric information. instead of line edges, where the pixel intensity change occurs This algorithm is based on two steps. The first one is a over a finite distance. Such gaps can be detected with the help l fi tering using curvelet transform (CT). Curvelet l fi ters were of die ff rential operators such as the Sobel operator (Sobel 1968), introduced by Starck et al. (2002) in order to overcome the the Laplacian and the Laplacian of Gaussians (also called Marr- limitation of highlighting lines and edges. The curvelet Hildreth operator) (Acharya and Ray 2005), the Prewitt opera- transform is a wavelet transform except that instead of the tor (Prewitt 1970), or the Kirsch operator (Kirsch 1971). More wavelets we use curved functions called curvelets. The sec- sophisticated techniques such as the Hough transform were also ond step consists in applying the fiber extraction algorithm used to determine image edges (Hough 1962). Once the edges FIRE developed by Stein et al. (2008) It describes the fibers are detected, mathematical morphology operators (erosion, dila- as a set of n vertices and p paths. Every path corresponds to i i i tion, opening, closing, etc.) are used to fill the targeted regions a fiber characterized by k vertex identifiers p = (n , n 1 2, …, and, thus, segment the image. n ). The image is smoothed using a Gaussian filter before Edge-based segmentation is a high-level segmentation segmenting it through thresholding. Then, for each pixel of approach similar to the way humans perceive an image. It works the segmented image, the Euclidean distance map is com- well on images with high contrast. However, it is highly sensitive puted. This map is used to identify the centerlines of the to noise. It is centered on local information and does not take fibers. Once the centerlines identified, short non-relevant into consideration the global view. In addition, it does not work fibers are deleted and close fibers are connected. well to detect corners and when the contrast is low. The CT step introduced in the CT-FIRE algorithm improved the result of the fiber extraction compared to the classic FIRE Fast marching method (FMM) The fast-marching algorithm algorithm. It provides better results when the collagen fibers (Malladi and Sethian 1996) allows to track object boundary. are densely packed. However, for highly noisy images, other It was initially developed to follow an interface or contour pre-processing techniques may be needed before applying the propagating under a speed function F and was then used in CT-FIRE algorithm. It also does not work well on images where medical applications (Cardinal 2010). The FMM is a discre- the fibers are wavy and present many intersections. tized and computationally optimized version of the level set method (Osher and Sethian 1988). It aims at spreading an ini- tial surface until it covers the entire surface of interest (the Statistical feature extraction collagen fibers in our case) by solving the Eikonal equation. It is based on computing a distance map between the initial sur- The analysis of an image texture covers the region-specific iden- face and its surroundings. The surrounding points are divided tification of higher-order properties which are hard to detect visu- into three regions: the accepted points, the narrow band, and ally. Texture analysis leads to the definition of statistically uniform the far region. Initially, the accepted point region is the initial regions of an image based on the intensity distribution (Dudenkova surface. The narrow band constitutes the closest pixel to the et al. 2019). Statistical approaches that have been used to analyze initial front. The far region is what is left of the image. The SHG collagen images can be divided into three categories: first- Eikonal equation is solved on the edge points of the initial sur- order statistics, second-order statistics, and directional statistics. face. The points that satisfy this equation are then added to the initial surface and the same steps are applied again until there First‑order statistics (FOS) First-order statistics estimates parame- are no more points that may be added to the accepted point set. ters derived directly from the image statistics. They are often used The algorithm gives good results when the image is very to simply describe the image intensity distribution. However, they distinct from its background. Besides, the use of such algo- ignore the spatial correlations between the pixels of the image. In rithm does not need a prior setting of the parametric repre- other terms, FOS describes the probability to observe a pixel hav- sentation of the surface contour to be followed: this tech- ing a certain intensity in any position in the image. In more details: nique is robust with respect of the topology to be analyzed. However, it relies entirely on a physical interpretation of the The intensity distribution histogram is a representation problem characterized by the isotropic front propagation of of the number of pixels in an image with respect to their the initial surface (Cristiani 2009). Besides, the use of the values. It is a useful tool to detect saturation effects in an first-order neighbors (only four neighbors) introduces errors image (i.e., presence of pixels with maximum intensity), in the computation of the travel time from a point to another. to deduce the brightness (the image is bright if the histo- gram values are more concentrated around high values), C T‑FIRE The CT-FIRE is an algorithm introduced by and to check the contrast (if the values of the histogram Bredfeldt et al. (2014) that enables the extraction of fibers are spread out without a noticeable peak). 1 3 Research on Biomedical Engineering • • The mean calculated from the pixels’ intensity or from The entropy focuses on the randomness of regions in the probability distribution of the pixels’ intensity is used an image with respect to its neighborhood in terms of to evaluate the presence of one texture in the image. intensity distribution. Low entropy values correspond to • The standard deviation captures how the pixels are a uniform and homogeneous image spread out with respect to their intensity. • The skewness evaluates the histogram’s lack of symmetry Texture analysis can be performed on an entire image, but and allows characterizing the slope of the image histogram it is more interesting on a localized area to capture morpho- with respect to the central line. The skewness of a normal logical changes. This technique allows seeing morphologi- distribution is equal to zero. A negative (resp. positive) cal changes of the collagen structure (for example, to make skewness denotes an image for which the majority of pix- a comparison between a benign and a malignant tumor), els have values smaller (resp. greater) than the mean value. but it does not give information about their geometric and • The kurtosis describes how much a distribution is con- composition information. centrated around a peak (the mean) and allows evaluating the efficiency of a denoizing algorithm. Directional statistics Directional statistics focuses on observations that have directions. These observations FOS are easy and fast to calculate. However, their inter- usually lie whether on the circumference of a circle (cir- pretation is not always simple. They give global information cular statistics) or on the surface of a sphere or a hyper- and cannot be used to quantify local information (unless the sphere (spherical statistics) (Ley and Verdebout 2017). initial image is divided into several ROIs). Statistical analysis of directional data became more used after Fisher’s paper (1953) where he explained the need Second‑order statistics (SOS) Second-order statistics estimate to consider the curved nature of the sample space. Sev- parameters from the matrix generated by performing a correla- eral directional distributions emanated from Fisher’s tion between the image pixels. It studies, in particular, the topol- contribution. They are based on the extension of classi- ogy of one region compared to the image. Here we talk about cal concepts from multivariate analysis (e.g., point esti- texture analysis. This technique is usually used to describe and mation, regression, multi-sample testing procedure) to characterize a local area in an image through the use of gray directional setting (Pewsey and Garcίa-Portugués 2020; level co-occurrence matrix (GLCM) and some statistics (Haral- Mardia et al. 2008; Mardia and Jupp 2000). In the follow- ick et al. 1973). ing, we will focus on the Von Mises distribution which has been used to extract quantitative information from • The GLCM evaluates the spatial relationships between SHG images of collagen fibers. the values of the pixel intensity. It is a squared matrix of The Von Mises distribution is considered a flexible cir - dimension equal to the number of gray levels in the con- cular distribution. It is useful for a circle from a statistical sidered image (for example, 256 for 8-bit images). The inference point of view (Mardia and Jupp 2000). It repre- parameters that will be presented subsequently (Iqbal sents the maximum entropy distribution for circular data et al. 2021) can be calculated from the initial image but when the first circular moment real and imaginary parts are they are more relevant when they are performed on the specified. It is characterized by two parameters, a location GLCM. parameter µ ∊ [− π, π] and a concentration parameter κ. κ is • The energy (also called uniformity) allows to evaluate the positive and it allows to regulate the concentration of the uniformity of the image. distribution around µ. This distribution was later generalized • The inverse difference moment (IDM) measures the to higher dimensions by Von Mises and Fisher and, thus, local homogeneity of an image. When the IDM value was named von Mises-Fisher distribution. The Von Mises increases, it means that the incidence of pixels’ pairs co- distribution can also be referred to as the circular normal occurrence is enhanced which means that IDM is high distribution. To characterize collagen in SHG images, it is when the image is homogeneous. possible to evaluate the fiber dispersion and its diameter by The inertia (also called contrast) allows studying local fitting a Von Mises distribution. variations in an image. It is highly sensitive to large dif- It is a good tool to study 3D images because it can be gen- ferences in the GLCM values and has a strong correlation eralized to high dimensions without using many parameters. with the lowest and highest values in a ROI. However, for SHG images of collagen, this method assumes The correlation characterizes the gray-level linear that all the fibers belong to a single family (i.e., having the dependency on specified pixels on an image (i.e., the same orientation). repetitive nature of the texture element position). 1 3 Research on Biomedical Engineering Table 3 Information selection and extraction methods Methods Output Advantages Drawbacks Thresholding Binary image -It is simple and fast -It is highly sensitive to noise (Yuheng and -It works well for images having a histo- Hao 2017) gram with distinct peaks -It is specific to the considered image -It is a global method Region-based segmentation Binary image -It allows to partition the image (Yuheng -It depends on the choice of the seed points and Hao 2017) -It is local technique with no global view -It works properly on smoothly varying regions Edge-based segmentation Binary image -It is a high-level segmentation approach -It is highly sensitive to noise (Gonzalez and Woods 2018) -It does a poor job detecting corners -It works well on images with good con- trast FMM Segmented image -It gives good results when the image is -It is a static technique (Malladi and Sethian very distinct from its background (Cris- 1996) tiani 2009) -The first-order nature introduces errors in -It is robust and fast computation (Cristiani 2009) CT-FIRE Fiber skeleton -It works well on images of densely packed -It needs sometimes some additional pre- collagen fibers (Bredfeldt et al. 2014) processing FOS Statistical information -It is fast and easy to implement -It gives global information SOS Statistical information -It captures changes in images -It only gives information on the fibers texture Directional statistics Mathematical function -It fits well the orientation distribution -It assumes that the fibers follow one direc- profile of collagen fibers tion -It can be generalized to higher dimensions with few parameters their arrangement has a strong impact on the tissue’s Quantities to measure and associated biomechanics. questions Scale of measure In order to analyze and understand how collagen fibers behave when they are under a mechanical load, it is necessary to quan- Orientation, waviness, and curvature are the important tify them using some relevant information. For example, for geometric information about collagen fibers. Orientation is arteries, the quantitative information extracted from correspond- usually calculated globally, but sometimes researchers focus ing SHG images can be introduced in previously developed on specific regions in an image and therefore on the local mechanical models to better characterize the behavior of arter- directions. On the other hand, waviness and curvature are ies (Holzapfel et al. 2000; Morin et al. 2021). In the literature, determined locally. researchers focused on three types of information that can be extracted from SHG images of collagen fibers: its geometry, its Local characterization The study of collagen fibers in bio- composition, and its morphology. However, they dealt with dif- logical tissues showed that those fibers are crimped and ferent types of input data (thus, the output data were different) undulated. Thus, it is important to characterize their shapes. at different scales of measure. In the present section, we will For this matter, several techniques have been proposed. exhibit how that information was extracted through the litera- For the estimation of the fiber waviness, one needs to ture. We summarize it in Table 4. start by extracting the fibers. Sugita and Matsumoto (2017) determined the centers of the fibers as the pixels with a local Geometric information (orientation, waviness) maximum intensity. Then, they computed the length of the fiber as the distance between all the centers of a same fiber. A strong attention in the biomedical community is paid The CT-FIRE algorithm (Bredfeldt et al. 2014) is one to geometric information of the collagen fibers. Changes of the techniques used to improve the images by extract- in their geometric characteristics when they are under ing the fibers. The developers used also their algorithm to a mechanical load can actually be seen with a naked eye extract the collagen fibers and then estimated the waviness. on SHG image; hence, the will to quantify it. Besides, 1 3 Research on Biomedical Engineering CT-FIRE was also used in Best et al. (2019) to extract the in a considered segment. Some other researchers used the collagen fibers in renal cell carcinoma and by Zhou et al. FFT to evaluate the local orientation (Sivaguru et al. 2010; (2017) in gastric cancer in order to characterize their organi- Rao et al. 2009; Ambekar et al. 2012a; Lau et al. 2012). For zation and their straightness. example, Rao et al. (2009) focused on the preferred orienta- It is also possible to segment the SHG images and extract tion and the maximum spatial frequency of some regions the collagen fibers using other methods such as the skel - in the SHG images. To determine those metrics, they com- etonization. Koch et al. (2014) proposed a new approach puted the 2D FFT of the considered regions. The FFT gives based on the application of several filters before segment- the perpendicular angle to the preferred direction. To have a ing the images. They used sequentially a CLAHE, a his- better quantitative approximation, one can fit the probability togram adjustment, and a Frangi filter to reduce the noise distributions of fiber orientations using one Gaussian func- and enhance the fibrous information. Then, a threshold was tion (Sugita and Matsumoto 2017). It is also possible to apply applied to recover a binary image where the fibers are well a 3D FFT on the entire stack to evaluate the fiber preferred defined. Finally, they applied mathematical morphology direction in the space (Lau et al. 2012). However, the poor operators to retrieve the fiber skeleton. resolution of the SHG images in the third dimension may Techniques which were not initially developed for quan- have a bad impact on the result of the 3D FFT to estimate the tifying collagen in SHG images were also used. The most fiber directions in space. known one is the NeuronJ plugin of ImageJ software (Mei- Wavelet transforms were also used for direction estimation jering et al. 2004). This plugin was designed to characterize (Tilbury et al. 2014). The properties of the wavelet transform neurons which have a linear shape. NeuronJ was used for allow capturing small details and thus estimating correctly the tracing the fibers and analyzing their waviness (Zyablitskaya orientation of the fibers. For this matter, the local coefficients et al. 2017; Chow et al. 2014; Zeinali-Davarani et al. 2013). of the wavelet transform were calculated and then clustered Besides, a 3D implementation of this technique was pro- using K-nearest neighbors (K-NN) (Altman 1992) and prin- posed and tested on SHG images. For example, to determine cipal component analysis (PCA) (Pearson 1901). the fiber arc length, Hill et al. (2012) proceeded to a recon- Image gradient is an efficient method to estimate orienta- struction of the SHG stack using a fast-marching algorithm tions. This technique was initially developed by Chaudhuri et al. to trace the fibers. (1993). It consists in computing the gradient of the image to Once an accurate extraction of the collagen fibers is detect its edges and then to keep only the most relevant direc- reached, it is possible to compute the waviness as a ratio tion. The proposed method is similar to the Hough transform. It between the Euclidean distance between the starting and was later applied to biological tissues (Karlon et al. 1998) and ending points of a fiber and its actual length (Ayyalasomaya- to SHG images in particular (Hill et al. 2012; Phillippi et al. jula et al. 2019; Hill et al. 2012; Koch et al. 2014). The 2014; Kabir et al. 2013; Sun et al. 2015). In Kabir et al. (2013), estimation of those distances is done manually using ImageJ the authors focused on a ROI from initial SHG image where the (National Institutes of Health, Bethesda, MD, USA) or Ima- fibers have a pronounced dominant direction and calculated its ris (Bitplane, CT, USA). 2D gradient to estimate the fiber orientation. A powerful ImageJ The waviness in the 3D space was also investigated by plugin that has proven its efficiency on biological images is Ori- Luo et al. (2017). They proceeded to a 3D skeletonization entationJ. It is based on computing the image’s gradient and its based on the fast-marching algorithm. The waviness com- related weighted 2D structure tensors at each pixel. Cavinato putation is similar to what has been explained before, except et al. (2017) used this plugin to extract the orientation distribu- that the considered points have 3D coordinates. tion histogram. Gaussian functions were then fitted to the his- Regarding the local orientation, some interesting tech- togram in order to quantify the dominant fiber directions. In niques were tested on collagen gels and showed their effi- Avila and Bueno (2015), the authors also used it on the image ciency. One can cite the work of Bayan et al. (2009) where structure tensor. they used the Hough transform on different small partitions Even though most of the proposed methods that have of the SHG image to determine the dominant local orienta- been used to quantify collagen fiber orientation were per - tion of the considered fiber. The size of the partitions is formed in 2D, some researcher such as Liu et al. (2018) took chosen such as they are likely to contain a linear fiber. The into consideration the collagen fiber distribution in the 3D SHG images were pre-processed to delete the noise through space. They used the 3D directional variance algorithm to an adaptive thresholding and the application of an erosion identify each pixel orientation and then estimate the entire and a dilation if needed. fiber orientation. It is also possible to evaluate orientations after fiber More recently with the emergence of deep learning extraction. In Koch et al. (2014), the authors used the seg- algorithms, some authors applied this technique to esti- mented skeleton to calculate the local orientation as the mate local orientations of collagen fibers. For example, in angle of the tangent line between the first and last points Schmarje et al. (2019), a comparison of different 2D and 1 3 Research on Biomedical Engineering 3D methods aiming at estimating local orientations was Directional filters were also used to determine the local proposed. Besides, the authors introduced a new modality orientation of the collagen fibers. Wen et al. (2014) proposed to transfer 2D weights to 3D weight in different-network an approach based on those filters with different scales to architecture to perform a segmentation of some images with determine the collagen fiber orientation in ovarian cancer. respect to local orientations. They extracted a histogram of the frequency of occurrence of individual patterns in an image. A nearest neighbor clas- Global characterization Most of the scientific contributions sification was then performed on the extracted histograms aiming at extracting quantitative information from SHG to distinguish between human normal and high-grade malig- images of collagen fibers in biological tissues focused on nant ovarian tissues. the fiber orientation. Some local techniques such as texture analysis have been It is possible to determine fiber orientation using the used to quantify and describe the main fiber orientation. FFT. It is the most used technique for this matter (Ayyala- They showed their efficiency and they may be also more somayajula et al. 2019; Bueno et al. 2013; Chiu 2010; Chow precise than the classic FFT. In fact, Hu et al. (2012) pro- et al. 2014; Forouhesh Tehrani et al. 2021; Lau et al. 2012; posed a new approach for texture analysis based on orienta- Lee et al. 2019; Pijanka et al. 2019; Robinson et al. 2016; tion-dependent gray level co-occurrence matrix. They used Sivaguru et al. 2010; Wu et al. 2011). This approach is also their algorithm on ex vivo rat tendons to study the dominant called FT-SHG imaging (Ambekar et al. 2012a). In Lee et al. collagen fiber direction. For this matter, they focused on the (2019), the authors used the FFT on the entire stack and then correlation feature of the GLCM. performed a segmentation on the transformed images to only keep the dominant fibers’ direction. Once the segmentation achieved, it is possible to recover the angles’ distribution that Input data nature (2D/projected 3D/3D) corresponds to each image. It is then possible to evaluate the variation of the angles while going deeper in the stack. The determination of the orientation and the waviness of Germann et al. (2018) used the same methodology as Bueno collagen fibers can be done using different types of input. et al. (2013), based on some pre-processing (noise reduction Multi-photon microscopes allow going deeper in the tissue, and edge sharpening) and a FFT to extract the orientation of and one can recover 3D stacks of images. However, most of collagen fibers in SHG corneal images. the proposed techniques in the literature were limited to the Usually, the use of the FFT is sufficient to estimate the image plane. fiber directions, but sometimes it is useful to make the pro- Generally, 2D images are used. For example, Zyablits- cedure more automated. For example, Ayyalasomayajula kaya et al. (2017) used 2D SHG image of rabbit sclera to et al. (2019) extracted the distribution using a finite mixture estimate the waviness of the collagen fibers. In addition, to of Von Mises distribution to fit the orientation distribution assess the accuracy of their measurements, they calculated extracted from the FFT in order to determine the global the average value on 10 SHG images. Ayyalasomayajula mean orientation. Others, such as Schriefl et al. (2013) and et al. (2019) used 2D images but limited their study to 10 Polzer et al. (2013), used a classical Von Mises distribution slices of the stack. Then, the global orientation was set as for the same purpose. It is also possible to use a Gaussian the average of the computed orientations. function for the fitting such as in Ambekar et al. (2012b). In However, for the computation of the waviness of the colla- some papers (Brisson et al. 2015; Kroger et al. 2021; Tang gen fibers, some papers processed 3D images such as in Hill et al. 2014; Wu et al. 2011), the focus was oriented toward et al. (2012), so they were able to characterize this metric in the result of the FFT where an ellipse was superposed. The 3D in arterial tissues. In this paper, the waviness was com- major axis of this ellipse corresponds to the orthogonal of puted from a 3D reconstruction of the SHG images by tracing the dominant direction if the ratio between the major and the the fibers using a 2D marching algorithm. This is possible minor axes is high. Otherwise, there is no preferable orien- because the waviness estimation is based on coordinates, tation. Besides, it may be useful to use the radon transform which can be deduced from 3D SHG images. Meanwhile, an on the 2D FFT of the SHG images (Mclean 2015; Mega accurate 3D reconstruction of the SHG image may be hard to et al. 2012) since, unlike the FFT, it provides quantitative get because of the poor data resolution in the third dimension. information for each discrete angle. It is also common to use Regarding the orientation measurements, Hill et al. wedge l fi ters after the FFT and then t fi the orientation distri - (2012) and Cavinato et al. (2020) used a 2D superimposed bution with a Von Mises distribution to better estimate the projection of the 3D stack of SHG images. Phillippi et al. orientation (Polzer et al. 2013; Schriefl et al. 2013; Niestraw - (2014) succeeded in evaluating both collagen and elastin ska et al. 2016). In Zeitoune et al. (2017) the author applied fibers in the aorta using superimposed 2D image stacks. The an FFT on the images. Then, they improved the result of the dominant orientation from a projection of all the SHG stack transform by smoothing and enhancing it. images can be extracted to recover a 2D image that contains 1 3 Research on Biomedical Engineering all information from the entire stack (Hristu et al. 2018). density, it is important to choose a ROI that covers up to 10 However, it is more common to use 2D images to estimate times the collagen fiber diameter. the orientation (Bueno et al. 2013; Kabir et al. 2013) and In order to evaluate the fiber density, it is mandatory to look at its evolution with respect to the stack depth (Lee enhance the SHG image by improving the signal-to-noise et al. 2019). ratio. For this matter, it is important to filter the image and Some studies showed that the collagen fiber orientation to recover an accurate representation of the fiber network in the axial–radial direction is negligible (Humphrey and through segmentation (Hompland et al. 2008) or b fi er extrac - Holzapfel 2012; Wagenseil and Mecham 2009). However, in tion (Wegner et al. 2017). Lau et al. (2012), the authors proposed a 3D FFT approach Gade et al. (2019) performed a segmentation on the SHG to evaluate the fibers preferred orientation in 3D stacks of stack using the Otsu thresholding. Then, the authors com- SHG images. SHG stacks were also used to determine the puted the area of segmented pixels in every slice and sum waviness of the fibers such as in Luo et al. (2017). Bivariate up the segmented area across the volume to calculate total Von Mises distribution was also used on 3D stacks of colla- areal density in the image stack. The same procedure was gen in the aorta to fit the in-plane and out-of-plane collagen followed by Balu et al. (2014) and Tjin et al. (2014) where fiber orientations (Niestrawska et al. 2016). they performed a segmentation using ImageJ and then com- puted the collagen density as the sum of the pixels that have Output data nature intensity values greater than a certain threshold. It is sometimes interesting to proceed to a complete image The outputs of all the methods cited above can be divided enhancement step because of the diminution of the pixel into two types: a single value or an orientation distribution intensity when we go deeper in the stack. For this matter, (i.e., a list). For example, the use of the FFT followed by it is useful to apply a CLAHE on the SHG images. Cai ellipse fitting (Brisson et al. 2015; Tang et al. 2014; Wu et al. et al. (2014) enhanced the dermal layer of human skin SHG 2011) gives one value corresponding to the dominant ori- images using the CLAHE algorithm. Then, they applied the entation in the considered stack or ROI. Single-orientation Frangi filter and a segmentation using Otsu’s tresholding in values can also be extracted using texture analysis, which is order to capture a representation of both the fibers and the applied locally (Hu et al. 2012). It is also possible to extract holes in the images. an orientation distribution histogram from the FFT by the CT-FIRE is another algorithm used to extract the col- application of a radon transform (Mclean 2015; Mega et al. lagen fibers (Best et al. 2019; Wegner et al. 2017; Zhou 2012). It is also possible to use a Von Mises distribution et al. 2017). For example, in Best et al. (2019), the authors and fit it to the orientation distribution obtained by the FFT extracted the collagen in renal cell carcinoma in order to (Ayyalasomayajula et al. 2019; Niestrawska et al. 2016; evaluate the density of the collagen in low- and high-grade Polzer et al. 2013; Schriefl et al. 2013). Histogram of the tumors. The density can be calculated as the number of pix- frequency of occurrence is another representation of the ori- els corresponding to the fiber network with respect to the entation (Wen et al. 2014). image or to the entire stack. Second-order statistics, in general, and the grey level co- Composition information (density) occurrence matrix, in particular, have been used to estimate the density of collagen fibers. In Kroger et al. (2021), the Fiber density estimation is important for collagen charac- authors used the GLCM and especially the homogeneity terization. In the literature, there are two ways to define the parameter to determine the density of features in an image. density: the volume occupied by the fibers in the stack (i.e., Some papers focused on the estimation of the ratio of volume fraction) or the number of fibers in a considered both collagen and elastin fibers in SHG stacks (Abraham and region. Hogg 2010; Lin et al. 2005; Koehler et al. 2006). In Abra- ham and Hogg (2010), the authors started by filtering the Scale of measure images to reduce the noise. Then, they segmented the images and estimated the volume fraction of the fiber network as The scale of measure depends on what has to be quantified. the sum of all the pixels belonging to the segmented region. For volume fraction estimation, the procedure is global and applied to the entire stack. It may be interesting for some Input data nature (2D/projected 3D/3D) applications to focus on a ROI in the stack and calculate its volume fraction (for example, to characterize the evolu- The computation of the fiber density (also referred to as the tion of tumors density). The same reasoning is applicable volume fraction) requires the entire stack. The evaluation of to calculate the density as the number of fibers in the entire the density can be done in 2D (i.e., slice per slice) or directly stack or in a ROI. However, for an accurate estimation of the 1 3 Research on Biomedical Engineering on the entire stack. It depends on how the segmentation is can be generalized to distinguish between the collagen fib- performed (in 2D or 3D). ers and, thus, estimate their diameter (Cicchi et al. 2009). Cai et al. (2014) focused on 2D virtual biopsy images It is also possible to deduce if there are linear fibers and a and not stacks. Therefore, they tested their approach only on fine structure through the computation of the entropy (Wu single 2D images. In Zhou et al. (2017), the authors focused et al. 2016). on single SHG images and, thus, calculated the collagen Some researchers showed interest in evaluating the density in the 2D plane. fiber length. In Sugita and Matsumoto (2017), the authors In general, SHG images are segmented separately. For extracted the centers of each fiber assuming that they cor - example, in Gade et al. (2019) and Balu et al. (2014), the respond to a maximum intensity value and then estimated authors segmented the SHG images using a thresholding the fiber length as the sum of the distances between their technique. Then, they calculated the number of white pixels centers. Fiber length can be evaluated manually from the in every image and summed them up across the volume to SHG images of collagen gels after segmentation and using calculate total 3D density. ImageJ drawing tool (Ajeti et al. 2011). A global overview of the stack gives a more accurate esti- Moreover, collagen fibers that are extracted using the mation of the collagen density. That is why some research- CT-FIRE algorithm can be used to extract manually the ers such as Abraham and Hogg (2010) implemented their length and the fiber diameter (Drifka et al. 2016; Rosen method on the entire SHG stack. et al. 2020; Wegner et al 2017; Zhou et al. 2017). In Rosen et al. (2020), the collagen fibers in every SHG Morphologic information (fiber’s size) image of feline mammary adenocarcinoma were identi- fied by the mean of the CT-FIRE algorithm. Once the In addition to geometric and composition information, it is fiber extraction is achieved, each fiber was analyzed and necessary to know the fiber morphology in order to have a its length and width were extracted in addition to the complete picture of the considered microstructure. For this percentage of straight fibers. matter, the intersections between the collagen fibers and the Some out-of-the-box techniques were used. For instance, fibers size have been investigated in the literature. Robinson et al. (2016) estimated the collagen fiber thick - ness using the BoneJ plugin (Doube et al. 2010) for ImageJ Scale of measure which was initially developed to measure bone geometry. This algorithm gives the thickness of a considered fiber. The study of the intersection between collagen fibers and even the estimation of their size is done locally because they Input data nature (2D/projected 3D/3D) are specific to a fiber (size) or a region (intersection). In order to be able to extract that information from the SHG For texture analysis, the application is usually performed images, it is important to enhance them. on 2D images. Indeed, Wu et al. (2016) were only inter- For example, Koch et al. (2014) used segmented SHG ested in investigating some layers of the dermis with images to estimate the fiber diameter. They performed some the strongest collagen intensity. In Cicchi et al. (2009), mathematical morphology operators (erosion and dilation) the investigation was also limited to 2D SHG images of to obtain a 1-pixel-thick fiber skeleton. This skeleton was human dermis. later used to calculate the fiber radii from the initial seg- In the case of a skeletonization, the authors of Koch mented image. et al. (2014) used two 2D images (one of the fiber skele- In some cases, depending on the application, only the ton and one of the enhanced image) to determine the fiber characterization of the evolution of the morphology is radii. In addition, to estimate the fiber length, they used needed. For this matter, texture analysis is used. In Wu the skeleton of the fibers. Sugita and Matsumoto (2017) et al. (2016), the authors used this technique to study the focused also on the fiber length and used 2D SHG images impact of aging on the skin microstructure. They computed since fiber centers were determined in the 2D plane. It is the contrast, correlation, and entropy from the GLCM of also possible to extract the fiber network using the CT- the image and analyzed them to characterize the fiber FIRE algorithm and determine the fiber length and diam- structure and morphology. The contrast was computed to eter (Drifka et al. 2016; Zhou et al. 2017). The papers that assess the presence of a fine structure of collagen fibrils. considered segmentation of the SHG images focused on Wu et al. (2016) characterized how the collagen matrix is each image individually and did not apply the segmenta- distinct from its surrounding and if there is loss in collagen tion to the entire stack (Ajeti et al. 2011). through time using the computation of correlation. This 1 3 Research on Biomedical Engineering Table 4 Main methods used in the literature to quantitatively analyze collagen fibers Measure Methods References Waviness Locally -Local maximum intensity (Sugita and Matsumoto 2017) -Manual (ImageJ, Imaris) (Hill et al. 2012; Ayyalasomayajula et al. 2019) -CT-FIRE (Best et al. 2019; Zhou et al. 2017; Bredfeldt et al. 2014) -Skeletonization (Koch et al. 2014; Luo et al. 2017) -NeuronJ (Zyablitskaya et al. 2017; Chow et al.2014; Zeinali-Davarani et al. 2013) -FMM (Hill et al. 2012) Orientation Locally -Segmentation + Hough transform (Bayan et al. 2009) -Skeletonization (Koch et al. 2014) -FFT (Sivaguru et al. 2010; Rao et al. 2009; Ambekar et al. 2012a; Lau et al. 2012; Sugita -Wavelet transform and Matsumoto 2017) -Gradient (Tilbury et al. 2014) -3D directional variance (Hill et al. 2012; Phillippi et al. 2014; Kabir et al. 2013; Cavinato et al. 2017; Avila and Bueno 2015) (Liu et al. 2018) Globally -FFT (Bueno et al. 2013; Chiu 2010; Chow et al. 2014; Lee et al. 2019; Lau et al. 2012; -FFT + Von Mises Robinson et al. 2016; Zeitoune et al. 2017; Sivaguru et al. 2010; Wu et al. 2011; -FFT + wedge filter + Von Mises Pijanka et al. 2019; Germann et al. 2018) -FFT + Gaussian (Ayyalasomayajula et al. 2019) -FFT + ellipse fitting (Polzer et al. 2013; Schriefl et al. 2013; Niestrawska et al. 2016) -FFT + Radon transform (Ambekar et al. 2012b) -Directional filters (Brisson et al. 2015; Tang et al. 2014; Wu et al. 2011) -Texture analysis (Mclean 2015; Mega et al. 2012) (Wen et al. 2014) (Hu et al. 2012) Density -Segmentation (Gade et al. 2019; Balu et al. 2014; Tjin et al. 2014; Cai et al. 2014; Abraham and -CT-FIRE Hogg 2010; Lin et al. 2005; Koehler et al. 2006) (Best et al. 2019; Wegner et al. 2017; Zhou et al. 2017) Size -Manual (Ajeti et al. 2011) -BoneJ (Robinson et al. 2016) -Skeletonization (Koch et al. 2014) -Local maximum intensity (Sugita and Matsumoto 2017) -Texture analysis (Wu et al. 2016; Cicchi et al. 2009) -CT-FIRE (Rosen et al. 2020; Zhou et al. 2017; Drifka et al. 2016; Wegner et al. 2017) The FFT of the SHG image was computed and a thresh- Comparison of some methods olding operation was applied on the FFT result. For this purpose, we used ImageJ. Then, an ellipse was fitted on the In the following, some of the collagen quantification meth- resulting image to recover the main direction of the collagen ods that were previously described will be tested on a case fibers. It corresponds to the direction perpendicular to the study SHG image. The choice of those methods is based on angle of the major axis of the ellipse. Figure 2.b exhibits their efficiency and how often they are used. Therefore, the the result of the FFT of the considered SHG image after tested methods are the FFT, the gradient through the Orien- thresholding. In our case study, the dominant orientation of tationJ plugin, and the CT-FIRE. The needed pre-processing the collagen fibers corresponds to 41.954°. is described. We performed all the cited methods on an SHG We applied the FFT on a ROI from the initial image. The image of the adventitia layer of a human aorta, Fig 2.a. In the ROI was chosen as an area containing only one fiber. The following, the estimated angles are expressed with respect ROI was smoothed using a median filter, Fig. 3.a. Then, the to the horizontal direction. FFT (Fig. 3.b) and the power spectrum (Fig. 3.c) of the ROI were calculated. As can be seen from both the FFT and the FFT power spectrum representations, the result emphasizes one angle that corresponds to the longest portion of the fiber. The FFT was applied on a SHG image of collagen fibers of Our calculations give an angle around 61° as the dominant human aorta. We first used the FFT on the entire image to orientation. extract the dominant orientation in the considered image. One can clearly see that in the presence of a distinct ori- Then, for comparison purposes, we focused on a ROI of entation, the result of the FFT gives a good estimation of the image where the fibers are aligned with respect to one that orientation. direction. 1 3 Research on Biomedical Engineering Fig. 2 a Initial SHG image; b FFT of the SHG image after thresholding Gradient CT‑FIRE To calculate the gradient of the SHG image presented in The CT-FIRE algorithm was first applied to the initial SHG Fig. 4.a, we used the OrientationJ plugin on ImageJ. First, image without any pre-processing. The result is exhibited in we tried a global approach to detect the dominant direc- Fig. 7. Because of the poor quality of the considered image, tion of all the collagen fibers. Figure 4.b shows the result the CT-FIRE could not provide a good extraction of the col- of orientation distribution. One can see that the preferred lagen fibers. orientation of the fibers is around 42.5°. As the fiber extraction was not good, the estimation of the To limit the non-relevant calculation due to noise, we global orientation (63.4°) is far from what has been com- applied a median filter on the initial SHG image to smooth it puted using the FFT and the gradient (around 42°). The same and to reduce the artifacts inside the fibers. Figure 5 shows the steps were applied on the smoothed version of the SHG filtered image and the new orientation distribution. The orienta- image and the results were similar. Indeed, the orientation tion distribution presents one major peak located around 43°. was estimated to be equal to 61.3°. The gradient method was also applied on the ROI previ- We then applied the CT-FIRE algorithm on the same ROI. ously considered. The results can be seen in Fig. 6. The orien- The algorithm was able to extract the skeleton of the fiber, tation distribution shows a dominant peak at the angle 66.5°. Fig. 8. It estimated the fiber orientation to be around 70°. Fig. 3 a ROI from the initial SHG image; b FFT of the smoothed ROI; c ROI power spectrum 1 3 Research on Biomedical Engineering Fig. 4 a Initial SHG image; b orientation distribution using OrientationJ Regarding the CT-FIRE algorithm, its application on a Comparison bad-quality image without any pre-processing showed its limitation in extracting the collagen fibers. In addition, this The experiments conducted previously showed that the FFT is an appropriate tool to estimate the main orientation of the technique looks like it computes the average of all the local orientations and not really the main global direction. The collagen fibers if the fibers are well organized. Otherwise, no distinct information can be retrieved from the FFT. Besides, estimated orientation given by the CT-FIRE algorithm is close to the orientations given by the FFT and the gradient the result is highly sensitive to the thresholding algorithm used. On our chosen SHG image, the Otsu thresholding did on a particular ROI of the image. It is, however, interesting to use the CT-FIRE algorithm not give an accurate orientation estimation. Moreover, for noisy images, the result of the FFT may be noisy too and, on a ROI. The experiments showed that, for small filtered ROI where there is only one fiber, the algorithm is able to thus, non-exploitable. The study of a ROI containing one fiber with two orienta- extract correctly the position of the fiber’s skeleton and thus estimate its orientation and its width and length. tions and different intensities showed that the FFT focuses only on the most “visible” portion of the image. Our tests allowed us to extract only one angle from the FFT. The global overview of the collagen fiber orientation Discussion using the gradient gives a result close to the one found using the FFT. This can be explained by the fact that the fibers in To our knowledge, reviews dealing with the quantitative analysis of collagen fibers from SHG images are not very the considered image are not very crimped and undulated. Because the gradient is locally calculated, it is very sensitive common. The only one that we identified treated the topic from a method point of view. In fact, the main contribution to the shape of the collagen fibers. Therefore, it considers the fiber geometry. This method will always provide an estima- of the present paper lies in the categorization of the image processing method with respect to the information that we tion of a main orientation even if this one does not really exist. The result of this method is highly dependent on the want to extract (geometry, composition, or morphology). This structure makes it easier to biomedical researchers to find the quality of the image. Indeed, since it is a pixel-wise tech- nique, it depends on the difference between the neighboring- most suitable method to the problem they are trying to solve. On another hand, the third part of the present review, which pixel intensities that can change while filtering the image. When computed on a smoothed ROI with just one fiber, compared three of the most used techniques to estimate col- lagen fiber orientations, already gives the user an idea about the gradient gives a good estimation of the orientation since it only involves information relevant to the considered fiber. how to use those methods and what to expect from them. 1 3 Research on Biomedical Engineering Fig. 5 a Smoothed SHG image using a median filter; b orientation distribution using OrientationJ Many image processing methods have been used in order to example, in Schmarje et al. (2019), the authors used convo- extract valuable information from collagen SHG images. How- lutional neural networks (CNN) to segment SHG images of ever, the choice of the methods to choose depends deeply on collagen fibers in order to quantify their local orientations. the quality of the images and, thus, on the used microscope. In most of the cases, a pre-processing phase is necessary. In addi- tion, the result of the pre-processing can affect, for example, the Conclusion dimension estimation. In fact, if the image is not well filtered, blur can be mistaken to be part of a fiber. In order to study the collagen fiber behavior in biologi- It may be interesting to use machine learning algorithms cal tissues, it is necessary to extract quantitative informa- to quantitatively analyze SHG images of collagen fiber. Some tion to characterize them. To analyze those fibers, several encouraging attempts can be found in the literature. For techniques (including pre-processing and information Fig. 6 a Smoothed ROI; b orientation distribution using OrientationJ 1 3 Research on Biomedical Engineering Fig. 7 Fiber extraction from a noisy image using CT-FIRE of these methods to discuss their actual abilities to quan- tify collagen orientation. The choice of the method still depends on the images that need to be processed, their quality, and the error tolerance rate. A proper quantitative analysis of collagen fibers needs a combination of some of the techniques presented previously. On the other hand, the quantitative analysis of collagen fibers in 3D is still not widely developed because of the limitations of the acquisition technique when going deep into the tissue and the poor imaging resolution in the third dimension. Further studies need to be oriented toward this issue especially because it is important to quantify the fiber network in the 3D space. Declarations Conflict of interest The authors declare no competing interests. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or for- Fig. 8 Fiber extraction from a noisy ROI using CT-FIRE mat, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third selection) can be used, each one of them having advan- party material in this article are included in the article's Creative tages and drawbacks. The choice of the method to use is Commons licence, unless indicated otherwise in a credit line to the highly dependent on the information that need to be quanti- material. If material is not included in the article's Creative Com- fied. In this paper, we exhibited the most used techniques mons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain to quantify collagen fibers and we discussed the types of permission directly from the copyright holder. To view a copy information that they allow to extract. As an illustration, of this licence, visit http:// creat iveco mmons. org/ license s/ by/4. 0/. we proposed a comparison of implementations of some 1 3 Research on Biomedical Engineering Brown E, McKee T, diTomaso E, et al. Dynamic imaging of collagen References and its modulation in tumors in vivo using second-harmonic gen- eration. Nat Med. 2003;9(6):796–800. Abraham T, Hogg J. Extracellular matrix remodeling of lung alveo- Bruun G. z-transform DFT filters and FFT’s. IEEE Trans Acoust lar walls in three-dimensional space identified using second Speech Signal Process. 1978;26(1):56–63. harmonic generation and multiphoton excitation fluorescence. Bueno JM, Palacios R, Chessey MK, Ginis H. Analysis of spatial lamel- J Struct Biol. 2010;171(2):189–96. lar distribution from adaptive-optics second harmonic generation Acharya T, Ray AK. Image processing: principles and applications. corneal images. Biomed Opt Express. 2013;4(7):1006–13. Hoboken: Wiley; 2005. Cai CS, Chen CF, Lee GG, et al. Density analysis of collagen fibers Adams R, Bischof L. Seeded region growing. IEEE Trans Pattern based on enhanced Frangi filter in second harmonic generation Anal Mach Intell. 1994;16(6):641–7. virtual biopsy images. IEEE China Summit & International Ajeti V, Nadiarnykh O, Ponik SM, Keely PJ, Eliceiri KW, Cam- Conference on Signal and Information Processing (ChinaSIP), pagnola PJ. Structural changes in mixed Col I/Col V collagen Xi’an, China. 2014;465–9. https:// doi. org/ 10. 1109/ China SIP . gels probed by SHG microscopy: implications for probing stro- 2014. 68892 86. mal alterations in human breast cancer. Biomed Opt Express. Campagnola PJ, Millard AC, Terasaki M, Hoppe PE, Malone CJ, 2011;2(8):2307–16. Mohler WA. Three-dimensional high-resolution second-har- Akansu AN. Multiresolution signal decomposition - transforms, sub- monic generation imaging of endogenous structural proteins in bands, and wavelets. 2nd ed. San Diego: Academic Press; 2001. biological tissues. Biophys J. 2002;82(1):493–508. https:// doi. Altman NS. An introduction to kernel and nearest-neighbor nonpara- org/ 10. 1016/ S0006- 3495(02) 75414-3. metric regression. Am Stat. 1992;46(3):175–85. Campagnola PJ. Second harmonic generation imaging micros- Ambekar R, Chittenden M, Jasiuk I, Toussaint KC. Quantitative copy: applications to diseases diagnostics. Anal Chem. second-harmonic generation microscopy for imaging porcine 2011;83(9):3224–31. cortical bone: comparison to SEM and its potential to investi- Cardinal MHR. Fast-marching segmentation of three-dimensional gate age-related changes. Bone. 2012a;50(3):643–50. intravascular ultrasound images: A pre-and post-intervention Ambekar R, Lau TY, Walsh M, Bhargava R, Toussaint KC. study. Med Phys. 2010;37(7):3633–47. Quantifying collagen structure in breast biopsies using sec- Cavinato C, Helfenstein-Didier C, Olivier T, du Roscoat SR, Laroche ond-harmonic generation imaging. Biomed Opt Express. N, Badel P. Biaxial loading of arterial tissues with 3D in situ 2012b;3(9):2021–35. observations of adventitia fibrous microstructure: a method cou- Avila FJ, Bueno JM. Analysis and quantification of collagen organiza - pling multi-photon confocal microscopy and bulge inflation test. tion with the structure tensor in second harmonic microscopy J Mech Behav Biomed Mater. 2017;74:488–98. images of ocular tissues. Appl Opt. 2015;54(33):9848–54. Cavinato C, Badel P, Kransy W, Avril S, Morin C. Experimental Ayyalasomayajula V, Pierrat B, Badel P. A computational model characterization of adventitial collagen fiber kinematics using for understanding the micro-mechanics of collagen fiber net - second-harmonic generation imaging microscopy: similarities work in the tunica adventitia. Biomech Model Mechanobiol. and differences across arteries, species and testing conditions. 2019;18(5):1507–28. In: Zhang Y, editor. Multi-scale extracellular matrix mechanics Babalola O, Mamalis A, Lev-Tov H, Jagdeo J. Optical coherence and mechanobiology. Switzerland: Springer; 2020. p. 123–64. tomography (OCT) of collagen in normal skin and skin fibrosis. Chaudhuri B, Kundu P, Sarkar N. Detection and gradation of oriented Arch Dermatol Res. 2014;306(1):1–9. texture. Pattern Recogn Lett. 1993;14(2):147–53. Bailly L, Cochereau T, Orgéas L, et al. 3D multiscale imaging of Chaudhuri D, Agrawal A. Split-and-merge procedure for image seg- human vocal folds using synchrotron X-ray microtomography in mentation using bimodality detection approach. Defense Science phase retrieval mode. Sci Rep. 2018;8:14003. Journal. 2010;60(3):290–301. Balu M, Kelly KM, Zachary CB, et al. Distinguishing between benign Chen X, Nadiarynkh O, Plotnikov S, Campagnola PJ. Second harmonic and malignant melanocytic nevi by in vivo multiphoton micros- generation microscopy for quantitative analysis of collagen fibril- copy. Can Res. 2014;74(10):2688–97. lar structure. Nat Protoc. 2012;7(4):654–69. Bamberger R, Smith M. A filter bank for the directional decomposi- Chiu YW. Second-harmonic generation imaging of collagen fibers tion of images: theory and design. IEEE Trans Signal Process. in myocardium for atrial fibrillation diagnosis. J Biomed Opt. 1992;40(4):882–93. 2010;15(2): 026002. Bayan C, Levitt JM, Miller E, Kaplan D, Georgakoudi I. Fully auto- Chow MJ, Turcotte R, Lin CP, Zhang Y. Arterial extracellular matrix: a mated, quantitative, noninvasive assessment of collagen fiber mechanobiological study of the contributions and interactions of content and organization in thick collagen gels. J Appl Phys. elastin and collagen. Biophys J. 2014;106(12):2684–92. 2009;105(10): 102042. Cicchi R, Kapsokalyvas D, De Giorgi V, et al. Scoring of collagen Bergland GD. A guided tour of the fast Fourier transform. IEEE Spectr. organization in healthy and diseased human dermis by multipho- 1969;6(7):41–52. ton microscopy. J Biophotonics. 2009;3(1–2):34–43. Best SL, Liu Y, Keikhosravi A, et al. Collagen organization of renal Cooley JW, Lewis PAW, Welch PD. The fast Fourier transform and its cell carcinoma differs between low and high grade tumors. BMC applications. IEEE Trans Educ. 1969;12(1):27–34. Cancer. 2019;19(1):490. Cristiani E. A fast marching method for Hamilton-Jacobi equa- Bluestein L. A linear filtering approach to the computation of dis- tions modeling monotone front propagations. J Sci Comput. crete Fourier transform. IEEE Trans Audio Electroacoust. 2009;39(2):189–205. 1970;18(4):451–5. Dahlke S, Maass P, Teschke, et al. Multiscale approximation. In: Bredfeldt JS, Liu Y, Pehlke CA, et al. Computational segmentation Dahlhaus R, Kurths J, Maass P, Timmer J, editors., et al., Math- of collagen fibers from second-harmonic generation images of ematical methods in signal processing and digital image analysis. breast cancer. J Biomed Opt. 2014;19(1): 016007. Berlin Heidelberg: Springer; 2008. p. 75–109. Brisson BK, Mauldin EA, Lei W, et al. Type III collagen directs stro- Damiand G, Resch P. Split-and-merge algorithms defined on topo- mal organization and limits metastasis in a murine model of logical maps for 3D image segmentation. Graph Models. breast cancer. Am J Pathol. 2015;185(5):1471–86. 2003;65(1–3):149–67. 1 3 Research on Biomedical Engineering Deans SR. The radon transform and some of its applications. Mineola: Hompland T, Erikson A, Lindgren M, Lindmo T, de Lange DC. Sec- Dover Publications; 2007. ond-harmonic generation in collagen as a potential cancer diag- Disney CM, Madi K, Bodey AJ, Lee PD, Hoyland JA, Sherratt MJ. Vis- nostic parameter. J Biomed Opt. 2008;13(5): 054050. ualising the 3D microstructure of stained and native interverte- Hough PVC, inventor; Method and means for recognizing complex bral discs using X-ray micro tomography. Sci Rep. 2017;7:16279. patterns. United States patent US 3069654. 1962 Dec 18. Doube M, losowski MMK, Arganda-Carreras I, et al. BoneJ: Hristu R, Eftimie LG, Stanciu SG, et al. Quantitative second harmonic free and extensible bone image analysis in ImageJ. Bone. generation microscopy for the structural characterization of 2010;47(6):1076–9. capsular collagen in thyroid neoplasms. Biomed Opt Express. Drifka CR, Loeffler AG, Mathewson K, et al. Comparison of pic- 2018;9(8):3923–36. rosirius red staining with second harmonic generation imag- Hu W, Li H, Wang C, Gou S, Fu L. Characterization of collagen fib- ing for the quantification of clinically relevant collagen fiber ers by means of texture analysis of second harmonic generation features in histopathology samples. J Histochem Cytochem. images using orientation-dependent gray level co-occurrence 2016;64(9):519–29. matrix method. J Biomed Opt. 2012;17(2): 026007. Duda RO, Hart PE. Use of the Hough transformation to detect lines and Huang T, Yang G, Tang G. A fast two-dimensional median filter - curves in pictures. Commun ACM. 1972;15(1):11–5. ing algorithm. IEEE Trans Acoust Speech Signal Process. Dudenkova VV, Shirmanova MV, Lukina MM, Feldshtein FI, Virkin 1979;27(1):13–8. A, Zagainova EV. Examination of collagen structure and state Humphrey J, Holzapfel G. Mechanics, mechanobiology, and mode- by the second harmonic generation microscopy. Biochem Mosc. ling of human abdominal aorta and aneurysms. J Biomechanics. 2019;84(S1):89–107. 2012;45(5):805–14. Fisher R. Dispersion on a sphere. Proc R Soc Lond A. Hyvärinen A, Hurri J, Hoyer PO. Natural image statistics. London: 1953;217(1130):295–305. Springer; 2009. Forouhesh Tehrani K, Pendleton EG, Southern WM, Call JA, Iqbal N, Mumtaz R, Shafi U, Zaidi SMH. Gray level co-occurrence Mortensen LJ. Spatial frequency metrics for analysis of matrix (GLCM) texture based crop classification using low alti- microscopic images of musculoskeletal tissues. Connect Tis- tude remote sensing platforms. PeerJ Comput Sci. 2021;7: e536. sue Res. 2021;62(1):4–14. https:// doi. org/ 10. 1080/ 03008 207. https:// doi. org/ 10. 7717/ peerj- cs. 536. 2020. 18283 81. Jaffar Iqbal Barbhuiya AHM, Hemachandran K. Wavelet tranforma- Frangi AF, Niessen WJ, Vincken KL, Viergever MA. Multiscale vessel tions & its major applications in digital image processing. IJERT. enhancement filtering. Wells WM, Colchester A, Delp S, edi- 2013;2(3). tors. Medical image computing and computer-assisted interven- Kabir MM, Inavalli VVGK, Lau TY, Toussaint KC. Application of tion — MICCAI’98. Lecture Notes in Computer Science. Berlin, quantitative second-harmonic generation microscopy to dynamic Heidelberg: Springer; 1998:130–137. conditions. Biomed Opt Express. 2013;4(11):2546–54. Franken PA, Hill AE, Peters CW, Weinreich G. Generation of optical Karlon WJ, Covell JW, Mcculloch AD, Hunter JJ, Omens JH. Auto- harmonics. Phys Rev Lett. 1961;7(4):118–9. https:// doi. org/ 10. mated measurement of myofiber disarray in transgenic mice with 1103/ PhysR evLett. 7. 118. ventricular expression of ras. Anat Rec. 1998;252(4):612–25. Freund I, Deutsch M, Sprecher A. Connective tissue polarity. Opti- Khattak SS, Saman G, Khan I, Salam A. Maximum entropy based cal second-harmonic microscopy, crossed-beam summa- image segmentation of human skin lesion. Int J Comput Control tion, and small-angle scattering in rat-tail tendon. Biophys J. Quantum Inf Eng. 2015;9(5):667–71. 1986;50(4):693–712. https:// doi. org/ 10. 1016/ S0006- 3495(86) Kirsch RA. Computer determination of the constituent structure of 83510-X. biological images. Comput Biomed Res. 1971;4(3):315–28. Fujimoto JG, Pitris C, Boppart SA, Brezinski ME. Optical coherence Kittler J, Illingworth J. Minimum error thresholding. Pattern Recogn. tomography: an emerging technology for biomedical imaging 1986;19(1):41–7. and optical biopsy. Neoplasia. 2000;2(1–2):9–25. Koch RG, Tsamis A, D’Amore A, et al. A custom image-based analysis Gade P. S, Robertson A. M, Chuang C-Y. Multiphoton Imaging of tool for quantifying elastin and collagen micro-architecture in Collagen Elastin and Calcification in Intact Soft-Tissue Samples. the wall of the human aorta from multi-photon microscopy. J Current Protocols in Cytometry. 2019;87(1) Biomech. 2014;47(5):935–43. Germann JA, Martinez-Enriquez E, Marcos S. Quantization of Col- Koehler MJ, König K, Elsner P, Bückle R, Kaatz M. In vivo assessment lagen organization in the stroma with a new order coefficient. of human skin aging by multiphoton laser scanning tomography. Biomed Opt Express. 2018;9(1):173–89. Opt Lett. 2006;31(19):2879. Gonzalez RC, Woods RE. Digital image processing. 4th ed. New York: Kroger M, Schleusener J, Jung S, Darvin ME. Characterization Pearson; 2018. of collagen I fiber thickness, density, and orientation in the Grossmann A, Morlet J. Decomposition of hardy functions into square human skin in vivo using second-harmonic generation imag- integrable wavelets of constant shape. SIAM J Math Anal. ing. Photonics. 2021;8:404. https://doi. or g/10. 3390/ pho tonics8 1984;15(4):723–36. 090404. Haralick RM, Shanmugam K, Dinstein I. Textural features Lau TY, Ambekar R, Toussaint KC. Quantification of collagen for image classification. IEEE Trans Syst Man Cybern. fiber organization using three-dimensional Fourier trans- 1973;SMC-3(6):610–21. form-second-harmonic generation imaging. Opt Express. Hill MR, Duan X, Gibson GA, Watkins S, Robertson AM. A theoretical 2012;20(19):21821–32. and non-destructive experimental approach for direct inclusion Leavers VF. Shape detection in computer vision using the hough trans- of measured collagen orientation and recruitment into mechani- form. London: Springer; 1992. cal models of the artery wall. J Biomech. 2012;45(5):762–71. Lee SL, Chen YF, Dong CY. Second harmonic generation imaging Holzapfel G. A, Gasser T. C, Ogden R. W. A new constitutive frame- reveals asymmetry in the rotational helicity of collagen lamellae work for arterial wall mechanics and a comparative study of in chicken corneas. Biomed Opt Express. 2019;10(10):5223–34. material models. J Elast. 2000;61:1–48. https:// doi. or g/ 10. Ley C, Verdebout T. Modern directional statistics. Boca Raton: CRC 1023/A: 10108 35316 564. Press; 2017. 1 3 Research on Biomedical Engineering Lin SJ, Wu RJ, Tan HY, et al. Evaluating cutaneous photoaging by use Pijanka JK, Markov PP, Midgett D, et al. Quantification of collagen fiber of multiphoton fluorescence and second-harmonic generation structure using second harmonic generation imaging and two- microscopy. Opt Lett. 2005;30(17):2275–7. dimensional discrete Fourier transform analysis: application to the Liu Z, Speroni L, Quinn KP, et al. 3D organizational mapping of col- human optic nerve head. J Biophotonics. 2019;12(5): e201800376. lagen fibers elucidates matrix remodeling in a hormone-sensitive Pizer SM, Amburn EP, Austin JD, et al. Adaptive histogram equali- 3D breast tissue model. Biomaterials. 2018;179:96–108. zation and its variations. Comput Vis Graphics Image Process. Lodish H, Berk A, Zipursky SL, Matsudaira P, Baltimore D, Darnell J. 1987;39:355–68. Molecular cell biology. 4th ed. New York: W.H. Freeman; 2000. Polzer S, Gasser TC, Forsell C, et al. Automatic identification and Lowe JS, Anderson PG, Stevens A. Stevens & Lowe’s human histology, validation of planar collagen organization in the aorta wall 4th ed. Philadelphia: Elsevier/Mosby: 2015. with application to abdominal aortic aneurysm. Microsc Luo T, Chen H, Kassab GS. Resliced image space construction for Microanal. 2013;19(6):1395–404. coronary artery collagen fibers. PLoS ONE. 2017;12(9):1–15. Prado IM, Di Dio LJ, Miranda-Neto MH, et al. Distribution of col- Luthon F, Liévin M, Faux F. On the use of entropy power for thresh- lagen fibers in the aggregated lymphoid follicles of swine ter - old selection. Signal Process. 2004;84(10):1789–804. minal ileum. Ann of Anat. 2003;185(1):73–80. Malladi R, Sethian J. Level set and fast marching methods in image Prewitt JMS. Object enhancement and extraction. In: Lipkin BS, processing and computer vision. Proceedings of 3rd IEEE editor. Picture processing and psychopictorics. Cambridge: International Conference on Image Processing; 1996 Sep 19; Academic Press; 1970. p. 75–149. Lausanne, Switzerland. IEEE, 1996. Rader C. Discrete Fourier transforms when the number of data sam- Mancas M, Gosselin B, Macq B. Segmentation using a region ples is prime. Proc IEEE. 1968;56(6):1107–8. growing thresholding. Image Process Algorithms Syst IV. Rao RAR, Mehta MR, Toussaint KC. Fourier transform-second-har- 2006;5672:388–98. monic generation imaging of biological tissues. Opt Express. Mardia KV, Jupp PE. Directional statistics. New York: J. Wiley; 2009;17(17):14534–42. 2000. Rivard G. Direct fast Fourier transform of bivariate functions. IEEE Mardia KV, Hughes G, Taylor CC, Singh H. A multivariate Von Trans Acoust Speech Signal Process. 1977;25(3):250–2. Mises distribution with applications to bioinformatics. Cana- Robinson BK, Cortes E, Rice AJ, Sarper M, del Rίo HA. Quantita- dian Journal of Statistics. 2008;36(1):99–109. tive analysis of 3D extracellular matrix remodelling by pancre- McLean J. A linear method for quantification of collagen fiber orien- atic stellate cells. Biol Open. 2016;5(6):875–82. tation in cornea. 41st Annual Northeast Biomedical Engineer- Rosen S, Brisson BK, Durham AC, et al. Intratumoral collagen sig- ing Conference (NEBEC). 2015;1–2. natures predict clinical outcomes in feline mammary carci- Mega Y, Robitaille M, Zareian R, McLean J, Ruberti J, DiMarzio noma. PLoS ONE. 2020;15(8): e0236516. C. Quantification of lamellar orientation in corneal col- Schmarje M, Zelenka C, Geisen U, Glüer CC, Koch R. 2D and 3D lagen using second harmonic generation images. Opt Lett. segmentation of uncertain local collagen fiber orientations in 2012;37(16):3312–4. SHG microscopy, arXiv: 1907. 12868 [cs], 2019;11824:374– Meijering E, Jacob M, Sarria JC, Steiner P, Hirling H, Unser M. Design 386. https:// doi. org/ 10. 1007/ 978-3- 030- 33676-9_ 26. and validation of a tool for neurite tracing and analysis in fluo- Schriefl AJ, Wolinski H, Regitnig P, Kohlwein SD, Holzapfel GA. rescence microscopy images. Cytometry. 2004;58A(2):167–76. An automated approach for three-dimensional quantification of Morin C, Hellmich C, Nejim Z, Avril S. Fiber rearrangement and fibrillar structures in optically cleared soft biological tissues. J matrix compression in soft tissues: multiscale hypoelasticity and Rl Soc Interface. 2013;10:20120760. application to tendon. Front Bioeng Biotechnol. 2021;9:725047. Senthilkumaran N, Rajesh R. Edge detection techniques for image https:// doi. org/ 10. 3389/ fbioe. 2021. 725047. segmentation – a survey of soft computing approaches. Int J Mustafa WA, Abdul Kader MMM. A review of histogram equalization Recent Trends Eng. 2009;1(2):250–4. techniques in image enhancement application. J Phys: Conf Ser. Simoncelli E, Farid H. Steerable wedge filters for local orientation 2018;1019: 012026. analysis. IEEE Trans Image Process. 1996;5(9):1377–82. Niestrawska JA, Viertler C, Regitnig P, Cohnert TU, Sommer G, Singer KD, Wu Y. Second harmonic generation (shg) as a charac- Holzapfel GA. Microstructure and mechanics of healthy and terization technique and phenomological probe for organic aneurysmatic abdominal aortas: experimental analysis and mod- materials. In: Ostroverkhova O, editor. Handbook of organic elling. J Rl Soc Interface. 2016;13:20160620. materials for optical and (opto)electronic devices. Sawston: Ohki M, Zervakis ME, Venetsanopoulos AN. 3-D digital filters. In: Woodhead Publishing Limited; 2013:442–69. https:// doi. org/ Leondes CT, editor. Control and dynamic systems. London: Aca-10. 1533/ 97808 57098 764. demic Press; 1995;69:49–88. Sivaguru M, Durgam S, Ambekar R, et al. Quantitative analysis of Orberg JW, Klein L, Hiltner A. Scanning electron microscopy of col- collagen fiber organization in injured tendons using Fourier lagen fibers in intestine. Connect Tissue Res. 1982;9(3):187–93. transform-second harmonic generation imaging. Opt Express. Osher S, Sethian JA. Fronts propagating with curvature-dependent 2010;18(24):24983–93. speed: algorithms based on Hamilton-Jacobi formulations. J So PTC. Two-photon light microscopy. In: Encyclopedia of life sci- Comput Phys. 1988;79:12–49. ences. New York: Macmillan Publishers Ltd; 2002. Otsu N. A threshold selection method from gray-level histograms. Sobel I. An isotropic 3x3 image gradient operator. Presentation at IEEE Trans Syst Man Cybern. 1979;9(1):62–6. Stanford A.I. Project. 1968. Pearson KLIII. On lines and planes of closest fit to systems of Starck JL, Candès EJ, Donoho DL. The curvelet transform for image points in space. London Edinburgh Dublin Philos Mag J Sci. denoising. IEEE Trans Image Process. 2002;11(6):670–84. 1901;2(11):559–72. Stein AM, Vader DA, Jawerth LM, Weitz DA, Sander LM. An algo- Pewsey A, Garcίa-Portugués E. Recent advances in directional statis- rithm for extracting the network geometry of three-dimensional tics. arXiv: 2005. 06889 [stat]. 2020. collagen gels. J Microsc. 2008;232(3):463–75. Phillippi JA, Green BR, Eskay MA, et al. Mechanism of aortic medial Strupler M, Pena AM, Hernest M, et al. Second harmonic imag- matrix remodeling is distinct in patients with bicuspid aortic ing and scoring of collagen in fibrotic tissues. Opt Express. valve. J Thorac Cardiovasc Surg. 2014;147(3):1056–64. 2007;15(7):4054–65. 1 3 Research on Biomedical Engineering Sugita S, Matsumoto T. Multiphoton microscopy observations of Wen BL, Brewer MA, Nadiarnykh O, et al. Texture analysis applied 3D elastin and collagen fiber microstructure changes during to second harmonic generation image data for ovarian cancer pressurization in aortic media. Biomech Model Mechanobiol. classification. J Biomed Opt. 2014;19(9): 096007. 2017;16:763–73. Wu J, Rajwa B, Filmer DL, et al. Quantification and reconstruc- Sun M, Bloom AB, Zaman MH. Rapid quantification of 3D collagen tion of collagen matrix from 3D confocal datasets. J Microsc. fiber alignment and fiber intersection correlations with high 2003;210(2):158–65. sensitivity. PLoS ONE. 2015;10(7): e0131814. Wu S, Li H, Yang H, Zhang X, Li Z, Xu S. Quantitative analysis on Svoboda K, Yasuda R. Principles of two-photon excitation microscopy collagen morphology in aging skin based on multiphoton micros- and its applications to neuroscience. Neuron. 2006;50(6):823–39. copy. J Biomed Opt. 2011;16(4): 040502. Tang SY, Monslow J, Todd L, Lawson J, Puré E, FitzGerald GA. Wu S, Peng Y, Hu L, Zhang X, Li H. Classification and recognition Cyclooxygenase-2 in endothelial and vascular smooth muscle of texture collagen obtaining by multiphoton microscope with cells restrains atherogenesis in hyperlipidemic mice. Circulation. neural network analysis. J Phys: Conf Ser. 2016;680: 012014. 2014;129(17):1761–9. Yamada M, Lin LL, Prow TW. Multiphoton microscopy applications Theodossiou TA, Thrasivoulou C, Ekwobi C, Becker DL. Second har- in biology. In: Conn M, Anda Cornea P, editors. Fluorescence monic generation confocal microscopy of collagen type I from rat microscopy. Cambridge: Academic Press; 2014. p. 185–97. tendon cryosections. Biophys J. 2006;91(12):4665–77. Yeh AT, Nassif N, Zoumi A, Tromberg BJ. Selective corneal imaging Thilagamani S, Shanthi N. A survey on image segmentation through using combined second-harmonic generation and two-photon clustering. Int J Res Rev Inf Sci. 2011;1(1):14–7. excited fluorescence. Opt Lett. 2002;27(23):2082–4. Tilbury K, Hocker J, Wen BL, Sandbo N, Singh V, Campagnola PJ. Yuheng S, Hao Y. Image segmentation algorithms overview. ArXiv, Second harmonic generation microscopy analysis of extracel- vol. Abs/1707.02051. 2017. lular matrix changes in human idiopathic pulmonary fibrosis. J Zeinali-Davarani S, Chow MJ, Turcotte R, Zhang Y. Characterization Biomed Opt. 2014;19(8): 086014. of biaxial mechanical behavior of porcine aorta under gradual Tizhoosh HR. Image thresholding using type II fuzzy sets. Pattern elastin degradation. Ann Biomed Eng. 2013;41(7):1528–38. Recogn. 2005;38(12):2363–72. Zeitoune AA, Erbes LA, Casco VH, Adur JF. Improvement of co- Tjin G, Xu P, Kable SH, Kable EPW, Burgess JK. Quantification of occurrence matrix calculation and collagen fibers orientation collagen I in airway tissues using second harmonic generation. J estimation. Proc. SPIE 10160, 12th International Symposium Biomed Opt. 2014;19(3): 036005. on Medical Information Processing and Analysis. 2017;101601B. Tsai WH. Moment-preserving thresolding: a new approach. Comput https:// doi. org/ 10. 1117/ 12. 22567 21. Vis Graphics Image Process. 1985;29(3):377–93. Zhou ZH, Ji CD, Xiao HL, Zhao HB, Cui YH, Bian XW. Reorganized Ugryumova N, Jacobs J, Bonesi M, Matcher S. Novel optical imaging collagen in the tumor microenvironment of gastric cancer and its technique to determine the 3-D orientation of collagen fibers in association with prognosis. J Cancer. 2017;8(8):1466–76. cartilage: variable-incidence angle polarization-sensitive opti- Zipfel WR, Williams RM, Webb WW. Nonlinear magic: multiphoton cal coherence tomography. Osteoarthr Cartil. 2009;17(1):33–42. microscopy in the biosciences. Nat Biotechnol. 2003;21(11):1369–77. Wagenseil JE, Mecham RP. Vascular extracellular matrix and arterial https:// doi. org/ 10. 1038/ nbt899. mechanics. Physiol Rev. 2009;89(3):957–89. Zyablitskaya M, Takaoka A, Munteanu EL, Nagasaki T, Trokel SL, Walton LA, Bradley RS, Withers PJ, et al. Morphological charac- Paik DC. Evaluation of Therapeutic Tissue Crosslinking (TXL) terization of unstained and intact tissue micro-architecture for Myopia using second harmonic generation signal microscopy by X-ray computed micro- and nano-tomography. Sci Rep. in rabbit sclera. Investigative Opthalmology & Visual Science. 2015;5(1):10074. https:// doi. org/ 10. 1038/ srep1 0074. 2017;58:21–9. Wegner KA, Keikhosravi A, Eliceiri KW, Vezina CM. Fluorescence of picrosirius red multiplexed with immunohistochemistry for the Publisher's note Springer Nature remains neutral with regard to quantitative assessment of collagen in tissue sections. J Histo- jurisdictional claims in published maps and institutional affiliations. chem Cytochem. 2017;65(8):479–90. 1 3
Journal
Research on Biomedical Engineering – Springer Journals
Published: Mar 1, 2023
Keywords: Information extraction; Image processing; Segmentation; Orientation; Density; Fiber morphology