Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Intraoperative Registration of 2D C-arm Images with Preoperative CT Data in Computer Assisted Spine Surgery: Motivation to Use Convolutional Neural Networks for Initial Pose Generator

Intraoperative Registration of 2D C-arm Images with Preoperative CT Data in Computer Assisted... Current Directions in Biomedical Engineering 2019;5(1):25-28 Julio Alvarez-Gomez*, Hubert Roth, Jürgen Wahrburg Intraoperative Registration of 2D C-arm Images with Preoperative CT Data in Computer Assisted Spine Surgery: Motivation to Use Convolutional Neural Networks for Initial Pose Generator https://doi.org/10.1515/cdbme-2019-0007 Abstract: In this paper, we present an approach for getting an 1 Introduction initial pose to use in a 2D/3D registration process for computer-assisted spine surgery. This is an iterative process 2D to 3D registration algorithms have been studied intensively that requires an initial pose close to the actual final pose. When based on intensity-base and stochastic-base methods. We use using a proper initial pose, we get registrations within two preoperative CT-Scans taken for planning and register them millimeters of accuracy. Consequently, we developed a fully using intraoperative C-arm images in our research on spine connected neural network (FCNN), which predicts the pose of surgery and implantation of pedicle screws. Previous works a specific 2D image within an acceptable range. Therefore, we using multimodality images have found that the initial guess can use this result as the initial pose for the registration used for the registration has a high impact on the outcome process. However, the inability of the FCNN for learning [1][2]. We experience the same phenomena when carrying out spatial attributes, and the decrease of the resolution of the some tests using two different lumbar spine phantoms from images before inserting them in the FCNN, make the variance whom we took CT-Scans and X-ray images. Using of the prediction large enough to make some of the predictions measurement methods and optimizing cost functions, we have entirely out of the acceptable range. Additionally, new achieved registrations that result in errors within ±2mm when researches in deep learning field have shown that the initial pose was no more than ±5° and ±10mm away in each convolutional neural networks (CNNs) offer high advantages axis respecting to the actual pose. At this point in our study, when the inputs of the net are images. We consider that using the initial pose is chosen manually, but it is necessary to CNNs can help to improve our results, generalizing the system remove user intervention to the minimum extent. for a greater variety of inputs, and facilitating the integration Consequently, using an image processing method, that gives a with our current workflow. Then we present an outline for a pose within our critical range, is our next step to support CNN for our application, and some further steps we need to intraoperative navigation based on CT-based planning. For complete to achieve this implementation. automating this process, we gathered a set of X-ray images of Keywords: 2D/3D registration, intraoperative registration, both phantoms from whom we know the C-arm poses. Starting computer assisted surgery, spine surgery, neural network, for predicting the rotation, we split the data set in training and convolutional neural network. testing sets, and used them for training and testing the fully connected neural network. The inputs of the FCNN were the https://doi.org/10.1515/cdbme-2019-XXXX X-ray image and CT-scans lateral and anteroposterior projection. We got pose predictions, which were within our range for being inputted as initial pose for the registration process, but others were completely out of range. Other ______ *Corresponding author: Julio Alvarez-Gomez: Institute of researches have proposed using CNNs [2] and Regression Control Engineering (RST) and Center for Sensor Systems learning [3] for developing the entire registration process (ZESS), University of Siegen, Siegen, Germany, e-mail: reporting high rates of successful registrations. In our case, to julio.agomez@uni-siegen.de. improve the prediction rate, and making a better Hubert Roth, Jürgen Wahrburg: Institute of Control Engineering generalization, we propose an approach using CNNs as a mean (RST) and Center for Sensor Systems (ZESS), University of Siegen, Siegen, Germany. Open Access. © 2019 Julio Alvarez-Gomez et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. J. Alvarez-Gomez et al., Intraoperative Registration of 2D C-arm Images with Preoperative CT Data — 26 for obtaining the initial pose for our registration procedure. 2.3 Data Preparation CNNs have been the most selected algorithm for computer Later, all of the images were registered, which means we vision since last years because they can learn autonomously computed the registration pose of all of them. We created the more features from input images than any other neural dataset file relating each image with its registration pose. networks [4]. That means making a better generalization over Moreover, the input images have a native resolution of a dataset and improving the results. Additionally, Python 568x568 pixels, but we reduced their resolution to 30x30 allows deploying trained CNN into C++, which makes the pixels using an average pooling layer. Later on, we used 70% predictor compatible with our current development. of the data for the training set and the remaining 30% as the testing set. 2 Materials and Methods 2.4 Defining the FCNN 2.1 Software Tools Stating the fact that we do not have enough samples for training a deep neural network, we decided to go for a fully We are working under the Mevislab framework, which connected neural network because it requires less parameters includes a digital reconstructed radiography (DRR) module. It to be computed. The reduction of input images resolution was has as user inputs the positions and orientation of the DICOM carried out consequently to keep a balance between the dataset volume respecting to the rendering element. We then called size and the number of weights to train, i.e., to avoid registration pose to the position and orientation obtained after underfitting. The inputs of the FCNN are one actual X-ray the registration process. It consists of 6 degrees of freedom image, and the lateral and AP images of the respective DICOM (DOF), three in translation {x,y,z} and three in rotation {αz, set created by the DRR module. All in all, our FCNN has 2700 αx, αy}. The order of the rotations is defined as so by the DRR inputs (30x30 pixels per image by 3 images). In the beginning, module, which applies a Eulerian ZXY rotation. From the we defined the outputs of the FCNN as the three angles of the perspective of the render interface, the last rotation is registration pose, but we decided to simplify them. Having performed over the rendered image after applying the Z and X only the predictions of the Z and X angles, we could render a rotation. In other words, once the render of Z and X rotation is DRR image and estimate the Y angle by traditional image made, the resulting image is rotated, which completes the Y processing methods. Therefore, we have only the Z and X rotation. rotation as outputs of our FCNN. The activation function used For developing the FCNN, we used Matlab. There we between hidden layers was the sigmoid, the used cost function used some native libraries for reading DICOM files, our X-ray was mean squared error with regularization, and the used images. optimization function was conjugate gradients. For the training, we grouped the dataset as follows: 80% of the samples as the training set and the 20% remaining as the 2.2 Phantoms and C-arm testing set. There are no equations for defining the proper topology of In our laboratory, we have two lumbar phantoms. One of a FCNN, or how many neurons per layer must be used. There them contains the vertebra L2-L5, the other L1-L5 including are, nevertheless, some experimental rules of thumbs that the sacrum. From both phantoms, we have their CT-scans. In show some ranges of values, which could lead to good results. our facilities, it is available a Ziehm Vario 3D C-arm, which For this reason, we followed some recommendation about the was used to take about 2500 X-ray images. They were taken number of neurons depending on the number of layers. using the auto scanning mode of the C-arm, which allows to However, there is still some uncertainties in the number of take up to 120 images while the C-arm rotates 130° around its hidden layers, and the exact number of neurons in each center. Playing with the location of the Phantoms respecting to specific layer. the C-arm’s center as well as their orientation, we consider We addressed this problem by creating different every captured image unique. That means every image is topologies, with different amount of neurons in each layer and different from any other from the dataset in at least one comparing the error prediction in each of these networks, but element of its 6 DOF. avoiding deeper layers to have more neurons than outer layers. We ran the tests changing the number of neurons in topologies J. Alvarez-Gomez et al., Intraoperative Registration of 2D C-arm Images with Preoperative CT Data — 27 containing one, two, and three hidden layers. That means, we  2700 - H1 - H2 - H3 – 2 tested the following topologies: H1, H2 = 1000  2700 - H1 - 2 H3 {200, 300, 400, 500, 600, 700, 800}  2700 - H1 - H2 – 2  2700 - H1 - H2 - H3 – 2 In this case, we collected the best results in Table 2. We where: H1 ≥ H2 ≥ H3 noticed that both topologies with three and two hidden layers report similar results. However, it is significant to reduce the Additionally, each layer was changed in intervals of 200 number of neurons in our case because of the available number neurons moving from 200 until 1000, so of our training samples, so we selected the topology 2700- H1,H2,H3  {200, 400, 600, 800, 1000}. 1400-800-2 as the most suitable for our application. After finding the best layout, we changed the number of epochs used for each training over the best network and also Table 2: Errors of the best topologies with extended range. the regularization value to avoid overfitting the network. The -7 regularization term (λ) was moved from 3x10 until 3 with Topology Error Z° Error X° Average logarithmic increments of 0.5, so Error -7 -7 -6 -6 λ  {3x10 , 1x10 , 3x10 , 1x10 ,…0.3,1,3}. 2700-1900-2 3.23 4.49 3.86 The number of epochs, on the other hand, were started in 2700-1400-800-2 2.48 4.11 3.30 800 increasing to the next one in roughly 66%, so 2700-1000-1000-500-2 2.46 4.19 3.33 Epochs{0.8, 1.3 ,2 ,3 ,4.5 ,6.8 ,10 ,15 ,22 ,34}x10 3 Results 3.1 Finding the FCNN Topology The best topologies for each hidden layer are shown in Table 1. For the topology with three hidden layers, there were two topologies with closed values, therefore, both are displayed. Table 1: Errors for the best topologies. Topology Error Z° Error X° Average Figure 1: Testing set predictions of the FCNN 2700-1400-800-2 Error 2700-1000-2 3.23 5.11 4.17 2700-1000-1000-2 2.55 4.12 3.34 2700-1000-1000-400-2 2.47 4.44 3.46 3.2 Finding regularization term and 2700-1000-1000-800-2 2.71 4.22 3.47 number of Epochs We noticed that the best result was always the greatest With this topology, we moved through the range of the value of the range, so we decided to adjust the ranges as regularization and using the epochs described in section 2.4. follows: We gathered the best results in Table 3. From these, the  2700 - H1 - 2 selected values were the regularization term equals to 3 and H1 {800, 900, 1000,…1900, 2000} 15000 epochs for training. Figure 1 shows the prediction and actual angle of each sample over the testing set as well as its  -2700 - H1 - H2 – 2 full-scale error. We noticed how most of the error values lay H1 {1000, 1100, 1200, 1300, 1400} below the average line, but there are some peaks reaching 15% H2 {600, 800, 900, 1000} of error. These outliers are a risk for the outcome of our application. They may lead the following registration process J. Alvarez-Gomez et al., Intraoperative Registration of 2D C-arm Images with Preoperative CT Data — 28 to wrong results, and as a consequence, to perform inaccurate performance when it is used with a regularization coefficient spine procedures. It makes us consider a radical change in our of 3 and trained in a range of around 15000 epochs. topology by using instead a CNN. Our results using a FCNN lay within the range that can be used as the initial pose by our registration algorithm for spine surgery. The average errors were 1.98% in the Z-axis and Table 3: Errors of the best regularization term and amount of 3.06% in X-axis, and when using the full-scale conversion, epochs of the topology 2700-1400-800-2. they are equivalent to 3.564° and 2.754 in Z- and X- axis respectively. However, the peaks of the errors reach 15%, Regularization Epochs Error Z° Error X° Average term (λ) which are equivalent to 27° and 13.5° so completely out of the Error acceptable range. Therefore, we consider that our next 3 15000 1.98 3.06 2.52 improvement will be using CNNs that have been reported to 3 22000 1.92 3.88 2.9 1 15000 2.17 3.52 2.84 produce better results with image processing. We expect the 1 22000 2.29 3.6 2.95 results with CNN to be more trustworthy and easy to integrate 1 34000 2.17 3.19 2.68 with our current workflow for computer-assisted spine surgery. Additionally, we want to increase our dataset size to have a better description of the problem, and also to train the network correctly. Such a task is time-consuming, yet it is the 4 Outlook for CNN main reason for our current dataset size. One of the reasons why the full-scale error of our results Author Statement does not decrease less than 2.68%, could be the reduction of Conflict of interest: Authors state no conflict of interest. the image resolution before inputting them to the FCNN. Informed consent: Informed consent has been obtained from Additionally, the FCNN can lack generalization regarding all individuals included in this study. Ethical approval: The spatial information of the image, i.e., the same image with the research related to human use complies with all the relevant same rotations but different translations could be wrongly national regulations, institutional policies and was performed predicted. For these reasons, we propose using a CNN instead in accordance with the tenets of the Helsinki Declaration, and our current FCNN for our initial pose generator. has been approved by the authors' institutional review board or Initially, images could be entered as full resolution and equivalent committee. reduce their resolution by using max-pooling layers. Additional convolutional kernels could be added to extract Acknowledgement: Part of this work is funded by the features. The final stage of the CNN would be a FCNN for German Federal Ministry of Education and Research (KMU- doing the final prediction. innovativ: contract number 13GW0175B) For training a bigger net, we will have to use a larger dataset. As a rule of thumb, a deep learning algorithm typically requires 5000 samples for having acceptable performance [5]. References Therefore, we must use some data augmentation techniques suitable for deep learning [6]. However, we must carefully [1] Miao S, Liao J, Lucas J, et al. Toward accurate and robust 2- evaluate them because applying a common technique as image D/3-D registration of implant models to single-plane fluoroscopy, in Augmented Reality Environments for Medical rotation will spoil the results instead of improving them. Imaging Computer-Assisted Interventions, 2013. Finally, once the CNN is trained and validated, it can be [2] Miao S, Wang Jane, Liao R, A CNN Regression Aprroach for deployed in C++, which is the language used for our Real-Time 2D/3D Registrtation, in IEEE Transactions on framework, and where the planning software for the spine Medical Imaging, Vol. 35, No. 5. 2016. surgery is done. [3] C. Chou, B. Frederick, et al. 2D/3D Image Registration Using Regression Learning. Comput Vis Image Underst. 2013 September 1; 117(9): 1095–1106. [4] Chollet F, Deep Learning with Python, Manning Publication Co. 2018 5 Discussion and Conclusion [5] Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press. We found out that for our application in spine surgeries, a [6] Perez L, Wang J. The Effectiveness of Data Augmentation in FCNN with a topology 2700-1400-800-2 gives the best Image Classification using Deep Learning. CoRR abs/1712.04621 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Current Directions in Biomedical Engineering de Gruyter

Intraoperative Registration of 2D C-arm Images with Preoperative CT Data in Computer Assisted Spine Surgery: Motivation to Use Convolutional Neural Networks for Initial Pose Generator

Download PDF
4 pages

Loading next page...
 
/lp/de-gruyter/intraoperative-registration-of-2d-c-arm-images-with-preoperative-ct-Lg952d0wgC
Publisher
de Gruyter
Copyright
© 2019 by Walter de Gruyter Berlin/Boston
eISSN
2364-5504
DOI
10.1515/cdbme-2019-0007
Publisher site
See Article on Publisher Site

Abstract

Current Directions in Biomedical Engineering 2019;5(1):25-28 Julio Alvarez-Gomez*, Hubert Roth, Jürgen Wahrburg Intraoperative Registration of 2D C-arm Images with Preoperative CT Data in Computer Assisted Spine Surgery: Motivation to Use Convolutional Neural Networks for Initial Pose Generator https://doi.org/10.1515/cdbme-2019-0007 Abstract: In this paper, we present an approach for getting an 1 Introduction initial pose to use in a 2D/3D registration process for computer-assisted spine surgery. This is an iterative process 2D to 3D registration algorithms have been studied intensively that requires an initial pose close to the actual final pose. When based on intensity-base and stochastic-base methods. We use using a proper initial pose, we get registrations within two preoperative CT-Scans taken for planning and register them millimeters of accuracy. Consequently, we developed a fully using intraoperative C-arm images in our research on spine connected neural network (FCNN), which predicts the pose of surgery and implantation of pedicle screws. Previous works a specific 2D image within an acceptable range. Therefore, we using multimodality images have found that the initial guess can use this result as the initial pose for the registration used for the registration has a high impact on the outcome process. However, the inability of the FCNN for learning [1][2]. We experience the same phenomena when carrying out spatial attributes, and the decrease of the resolution of the some tests using two different lumbar spine phantoms from images before inserting them in the FCNN, make the variance whom we took CT-Scans and X-ray images. Using of the prediction large enough to make some of the predictions measurement methods and optimizing cost functions, we have entirely out of the acceptable range. Additionally, new achieved registrations that result in errors within ±2mm when researches in deep learning field have shown that the initial pose was no more than ±5° and ±10mm away in each convolutional neural networks (CNNs) offer high advantages axis respecting to the actual pose. At this point in our study, when the inputs of the net are images. We consider that using the initial pose is chosen manually, but it is necessary to CNNs can help to improve our results, generalizing the system remove user intervention to the minimum extent. for a greater variety of inputs, and facilitating the integration Consequently, using an image processing method, that gives a with our current workflow. Then we present an outline for a pose within our critical range, is our next step to support CNN for our application, and some further steps we need to intraoperative navigation based on CT-based planning. For complete to achieve this implementation. automating this process, we gathered a set of X-ray images of Keywords: 2D/3D registration, intraoperative registration, both phantoms from whom we know the C-arm poses. Starting computer assisted surgery, spine surgery, neural network, for predicting the rotation, we split the data set in training and convolutional neural network. testing sets, and used them for training and testing the fully connected neural network. The inputs of the FCNN were the https://doi.org/10.1515/cdbme-2019-XXXX X-ray image and CT-scans lateral and anteroposterior projection. We got pose predictions, which were within our range for being inputted as initial pose for the registration process, but others were completely out of range. Other ______ *Corresponding author: Julio Alvarez-Gomez: Institute of researches have proposed using CNNs [2] and Regression Control Engineering (RST) and Center for Sensor Systems learning [3] for developing the entire registration process (ZESS), University of Siegen, Siegen, Germany, e-mail: reporting high rates of successful registrations. In our case, to julio.agomez@uni-siegen.de. improve the prediction rate, and making a better Hubert Roth, Jürgen Wahrburg: Institute of Control Engineering generalization, we propose an approach using CNNs as a mean (RST) and Center for Sensor Systems (ZESS), University of Siegen, Siegen, Germany. Open Access. © 2019 Julio Alvarez-Gomez et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. J. Alvarez-Gomez et al., Intraoperative Registration of 2D C-arm Images with Preoperative CT Data — 26 for obtaining the initial pose for our registration procedure. 2.3 Data Preparation CNNs have been the most selected algorithm for computer Later, all of the images were registered, which means we vision since last years because they can learn autonomously computed the registration pose of all of them. We created the more features from input images than any other neural dataset file relating each image with its registration pose. networks [4]. That means making a better generalization over Moreover, the input images have a native resolution of a dataset and improving the results. Additionally, Python 568x568 pixels, but we reduced their resolution to 30x30 allows deploying trained CNN into C++, which makes the pixels using an average pooling layer. Later on, we used 70% predictor compatible with our current development. of the data for the training set and the remaining 30% as the testing set. 2 Materials and Methods 2.4 Defining the FCNN 2.1 Software Tools Stating the fact that we do not have enough samples for training a deep neural network, we decided to go for a fully We are working under the Mevislab framework, which connected neural network because it requires less parameters includes a digital reconstructed radiography (DRR) module. It to be computed. The reduction of input images resolution was has as user inputs the positions and orientation of the DICOM carried out consequently to keep a balance between the dataset volume respecting to the rendering element. We then called size and the number of weights to train, i.e., to avoid registration pose to the position and orientation obtained after underfitting. The inputs of the FCNN are one actual X-ray the registration process. It consists of 6 degrees of freedom image, and the lateral and AP images of the respective DICOM (DOF), three in translation {x,y,z} and three in rotation {αz, set created by the DRR module. All in all, our FCNN has 2700 αx, αy}. The order of the rotations is defined as so by the DRR inputs (30x30 pixels per image by 3 images). In the beginning, module, which applies a Eulerian ZXY rotation. From the we defined the outputs of the FCNN as the three angles of the perspective of the render interface, the last rotation is registration pose, but we decided to simplify them. Having performed over the rendered image after applying the Z and X only the predictions of the Z and X angles, we could render a rotation. In other words, once the render of Z and X rotation is DRR image and estimate the Y angle by traditional image made, the resulting image is rotated, which completes the Y processing methods. Therefore, we have only the Z and X rotation. rotation as outputs of our FCNN. The activation function used For developing the FCNN, we used Matlab. There we between hidden layers was the sigmoid, the used cost function used some native libraries for reading DICOM files, our X-ray was mean squared error with regularization, and the used images. optimization function was conjugate gradients. For the training, we grouped the dataset as follows: 80% of the samples as the training set and the 20% remaining as the 2.2 Phantoms and C-arm testing set. There are no equations for defining the proper topology of In our laboratory, we have two lumbar phantoms. One of a FCNN, or how many neurons per layer must be used. There them contains the vertebra L2-L5, the other L1-L5 including are, nevertheless, some experimental rules of thumbs that the sacrum. From both phantoms, we have their CT-scans. In show some ranges of values, which could lead to good results. our facilities, it is available a Ziehm Vario 3D C-arm, which For this reason, we followed some recommendation about the was used to take about 2500 X-ray images. They were taken number of neurons depending on the number of layers. using the auto scanning mode of the C-arm, which allows to However, there is still some uncertainties in the number of take up to 120 images while the C-arm rotates 130° around its hidden layers, and the exact number of neurons in each center. Playing with the location of the Phantoms respecting to specific layer. the C-arm’s center as well as their orientation, we consider We addressed this problem by creating different every captured image unique. That means every image is topologies, with different amount of neurons in each layer and different from any other from the dataset in at least one comparing the error prediction in each of these networks, but element of its 6 DOF. avoiding deeper layers to have more neurons than outer layers. We ran the tests changing the number of neurons in topologies J. Alvarez-Gomez et al., Intraoperative Registration of 2D C-arm Images with Preoperative CT Data — 27 containing one, two, and three hidden layers. That means, we  2700 - H1 - H2 - H3 – 2 tested the following topologies: H1, H2 = 1000  2700 - H1 - 2 H3 {200, 300, 400, 500, 600, 700, 800}  2700 - H1 - H2 – 2  2700 - H1 - H2 - H3 – 2 In this case, we collected the best results in Table 2. We where: H1 ≥ H2 ≥ H3 noticed that both topologies with three and two hidden layers report similar results. However, it is significant to reduce the Additionally, each layer was changed in intervals of 200 number of neurons in our case because of the available number neurons moving from 200 until 1000, so of our training samples, so we selected the topology 2700- H1,H2,H3  {200, 400, 600, 800, 1000}. 1400-800-2 as the most suitable for our application. After finding the best layout, we changed the number of epochs used for each training over the best network and also Table 2: Errors of the best topologies with extended range. the regularization value to avoid overfitting the network. The -7 regularization term (λ) was moved from 3x10 until 3 with Topology Error Z° Error X° Average logarithmic increments of 0.5, so Error -7 -7 -6 -6 λ  {3x10 , 1x10 , 3x10 , 1x10 ,…0.3,1,3}. 2700-1900-2 3.23 4.49 3.86 The number of epochs, on the other hand, were started in 2700-1400-800-2 2.48 4.11 3.30 800 increasing to the next one in roughly 66%, so 2700-1000-1000-500-2 2.46 4.19 3.33 Epochs{0.8, 1.3 ,2 ,3 ,4.5 ,6.8 ,10 ,15 ,22 ,34}x10 3 Results 3.1 Finding the FCNN Topology The best topologies for each hidden layer are shown in Table 1. For the topology with three hidden layers, there were two topologies with closed values, therefore, both are displayed. Table 1: Errors for the best topologies. Topology Error Z° Error X° Average Figure 1: Testing set predictions of the FCNN 2700-1400-800-2 Error 2700-1000-2 3.23 5.11 4.17 2700-1000-1000-2 2.55 4.12 3.34 2700-1000-1000-400-2 2.47 4.44 3.46 3.2 Finding regularization term and 2700-1000-1000-800-2 2.71 4.22 3.47 number of Epochs We noticed that the best result was always the greatest With this topology, we moved through the range of the value of the range, so we decided to adjust the ranges as regularization and using the epochs described in section 2.4. follows: We gathered the best results in Table 3. From these, the  2700 - H1 - 2 selected values were the regularization term equals to 3 and H1 {800, 900, 1000,…1900, 2000} 15000 epochs for training. Figure 1 shows the prediction and actual angle of each sample over the testing set as well as its  -2700 - H1 - H2 – 2 full-scale error. We noticed how most of the error values lay H1 {1000, 1100, 1200, 1300, 1400} below the average line, but there are some peaks reaching 15% H2 {600, 800, 900, 1000} of error. These outliers are a risk for the outcome of our application. They may lead the following registration process J. Alvarez-Gomez et al., Intraoperative Registration of 2D C-arm Images with Preoperative CT Data — 28 to wrong results, and as a consequence, to perform inaccurate performance when it is used with a regularization coefficient spine procedures. It makes us consider a radical change in our of 3 and trained in a range of around 15000 epochs. topology by using instead a CNN. Our results using a FCNN lay within the range that can be used as the initial pose by our registration algorithm for spine surgery. The average errors were 1.98% in the Z-axis and Table 3: Errors of the best regularization term and amount of 3.06% in X-axis, and when using the full-scale conversion, epochs of the topology 2700-1400-800-2. they are equivalent to 3.564° and 2.754 in Z- and X- axis respectively. However, the peaks of the errors reach 15%, Regularization Epochs Error Z° Error X° Average term (λ) which are equivalent to 27° and 13.5° so completely out of the Error acceptable range. Therefore, we consider that our next 3 15000 1.98 3.06 2.52 improvement will be using CNNs that have been reported to 3 22000 1.92 3.88 2.9 1 15000 2.17 3.52 2.84 produce better results with image processing. We expect the 1 22000 2.29 3.6 2.95 results with CNN to be more trustworthy and easy to integrate 1 34000 2.17 3.19 2.68 with our current workflow for computer-assisted spine surgery. Additionally, we want to increase our dataset size to have a better description of the problem, and also to train the network correctly. Such a task is time-consuming, yet it is the 4 Outlook for CNN main reason for our current dataset size. One of the reasons why the full-scale error of our results Author Statement does not decrease less than 2.68%, could be the reduction of Conflict of interest: Authors state no conflict of interest. the image resolution before inputting them to the FCNN. Informed consent: Informed consent has been obtained from Additionally, the FCNN can lack generalization regarding all individuals included in this study. Ethical approval: The spatial information of the image, i.e., the same image with the research related to human use complies with all the relevant same rotations but different translations could be wrongly national regulations, institutional policies and was performed predicted. For these reasons, we propose using a CNN instead in accordance with the tenets of the Helsinki Declaration, and our current FCNN for our initial pose generator. has been approved by the authors' institutional review board or Initially, images could be entered as full resolution and equivalent committee. reduce their resolution by using max-pooling layers. Additional convolutional kernels could be added to extract Acknowledgement: Part of this work is funded by the features. The final stage of the CNN would be a FCNN for German Federal Ministry of Education and Research (KMU- doing the final prediction. innovativ: contract number 13GW0175B) For training a bigger net, we will have to use a larger dataset. As a rule of thumb, a deep learning algorithm typically requires 5000 samples for having acceptable performance [5]. References Therefore, we must use some data augmentation techniques suitable for deep learning [6]. However, we must carefully [1] Miao S, Liao J, Lucas J, et al. Toward accurate and robust 2- evaluate them because applying a common technique as image D/3-D registration of implant models to single-plane fluoroscopy, in Augmented Reality Environments for Medical rotation will spoil the results instead of improving them. Imaging Computer-Assisted Interventions, 2013. Finally, once the CNN is trained and validated, it can be [2] Miao S, Wang Jane, Liao R, A CNN Regression Aprroach for deployed in C++, which is the language used for our Real-Time 2D/3D Registrtation, in IEEE Transactions on framework, and where the planning software for the spine Medical Imaging, Vol. 35, No. 5. 2016. surgery is done. [3] C. Chou, B. Frederick, et al. 2D/3D Image Registration Using Regression Learning. Comput Vis Image Underst. 2013 September 1; 117(9): 1095–1106. [4] Chollet F, Deep Learning with Python, Manning Publication Co. 2018 5 Discussion and Conclusion [5] Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press. We found out that for our application in spine surgeries, a [6] Perez L, Wang J. The Effectiveness of Data Augmentation in FCNN with a topology 2700-1400-800-2 gives the best Image Classification using Deep Learning. CoRR abs/1712.04621

Journal

Current Directions in Biomedical Engineering – de Gruyter

Published: Sep 1, 2019

Keywords: 2D/3D registration; intraoperative registration; computer assisted surgery; spine surgery; neural network; convolutional neural network

There are no references for this article.