Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Learning-based three-dimensional registration with weak bounding box supervision

Learning-based three-dimensional registration with weak bounding box supervision Abstract.Purpose: Image registration is the process of aligning images, and it is a fundamental task in medical image analysis. While many tasks in the field of image analysis, such as image segmentation, are handled almost entirely with deep learning and exceed the accuracy of conventional algorithms, currently available deformable image registration methods are often still conventional. Deep learning methods for medical image registration have recently reached the accuracy of conventional algorithms. However, they are often based on a weakly supervised learning scheme using multilabel image segmentations during training. The creation of such detailed annotations is very time-consuming.Approach: We propose a weakly supervised learning scheme for deformable image registration. By calculating the loss function based on only bounding box labels, we are able to train an image registration network for large displacement deformations without using densely labeled images. We evaluate our model on interpatient three-dimensional abdominal CT and MRI images.Results: The results show an improvement of ∼10  %   (for CT images) and 20% (for MRI images) in comparison to the unsupervised method. When taking into account the reduced annotation effort, the performance also exceeds the performance of weakly supervised training using detailed image segmentations.Conclusion: We show that the performance of image registration methods can be enhanced with little annotation effort using our proposed method. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Medical Imaging SPIE

Learning-based three-dimensional registration with weak bounding box supervision

Download PDF
10 pages

Loading next page...
 
/lp/spie/learning-based-three-dimensional-registration-with-weak-bounding-box-9xvSbhfmr0
Publisher
SPIE
Copyright
© 2022 Society of Photo-Optical Instrumentation Engineers (SPIE)
ISSN
2329-4302
eISSN
2329-4310
DOI
10.1117/1.jmi.9.4.044001
Publisher site
See Article on Publisher Site

Abstract

Abstract.Purpose: Image registration is the process of aligning images, and it is a fundamental task in medical image analysis. While many tasks in the field of image analysis, such as image segmentation, are handled almost entirely with deep learning and exceed the accuracy of conventional algorithms, currently available deformable image registration methods are often still conventional. Deep learning methods for medical image registration have recently reached the accuracy of conventional algorithms. However, they are often based on a weakly supervised learning scheme using multilabel image segmentations during training. The creation of such detailed annotations is very time-consuming.Approach: We propose a weakly supervised learning scheme for deformable image registration. By calculating the loss function based on only bounding box labels, we are able to train an image registration network for large displacement deformations without using densely labeled images. We evaluate our model on interpatient three-dimensional abdominal CT and MRI images.Results: The results show an improvement of ∼10  %   (for CT images) and 20% (for MRI images) in comparison to the unsupervised method. When taking into account the reduced annotation effort, the performance also exceeds the performance of weakly supervised training using detailed image segmentations.Conclusion: We show that the performance of image registration methods can be enhanced with little annotation effort using our proposed method.

Journal

Journal of Medical Imaging – SPIE

Published: Jul 1, 2022

Keywords: deformable image registration; weak supervision; deep learning

References

  • Temporal subtraction CT with nonrigid image registration improves detection of bone metastases by radiologists: results of a large-scale observer study
    Onoue, K.
  • Longitudinal bone microarchitectural changes are best detected using image registration
    Kemp, T. D.
  • Large deformation three-dimensional image registration in image-guided radiation therapy
    Foskey, M.
  • Automatic construction of multiple-object three-dimensional statistical shape models: application to cardiac modeling
    Frangi, A. F.
  • Automatic multi-resolution shape modeling of multi-organ structures
    Cerrolaza, J. J.
  • Fast free-form deformation using graphics processing units
    Modat, M.
  • Towards realtime multimodal fusion for image-guided interventions using self-similarities
    Heinrich, M. P.
  • A toolbox for intensity-based medical image registration
    Klein, S.
  • U-Net: convolutional networks for biomedical image segmentation
    Ronneberger, O.; Fischer, P.; Brox, T.
  • nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
    Isensee, F.
  • 3D U-Net: learning dense volumetric segmentation from sparse annotation
    Çiçek, Ö.
  • A deep learning framework for unsupervised affine and deformable image registration
    de Vos, B. D.
  • Pulmonary CT registration through supervised learning with convolutional neural networks
    Eppenhof, K. A.; Pluim, J. P.
  • GDL-fire 4D: deep learning-based fast 4D CT image registration
    Sentker, T.; Madesta, F.; Werner, R.
  • Architecture matters: evaluating design choices for deep learning registration networks
    Siebert, H.; Hansen, L.; Heinrich, M.
  • Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning
    Hering, A.
  • Large deformation diffeomorphic image registration with Laplacian pyramid networks
    Mok, T. C.; Chung, A. C.
  • Deep learning-based concurrent brain registration and tumor segmentation
    Estienne, T.
  • Enhancing label-driven deep deformable image registration with local distance metrics for state-of-the-art cardiac motion tracking,
    Hering, A.
  • Weakly supervised instance segmentation using the bounding box tightness prior
    Hsu, C. C.
  • Bounding boxes for weakly supervised segmentation: global constraints get close to full supervision
    Kervadec, H.
  • Deepcut: object segmentation from bounding box annotations using convolutional neural networks
    Rajchl, M.
  • Memory-efficient 2.5 D convolutional transformer networks for multi-modal deformable registration with weak label supervision applied to whole-heart CT and MRI scans
    Hering, A.
  • Non-parametric discrete registration with convex optimisation
    Heinrich, M. P.
  • Learn2Reg: 2020 MICCAI registration challenge
    Hansen, L.
  • Evaluation of six registration methods for the human abdomen on clinically acquired CT
    Xu, Z.
  • CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation
    Kavur, A. E.