Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Efficient Body Registration Using Single-View Range Imaging and Generic Shape Templates

Efficient Body Registration Using Single-View Range Imaging and Generic Shape Templates DE GRUYTER Current Directions in Biomedical Engineering 2020;6(3): 20203031 Tolga-Can Çallar*, Elmar Rueckert, and Sven Böttger Efficient Body Registration Using Single-View Range Imaging and Generic Shape Templates https://doi.org/10.1515/cdbme-2020-3031 teraction is the requirement of a patient body model for the definition and localization of task-specific anatomic target re- Abstract: Computer-aided medical systems, e.g. in the fields gions. Conventionally, established 3D imaging modalities with of medical robotics or image-based assistance, are continu- varying image quality, acquisition speed, and nocuous poten- ously investigated to overcome human limitations concerning tial, e.g. CT and MRI have been utilized in combination with perception, memory or dexterity. A common requirement of range imaging techniques to perform on-site registration of such systems is the availability of a digital model describ- body parts with previously acquired 3D image data [3]. How- ing the patient’s position and morphology during a procedure. ever, the associated health risks, procedural complexity and Operational complexity and technical limitations of estab- technical limitations of the image acquisition and registra- lished 3D imaging methods leave clinical settings in need of tion process render these approaches infeasible in cases where a method for the fast acquisition of a three-dimensional body large body surface coverage is required, visualization of inter- surface representation. For this purpose, we propose an unsu- nal anatomy is not desired or no prior suitable imaging stud- pervised and efficient body registration pipeline based on the ies have been done. Furthermore, unavoidable positional dif- markerless elastic registration and completion of single-view ferences between the subject in the source image and during stereo range images of the body surface with statistical para- registration may reduce the obtained model’s representative metric body shape templates. Initial results show a promising capacity regarding body pose and physiognomy at the mo- representative quality of the models generated through the reg- ment of a medical procedure. This encouraged us to inves- istration process with submillimetric fitting accuracy and real- tigate single-view range imaging with active infrared stereo istic surface morphology, indicating the general feasibility of cameras as a low-cost and stand-alone technique for this pur- our approach as an instant body registration method for auto- pose, due to its inherent advantages effectively mitigating the mated medical and biometric applications. mentioned drawbacks: The instant optical image acquisition Keywords: body registration, range imaging, 3D modeling, increases the temporal representative quality of acquired im- computer vision, medical robotics, automation age data while facilitating an efficient and innocuous regis- tration process. When compared to other range imaging tech- niques, e.g. ToF-cameras, infrared stereo imaging provides a 1 Introduction larger field of view and higher depth resolution at medium range, which is crucial for clinical applications. The major dif- The clinical success of surgical, diagnostic and therapeutic ficulty with this technology is the occurrence of optical occlu- procedures exhibits a high degree of dependency upon the in- sions restricting the retrievable body surface area from a given teractions and capabilities of medical personnel. Through the acquisition perspective. Proposed techniques to resolve this is- emergence of computer-aided medicine and medical automa- sue generally utilize multi-view acquisition settings to increase tion, many conceptual solutions for negative effects caused by the amount of captured surface area. Classically, overlapping user-dependency and human limitations of perception, cogni- partial scans are acquired and rigidly registered [4] to recon- tion, physical strength or dexterity have been developed [1]. struct a body model. To recover unobserved regions, promis- Currently studied use cases within this research field includ- ing results were achieved through the non-rigid registration ing robotic medical ultrasound [2] indicate improvements in of densely acquired full-body scans with template models [5] reliability and consistency of outcome when compared to con- or sparse surface data with individual shape priors in form ventional medical practice. A commonality of virtually all au- of handcrafted templates [6] or shape representations learned tomated medical systems employed for physical patient in- from subject-specific multi-view data [7]. As these approaches involve significant user-interaction and technical complexities *Corresponding author: Tolga-Can Çallar, Institute for Robotics decreasing the registration efficiency, we present a method for and Cognitive Systems, Universität zu Lübeck, Ratzeburger Allee the unsupervised elastic registration of single-view range im- 160, Lübeck, Germany, e-mail: ages with generic templates allowing the rapid and accurate tolgacan.callar@student.uni-luebeck.de digitization of the surface anatomy in clinical settings. Elmar Rueckert, Sven Böttger, Institute for Robotics and Cogni- tive Systems, Universität zu Lübeck, Germany Open Access. © 2020 Tolga-Can Callar et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. 2 T.-C. Çallar et al., Body Registration Using Single-View Range Imaging and Shape Templates 2.2 Body Segmentation 2 Methods To define the area of the range scan to be registered with the Our proposed approach for the registration of the body sur- template, we perform the segmentation of the subject in the face consists of three main elements: First, we acquire one triangulated point cloud of the scene. For this purpose we single-view range image of a subject on an examination couch implemented a background subtraction variant utilizing the with the low-cost Intel RealSense D415 active IR stereo cam- VTK-library: We generate k-d-tree-based space-partitionings era (Intel Corporation, Santa Clara, USA). Afterward, we per- of both the background scenery captured in advance and the form the segmentation of the subject’s body and the reduc- scene including the subject. On these, we perform a nearest- tion of range imaging artifacts. In the last step, the segmented neighbor search to identify those vertices in the subject’s scene body surface is elastically registered with a parametric tem- without any corresponding vertex from the background mesh plate body model, which has been modified in advance to re- in their vicinity. Due to optical occlusions causing regions with semble the subject’s anthropometric features. The segmenta- sparsely imaged steep depth gradients around border regions tion and registration processes of our framework are imple- of the body (see Figure 2), the segmentation result still con- mented using the 3D-processing software VTK (Kitware, Inc., tains artifacts visible in the form of local elongation of the Clifton Park, USA). triangular faces in these regions of the mesh. We exploit the large ratio between the perimeter of the triangular faces and their area to filter out the elongated faces. 2.1 Range Image Acquisition A range image of the complete scenery is captured from which the sections belonging to the subject are extracted and used as a target input for the registration of the body surface. The stereo camera is placed lateral to the examination couch and oriented to align the x-axis of the camera frame with the ex- Fig. 2: Segmented body surface with elongated triangular faces pected longitudinal axis of the body of the subject to be regis- caused by locally steep depth gradients. tered. The camera’s height and vertical tilt are adjusted, until the field of view spans the entire space to be occupied by the body, with the image frame being centered around the center 2.3 Template-Based Registration of the examination couch. In preparation for the subsequent segmentation, we first capture the scenery without the subject Regardless of optical occlusions, single-view range images al- and then with the subject in the supine position. As visual- ways retrieve less than half of the body surface. This renders ized in Figure 1, we acquire a frontal and oblique image for a solely segmentation-based approach insufficient for the spa- every subject to generate source data for evaluation of our reg- tial registration of complete 3D body surfaces. Therefore, we istration process under varying acquisition conditions. All of propose the elastic registration of a parameterized body shape the obtained range images are converted to point clouds and template onto the segmented range image to augment the exported as a polygonal mesh through the internal Delaunay- anatomically incomplete range image source while preserving triangulation function of the RealSense Viewer application. the captured anatomic details. To ensure the template’s mor- phological resemblance of the subject, we employ adjustable Frontal polygonal body models originating from statistical full-body laser scans from UMTRI Human Shape [8]. This allows us to create individual base shapes accommodating for the respec- tive anthropometric properties of a subject that may also allow for semantic pre-annotations, e.g. for task-specific target re- gions. We implemented an affine variation of the point set reg- istration algorithm ICP derived from the method described by [5]. Our algorithm iteratively optimizes transformations of the a b template to minimize the distance to the nearest-neighbor cor- Fig. 1: a Acquisition setup and orientation of the camera and body respondences between the template and target vertices while frame. b Utilized image acquisition perspectives. enforcing locally stiff deformations. In contrast to classical ICP we do not utilize global transformations on our template T.-C. Çallar et al., Body Registration Using Single-View Range Imaging and Shape Templates 3 model. Instead we define 𝑛 individual affine transformations mesh with the x-axis of the camera (see Figure 1), confining 𝑋 for every vertex 𝑣 ∈ 𝑉 of the template mesh 𝑇 . These the rotational difference to this axis. Exploiting the approxi- 𝑖 𝑖 are arranged in a matrix 𝑋 := [𝑋𝑖 . . .𝑋𝑛] representing the mate coincidence of the surface normal of the couch with the entirety of the vertex-wise transformations regularized by the body’s sagittal axis in the supine position, we infer the body distance to closest points on the target surface 𝒯 as expressed orientation from the average direction of the vertex normals by the energy term of the couch nearest to its intersection point with the camera’s ∑︁ z-axis. The template is rotated about its longitudinal axis by 𝐸 (𝑋 ) := 𝑤 dist (𝒯 ,𝑋 𝑣 ) , (1) 𝑑 𝑖 𝑖 𝑖 the angle Φ that the subject’s estimated sagittal axis deviates 𝑣 ∈𝑉 from the camera’s optical axis, which is equal to the acqui- where 𝑤 denotes a binary reliability weighting for the nearest- 𝑖 sition angle. The translational misalignment is resolved by neighbour correspondences. Two reliability tests are per- formed on every candidate vertex pair (𝑣 ∈ 𝑇 ,𝑣 ∈ 𝒯 ) after 𝑖 𝑗 finding the initial set of correspondences. A correspondence is rejected, if the template’s vertex normal 𝑛 exceeds an an- 𝑣 ∈𝑇 𝑖 ϕ gular deviation threshold of 𝜃 = 90° with respect to the closest target vertex normal 𝑛 , i.e. 𝑣 ∈𝒯 (︂ )︂ 𝑛 · 𝑛 𝑣 𝑣 𝑖 𝑗 arccos > 𝜃 , (2) ‖𝑛 ‖·‖ 𝑛 ‖ 𝑣 𝑣 𝑖 𝑗 a b c d or if the nearest template vertex is located on the border of Fig. 3: a Rotational alignment of the template through the acqui- sition angle Φ. b Definition of a COM for the simulated region of the template mesh. This avoids deformations towards topo- the template visible at the acquisition angle, which semantically logically incompatible vertices and the mapping of multiple corresponds with c the target’s COM. d Alignment after translation template vertices onto the same target vertex. We penalize with the vector between the COMs. the dissimilarity of the transformations of vertices connected via an edge ℰ in the template mesh, based on the Frobe- finding the displacement between the centers of mass (COMs) nius norm || · || defined as the square root of the sum of of the template and target. Due to missing regions in the target the squares of the matrix elements, with a weighting matrix mesh, the COMs of the template and target do not correspond 𝐺 := diag(1, 1, 1, 𝛾 ), where 𝛾 can be controlled to emphasize to the same anatomical location. To compensate this, we recal- rotational over translational similarity in the transformations: culate the template’s COM for the surface area being visible ∑︁ 2 2 𝐸 (𝑋 ) := ||(𝑋 − 𝑋 )𝐺|| dist (𝒯 ,𝑋 𝑣 ) (3) 𝑠 𝑖 𝑗 𝑖 𝑖 from the acquisition angle Φ (see Figure 3). The anatomically {𝑖,𝑗}∈ℰ corresponding COMs are then used to define the translational displacement and obtain an initialization for the elastic regis- This ensures the transformations to retain local stiffness, re- tration. sulting in a globally smooth deformation and infilling of holes in the registered mesh to ensure robustness against missing shape topology while approximating the target surface. We deliberately omitted a penalty for the distance between land- 3 Results marks on the mesh surfaces, to avoid complicating the regis- tration process by setting landmark correspondences. The final We evaluated the performance of our method by registering cost function assumes the shape, frontal and oblique range images of four male volunteers cov- ering a spectrum of body morphologies with respect to age 𝐸(X) := 𝐸 (𝑋 ) + 𝛼𝐸 (𝑋 ) (4) (M = 36.5 a, SD = 12.69 a), height (M = 180.25 cm, SD = and constrains a conjugate gradient sparse solver (Eigen 5.89 cm) and weight (M = 100.75 kg, SD = 13.63 kg). Using v3.3.7), which we employ to solve for an optimal set of trans- template models with approximately 10000 polygons, the av- formations 𝑋 . As transformations for regions containing re- erage runtime per registration pass amounted to 30 seconds on jected correspondences are not subject to the distance con- a mobile 2.0 GHz quad-core CPU. The registration quality is straint 𝐸 (𝑋 ) and thus would remain unchanged, we enforce assessed by the morphological realism and the residual RMS smooth transformation of these vertices by emphasizing the Euclidean distances d (𝒯 ,𝑇 ) between the vertices of the RMS similarity constraint by through the parameter 𝛼. To initialize source meshes and their respective closest vertices in the de- the registration with semantically correct point-to-point cor- formed templates (see Table 1). Our method preserves anatom- respondences, we align the longitudinal axis of the template ical details by accurately approximating overlapping regions 4 T.-C. Çallar et al., Body Registration Using Single-View Range Imaging and Shape Templates Tab. 1: Averge residual RMS Euclidean distance between the range images with parametric shape templates. Our method target and template for the total meshes and the region of the is designed to be applicable within clinical environments, re- torso for the respective acquisition views. quiring minimal technical equipment, source data and user interaction for the registration process. Initial experiments in- View d (𝒯 , 𝑇 ) d (𝒯 , 𝑇 ) 𝑡𝑜𝑟𝑠𝑜 RMS RMS dicate the general feasibility of this technique, validating its Frontal 12.1357 mm 0.8015 mm robustness and accuracy under variable acquisition settings. Oblique 13.3294 mm 0.6562 mm This work represents a contribution to alleviate the establish- ment of automated medical systems, by providing the ability a b c d to rapidly obtain complete digital surface anatomy represen- tations to be used for targeting and motion planning purposes, e.g. within robotic systems. More generally, we demonstrate an alternative imaging technique, suitable for the assessment of the external morphology in humans. Based on these prelim- > 5 mm inary results we intend to further improve our method’s shape completion accuracy and robustness against body pose varia- tions by exploiting anatomic symmetries to enforce mirroring 0 mm of transformations about the sagittal plane. Author Statement Research funding: Authors state no funding involved. Con- flict of interest: Authors state no conflict of interest. Informed consent: Informed consent has been obtained from all individ- Fig. 4: Registration results obtained for frontal and oblique range uals included in this study. Ethical approval: The conducted images of one subject: a Segmented body surface. b Anterior and research is not related to either human or animal use. c posterior deformations of the template models. d Residual local minimal Euclidean distances to the respective target surface. between template and target and completes missing regions in References a statistically probable manner concerning principal anthropo- [1] Marc T, Computer-aided medicine and surgery. In: Digital metric features of the subject (see Figure 4). With submillimet- Medicine. 2.3;2016:81-87. ric deviations, the registration of the torso can be interpreted [2] Al-Badri M, Ipsen S, Böttger S, Ernst F. Robotic 4D ultra- as a reparameterization of the source image. The registration sound solution for real-time visualization and teleoperation. fit of articulated regions suffers from positioning inconsisten- Current Directions in Biomedical Engineering 2017;3(2);559- cies, which are partially recovered (see the right arm in Fig- [3] Placht S, Stancanello J, Schaller C, Balda M, Angelopoulou ure 4). As these regions often are not of direct medical inter- E. Fast time-of-flight camera based surface registra- est, this is irrelevant for most applications. Regardless of sim- tion for radiotherapy patient positioning. Medical physics ilar numerical results, the overall realism achievable through 2012;39(1);4-17. frontal views appears to be superior, as the symmetric source [4] Chen C, Hung Y, Cheng JB. A fast automatic method for data guides the deformation evenly. For practical applications, registration of partially-overlapping range images. IEEE Sixth the viewing angle capturing most of the respective application- International Conference on Computer Vision 1998;242-248. [5] Amberg B, Romdhani S, Vetter T. Optimal step nonrigid specific regions of interest still should be preferred. As missing ICP algorithms for surface registration. IEEE Conference regions in the target mesh are filled in based on the enforced on Computer Vision and Pattern Recognition 2007;1-8. similarity local transformations, surface models produced by [6] Li H, Adams B, Guibas LJ, Pauly M. Robust single-view ge- the registration of oblique range images may be partially in- ometry and motion reconstruction. ACM Transactions on sufficient for high accuracy applications. Graphics 2009;28(5):1-10. [7] Anguelov D, Srinivasan P, Koller D, Thrun S, Rodgers, J, Davis J. SCAPE: shape completion and animation of people. ACM SIGGRAPH Papers 2005;408-416. 4 Conclusion [8] Reed M, Raschke U, Tirumali R, Parkinson M. Developing and implementing parametric human body shape models in ergonomics software. Proceedings of the 3rd international We presented an efficient 3D body registration framework digital human modeling conference 2014. based on the markerless elastic registration of single-view Oblique Frontal http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Current Directions in Biomedical Engineering de Gruyter

Efficient Body Registration Using Single-View Range Imaging and Generic Shape Templates

Loading next page...
 
/lp/de-gruyter/efficient-body-registration-using-single-view-range-imaging-and-YNZcukseyV

References (8)

Publisher
de Gruyter
Copyright
© 2020 by Walter de Gruyter Berlin/Boston
eISSN
2364-5504
DOI
10.1515/cdbme-2020-3031
Publisher site
See Article on Publisher Site

Abstract

DE GRUYTER Current Directions in Biomedical Engineering 2020;6(3): 20203031 Tolga-Can Çallar*, Elmar Rueckert, and Sven Böttger Efficient Body Registration Using Single-View Range Imaging and Generic Shape Templates https://doi.org/10.1515/cdbme-2020-3031 teraction is the requirement of a patient body model for the definition and localization of task-specific anatomic target re- Abstract: Computer-aided medical systems, e.g. in the fields gions. Conventionally, established 3D imaging modalities with of medical robotics or image-based assistance, are continu- varying image quality, acquisition speed, and nocuous poten- ously investigated to overcome human limitations concerning tial, e.g. CT and MRI have been utilized in combination with perception, memory or dexterity. A common requirement of range imaging techniques to perform on-site registration of such systems is the availability of a digital model describ- body parts with previously acquired 3D image data [3]. How- ing the patient’s position and morphology during a procedure. ever, the associated health risks, procedural complexity and Operational complexity and technical limitations of estab- technical limitations of the image acquisition and registra- lished 3D imaging methods leave clinical settings in need of tion process render these approaches infeasible in cases where a method for the fast acquisition of a three-dimensional body large body surface coverage is required, visualization of inter- surface representation. For this purpose, we propose an unsu- nal anatomy is not desired or no prior suitable imaging stud- pervised and efficient body registration pipeline based on the ies have been done. Furthermore, unavoidable positional dif- markerless elastic registration and completion of single-view ferences between the subject in the source image and during stereo range images of the body surface with statistical para- registration may reduce the obtained model’s representative metric body shape templates. Initial results show a promising capacity regarding body pose and physiognomy at the mo- representative quality of the models generated through the reg- ment of a medical procedure. This encouraged us to inves- istration process with submillimetric fitting accuracy and real- tigate single-view range imaging with active infrared stereo istic surface morphology, indicating the general feasibility of cameras as a low-cost and stand-alone technique for this pur- our approach as an instant body registration method for auto- pose, due to its inherent advantages effectively mitigating the mated medical and biometric applications. mentioned drawbacks: The instant optical image acquisition Keywords: body registration, range imaging, 3D modeling, increases the temporal representative quality of acquired im- computer vision, medical robotics, automation age data while facilitating an efficient and innocuous regis- tration process. When compared to other range imaging tech- niques, e.g. ToF-cameras, infrared stereo imaging provides a 1 Introduction larger field of view and higher depth resolution at medium range, which is crucial for clinical applications. The major dif- The clinical success of surgical, diagnostic and therapeutic ficulty with this technology is the occurrence of optical occlu- procedures exhibits a high degree of dependency upon the in- sions restricting the retrievable body surface area from a given teractions and capabilities of medical personnel. Through the acquisition perspective. Proposed techniques to resolve this is- emergence of computer-aided medicine and medical automa- sue generally utilize multi-view acquisition settings to increase tion, many conceptual solutions for negative effects caused by the amount of captured surface area. Classically, overlapping user-dependency and human limitations of perception, cogni- partial scans are acquired and rigidly registered [4] to recon- tion, physical strength or dexterity have been developed [1]. struct a body model. To recover unobserved regions, promis- Currently studied use cases within this research field includ- ing results were achieved through the non-rigid registration ing robotic medical ultrasound [2] indicate improvements in of densely acquired full-body scans with template models [5] reliability and consistency of outcome when compared to con- or sparse surface data with individual shape priors in form ventional medical practice. A commonality of virtually all au- of handcrafted templates [6] or shape representations learned tomated medical systems employed for physical patient in- from subject-specific multi-view data [7]. As these approaches involve significant user-interaction and technical complexities *Corresponding author: Tolga-Can Çallar, Institute for Robotics decreasing the registration efficiency, we present a method for and Cognitive Systems, Universität zu Lübeck, Ratzeburger Allee the unsupervised elastic registration of single-view range im- 160, Lübeck, Germany, e-mail: ages with generic templates allowing the rapid and accurate tolgacan.callar@student.uni-luebeck.de digitization of the surface anatomy in clinical settings. Elmar Rueckert, Sven Böttger, Institute for Robotics and Cogni- tive Systems, Universität zu Lübeck, Germany Open Access. © 2020 Tolga-Can Callar et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. 2 T.-C. Çallar et al., Body Registration Using Single-View Range Imaging and Shape Templates 2.2 Body Segmentation 2 Methods To define the area of the range scan to be registered with the Our proposed approach for the registration of the body sur- template, we perform the segmentation of the subject in the face consists of three main elements: First, we acquire one triangulated point cloud of the scene. For this purpose we single-view range image of a subject on an examination couch implemented a background subtraction variant utilizing the with the low-cost Intel RealSense D415 active IR stereo cam- VTK-library: We generate k-d-tree-based space-partitionings era (Intel Corporation, Santa Clara, USA). Afterward, we per- of both the background scenery captured in advance and the form the segmentation of the subject’s body and the reduc- scene including the subject. On these, we perform a nearest- tion of range imaging artifacts. In the last step, the segmented neighbor search to identify those vertices in the subject’s scene body surface is elastically registered with a parametric tem- without any corresponding vertex from the background mesh plate body model, which has been modified in advance to re- in their vicinity. Due to optical occlusions causing regions with semble the subject’s anthropometric features. The segmenta- sparsely imaged steep depth gradients around border regions tion and registration processes of our framework are imple- of the body (see Figure 2), the segmentation result still con- mented using the 3D-processing software VTK (Kitware, Inc., tains artifacts visible in the form of local elongation of the Clifton Park, USA). triangular faces in these regions of the mesh. We exploit the large ratio between the perimeter of the triangular faces and their area to filter out the elongated faces. 2.1 Range Image Acquisition A range image of the complete scenery is captured from which the sections belonging to the subject are extracted and used as a target input for the registration of the body surface. The stereo camera is placed lateral to the examination couch and oriented to align the x-axis of the camera frame with the ex- Fig. 2: Segmented body surface with elongated triangular faces pected longitudinal axis of the body of the subject to be regis- caused by locally steep depth gradients. tered. The camera’s height and vertical tilt are adjusted, until the field of view spans the entire space to be occupied by the body, with the image frame being centered around the center 2.3 Template-Based Registration of the examination couch. In preparation for the subsequent segmentation, we first capture the scenery without the subject Regardless of optical occlusions, single-view range images al- and then with the subject in the supine position. As visual- ways retrieve less than half of the body surface. This renders ized in Figure 1, we acquire a frontal and oblique image for a solely segmentation-based approach insufficient for the spa- every subject to generate source data for evaluation of our reg- tial registration of complete 3D body surfaces. Therefore, we istration process under varying acquisition conditions. All of propose the elastic registration of a parameterized body shape the obtained range images are converted to point clouds and template onto the segmented range image to augment the exported as a polygonal mesh through the internal Delaunay- anatomically incomplete range image source while preserving triangulation function of the RealSense Viewer application. the captured anatomic details. To ensure the template’s mor- phological resemblance of the subject, we employ adjustable Frontal polygonal body models originating from statistical full-body laser scans from UMTRI Human Shape [8]. This allows us to create individual base shapes accommodating for the respec- tive anthropometric properties of a subject that may also allow for semantic pre-annotations, e.g. for task-specific target re- gions. We implemented an affine variation of the point set reg- istration algorithm ICP derived from the method described by [5]. Our algorithm iteratively optimizes transformations of the a b template to minimize the distance to the nearest-neighbor cor- Fig. 1: a Acquisition setup and orientation of the camera and body respondences between the template and target vertices while frame. b Utilized image acquisition perspectives. enforcing locally stiff deformations. In contrast to classical ICP we do not utilize global transformations on our template T.-C. Çallar et al., Body Registration Using Single-View Range Imaging and Shape Templates 3 model. Instead we define 𝑛 individual affine transformations mesh with the x-axis of the camera (see Figure 1), confining 𝑋 for every vertex 𝑣 ∈ 𝑉 of the template mesh 𝑇 . These the rotational difference to this axis. Exploiting the approxi- 𝑖 𝑖 are arranged in a matrix 𝑋 := [𝑋𝑖 . . .𝑋𝑛] representing the mate coincidence of the surface normal of the couch with the entirety of the vertex-wise transformations regularized by the body’s sagittal axis in the supine position, we infer the body distance to closest points on the target surface 𝒯 as expressed orientation from the average direction of the vertex normals by the energy term of the couch nearest to its intersection point with the camera’s ∑︁ z-axis. The template is rotated about its longitudinal axis by 𝐸 (𝑋 ) := 𝑤 dist (𝒯 ,𝑋 𝑣 ) , (1) 𝑑 𝑖 𝑖 𝑖 the angle Φ that the subject’s estimated sagittal axis deviates 𝑣 ∈𝑉 from the camera’s optical axis, which is equal to the acqui- where 𝑤 denotes a binary reliability weighting for the nearest- 𝑖 sition angle. The translational misalignment is resolved by neighbour correspondences. Two reliability tests are per- formed on every candidate vertex pair (𝑣 ∈ 𝑇 ,𝑣 ∈ 𝒯 ) after 𝑖 𝑗 finding the initial set of correspondences. A correspondence is rejected, if the template’s vertex normal 𝑛 exceeds an an- 𝑣 ∈𝑇 𝑖 ϕ gular deviation threshold of 𝜃 = 90° with respect to the closest target vertex normal 𝑛 , i.e. 𝑣 ∈𝒯 (︂ )︂ 𝑛 · 𝑛 𝑣 𝑣 𝑖 𝑗 arccos > 𝜃 , (2) ‖𝑛 ‖·‖ 𝑛 ‖ 𝑣 𝑣 𝑖 𝑗 a b c d or if the nearest template vertex is located on the border of Fig. 3: a Rotational alignment of the template through the acqui- sition angle Φ. b Definition of a COM for the simulated region of the template mesh. This avoids deformations towards topo- the template visible at the acquisition angle, which semantically logically incompatible vertices and the mapping of multiple corresponds with c the target’s COM. d Alignment after translation template vertices onto the same target vertex. We penalize with the vector between the COMs. the dissimilarity of the transformations of vertices connected via an edge ℰ in the template mesh, based on the Frobe- finding the displacement between the centers of mass (COMs) nius norm || · || defined as the square root of the sum of of the template and target. Due to missing regions in the target the squares of the matrix elements, with a weighting matrix mesh, the COMs of the template and target do not correspond 𝐺 := diag(1, 1, 1, 𝛾 ), where 𝛾 can be controlled to emphasize to the same anatomical location. To compensate this, we recal- rotational over translational similarity in the transformations: culate the template’s COM for the surface area being visible ∑︁ 2 2 𝐸 (𝑋 ) := ||(𝑋 − 𝑋 )𝐺|| dist (𝒯 ,𝑋 𝑣 ) (3) 𝑠 𝑖 𝑗 𝑖 𝑖 from the acquisition angle Φ (see Figure 3). The anatomically {𝑖,𝑗}∈ℰ corresponding COMs are then used to define the translational displacement and obtain an initialization for the elastic regis- This ensures the transformations to retain local stiffness, re- tration. sulting in a globally smooth deformation and infilling of holes in the registered mesh to ensure robustness against missing shape topology while approximating the target surface. We deliberately omitted a penalty for the distance between land- 3 Results marks on the mesh surfaces, to avoid complicating the regis- tration process by setting landmark correspondences. The final We evaluated the performance of our method by registering cost function assumes the shape, frontal and oblique range images of four male volunteers cov- ering a spectrum of body morphologies with respect to age 𝐸(X) := 𝐸 (𝑋 ) + 𝛼𝐸 (𝑋 ) (4) (M = 36.5 a, SD = 12.69 a), height (M = 180.25 cm, SD = and constrains a conjugate gradient sparse solver (Eigen 5.89 cm) and weight (M = 100.75 kg, SD = 13.63 kg). Using v3.3.7), which we employ to solve for an optimal set of trans- template models with approximately 10000 polygons, the av- formations 𝑋 . As transformations for regions containing re- erage runtime per registration pass amounted to 30 seconds on jected correspondences are not subject to the distance con- a mobile 2.0 GHz quad-core CPU. The registration quality is straint 𝐸 (𝑋 ) and thus would remain unchanged, we enforce assessed by the morphological realism and the residual RMS smooth transformation of these vertices by emphasizing the Euclidean distances d (𝒯 ,𝑇 ) between the vertices of the RMS similarity constraint by through the parameter 𝛼. To initialize source meshes and their respective closest vertices in the de- the registration with semantically correct point-to-point cor- formed templates (see Table 1). Our method preserves anatom- respondences, we align the longitudinal axis of the template ical details by accurately approximating overlapping regions 4 T.-C. Çallar et al., Body Registration Using Single-View Range Imaging and Shape Templates Tab. 1: Averge residual RMS Euclidean distance between the range images with parametric shape templates. Our method target and template for the total meshes and the region of the is designed to be applicable within clinical environments, re- torso for the respective acquisition views. quiring minimal technical equipment, source data and user interaction for the registration process. Initial experiments in- View d (𝒯 , 𝑇 ) d (𝒯 , 𝑇 ) 𝑡𝑜𝑟𝑠𝑜 RMS RMS dicate the general feasibility of this technique, validating its Frontal 12.1357 mm 0.8015 mm robustness and accuracy under variable acquisition settings. Oblique 13.3294 mm 0.6562 mm This work represents a contribution to alleviate the establish- ment of automated medical systems, by providing the ability a b c d to rapidly obtain complete digital surface anatomy represen- tations to be used for targeting and motion planning purposes, e.g. within robotic systems. More generally, we demonstrate an alternative imaging technique, suitable for the assessment of the external morphology in humans. Based on these prelim- > 5 mm inary results we intend to further improve our method’s shape completion accuracy and robustness against body pose varia- tions by exploiting anatomic symmetries to enforce mirroring 0 mm of transformations about the sagittal plane. Author Statement Research funding: Authors state no funding involved. Con- flict of interest: Authors state no conflict of interest. Informed consent: Informed consent has been obtained from all individ- Fig. 4: Registration results obtained for frontal and oblique range uals included in this study. Ethical approval: The conducted images of one subject: a Segmented body surface. b Anterior and research is not related to either human or animal use. c posterior deformations of the template models. d Residual local minimal Euclidean distances to the respective target surface. between template and target and completes missing regions in References a statistically probable manner concerning principal anthropo- [1] Marc T, Computer-aided medicine and surgery. In: Digital metric features of the subject (see Figure 4). With submillimet- Medicine. 2.3;2016:81-87. ric deviations, the registration of the torso can be interpreted [2] Al-Badri M, Ipsen S, Böttger S, Ernst F. Robotic 4D ultra- as a reparameterization of the source image. The registration sound solution for real-time visualization and teleoperation. fit of articulated regions suffers from positioning inconsisten- Current Directions in Biomedical Engineering 2017;3(2);559- cies, which are partially recovered (see the right arm in Fig- [3] Placht S, Stancanello J, Schaller C, Balda M, Angelopoulou ure 4). As these regions often are not of direct medical inter- E. Fast time-of-flight camera based surface registra- est, this is irrelevant for most applications. Regardless of sim- tion for radiotherapy patient positioning. Medical physics ilar numerical results, the overall realism achievable through 2012;39(1);4-17. frontal views appears to be superior, as the symmetric source [4] Chen C, Hung Y, Cheng JB. A fast automatic method for data guides the deformation evenly. For practical applications, registration of partially-overlapping range images. IEEE Sixth the viewing angle capturing most of the respective application- International Conference on Computer Vision 1998;242-248. [5] Amberg B, Romdhani S, Vetter T. Optimal step nonrigid specific regions of interest still should be preferred. As missing ICP algorithms for surface registration. IEEE Conference regions in the target mesh are filled in based on the enforced on Computer Vision and Pattern Recognition 2007;1-8. similarity local transformations, surface models produced by [6] Li H, Adams B, Guibas LJ, Pauly M. Robust single-view ge- the registration of oblique range images may be partially in- ometry and motion reconstruction. ACM Transactions on sufficient for high accuracy applications. Graphics 2009;28(5):1-10. [7] Anguelov D, Srinivasan P, Koller D, Thrun S, Rodgers, J, Davis J. SCAPE: shape completion and animation of people. ACM SIGGRAPH Papers 2005;408-416. 4 Conclusion [8] Reed M, Raschke U, Tirumali R, Parkinson M. Developing and implementing parametric human body shape models in ergonomics software. Proceedings of the 3rd international We presented an efficient 3D body registration framework digital human modeling conference 2014. based on the markerless elastic registration of single-view Oblique Frontal

Journal

Current Directions in Biomedical Engineeringde Gruyter

Published: Sep 1, 2020

Keywords: body registration; range imaging; 3D modeling; computer vision; medical robotics; automation

There are no references for this article.