Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Integrating 3D cameras into sterile surgical environments: A comparison of different protective materials regarding scan accuracy

Integrating 3D cameras into sterile surgical environments: A comparison of different protective... DE GRUYTER Current Directions in Biomedical Engineering 2022;8(1): 25-29 Lars Wagner*, Lukas Bernhard, Jonas Fuchtmann, Mert Asim Karaoglu, Alexander Ladikos, Hubertus Feußner and Dirk Wilhelm Integrating 3D cameras into sterile surgical environments: A comparison of different protective materials regarding scan accuracy Development of a sterile camera enclosure for the detection of laparoscopic instruments https://doi.org/10.1515/cdbme-2022-0007 When analyzing multiple pane types, polymethyl methacrylate with 0.5 mm thickness (PMMA 0.5) provided the best results. Abstract: This work presents a sterile concept for 3D cameras At a scan distance of 560 mm to the surface center, which is within sterile surgical environments. In the digital operating required for the complete acquisition of a laparoscopic room (OR), such cameras can serve as a valuable data source instrument, PMMA 0.5 achieved the smallest Chamfer for cognitive workflow assistance systems, e.g decision or distance (CD) values for both the scans with the laparoscopic mechatronic support systems. One recent example are robotic instruments in the instrument holder (0.23 ± 1.52 mm) and in assistants for instrument handling, such as the robotic scrub the drop zone (0.12 ± 0.25 mm). nurse currently developed in the framework of the SASHA-OR research project . In this context, we detect laparoscopic Keywords: Laparoscopic Instrument Detection, Point instruments and the surgical environment with a 3D camera, Clouds, Sterility, Optical Plastic whereby hygienic requirements need to be met. Using a Zivid Two sensor, we generated point clouds of the laparoscopic instruments located in an instrument holder and 1 Introduction a drop zone. We compared the effect of using different pane types and thicknesses for the sterile camera enclosure and Nurse staffing shortages are an enduring problem, that is compared the performance with and without protective pane expected to grow in the upcoming years. Especially scrub and in terms of the point cloud accuracy. circulating nurse understaffing limits the use of available operating capacities in hospitals [1]. Due to the shortage of personnel, there is increasing recourse to unqualified staff, whose inexperience has a significant influence on the ______ workflow of a surgery. While the vision of the cognitive *Corresponding author: Lars Wagner: Minimally Invasive Interdisciplinary Therapeutical Intervention, University Hospital operating room [2] aims at mitigating these effects by means rechts der Isar, Technical University Munich, Munich, Germany, e- of smart assistance and decision support systems, extensive mail: lars.wagner@tum.de sensor input is needed as a foundation for understanding the Lukas Bernhard, Jonas Fuchtmann, Dirk Wilhelm, Hubertus current needs of the surgical team. In this context, 3D cameras Feußner: Minimally Invasive Interdisciplinary Therapeutical are an especially feature-rich modality and therefore highly Intervention, University Hospital rechts der Isar, Technical University Munich, Munich, Germany desirable for various applications, such as assistive robotics. Mert Asim Karaoglu: Chair of Computer Aided Medical However, integrating such sensors can be challenging, Procedures and Augmented Reality, Technical University Munich, especially if they need to be mounted in proximity of the Munich, Germany operative field thus affect hygienic requirements. Mert Asim Karaoglu, Alexander Ladikos: ImFusion GmbH, To address the problem of nurse understaffing, several Munich, Germany robotic scrub nurse concepts have been introduced within the Dirk Wilhelm, Hubertus Feußner: Department of Surgery, University Hospital rechts der Isar, Technical University Munich, scientific community [3-8]. Previous work has focused Munich, Germany primarily on open surgical procedures. However, due to its Technical University Munich: SASHA-OR. URL: web.med.tum.de/en/miti/research/projects/sasha-or/ [12.05.2022] Open Access. © 2022 The Author(s), published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 International License. 25 significant advantages over open surgery, laparoscopic 2.1 Models of surgical instruments surgery has become the gold standard for a range of surgical Most laparoscopic instruments are built upon three parts: a procedures, including cholecystectomies, appendectomies, handle, an elongated sheath and an effector [11]. Some of the inguinal hernia repair and some types of colon surgery [9] and instruments differ only in the instrument tip. A thumbwheel provides great potential for the use of robotic scrub nurse allows the effector to be rotated 360° for optimal systems. As a part of the StMWi-funded SASHA-OR project maneuverability at the surgical site. However, the available we are developing a context-aware robotic assistance system features make it difficult to identify an instrument using a 3D for flexible instrument and object management within the camera. For robust recognition, the instruments were scanned sterile area of an OR. For the handover of instruments and using a 3D scanner (EinScan-Sp, Shining 3D, China) and sterile goods to the surgeon, we have designed a platform saved as a Standard Transformation Language file. Using the where a robotic handling arm (Panda, Franka-Emika Gmbh, 3D model, visual features can be learned to enable the Germany) is intended to take over the tasks of a human scrub instruments to be recognized on the robot platform. nurse. The platform consists of an instrument holder, in which the laparoscopic instruments are stored, and a drop zone, which allows the surgeon to return the instruments. A 3D 2.2 Camera enclosure camera (Zivid Two, Zivid, Norway) was installed on the robotic arm for object recognition of the laparoscopic For compliance with hygiene regulations in the sterile area of instruments. the OR, we have developed a device for the Zivid Two sensor. Since the robotic platform will be used in the sterile area In daily clinical practice, sensitive equipment with electronic of the OR, specific hygienic regulations have to be met. components must be quickly covered with sterile surgical Reusable medical devices used during a surgical procedure are drapes, covers or foils for single use. To ensure that scanning considered potentially contaminated, and as such must be of the instruments and the surgical environment is still reprocessed after use to prevent surgical site infections. The possible, a transparent pane must be installed in the device in reprocessing method used is based on the criticality of the front of the sensor. Foils that are very vulnerable to visual device, and ranges from simple disinfection to a combination obstructions can be excluded. Plastics such as polymethyl of cleaning, thermal disinfection, and moist heat sterilization methacrylate (PMMA), polycarbonate (PC), and polyvinyl in an autoclave [10]. As some medical devices, particularly chloride (PVC) are more suitable for use in the OR than optical larger equipment with electronic parts such as robotic handling glass because of their properties of being less fragile and arms, are not suitable for reprocessing due to their size and brittle. Acrylic has the lowest refractive index and the highest components, other methods of ensuring sterility have to be Abbe number of these plastics. The material is very scratch developed. Especially, maintaining the sterility of the 3D resistant and not very water absorbent, which is why it is used camera is problematic, as a clear field of view must be ensured in many plastic optical applications. PC stands out with its for the instruments to be recognized. In this paper, we propose high impact resistance. [12] The plastic pane was integrated a concept for a sterile enclosure of the 3D camera and into the sterile drape and fixed by magnetic locking. In order demonstrate the effectiveness of our approach in the context to obtain the best possible scan result, we want to examine the of laparoscopic instrument detection. effect of using different plastic pane types and thicknesses regarding their performance and compare their performance with and without the protective pane in terms of point cloud 2 Material and Methods accuracy. The plastic types used together with their thickness and labeling are shown in Table 1. The method section is split into four parts. In the first part, we explain the 3D model acquisition of the surgical tools. These models are to be employed for vision-based detection and 2.3 Scan Setup localization of the objects in the OR. This is followed by the The Zivid Two sensor uses structured light as 3D technology, description of the camera enclosure and the presentation of the whereby the optimal working distance of this sensor is scan setup to generate point clouds. Finally, we describe our between 500 mm and 1100 mm. To investigate the effect of evaluation metric used. using different pane types with respect to the scan results, we chose different scan distances from the instrumentation table what we refer to as scan depth. Since the maximum range of 26 motion of the Panda robot arm is 855 mm, we decided to use on a grid. A common method for comparing point clouds scan depths of 560 mm and 810 mm. We scanned selected directly is the Chamfer distance, where the individual sample laparoscopic instruments (clip applicator, grasping forceps, points from two point clouds are compared. [13] scissors and DaVinci scissors) both in the instrument holder We consider the point cloud of the scan without the pane as and lying in the drop zone with automatic calibration (auto) the ground truth or reference for calculating the Chamfer and with the calibration setting of a scan without glass (no distance. For each point in the respective point cloud, the pane). For a scan depth of 560 mm, a laparoscopic instrument Chamfer Distance (CD) finds the closest point in the other is fully located within the field of view of the sensor. A scan point cloud and sums up the square of the distance. The metric depth of 810 mm represents the maximum possible distance of between two point clouds 𝑆 and 𝑆 is defined as ! " the sensor from the platform. To verify the optimal working 𝑪𝑫(𝑺 , 𝑺 ) = ∑ 𝐦𝐢𝐧‖𝒙 − 𝒚‖ distance of the sensor, we additionally performed scans at a 𝟏 𝟐 𝒙∈𝑺 |𝑺 | 𝟏 𝒚∈𝑺 height of 950 mm. + ∑ 𝐦𝐢𝐧‖𝒙 − 𝒚‖ . (1) 𝒚∈𝑺 𝟐 |𝑺 | 𝟐 𝒙∈𝑺 Table 1: Overview of the different plastic pane types and their For evaluation we use the mean CD between the respective thickness used during testing point clouds with the different pane types and the reference point cloud averaged over the number of 3D points. Label Plastic type Thickness PMMA 0.5 Polymethyl methacrylate 0,5 mm PMMA 1 Polymethyl methacrylate 1,0 mm 3 Results PMMA 1.5 Polymethyl methacrylate 1,5 mm PMMA 2.0 Polymethyl methacrylate 2,0 mm Figure 2 shows the CD values with the laparoscopic instruments in the instrument holder, while Figure 3 shows the PC 1 Polycarbonate 1,0 mm results with the instruments in the drop zone. The x-axis PVC 1 Polyvinyl chloride 1,0 mm displays the different plastic pane types and the y-axis the mean Chamfer distance. The different scan depths are highlighted in different colours, while the dashed line indicates 2.4 Measurements the metric values based on the auto-calibration setting and the solid line the results based on the calibration setting without Point clouds consist of a set of 3D points sampled on surfaces the pane. in the scene and are a common 3D data representation. The Achieving lower mean CD values indicates smaller deviation point cloud obtained from a scan of the laparoscopic of the 3D points compared to the 3D points of the scene instruments in the drop zone with a depth of 560 mm is shown without a pane. Thus, the smaller the CD value, the smaller the in Figure 1. However, comparison of point clouds is negative influence of the pane on the accuracy of the scene challenging because they cannot be compared using a common representation. metric such as Euclidian metric since they are not a function By comparing the scan results of the surgical instruments in the instrument holder (Figure 2) we notice that PMMA 1 shows very constant CD values, while the values for PMMA 0.5 are more dispersed. However, PMMA 0.5 has the lowest CD value with 0.23 ± 1.52 mm at a scan depth of 560 mm with autocalibration. For a scan depth of 810 mm, PMMA 1.5 achieves the lowest CD value of 0.63 ± 0.32 mm with auto- calibration. The CD values of PC 1 are comparable to those of PMMA 1.5. PVC 1 delivers slightly lower CD values than PC 1. When examining the scan results with the laparoscopic instruments in the drop zone (Figure 3), we see that PMMA 0.5 shows clearly better values this time compared to the other Figure 1: Point cloud of laparoscopic instruments with a scan pane types. The lowest CD value is again obtained for a scan depth of 560 mm. Instruments from top to bottom: clip depth of 560 mm with auto-calibration with 0.12 ± 0.25 mm. applicator, grasping forceps, scissors and DaVinci scissors. PMMA 2 once again shows clearly higher CD values 27 compared to the other pane types and is therefore not suitable on the performance but also the pane thickness. Figure 2 and for sterile housing of the Zivid Two sensor. PC 1 delivers better Figure 3 reveal that PMMA 2 gives the worst results, which results than PVC 1 at scan depths of 810 mm and 950 mm, can be attributed to this parameter. The PMMA 0.5 pane type however, both show poorer results in comparison to PMMA achieves the best results for a scan depth of 560 mm, both for 0.5. the instruments in the instrument holder and in the drop zone. Since the laparoscopic instruments are fully located in the field of view at this scan depth, we decide to use the PMMA 0.5 type within our concept. As a limitation of our experimental method, it must be noted that the evaluation of the scans using the CD does not involve a comparison with the underlying surface of the considered point cloud, but with the point cloud generated by the scan without a pane, which we have assumed to be the ground truth. Thus, we compare generated sample points, which may have a sensitive effect on the CD. In the future, we plan to incorporate further materials and thicknesses to allow for an even more comprehensive comparison. Figure 2: Mean chamfer distance and standard deviation between the respective point clouds of the scans with different plastic panes and the reference point cloud. Instruments are placed in the instrument holder. 5 Conclusion Due to the requirements in the sterile zone of the OR, cameras have to be enclosed adequately. Therefore, we developed a sterile concept for a 3D camera maintaining vision-based recognition of laparoscopic instruments and the surgical environment. For the vision-based tasks in the sterile zone, a clear-transparent pane has to be installed in front of the 3D camera, which do not influence the emitting structured light of a 3D camera. For this purpose, we tested different types of plastic panes and evaluated them using the mean CD values regarding the generated point clouds of laparoscopic instruments. Based on our results, we recommend the use of PMMA with a thickness of 0.5 mm. As a next step, we want to extract the visual features of the 3D models of the Figure 3: Mean chamfer distance and standard deviation between instruments described in section 2.1 in order to be able to the respective point clouds of the scans with different plastic classify them using computer vision algorithms. We will then panes and the reference point cloud. Instruments are located perform rescans and evaluate the computer vision recognition in the drop zone. rate of the instruments with and without camera enclosure. Author Statement Research funding: The project was funded by the Bavarian 4 Discussion Ministry of Economic Affairs, Regional Development and Energy (StMWi) (grant number: DIK0372). The results regarding the scan accuracy indicate that material Conflict of interest: Authors state no conflict of interest. properties have a notable influence on the quality of the scans. As expected, the PMMA panes yields the best results, since this type has the highest Abbe number and the lowest References refractive index. As PMMA has the lowest absorption rate, [1] Marć M et al. A nursing shortage - a prospect of global and primarily PMMA panes were investigated in this test setup. local policies. Int Nurs Rev. 2019;66:9-16 Nevertheless, not only material properties have an influence 28 [2] Maier-Hein L, Eisenmann M, Sarikaya D, et al. Surgical data science - from concepts toward clinical translation. Med Image Anal. 2022;76:102306. [3] Kochan A. Scalpel please, robot: Penelope’s debut in the operating room. Industrial Robot 2005;32:449–451 [4] Treat MR et al. Initial clinical experience with a partly autonomous robotic surgical instrument server. Surg Endosc 2006;20:1310–1314 [5] Jacob M et al. Gestonurse: a robotic surgical nurse for handling surgical instruments in the operating room. J Robotic Surg 2012;6:53–63 [6] Ezzat A et al. An eye-tracking based robotic scrub nurse: proof of concept. Surg Endosc 2021;35:5381–5391 [7] Muralidhar D et al. Collaborative Robot as Scrub Nurse. Current Directions in Biomedical Engineering 2021;7:162– [8] Carpintero E et al. Development of a robotic scrub nurse for the operating theatre. In: 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics. Tokyo;2010:504–509 [9] Buia A et al. Laparoscopic surgery: A qualified systematic review. World J Methodol. 2015;5:238-254. [10] Robert Koch Institute (RKI). Hygiene Requirements for the Reprocessing of Medical Devices. Bundesgesundheitsbl 2012;55:1244-1310 [11] Anderson PL et al. Robot-like dexterity without computers and motors: a review of hand-held laparoscopic instruments with wrist-like tip articulation. Expert Rev Med Devices. 2016;13:661-672 [12] Träger F. Springer Handbook of Lasers and Optics. Springer New York. 2007 [13] Urbach D et al. DPDist: Comparing point clouds using deep point cloud distance. European Conference on Computer Vision 2020: 545-560 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Current Directions in Biomedical Engineering de Gruyter

Integrating 3D cameras into sterile surgical environments: A comparison of different protective materials regarding scan accuracy

Download PDF
5 pages

Loading next page...
 
/lp/de-gruyter/integrating-3d-cameras-into-sterile-surgical-environments-a-comparison-i49wzuQj9Z
Publisher
de Gruyter
Copyright
© 2022 by Walter de Gruyter Berlin/Boston
eISSN
2364-5504
DOI
10.1515/cdbme-2022-0007
Publisher site
See Article on Publisher Site

Abstract

DE GRUYTER Current Directions in Biomedical Engineering 2022;8(1): 25-29 Lars Wagner*, Lukas Bernhard, Jonas Fuchtmann, Mert Asim Karaoglu, Alexander Ladikos, Hubertus Feußner and Dirk Wilhelm Integrating 3D cameras into sterile surgical environments: A comparison of different protective materials regarding scan accuracy Development of a sterile camera enclosure for the detection of laparoscopic instruments https://doi.org/10.1515/cdbme-2022-0007 When analyzing multiple pane types, polymethyl methacrylate with 0.5 mm thickness (PMMA 0.5) provided the best results. Abstract: This work presents a sterile concept for 3D cameras At a scan distance of 560 mm to the surface center, which is within sterile surgical environments. In the digital operating required for the complete acquisition of a laparoscopic room (OR), such cameras can serve as a valuable data source instrument, PMMA 0.5 achieved the smallest Chamfer for cognitive workflow assistance systems, e.g decision or distance (CD) values for both the scans with the laparoscopic mechatronic support systems. One recent example are robotic instruments in the instrument holder (0.23 ± 1.52 mm) and in assistants for instrument handling, such as the robotic scrub the drop zone (0.12 ± 0.25 mm). nurse currently developed in the framework of the SASHA-OR research project . In this context, we detect laparoscopic Keywords: Laparoscopic Instrument Detection, Point instruments and the surgical environment with a 3D camera, Clouds, Sterility, Optical Plastic whereby hygienic requirements need to be met. Using a Zivid Two sensor, we generated point clouds of the laparoscopic instruments located in an instrument holder and 1 Introduction a drop zone. We compared the effect of using different pane types and thicknesses for the sterile camera enclosure and Nurse staffing shortages are an enduring problem, that is compared the performance with and without protective pane expected to grow in the upcoming years. Especially scrub and in terms of the point cloud accuracy. circulating nurse understaffing limits the use of available operating capacities in hospitals [1]. Due to the shortage of personnel, there is increasing recourse to unqualified staff, whose inexperience has a significant influence on the ______ workflow of a surgery. While the vision of the cognitive *Corresponding author: Lars Wagner: Minimally Invasive Interdisciplinary Therapeutical Intervention, University Hospital operating room [2] aims at mitigating these effects by means rechts der Isar, Technical University Munich, Munich, Germany, e- of smart assistance and decision support systems, extensive mail: lars.wagner@tum.de sensor input is needed as a foundation for understanding the Lukas Bernhard, Jonas Fuchtmann, Dirk Wilhelm, Hubertus current needs of the surgical team. In this context, 3D cameras Feußner: Minimally Invasive Interdisciplinary Therapeutical are an especially feature-rich modality and therefore highly Intervention, University Hospital rechts der Isar, Technical University Munich, Munich, Germany desirable for various applications, such as assistive robotics. Mert Asim Karaoglu: Chair of Computer Aided Medical However, integrating such sensors can be challenging, Procedures and Augmented Reality, Technical University Munich, especially if they need to be mounted in proximity of the Munich, Germany operative field thus affect hygienic requirements. Mert Asim Karaoglu, Alexander Ladikos: ImFusion GmbH, To address the problem of nurse understaffing, several Munich, Germany robotic scrub nurse concepts have been introduced within the Dirk Wilhelm, Hubertus Feußner: Department of Surgery, University Hospital rechts der Isar, Technical University Munich, scientific community [3-8]. Previous work has focused Munich, Germany primarily on open surgical procedures. However, due to its Technical University Munich: SASHA-OR. URL: web.med.tum.de/en/miti/research/projects/sasha-or/ [12.05.2022] Open Access. © 2022 The Author(s), published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 International License. 25 significant advantages over open surgery, laparoscopic 2.1 Models of surgical instruments surgery has become the gold standard for a range of surgical Most laparoscopic instruments are built upon three parts: a procedures, including cholecystectomies, appendectomies, handle, an elongated sheath and an effector [11]. Some of the inguinal hernia repair and some types of colon surgery [9] and instruments differ only in the instrument tip. A thumbwheel provides great potential for the use of robotic scrub nurse allows the effector to be rotated 360° for optimal systems. As a part of the StMWi-funded SASHA-OR project maneuverability at the surgical site. However, the available we are developing a context-aware robotic assistance system features make it difficult to identify an instrument using a 3D for flexible instrument and object management within the camera. For robust recognition, the instruments were scanned sterile area of an OR. For the handover of instruments and using a 3D scanner (EinScan-Sp, Shining 3D, China) and sterile goods to the surgeon, we have designed a platform saved as a Standard Transformation Language file. Using the where a robotic handling arm (Panda, Franka-Emika Gmbh, 3D model, visual features can be learned to enable the Germany) is intended to take over the tasks of a human scrub instruments to be recognized on the robot platform. nurse. The platform consists of an instrument holder, in which the laparoscopic instruments are stored, and a drop zone, which allows the surgeon to return the instruments. A 3D 2.2 Camera enclosure camera (Zivid Two, Zivid, Norway) was installed on the robotic arm for object recognition of the laparoscopic For compliance with hygiene regulations in the sterile area of instruments. the OR, we have developed a device for the Zivid Two sensor. Since the robotic platform will be used in the sterile area In daily clinical practice, sensitive equipment with electronic of the OR, specific hygienic regulations have to be met. components must be quickly covered with sterile surgical Reusable medical devices used during a surgical procedure are drapes, covers or foils for single use. To ensure that scanning considered potentially contaminated, and as such must be of the instruments and the surgical environment is still reprocessed after use to prevent surgical site infections. The possible, a transparent pane must be installed in the device in reprocessing method used is based on the criticality of the front of the sensor. Foils that are very vulnerable to visual device, and ranges from simple disinfection to a combination obstructions can be excluded. Plastics such as polymethyl of cleaning, thermal disinfection, and moist heat sterilization methacrylate (PMMA), polycarbonate (PC), and polyvinyl in an autoclave [10]. As some medical devices, particularly chloride (PVC) are more suitable for use in the OR than optical larger equipment with electronic parts such as robotic handling glass because of their properties of being less fragile and arms, are not suitable for reprocessing due to their size and brittle. Acrylic has the lowest refractive index and the highest components, other methods of ensuring sterility have to be Abbe number of these plastics. The material is very scratch developed. Especially, maintaining the sterility of the 3D resistant and not very water absorbent, which is why it is used camera is problematic, as a clear field of view must be ensured in many plastic optical applications. PC stands out with its for the instruments to be recognized. In this paper, we propose high impact resistance. [12] The plastic pane was integrated a concept for a sterile enclosure of the 3D camera and into the sterile drape and fixed by magnetic locking. In order demonstrate the effectiveness of our approach in the context to obtain the best possible scan result, we want to examine the of laparoscopic instrument detection. effect of using different plastic pane types and thicknesses regarding their performance and compare their performance with and without the protective pane in terms of point cloud 2 Material and Methods accuracy. The plastic types used together with their thickness and labeling are shown in Table 1. The method section is split into four parts. In the first part, we explain the 3D model acquisition of the surgical tools. These models are to be employed for vision-based detection and 2.3 Scan Setup localization of the objects in the OR. This is followed by the The Zivid Two sensor uses structured light as 3D technology, description of the camera enclosure and the presentation of the whereby the optimal working distance of this sensor is scan setup to generate point clouds. Finally, we describe our between 500 mm and 1100 mm. To investigate the effect of evaluation metric used. using different pane types with respect to the scan results, we chose different scan distances from the instrumentation table what we refer to as scan depth. Since the maximum range of 26 motion of the Panda robot arm is 855 mm, we decided to use on a grid. A common method for comparing point clouds scan depths of 560 mm and 810 mm. We scanned selected directly is the Chamfer distance, where the individual sample laparoscopic instruments (clip applicator, grasping forceps, points from two point clouds are compared. [13] scissors and DaVinci scissors) both in the instrument holder We consider the point cloud of the scan without the pane as and lying in the drop zone with automatic calibration (auto) the ground truth or reference for calculating the Chamfer and with the calibration setting of a scan without glass (no distance. For each point in the respective point cloud, the pane). For a scan depth of 560 mm, a laparoscopic instrument Chamfer Distance (CD) finds the closest point in the other is fully located within the field of view of the sensor. A scan point cloud and sums up the square of the distance. The metric depth of 810 mm represents the maximum possible distance of between two point clouds 𝑆 and 𝑆 is defined as ! " the sensor from the platform. To verify the optimal working 𝑪𝑫(𝑺 , 𝑺 ) = ∑ 𝐦𝐢𝐧‖𝒙 − 𝒚‖ distance of the sensor, we additionally performed scans at a 𝟏 𝟐 𝒙∈𝑺 |𝑺 | 𝟏 𝒚∈𝑺 height of 950 mm. + ∑ 𝐦𝐢𝐧‖𝒙 − 𝒚‖ . (1) 𝒚∈𝑺 𝟐 |𝑺 | 𝟐 𝒙∈𝑺 Table 1: Overview of the different plastic pane types and their For evaluation we use the mean CD between the respective thickness used during testing point clouds with the different pane types and the reference point cloud averaged over the number of 3D points. Label Plastic type Thickness PMMA 0.5 Polymethyl methacrylate 0,5 mm PMMA 1 Polymethyl methacrylate 1,0 mm 3 Results PMMA 1.5 Polymethyl methacrylate 1,5 mm PMMA 2.0 Polymethyl methacrylate 2,0 mm Figure 2 shows the CD values with the laparoscopic instruments in the instrument holder, while Figure 3 shows the PC 1 Polycarbonate 1,0 mm results with the instruments in the drop zone. The x-axis PVC 1 Polyvinyl chloride 1,0 mm displays the different plastic pane types and the y-axis the mean Chamfer distance. The different scan depths are highlighted in different colours, while the dashed line indicates 2.4 Measurements the metric values based on the auto-calibration setting and the solid line the results based on the calibration setting without Point clouds consist of a set of 3D points sampled on surfaces the pane. in the scene and are a common 3D data representation. The Achieving lower mean CD values indicates smaller deviation point cloud obtained from a scan of the laparoscopic of the 3D points compared to the 3D points of the scene instruments in the drop zone with a depth of 560 mm is shown without a pane. Thus, the smaller the CD value, the smaller the in Figure 1. However, comparison of point clouds is negative influence of the pane on the accuracy of the scene challenging because they cannot be compared using a common representation. metric such as Euclidian metric since they are not a function By comparing the scan results of the surgical instruments in the instrument holder (Figure 2) we notice that PMMA 1 shows very constant CD values, while the values for PMMA 0.5 are more dispersed. However, PMMA 0.5 has the lowest CD value with 0.23 ± 1.52 mm at a scan depth of 560 mm with autocalibration. For a scan depth of 810 mm, PMMA 1.5 achieves the lowest CD value of 0.63 ± 0.32 mm with auto- calibration. The CD values of PC 1 are comparable to those of PMMA 1.5. PVC 1 delivers slightly lower CD values than PC 1. When examining the scan results with the laparoscopic instruments in the drop zone (Figure 3), we see that PMMA 0.5 shows clearly better values this time compared to the other Figure 1: Point cloud of laparoscopic instruments with a scan pane types. The lowest CD value is again obtained for a scan depth of 560 mm. Instruments from top to bottom: clip depth of 560 mm with auto-calibration with 0.12 ± 0.25 mm. applicator, grasping forceps, scissors and DaVinci scissors. PMMA 2 once again shows clearly higher CD values 27 compared to the other pane types and is therefore not suitable on the performance but also the pane thickness. Figure 2 and for sterile housing of the Zivid Two sensor. PC 1 delivers better Figure 3 reveal that PMMA 2 gives the worst results, which results than PVC 1 at scan depths of 810 mm and 950 mm, can be attributed to this parameter. The PMMA 0.5 pane type however, both show poorer results in comparison to PMMA achieves the best results for a scan depth of 560 mm, both for 0.5. the instruments in the instrument holder and in the drop zone. Since the laparoscopic instruments are fully located in the field of view at this scan depth, we decide to use the PMMA 0.5 type within our concept. As a limitation of our experimental method, it must be noted that the evaluation of the scans using the CD does not involve a comparison with the underlying surface of the considered point cloud, but with the point cloud generated by the scan without a pane, which we have assumed to be the ground truth. Thus, we compare generated sample points, which may have a sensitive effect on the CD. In the future, we plan to incorporate further materials and thicknesses to allow for an even more comprehensive comparison. Figure 2: Mean chamfer distance and standard deviation between the respective point clouds of the scans with different plastic panes and the reference point cloud. Instruments are placed in the instrument holder. 5 Conclusion Due to the requirements in the sterile zone of the OR, cameras have to be enclosed adequately. Therefore, we developed a sterile concept for a 3D camera maintaining vision-based recognition of laparoscopic instruments and the surgical environment. For the vision-based tasks in the sterile zone, a clear-transparent pane has to be installed in front of the 3D camera, which do not influence the emitting structured light of a 3D camera. For this purpose, we tested different types of plastic panes and evaluated them using the mean CD values regarding the generated point clouds of laparoscopic instruments. Based on our results, we recommend the use of PMMA with a thickness of 0.5 mm. As a next step, we want to extract the visual features of the 3D models of the Figure 3: Mean chamfer distance and standard deviation between instruments described in section 2.1 in order to be able to the respective point clouds of the scans with different plastic classify them using computer vision algorithms. We will then panes and the reference point cloud. Instruments are located perform rescans and evaluate the computer vision recognition in the drop zone. rate of the instruments with and without camera enclosure. Author Statement Research funding: The project was funded by the Bavarian 4 Discussion Ministry of Economic Affairs, Regional Development and Energy (StMWi) (grant number: DIK0372). The results regarding the scan accuracy indicate that material Conflict of interest: Authors state no conflict of interest. properties have a notable influence on the quality of the scans. As expected, the PMMA panes yields the best results, since this type has the highest Abbe number and the lowest References refractive index. As PMMA has the lowest absorption rate, [1] Marć M et al. A nursing shortage - a prospect of global and primarily PMMA panes were investigated in this test setup. local policies. Int Nurs Rev. 2019;66:9-16 Nevertheless, not only material properties have an influence 28 [2] Maier-Hein L, Eisenmann M, Sarikaya D, et al. Surgical data science - from concepts toward clinical translation. Med Image Anal. 2022;76:102306. [3] Kochan A. Scalpel please, robot: Penelope’s debut in the operating room. Industrial Robot 2005;32:449–451 [4] Treat MR et al. Initial clinical experience with a partly autonomous robotic surgical instrument server. Surg Endosc 2006;20:1310–1314 [5] Jacob M et al. Gestonurse: a robotic surgical nurse for handling surgical instruments in the operating room. J Robotic Surg 2012;6:53–63 [6] Ezzat A et al. An eye-tracking based robotic scrub nurse: proof of concept. Surg Endosc 2021;35:5381–5391 [7] Muralidhar D et al. Collaborative Robot as Scrub Nurse. Current Directions in Biomedical Engineering 2021;7:162– [8] Carpintero E et al. Development of a robotic scrub nurse for the operating theatre. In: 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics. Tokyo;2010:504–509 [9] Buia A et al. Laparoscopic surgery: A qualified systematic review. World J Methodol. 2015;5:238-254. [10] Robert Koch Institute (RKI). Hygiene Requirements for the Reprocessing of Medical Devices. Bundesgesundheitsbl 2012;55:1244-1310 [11] Anderson PL et al. Robot-like dexterity without computers and motors: a review of hand-held laparoscopic instruments with wrist-like tip articulation. Expert Rev Med Devices. 2016;13:661-672 [12] Träger F. Springer Handbook of Lasers and Optics. Springer New York. 2007 [13] Urbach D et al. DPDist: Comparing point clouds using deep point cloud distance. European Conference on Computer Vision 2020: 545-560

Journal

Current Directions in Biomedical Engineering – de Gruyter

Published: Jul 1, 2022

Keywords: Laparoscopic Instrument Detection; Point Clouds; Sterility; Optical Plastic

There are no references for this article.