Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows

Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical... Current Directions in Biomedical Engineering 2019;5(1):37-40 Richard Bieck*, Reinhard Fuchs, Thomas Neumuth Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows https://doi.org/10.1515/cdbme-2019-0010 Abstract: We introduce a wearable-based recognition 1 Introduction system for the classification of natural hand gestures during dynamic activities with surgical instruments. An armband- The digital operating room (OR) is a highly outcome- and based circular setup of eight EMG-sensors was used to cost-driven environment and personnel, and hospital superficially measure the muscle activation signals over the providers are continuously challenged to improve operation broadest cross-section of the lower arm. Instrument-specific quality and efficiency. In response, one aspect of surface EMG (sEMG) data acquisition was performed for 5 optimization is to recognize the intraoperative workflow distinct instruments. In a first proof-of-concept study, EMG based on the surgical work steps or used resources, like data were analyzed for unique signal courses and features, surgical instruments. There exists a wide range of employed and in a subsequent classification, both decision tree (DTR) technologies, e.g. cameras, scales as well as force, RFID and and shallow artificial neural network (ANN) classifiers were acceleration sensor, that involve some form of mounting or trained. For DTR, an ensemble bagging approach reached fixation to the OR instrumentation [1,2]. The main precision and recall rates of 0.847 and 0.854, respectively. drawbacks are the limited line of sight into the work area, The ANN network architecture was configured to mimic the sensor contamination, as well as limited stationary ensemble-like structure of the DTR and achieved 0.952 and acquisition capabilities and handling overhead for medical 0.953 precision and recall rates, respectively. In a subsequent personnel. With already limited space and time resources in multi-user study, classification achieved 70 % precision. the OR, touchless applications were introduced for the Main errors potentially arise for instruments with similar various tasks in the medical domain [3]. Yet in the operating gripping style and performed actions, interindividual room, the functionalization of wearables for the identification variations in the acquisition procedure as well as muscle tone of surgical tasks, e.g. using the recognition of natural user and activation magnitude. Compared to hand-mounted sensor gestures, is still unexploited. We, therefore, present an systems, the lower arm setup does not alter the haptic approach for a wearable-based recognition system using experience or the instrument gripping, which is critical, multi-EMG data of the surgeon’s lower arm movement. The especially in an intraoperative environment. Currently, system uses a circular sensor setup to acquire activity data of drawbacks of the fixed consumer product setup are the the surgeon through grasping information of the lower arm limited data sampling rate and the denial of frequency during usage of surgical instruments. We assume that the features into the processing pipeline. usage of a specific surgical instrument induces distinguishable recruitment of muscle groups, that is superficially measurable. Keywords: Natural User Interface, Gesture Recognition, We provide a proof-of-concept study and first multi-user Workflow Recognition, EMG Processing, Machine Learning classification, outlining the potential of the classification approach. 2 Methods & Material ______ 2.1 Instrument Selection *Corresponding author: Richard Bieck: Innovation Center Computer Assisted Surgery, University of Leipzig, Semmelweisstraße 14, 04103 Leipzig, Germany, e-mail: Since the number of surgical instruments to cover is far too richard.bieck@iccas.de large and would exceed and diminish the explorational Reinhard Fuchs, Thomas Neumuth: Innovation Center character of the work, we limited the selection to instruments Computer Assisted Surgery, Leipzig, Germany Open Access. © 2019 Richard Bieck et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. R. Bieck et al., Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows — 38 with distinct types of usage (Tab.1). Following the taxonomy from [4], we selected instruments from general surgery groups for (I) cutting and dissecting, (II) clamping and occluding and (III) grasping and holding. Additionally, we differentiated between a passive and active mode of instrument usage. The passive mode is defined by the characteristic holding gesture performed with the hand after picking up the instrument without additional actions to exclude hand movement usually covered by interacting with the scrub nurse. The active mode is defined by the characteristic hand gestures performed to apply the instrument functionality to the situs. All selected instruments can be distinguished in terms of occupied hand parts as well as the duration of active usage. Table 1: Overview of used surgical instruments and their corresponding type and movement characteristics Instrument Instrument Type Hand Movement Figure 1: The signal processing pipeline with the main work [4] Characteristic steps: (a) Instrument selection, (b) lower arm signal (Gripping Force acquisition, (c) EMG channel correspondence, (d) signal Duration) processing and feature extraction and (e) classification. Straight Clamp Clamping & Ring finger against Bluetooth master client connection. Using the circular EMG Occluding, Grasping thumb (spontaneous) setup of the Myo, we were able to measure the superficial & Holding / Hook lower arm muscle activity of interest occupied during hand, Forceps Grasping & Holding Index and middle and finger and wrist movements. The use of a specific / Hold finger against the thumb (continuous) surgical instrument and the corresponding generation of Perforator Cutting & Dissecting Index finger against sEMG-signals will subsequently be called as instrument / Pinch thumb (continuous) activity. Surgical instrument activities were measured for one user in a Bone Punch Cutting & Dissecting All fingers against / Punch thumb (spontaneous) proof-of-concept study followed by six users in a follow-up study. For each user, the acquisition was performed in one Sharp Spoon Cutting & Dissecting Index finger against / Carve the thumb, extensive session and with minimal pauses to compensate for sEMG wrist movement variability and sensibility as well as intrinsic acquisition (continuous) errors [7]. Each instrument activity was performed within a timespan of four seconds, followed by a rest phase of four seconds to improve signal separation leading to 15 unique 2.2 Signal Acquisition datasets per instrument and usage mode, respectively. We For the signal acquisition, we used a commercially available used the quadrated, absolute raw channel data and smoothed Myo Gesture Control Armband (Thalmic Labs, USA), a with a moving average-filter (W=50). All signal processing wireless multi-sensor armband with an 8-channel circular actions were performed using MATLAB 2016b. EMG sensor setup. Each sensor uses three dry electrodes of medical grade stainless steel for reference mass-based surface 2.3 Feature Selection & Classification measurement with a fixed sample rate of 200 sps. Concerning Based on the understanding of muscle physiology, we sEMG measurement standards, the armband is placed on the expected a causal relationship between different instrument prominent bulge of the lower arm where the main muscle activities and generated lower arm muscle potentials. mass is formed [6]. With an expandable band diameter Initially, we investigated the occurrence of unique sEMG between 19 and 34 cm and less than 100g weight, the signal curves with minimal cross-correlation of EMG armband is highly flexible and unobtrusive for the user. The channels with a standard algorithm for discrete cross- Myo supports raw EMG sensor data acquisition based on a R. Bieck et al., Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows — 39 correlation. Subsequently, feature definition and analysis behaviour to Bootstrapping-Ensemble-Approaches for DTR steps were performed on these unique channels based on learning. established feature sets for gesture-based sEMG-signal classification [8]. We tested ten different features in the time 3 Results domain to characterize statistical as well as shape-related signal properties. Power spectra and density functions were The investigation of channel-related signals during investigated, however, since spectral components of muscle instrument usage showed the most prominent activations for sEMG-signals range between 0 to 500 Hz [9] a significant channels 3, 4 and 5 for all instruments. Channels 6, 7 and 8 part of higher frequency information could not be acquired showed increased activation mainly for the forceps, straight due to the limited sampling rate of the EMG sensors. clamp and bone punch. Channel 2 was most sensitive for Emphasis was thus put on the time series analysis of the perforator and sharp spoon activities. Channel 1 noticeably instrument activities. For the identification of key features for had no traceable signal activation over all recorded activities. instrument classification, dependent two-sided t-tests and The cross-correlation of channel data showed that channels one-way variance (ANOVA) testing over all calculated 4-7 had the lowest similarity values across all instrument feature sets for all instrument activities were performed. For activities. The mean and variance analysis for signal features the signal classification, observation sets were generated identified the mean waveform length (mWL) as the most using the identified unique features (7 features x 5 channels). expressive property for all instruments, followed by a log Each classifier was then trained and validated based on a 5- detector (LOG) and signal variance (VAR). Overall, seven fold cross-validation method. We initially chose a Bagging- features were then used to form the feature vector for the Ensemble decision tree approach with a forest set of C = 30 subsequent classification studies. trees (Fig.2) due to the algorithms` robustness and low computational cost. Table 2: Results for the signal analysis of unique EMG channels and most expressive signal features. (mWL – mean waveform length, LOG – log detector, SSC – sign slope change, VAR – variance) Instrument Active EMG Unique Unique Channels Channels Features Straight Clamp 2-6 6 mWL, LOG, SSC Forceps 2-5, 7 7 mWL, VAR Perforator 2-6 5 mWL, LOG Bone Punch 2-6 4 mWL, LOG Sharp Spoon 2-5 5 mWL, LOG Figure 2: Overview of the employed artificial neural network architecture. A hidden layer is locally connected For the classification approaches, precision and recall rates to a following hidden layer, forming a smaller subnetwork of were calculated from the 5-fold cross-validation for both the 2 layers inside the ANN superstructure. proof-of-concept and the follow-up group. In both studies, Additionally, we investigated a way to introduce DTR- the ANN performed better than the DTR across all Ensemble-like training behaviour into a multi-layer ANN. instruments. For both studies, the highest precision and recall Similarly, to CNN architectures, we used local connectivity rates were achieved for active modes of the bone punch of consecutively connecting hidden layers forming parallel followed by the forceps. Lowest classification rates were subnetworks of layers that feed into a final output layer (Fig. reached for the perforator and the sharp spoon. Overall, the 3). ANN training was performed with a tanh-transfer function classifiers showed considerable precision and recall loss in for neurons and Bayesian Regularization (BR) as the the follow-up study. Main precision deficits were identified objective function. Input vectors consisted of the feature in the classification of passive bone punch and active, sharp subsets, and feature values were normalised to a range of [- spoon activity. 1,+1] to support transfer function properties and to avoid value training errors. We hypothesised a similar training R. Bieck et al., Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows — 40 Table 3: Results for the signal classification for the proof-of- multi-EMG-sensor armband. To our knowledge, no concept and follow-up studies. Precision and recall rates are recognition system currently employs EMG sensors for the averaged over both activity modes. classification of surgical activity. The disadvantages of the armband system as a consumer product lie in the limited Study Classifier Precision Recall biosignal acquisition capabilities and need to be compensated Proof-of- DTR 0.847 0.854 using more sophisticated classification approaches. Future Concept (n=1) work will address classification improvements for larger ANN 0.952 0.953 study groups. Follow-Up DTR 0.622 0.640 (n=6) ANN 0.701 0.712 Author Statement Research funding: The author state that no funding was involved. Conflict of interest: Authors state no conflict of interest. Informed consent: Informed consent has been 4 Discussion obtained from all individuals included in this study. Ethical approval: The research related to human use complied with Compared to exclusively hand-mounted sensor systems, the all the relevant national regulations, institutional policies and armband setup does not alter the haptic experience or the was performed in accordance with the tenets of the Helsinki instrument gripping, which is critical, especially in an Declaration and has been approved by the authors' intraoperative environment [3]. The number of employed institutional review board or equivalent committee. sensors is comparable to other setups for sEMG-based muscle activity classification [8]. Preparation of skin should be considered for future studies but is ultimately a drawback References for ease-of-use in the preparation phase for surgery. Raw signal data was not normalised due to the constraints of [1] Meißner, C., Meixensberger, J., Pretschner, A., & Neumuth, T. intraindividual measurement in a single recording session. (2014). Sensor-based surgical activity recognition in unconstrained environments. Minimally Invasive Therapy & Allied Technologies, However, noise generation due to lower arm movement, 23(4), 198-205. accumulating muscle fatigue during the recording session as [2] Kranzfelder, M., Schneider, A., Fiolka, A., Schwan, E., Gillen, S., well as untreated skin preparation could potentially affect Wilhelm, D., ... & Feussner, H. (2013). Real-time instrument signal acquisition quality. The predetermined sample rate detection in minimally invasive surgery using radiofrequency identification technology. journal of surgical research, 185(2), 704- denies the acquisition of muscle activity signals [8]. Thus, a considerable part of the expected signal frequency spectrum [3] Kolodzey, L., Grantcharov, P. D., Rivas, H., Schijven, M. P., & could not be included for signal analysis. The classification Grantcharov, T. P. (2017). Wearable technology in the operating results of the proof-of-concept study reveal that the apparent room: a systematic review. BMJ Innovations, 3(1), 55-63. physiological difference for using surgical instruments can be [4] [5] Nemitz, R. (2017). Surgical Instrumentation-eBook: An Interactive identified using an EMG sensor setup. However, the follow- Approach. Elsevier Health Sciences. up study revealed that likely due to strong interindividual [6] Stegeman, D., & Hermens, H. (2007). Standards for surface differences, the current DTR and ANN classifiers are not electromyography: The European project Surface EMG for non- adequately tailored towards a multi-user recognition invasive assessment of muscles (SENIAM). Enschede: Roessingh Research and Development, 108-12. application. Consequently, alternate classifier approaches, [7] Halaki, M., & Ginn, K. (2012). Normalization of EMG signals: To e.g. for time series prediction, should be employed to exploit normalize or not to normalize and what to normalize to?. In sequential information and connectedness of signal features Computational intelligence in electromyography analysis-a during surgical instrument usage. perspective on current applications and future challenges. IntechOpen. [8] Chowdhury, R., Reaz, M., Ali, M., Bakar, A., Chellappan, K., & Chang, T. (2013). Surface electromyography signal processing and 5 Conclusion classification techniques. Sensors, 13(9), 12431-12466. [9] De Luca, C. J. (1993). Use of the surface EMG signal for performance evaluation of back muscles. Muscle & Nerve: Official We successfully implemented a signal analysis and Journal of the American Association of Electrodiagnostic classification pipeline that enables the recognition of specific Medicine, 16(2), 210-216. used surgical instruments using a commercially available http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Current Directions in Biomedical Engineering de Gruyter

Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows

Download PDF
4 pages

Loading next page...
 
/lp/de-gruyter/surface-emg-based-surgical-instrument-classification-for-dynamic-xnQ47Ky9o3

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
de Gruyter
Copyright
© 2019 by Walter de Gruyter Berlin/Boston
eISSN
2364-5504
DOI
10.1515/cdbme-2019-0010
Publisher site
See Article on Publisher Site

Abstract

Current Directions in Biomedical Engineering 2019;5(1):37-40 Richard Bieck*, Reinhard Fuchs, Thomas Neumuth Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows https://doi.org/10.1515/cdbme-2019-0010 Abstract: We introduce a wearable-based recognition 1 Introduction system for the classification of natural hand gestures during dynamic activities with surgical instruments. An armband- The digital operating room (OR) is a highly outcome- and based circular setup of eight EMG-sensors was used to cost-driven environment and personnel, and hospital superficially measure the muscle activation signals over the providers are continuously challenged to improve operation broadest cross-section of the lower arm. Instrument-specific quality and efficiency. In response, one aspect of surface EMG (sEMG) data acquisition was performed for 5 optimization is to recognize the intraoperative workflow distinct instruments. In a first proof-of-concept study, EMG based on the surgical work steps or used resources, like data were analyzed for unique signal courses and features, surgical instruments. There exists a wide range of employed and in a subsequent classification, both decision tree (DTR) technologies, e.g. cameras, scales as well as force, RFID and and shallow artificial neural network (ANN) classifiers were acceleration sensor, that involve some form of mounting or trained. For DTR, an ensemble bagging approach reached fixation to the OR instrumentation [1,2]. The main precision and recall rates of 0.847 and 0.854, respectively. drawbacks are the limited line of sight into the work area, The ANN network architecture was configured to mimic the sensor contamination, as well as limited stationary ensemble-like structure of the DTR and achieved 0.952 and acquisition capabilities and handling overhead for medical 0.953 precision and recall rates, respectively. In a subsequent personnel. With already limited space and time resources in multi-user study, classification achieved 70 % precision. the OR, touchless applications were introduced for the Main errors potentially arise for instruments with similar various tasks in the medical domain [3]. Yet in the operating gripping style and performed actions, interindividual room, the functionalization of wearables for the identification variations in the acquisition procedure as well as muscle tone of surgical tasks, e.g. using the recognition of natural user and activation magnitude. Compared to hand-mounted sensor gestures, is still unexploited. We, therefore, present an systems, the lower arm setup does not alter the haptic approach for a wearable-based recognition system using experience or the instrument gripping, which is critical, multi-EMG data of the surgeon’s lower arm movement. The especially in an intraoperative environment. Currently, system uses a circular sensor setup to acquire activity data of drawbacks of the fixed consumer product setup are the the surgeon through grasping information of the lower arm limited data sampling rate and the denial of frequency during usage of surgical instruments. We assume that the features into the processing pipeline. usage of a specific surgical instrument induces distinguishable recruitment of muscle groups, that is superficially measurable. Keywords: Natural User Interface, Gesture Recognition, We provide a proof-of-concept study and first multi-user Workflow Recognition, EMG Processing, Machine Learning classification, outlining the potential of the classification approach. 2 Methods & Material ______ 2.1 Instrument Selection *Corresponding author: Richard Bieck: Innovation Center Computer Assisted Surgery, University of Leipzig, Semmelweisstraße 14, 04103 Leipzig, Germany, e-mail: Since the number of surgical instruments to cover is far too richard.bieck@iccas.de large and would exceed and diminish the explorational Reinhard Fuchs, Thomas Neumuth: Innovation Center character of the work, we limited the selection to instruments Computer Assisted Surgery, Leipzig, Germany Open Access. © 2019 Richard Bieck et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. R. Bieck et al., Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows — 38 with distinct types of usage (Tab.1). Following the taxonomy from [4], we selected instruments from general surgery groups for (I) cutting and dissecting, (II) clamping and occluding and (III) grasping and holding. Additionally, we differentiated between a passive and active mode of instrument usage. The passive mode is defined by the characteristic holding gesture performed with the hand after picking up the instrument without additional actions to exclude hand movement usually covered by interacting with the scrub nurse. The active mode is defined by the characteristic hand gestures performed to apply the instrument functionality to the situs. All selected instruments can be distinguished in terms of occupied hand parts as well as the duration of active usage. Table 1: Overview of used surgical instruments and their corresponding type and movement characteristics Instrument Instrument Type Hand Movement Figure 1: The signal processing pipeline with the main work [4] Characteristic steps: (a) Instrument selection, (b) lower arm signal (Gripping Force acquisition, (c) EMG channel correspondence, (d) signal Duration) processing and feature extraction and (e) classification. Straight Clamp Clamping & Ring finger against Bluetooth master client connection. Using the circular EMG Occluding, Grasping thumb (spontaneous) setup of the Myo, we were able to measure the superficial & Holding / Hook lower arm muscle activity of interest occupied during hand, Forceps Grasping & Holding Index and middle and finger and wrist movements. The use of a specific / Hold finger against the thumb (continuous) surgical instrument and the corresponding generation of Perforator Cutting & Dissecting Index finger against sEMG-signals will subsequently be called as instrument / Pinch thumb (continuous) activity. Surgical instrument activities were measured for one user in a Bone Punch Cutting & Dissecting All fingers against / Punch thumb (spontaneous) proof-of-concept study followed by six users in a follow-up study. For each user, the acquisition was performed in one Sharp Spoon Cutting & Dissecting Index finger against / Carve the thumb, extensive session and with minimal pauses to compensate for sEMG wrist movement variability and sensibility as well as intrinsic acquisition (continuous) errors [7]. Each instrument activity was performed within a timespan of four seconds, followed by a rest phase of four seconds to improve signal separation leading to 15 unique 2.2 Signal Acquisition datasets per instrument and usage mode, respectively. We For the signal acquisition, we used a commercially available used the quadrated, absolute raw channel data and smoothed Myo Gesture Control Armband (Thalmic Labs, USA), a with a moving average-filter (W=50). All signal processing wireless multi-sensor armband with an 8-channel circular actions were performed using MATLAB 2016b. EMG sensor setup. Each sensor uses three dry electrodes of medical grade stainless steel for reference mass-based surface 2.3 Feature Selection & Classification measurement with a fixed sample rate of 200 sps. Concerning Based on the understanding of muscle physiology, we sEMG measurement standards, the armband is placed on the expected a causal relationship between different instrument prominent bulge of the lower arm where the main muscle activities and generated lower arm muscle potentials. mass is formed [6]. With an expandable band diameter Initially, we investigated the occurrence of unique sEMG between 19 and 34 cm and less than 100g weight, the signal curves with minimal cross-correlation of EMG armband is highly flexible and unobtrusive for the user. The channels with a standard algorithm for discrete cross- Myo supports raw EMG sensor data acquisition based on a R. Bieck et al., Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows — 39 correlation. Subsequently, feature definition and analysis behaviour to Bootstrapping-Ensemble-Approaches for DTR steps were performed on these unique channels based on learning. established feature sets for gesture-based sEMG-signal classification [8]. We tested ten different features in the time 3 Results domain to characterize statistical as well as shape-related signal properties. Power spectra and density functions were The investigation of channel-related signals during investigated, however, since spectral components of muscle instrument usage showed the most prominent activations for sEMG-signals range between 0 to 500 Hz [9] a significant channels 3, 4 and 5 for all instruments. Channels 6, 7 and 8 part of higher frequency information could not be acquired showed increased activation mainly for the forceps, straight due to the limited sampling rate of the EMG sensors. clamp and bone punch. Channel 2 was most sensitive for Emphasis was thus put on the time series analysis of the perforator and sharp spoon activities. Channel 1 noticeably instrument activities. For the identification of key features for had no traceable signal activation over all recorded activities. instrument classification, dependent two-sided t-tests and The cross-correlation of channel data showed that channels one-way variance (ANOVA) testing over all calculated 4-7 had the lowest similarity values across all instrument feature sets for all instrument activities were performed. For activities. The mean and variance analysis for signal features the signal classification, observation sets were generated identified the mean waveform length (mWL) as the most using the identified unique features (7 features x 5 channels). expressive property for all instruments, followed by a log Each classifier was then trained and validated based on a 5- detector (LOG) and signal variance (VAR). Overall, seven fold cross-validation method. We initially chose a Bagging- features were then used to form the feature vector for the Ensemble decision tree approach with a forest set of C = 30 subsequent classification studies. trees (Fig.2) due to the algorithms` robustness and low computational cost. Table 2: Results for the signal analysis of unique EMG channels and most expressive signal features. (mWL – mean waveform length, LOG – log detector, SSC – sign slope change, VAR – variance) Instrument Active EMG Unique Unique Channels Channels Features Straight Clamp 2-6 6 mWL, LOG, SSC Forceps 2-5, 7 7 mWL, VAR Perforator 2-6 5 mWL, LOG Bone Punch 2-6 4 mWL, LOG Sharp Spoon 2-5 5 mWL, LOG Figure 2: Overview of the employed artificial neural network architecture. A hidden layer is locally connected For the classification approaches, precision and recall rates to a following hidden layer, forming a smaller subnetwork of were calculated from the 5-fold cross-validation for both the 2 layers inside the ANN superstructure. proof-of-concept and the follow-up group. In both studies, Additionally, we investigated a way to introduce DTR- the ANN performed better than the DTR across all Ensemble-like training behaviour into a multi-layer ANN. instruments. For both studies, the highest precision and recall Similarly, to CNN architectures, we used local connectivity rates were achieved for active modes of the bone punch of consecutively connecting hidden layers forming parallel followed by the forceps. Lowest classification rates were subnetworks of layers that feed into a final output layer (Fig. reached for the perforator and the sharp spoon. Overall, the 3). ANN training was performed with a tanh-transfer function classifiers showed considerable precision and recall loss in for neurons and Bayesian Regularization (BR) as the the follow-up study. Main precision deficits were identified objective function. Input vectors consisted of the feature in the classification of passive bone punch and active, sharp subsets, and feature values were normalised to a range of [- spoon activity. 1,+1] to support transfer function properties and to avoid value training errors. We hypothesised a similar training R. Bieck et al., Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows — 40 Table 3: Results for the signal classification for the proof-of- multi-EMG-sensor armband. To our knowledge, no concept and follow-up studies. Precision and recall rates are recognition system currently employs EMG sensors for the averaged over both activity modes. classification of surgical activity. The disadvantages of the armband system as a consumer product lie in the limited Study Classifier Precision Recall biosignal acquisition capabilities and need to be compensated Proof-of- DTR 0.847 0.854 using more sophisticated classification approaches. Future Concept (n=1) work will address classification improvements for larger ANN 0.952 0.953 study groups. Follow-Up DTR 0.622 0.640 (n=6) ANN 0.701 0.712 Author Statement Research funding: The author state that no funding was involved. Conflict of interest: Authors state no conflict of interest. Informed consent: Informed consent has been 4 Discussion obtained from all individuals included in this study. Ethical approval: The research related to human use complied with Compared to exclusively hand-mounted sensor systems, the all the relevant national regulations, institutional policies and armband setup does not alter the haptic experience or the was performed in accordance with the tenets of the Helsinki instrument gripping, which is critical, especially in an Declaration and has been approved by the authors' intraoperative environment [3]. The number of employed institutional review board or equivalent committee. sensors is comparable to other setups for sEMG-based muscle activity classification [8]. Preparation of skin should be considered for future studies but is ultimately a drawback References for ease-of-use in the preparation phase for surgery. Raw signal data was not normalised due to the constraints of [1] Meißner, C., Meixensberger, J., Pretschner, A., & Neumuth, T. intraindividual measurement in a single recording session. (2014). Sensor-based surgical activity recognition in unconstrained environments. Minimally Invasive Therapy & Allied Technologies, However, noise generation due to lower arm movement, 23(4), 198-205. accumulating muscle fatigue during the recording session as [2] Kranzfelder, M., Schneider, A., Fiolka, A., Schwan, E., Gillen, S., well as untreated skin preparation could potentially affect Wilhelm, D., ... & Feussner, H. (2013). Real-time instrument signal acquisition quality. The predetermined sample rate detection in minimally invasive surgery using radiofrequency identification technology. journal of surgical research, 185(2), 704- denies the acquisition of muscle activity signals [8]. Thus, a considerable part of the expected signal frequency spectrum [3] Kolodzey, L., Grantcharov, P. D., Rivas, H., Schijven, M. P., & could not be included for signal analysis. The classification Grantcharov, T. P. (2017). Wearable technology in the operating results of the proof-of-concept study reveal that the apparent room: a systematic review. BMJ Innovations, 3(1), 55-63. physiological difference for using surgical instruments can be [4] [5] Nemitz, R. (2017). Surgical Instrumentation-eBook: An Interactive identified using an EMG sensor setup. However, the follow- Approach. Elsevier Health Sciences. up study revealed that likely due to strong interindividual [6] Stegeman, D., & Hermens, H. (2007). Standards for surface differences, the current DTR and ANN classifiers are not electromyography: The European project Surface EMG for non- adequately tailored towards a multi-user recognition invasive assessment of muscles (SENIAM). Enschede: Roessingh Research and Development, 108-12. application. Consequently, alternate classifier approaches, [7] Halaki, M., & Ginn, K. (2012). Normalization of EMG signals: To e.g. for time series prediction, should be employed to exploit normalize or not to normalize and what to normalize to?. In sequential information and connectedness of signal features Computational intelligence in electromyography analysis-a during surgical instrument usage. perspective on current applications and future challenges. IntechOpen. [8] Chowdhury, R., Reaz, M., Ali, M., Bakar, A., Chellappan, K., & Chang, T. (2013). Surface electromyography signal processing and 5 Conclusion classification techniques. Sensors, 13(9), 12431-12466. [9] De Luca, C. J. (1993). Use of the surface EMG signal for performance evaluation of back muscles. Muscle & Nerve: Official We successfully implemented a signal analysis and Journal of the American Association of Electrodiagnostic classification pipeline that enables the recognition of specific Medicine, 16(2), 210-216. used surgical instruments using a commercially available

Journal

Current Directions in Biomedical Engineering – de Gruyter

Published: Sep 1, 2019

Keywords: Natural User Interface; Gesture Recognition; Workflow Recognition; EMG Processing; Machine Learning

There are no references for this article.