Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Deep Learning for Prediction of Diaphragm Activity from the Surface Electromyogram

Deep Learning for Prediction of Diaphragm Activity from the Surface Electromyogram Current Directions in Biomedical Engineering 2019;5(1):17–20 Niclas Bockelmann, Jan Graßhoff, Lasse Hansen, Giacomo Bellani, Mattias P. Heinrich, and Philipp Rostalski Deep Learning for Prediction of Diaphragm Activity from the Surface Electromyogram https://doi.org/10.1515/cdbme-2019-0005 particular it was shown that PVA is associated with adverse outcomes such as a higher mortality and a prolonged duration Abstract: The electrical activity of the diaphragm (EAdi) is a of mechanical ventilation [2, 12]. The electrical activity of the novel monitoring parameter for patients under assisted venti- diaphragm (EAdi) was introduced as a novel monitoring pa- lation and is used for assessing the patient’s neural respiratory rameter for the assessment of diaphragmatic function. It repre- drive. It is recorded by an array of electrodes placed inside the sents the envelope of the crural diaphragm’s electromyogram esophagus at the level of the diaphragm. A noninvasive alter- (EMG), measured by a special nasogastric catheter equipped native is the measurement of the electromyogram by means of with an array of electrodes placed inside the esophagus. It skin surface electrodes (sEMG). The respiratory sEMG signal, was suggested that the amplitude of EAdi directly reflects the however, is subject to electrocardiographic interference and breath-to-breath neural output of the respiratory center [5]. In crosstalk from other muscles and may also pick up a different daily clinical practice, the insertion and proper positioning of part of the muscular activity. In this work, we propose to use the esophageal catheter can be cumbersome, which limits the a deep neural network to predict the electrical activity of the routine application of EAdi. A non-invasive alternative is the diaphragm as measured by a nasogastric catheter from sEMG skin surface electromyogram (sEMG), measured on the pa- measurements. We use a ResNet based architecture and train tients thorax. Despite enabling a direct clinical application, the the network to directly regress the EAdi as a supervised learn- respiratory sEMG comes with its own intricacies: the signal- ing task – we further investigate a heatmap based regression to-noise ratio is small and the measured signal is subject to approach. The proposed methods are evaluated on a clinical strong electrocardiographic interference and muscle crosstalk. dataset consisting of 77 recordings from mechanically ven- Also the electrodes are located at a different spatial position tilated patients. For the direct regression task, the network’s in comparison to the internally placed electrodes. Some re- predictions reach a Pearson correlation coefficient (PCC) of searchers have considered the sEMG as a measure for pa- 0.818 with EAdi on the hold-out set. The heatmap regression tient activity [5] but it is still a subject of ongoing research increases the PCC to 0.830 while at the same time achieving a to what extent it is a viable surrogate for EAdi. In [1] the lower mean absolute error, indicating a superior performance. sEMG data were processed by detecting and gating out car- From our results we conclude that sEMG measurements may diac artifacts, then the amplitude of the sEMG’s envelope was be used to predict the internal activity of the diaphragm as compared to EAdi. The authors found a good correlation be- measured invasively using a nasogastric catheter. tween EAdi and one of the sEMG channels when averaging Keywords: diaphragm activity, sEMG, deep learning over several breaths. In this work, we consider the problem of predicting the EAdi signal on a sample-by-sample basis from all sEMG channels. We use a deep neural network to learn the 1 Introduction relation between the measured sEMG signals and the internal diaphragm activity. We train a ResNet based architecture and, In the light of recent studies regarding the prevalence and con- given the high noise level of the input signals, we further in- sequence of patient-ventilator asynchrony (PVA), researchers vestigate a heatmap regression approach. have emphasized the importance of monitoring the respiratory activity of patients under assisted spontaneous ventilation. In 2 Material and Methods Niclas Bockelmann, Jan Graßhoff, Philipp Rostalski, Institute for Electrical Engineering in Medicine, Universität zu Lübeck niclas.bockelmann@student.uni-luebeck.de 2.1 Data Lasse Hansen, Mattias P. Heinrich, Institute of Medical Informatics, Universität zu Lübeck The data consist of 77 clinical recordings from a total of 23 hansen@imi.uni-luebeck.de patients receiving pressure support ventilation, with record- Giacomo Bellani, Università degli Studi Milano Bic- ing lengths varying between 2 and 15 minutes. The data were occa, Dipartimento di Medicina e Chirurgia, Milano, Italy giacomo.bellani1@unimib.it Open Access. © 2019 Niclas Bockelmann, Jan Graß hoff, Lasse Hansen, Giacomo Bellani, Mattias P. Heinrich, Philipp Rostalski published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. 5 256 256 18 9 N. Bockelmann et al., Deep Learning for Prediction of Diaphragm Activity recorded in an 8-bed ICU at the San Gerardo Hospital (Monza, Italy), see [1] for further details on the data collection. Each dataset contains four raw sEMG channels and EAdi as the re- gression target, respectively sampled at 500 Hz and 100 Hz. The sEMG channels were recorded with pairs of electrodes positioned (1) bilaterally at the lower costal margin on the midclavicular line ("costmar"), (2) bilaterally in the second in- tercostal space ("intercost"), (3) parallel to the muscle fibers above the rectus abdominis muscle ("rect"), (4) parallel to the muscle fibers above the sternocleidomastoid muscle ("sterno") and (5) bilaterally 5 cm below the costmar-electrodes ("below- 7 x3 cost"). The first three channels are included in all recordings, while from the last two channels only one is available in each 7 x3 patient – the respective other channel was set to zero prior to training. An exemplary segment of the raw sEMG data is de- 7 x3 picted in the top part of Figure 1. x3 2.2 Architecture x3 For the proposed supervised learning task we use a deep con- volutional neural network (DCNN) with 34 layers (cf. Fig- ure 1). The number of layers of the DCNN was empirically determined to yield a sufficient receptive field and degree of semantic abstraction considering the challenging problem of predicting diaphragm activity from raw sEMG signals. To ease the training of the CNN we make use of residual blocks by incorporating identity skip connections [6] between convolu- Fig. 1: Overview of the proposed network architecture. Light gray tional layers. Features are downsampled after every third resid- boxes represent residual blocks containing two convolution lay- ual block using strided convolutions. At the same time we dou- ers and a skip connection. Dark gray boxes represent upsampling ble the number of feature channels to enrich semantic infor- modules containing an upsampling operator and additional con- mation. Motivated by [3], a kernel size of seven is used in volutional layers. Numbers denote the kernel size (on box), signal all residual blocks, whereas in the first layer of the network size (left of box) and the number of feature channels (right of box). The numbers next to the curly bracket indicate repetition of resid- a large kernel of size 51 is employed to enlarge the receptive ual blocks. Convolutional layers are followed by batch normaliza- field and capture and process low level signal features. Follow- tion and non-linear activations. ing the residual blocks, upsampling and convolution operators are used to propagate semantic features to higher resolution layers. A final convolutional layer with kernel size seven maps to time intervals of approx. 10 seconds), that are randomly the features to the number of output channels. Rectified Linear cropped from the input data. The corresponding EAdi signal Units (ReLUs) are used throughout the network as non-linear of size 1024 serves as the regression target. Prior to training, activation functions. For faster training convergence and im- sEMG signals are normalized to have standard deviation of proved generalization we employ batch normalization [7]. one. One of the challenges in predicting the EAdi is that it may be arbitrarily scaled against other EMG channels due to differences in the electrode/tissue impedance [1]. These scal- 2.3 Training ing factors between the EMG modalities can not be predicted from the data itself. Therefore, prior to training, we normalize Input to the DCNN are the previously described sEMG signals the EAdi amplitude to either the intercost or the costmar enve- (costmar, intercost, rect, belowcost and sterno). To deal with lope (depending on which of the two has a higher correlation the missing channels in the recordings (belowcost or sterno) to EAdi), and thereby remove the scaling ambiguity. We train we set the respective input channel to zero. The network is the proposed DCNN architecture with two different regression trained with fixed length patches of size 5120 (corresponding tasks: (1) direct regression of the EAdi signal (output channel 640 640 320 320 160 19 N. Bockelmann et al., Deep Learning for Prediction of Diaphragm Activity Tab. 1: Results of different methods on the hold-out set. the direct regression and the heatmap regression as DirectNet and HeatNet, respectively. As a further baseline we evaluate Method PCC MAE HS the same performance metrics on the conventional envelope of DirectNet 0.818 ± 0.198 1.071 ± 0.835 0.856 ± 0.064 the costmar sEMG channel (calculated by gating out cardiac HeatNet 0.830 ± 0.193 0.835 ± 0.700 0.888 ± 0.044 artifacts first and then using a running root-mean-square filter Envelope 0.799 ± 0.197 1.219 ± 0.932 0.765 ± 0.066 as described in [1]). In an additional experiment we investigate the relevance of the different input signals for the regression Tab. 2: Results of the trained networks on the hold-out set for only task. Therefore, we evaluate our trained networks using only a using a single input channel while other channels are set to zero. single channel as the input while setting the remaining chan- nels to zero and report the corresponding evaluation metrics Method Channel PCC MAE HS for each sEMG channel separately. Lastly, we report the infer- DirectNet Costmar 0.787 ± 0.224 1.400 ± 1.245 0.828 ± 0.074 DirectNet Intercostn 0.176 ± 0.269 1.560 ± 1.158 0.711 ± 0.032 ence times of our regression model: One sEMG signal patch of DirectNet Rect -0.003 ± 0.196 1.570 ± 1.166 0.707 ± 0.031 10 s is processed in approximately 6 ms on an NVIDIA RTX DirectNet Belowcost -0.035 ± 0.220 1.570 ± 1.163 0.706 ± 0.032 2070 GPU and 40 ms on an INTEL Core i7-6700K CPU. DirectNet Sterno 0.020 ± 0.207 1.566 ± 1.159 0.707 ± 0.031 HeatNet Costmar 0.812 ± 0.208 0.895 ± 0.719 0.860 ± 0.066 HeatNet Intercostn 0.000 ± 0.208 1.576 ± 1.163 0.707 ± 0.031 HeatNet Rect -0.003 ± 0.202 1.578 ± 1.164 0.703 ± 0.035 HeatNet Belowcost -0.042 ± 0.241 1.581 ± 1.163 0.699 ± 0.031 3 Results and Discussion HeatNet Sterno -0.001 ± 0.212 1.574 ± 1.158 0.705 ± 0.031 The quantitative results of the different sEMG-based predic- 𝑛 = 1) and (2) heatmap regression (output channel 𝑛 = 128), tors are provided in Table 1. For the prediction of diaphragm where the regression target is a quantized Gaussian distribu- activity, both neural network approaches clearly outperform tion with a standard deviation 𝜎 = 2 µV and the mean taken the simple costmar envelope function, that has been used in from the EAdi signal. To obtain the final single channel sig- previous publications (e.g. [1]). In comparison to the direct nal we take a Gaussian smoothed argmax from the quantized regression, the application of a heatmap regression procedure heatmap. Regressing a heatmap target rather than single val- showed an improvement by 1.5% in PCC and by 22.0% in ues has been proven advantageous in terms of robustness and MAE, while the HS is improved by 3.2 percentage points. convergence of training in many different deep learning tasks These results indicate a superior performance of the heatmap [9, 11]. We implement our regression framework in PyTorch regression over the direct regression for our given task. The [10]. All models are trained for 650 epochs with a batch size performance metrics for evaluating the neural networks on a of 26 using Adam optimization [8] with an initial learning rate single sEMG channel are given in Table 2. It turns out, that for of 0.01. L1 and L2 loss is used for the direct and heatmap re- both architectures, the costmar channel is the best predictor gression, respectively. for EAdi. Still, we achieve the best performance, when using all sEMG channels at once, which might indicate, that all of the considered electrode positions on the thorax provide some 2.4 Experiments information on the internal diaphragm activity. An exemplary segment from a patient in the hold-out set is depicted in Fig- The recordings of the 77 patients are split into a training and ure 2 – in that segment the sEMG data had a particularly low test set with 52 and 25 recordings, respectively. To evaluate signal-to-noise ratio, which is reflected by the substantial noise the performance of the regressed predictions we report results visible in the costmar envelope. Both neural network meth- for three different metrics: the Pearson correlation coefficient ods nicely capture the internal diaphragm activity. Visually, (PCC), the mean absolute error (MAE) and the Hamming sim- the HeatNet appears to better match the on-off characteristic ilarity (HS). The reference for the evaluation is again the EAdi of the EAdi signal when comparing it to the direct regression signal. The Hamming similarity was calculated as a metric of approach (see seconds 12 to 15). Both neural networks have how well the derived signals can detect the timing of inspira- a high sensitivity to crosstalk artifacts in the sEMG signals tions and expirations compared to EAdi. To this end, we used (e.g. motion artifacts), which is exemplified in Figure 3. In a simple breath detector (based on an adaptive thresholding al- that segment, both the HeatNet regression and the sEMG en- gorithm similar to [4]) and calculated the Hamming distance velope show large artifacts, that do not correspond to internal of the detected inspiratory/expiratory samples to the inspira- diaphragm activity. This problem will be addressed in further tory/expiratory samples detected in EAdi. All results are ob- studies. tained from the recordings of the hold-out set and we refer to 20 N. Bockelmann et al., Deep Learning for Prediction of Diaphragm Activity 6 might be a viable method for noninvasively assessing respira- tion and neural output of mechanically ventilated patients. Fur- thermore, the possibility of doing inference on unseen sEMG data in real-time, even on CPU, might promote the feasibility of our approach in the clinical arena, e.g. for the detection of PVA. One of the remaining challenges is the rejection of mus- cle crosstalk in the sEMG, which we will further investigate in our future work. References 0 5 10 15 20 25 30 [1] G. Bellani et al. Measurement of diaphragmatic electrical time (s) activity by surface electromyography in intubated subjects and its relationship with inspiratory effort. Respiratory Care, Fig. 2: Exemplary result on a patient from the hold-out set. The 63(11):1341–1349, 2018. DirectNet and HeatNet results are shown for three concatenated [2] L. Blanch et al. Asynchronies during mechanical ventila- patches (each approx. ten seconds long). The conventional sEMG tion are associated with mortality. Intensive Care Medicine, envelope (costmar channel) and EAdi are given as a reference. 41(4):633–641, 2015. [3] W. Dai, C. Dai, S. Qu, J. Li, and S. Das. Very deep convo- lutional neural networks for raw waveforms. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 421–425. IEEE, 2017. [4] L. Estrada, A. Torres, L. Sarlabous, and R. Jané. Onset and offset estimation of the neural inspiratory time in surface diaphragm electromyography: A pilot study in healthy sub- jects. IEEE Journal of Biomedical and Health Informatics, 22(1):67–76, 2018. [5] G. Grasselli, M. Pozzi, and G. Bellani. Monitoring Respiratory Effort by Means of the Electrical Activity of the Diaphragm, pages 299–310. Springer, Cham, 2016. 52 54 56 58 60 [6] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning time (s) for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, Fig. 3: An instance of failure for the neural network predictor at 𝑡 = 52 s: Both the envelope and the HeatNet signal contain large [7] S. Ioffe and C. Szegedy. Batch normalization: Accelerating artifacts, which can probably be prescribed to muscle crosstalk. deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015. [8] Diederik P Kingma and Jimmy Ba. Adam: A method for 4 Conclusion stochastic optimization. arXiv preprint arXiv:1412.6980, [9] A. Newell, K. Yang, and J. Deng. Stacked hourglass net- In this paper, different architectures have been evaluated for works for human pose estimation. In European Conference the task of predicting the internal diaphragm activity from on Computer Vision, pages 483–499. Springer, 2016. multiple surface EMG signals. The heatmap-based regression [10] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. De- approach outperformed the direct regression architecture, with Vito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017. a PCC of 0.830 ± 0.193, a MAE of 0.835 ± 0.700 and a [11] C. Payer, D. Štern, H. Bischof, and M. Urschler. Regressing HS of 0.888 ± 0.044 compared to EAdi. The prediction of heatmaps for multiple landmark localization using cnns. In diaphragm activity was substantially improved compared to International Conference on Medical Image Computing and the conventional single-channel sEMG envelope. Based on the Computer-Assisted Intervention, pages 230–238. Springer, learned relationship between the externally measured sEMG signals and the internal diaphragm acitvity measured by a na- [12] A. W. Thille, Pablo Rodriguez, Belen Cabello, François Lel- louche, and Laurent Brochard. Patient-ventilator asynchrony sogastric catheter we hypothesize, that the former may be used during assisted mechanical ventilation. Intensive Care as a surrogate for EAdi in clinical applications. The results fur- Medicine, 32(10):1515–1522, 2006. ther confirm the findings of Bellani et al. [1] that the sEMG sEMG sEMG EAdi envelope envelope HeatNet EAdi HeatNet DirectNet (µV) (µV) (µV) (a.u.) (µV) (a.u.) (a.u.) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Current Directions in Biomedical Engineering de Gruyter

Deep Learning for Prediction of Diaphragm Activity from the Surface Electromyogram

Download PDF
4 pages

Loading next page...
 
/lp/de-gruyter/deep-learning-for-prediction-of-diaphragm-activity-from-the-surface-ZNopCvrZIW
Publisher
de Gruyter
Copyright
© 2019 by Walter de Gruyter Berlin/Boston
eISSN
2364-5504
DOI
10.1515/cdbme-2019-0005
Publisher site
See Article on Publisher Site

Abstract

Current Directions in Biomedical Engineering 2019;5(1):17–20 Niclas Bockelmann, Jan Graßhoff, Lasse Hansen, Giacomo Bellani, Mattias P. Heinrich, and Philipp Rostalski Deep Learning for Prediction of Diaphragm Activity from the Surface Electromyogram https://doi.org/10.1515/cdbme-2019-0005 particular it was shown that PVA is associated with adverse outcomes such as a higher mortality and a prolonged duration Abstract: The electrical activity of the diaphragm (EAdi) is a of mechanical ventilation [2, 12]. The electrical activity of the novel monitoring parameter for patients under assisted venti- diaphragm (EAdi) was introduced as a novel monitoring pa- lation and is used for assessing the patient’s neural respiratory rameter for the assessment of diaphragmatic function. It repre- drive. It is recorded by an array of electrodes placed inside the sents the envelope of the crural diaphragm’s electromyogram esophagus at the level of the diaphragm. A noninvasive alter- (EMG), measured by a special nasogastric catheter equipped native is the measurement of the electromyogram by means of with an array of electrodes placed inside the esophagus. It skin surface electrodes (sEMG). The respiratory sEMG signal, was suggested that the amplitude of EAdi directly reflects the however, is subject to electrocardiographic interference and breath-to-breath neural output of the respiratory center [5]. In crosstalk from other muscles and may also pick up a different daily clinical practice, the insertion and proper positioning of part of the muscular activity. In this work, we propose to use the esophageal catheter can be cumbersome, which limits the a deep neural network to predict the electrical activity of the routine application of EAdi. A non-invasive alternative is the diaphragm as measured by a nasogastric catheter from sEMG skin surface electromyogram (sEMG), measured on the pa- measurements. We use a ResNet based architecture and train tients thorax. Despite enabling a direct clinical application, the the network to directly regress the EAdi as a supervised learn- respiratory sEMG comes with its own intricacies: the signal- ing task – we further investigate a heatmap based regression to-noise ratio is small and the measured signal is subject to approach. The proposed methods are evaluated on a clinical strong electrocardiographic interference and muscle crosstalk. dataset consisting of 77 recordings from mechanically ven- Also the electrodes are located at a different spatial position tilated patients. For the direct regression task, the network’s in comparison to the internally placed electrodes. Some re- predictions reach a Pearson correlation coefficient (PCC) of searchers have considered the sEMG as a measure for pa- 0.818 with EAdi on the hold-out set. The heatmap regression tient activity [5] but it is still a subject of ongoing research increases the PCC to 0.830 while at the same time achieving a to what extent it is a viable surrogate for EAdi. In [1] the lower mean absolute error, indicating a superior performance. sEMG data were processed by detecting and gating out car- From our results we conclude that sEMG measurements may diac artifacts, then the amplitude of the sEMG’s envelope was be used to predict the internal activity of the diaphragm as compared to EAdi. The authors found a good correlation be- measured invasively using a nasogastric catheter. tween EAdi and one of the sEMG channels when averaging Keywords: diaphragm activity, sEMG, deep learning over several breaths. In this work, we consider the problem of predicting the EAdi signal on a sample-by-sample basis from all sEMG channels. We use a deep neural network to learn the 1 Introduction relation between the measured sEMG signals and the internal diaphragm activity. We train a ResNet based architecture and, In the light of recent studies regarding the prevalence and con- given the high noise level of the input signals, we further in- sequence of patient-ventilator asynchrony (PVA), researchers vestigate a heatmap regression approach. have emphasized the importance of monitoring the respiratory activity of patients under assisted spontaneous ventilation. In 2 Material and Methods Niclas Bockelmann, Jan Graßhoff, Philipp Rostalski, Institute for Electrical Engineering in Medicine, Universität zu Lübeck niclas.bockelmann@student.uni-luebeck.de 2.1 Data Lasse Hansen, Mattias P. Heinrich, Institute of Medical Informatics, Universität zu Lübeck The data consist of 77 clinical recordings from a total of 23 hansen@imi.uni-luebeck.de patients receiving pressure support ventilation, with record- Giacomo Bellani, Università degli Studi Milano Bic- ing lengths varying between 2 and 15 minutes. The data were occa, Dipartimento di Medicina e Chirurgia, Milano, Italy giacomo.bellani1@unimib.it Open Access. © 2019 Niclas Bockelmann, Jan Graß hoff, Lasse Hansen, Giacomo Bellani, Mattias P. Heinrich, Philipp Rostalski published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0 License. 5 256 256 18 9 N. Bockelmann et al., Deep Learning for Prediction of Diaphragm Activity recorded in an 8-bed ICU at the San Gerardo Hospital (Monza, Italy), see [1] for further details on the data collection. Each dataset contains four raw sEMG channels and EAdi as the re- gression target, respectively sampled at 500 Hz and 100 Hz. The sEMG channels were recorded with pairs of electrodes positioned (1) bilaterally at the lower costal margin on the midclavicular line ("costmar"), (2) bilaterally in the second in- tercostal space ("intercost"), (3) parallel to the muscle fibers above the rectus abdominis muscle ("rect"), (4) parallel to the muscle fibers above the sternocleidomastoid muscle ("sterno") and (5) bilaterally 5 cm below the costmar-electrodes ("below- 7 x3 cost"). The first three channels are included in all recordings, while from the last two channels only one is available in each 7 x3 patient – the respective other channel was set to zero prior to training. An exemplary segment of the raw sEMG data is de- 7 x3 picted in the top part of Figure 1. x3 2.2 Architecture x3 For the proposed supervised learning task we use a deep con- volutional neural network (DCNN) with 34 layers (cf. Fig- ure 1). The number of layers of the DCNN was empirically determined to yield a sufficient receptive field and degree of semantic abstraction considering the challenging problem of predicting diaphragm activity from raw sEMG signals. To ease the training of the CNN we make use of residual blocks by incorporating identity skip connections [6] between convolu- Fig. 1: Overview of the proposed network architecture. Light gray tional layers. Features are downsampled after every third resid- boxes represent residual blocks containing two convolution lay- ual block using strided convolutions. At the same time we dou- ers and a skip connection. Dark gray boxes represent upsampling ble the number of feature channels to enrich semantic infor- modules containing an upsampling operator and additional con- mation. Motivated by [3], a kernel size of seven is used in volutional layers. Numbers denote the kernel size (on box), signal all residual blocks, whereas in the first layer of the network size (left of box) and the number of feature channels (right of box). The numbers next to the curly bracket indicate repetition of resid- a large kernel of size 51 is employed to enlarge the receptive ual blocks. Convolutional layers are followed by batch normaliza- field and capture and process low level signal features. Follow- tion and non-linear activations. ing the residual blocks, upsampling and convolution operators are used to propagate semantic features to higher resolution layers. A final convolutional layer with kernel size seven maps to time intervals of approx. 10 seconds), that are randomly the features to the number of output channels. Rectified Linear cropped from the input data. The corresponding EAdi signal Units (ReLUs) are used throughout the network as non-linear of size 1024 serves as the regression target. Prior to training, activation functions. For faster training convergence and im- sEMG signals are normalized to have standard deviation of proved generalization we employ batch normalization [7]. one. One of the challenges in predicting the EAdi is that it may be arbitrarily scaled against other EMG channels due to differences in the electrode/tissue impedance [1]. These scal- 2.3 Training ing factors between the EMG modalities can not be predicted from the data itself. Therefore, prior to training, we normalize Input to the DCNN are the previously described sEMG signals the EAdi amplitude to either the intercost or the costmar enve- (costmar, intercost, rect, belowcost and sterno). To deal with lope (depending on which of the two has a higher correlation the missing channels in the recordings (belowcost or sterno) to EAdi), and thereby remove the scaling ambiguity. We train we set the respective input channel to zero. The network is the proposed DCNN architecture with two different regression trained with fixed length patches of size 5120 (corresponding tasks: (1) direct regression of the EAdi signal (output channel 640 640 320 320 160 19 N. Bockelmann et al., Deep Learning for Prediction of Diaphragm Activity Tab. 1: Results of different methods on the hold-out set. the direct regression and the heatmap regression as DirectNet and HeatNet, respectively. As a further baseline we evaluate Method PCC MAE HS the same performance metrics on the conventional envelope of DirectNet 0.818 ± 0.198 1.071 ± 0.835 0.856 ± 0.064 the costmar sEMG channel (calculated by gating out cardiac HeatNet 0.830 ± 0.193 0.835 ± 0.700 0.888 ± 0.044 artifacts first and then using a running root-mean-square filter Envelope 0.799 ± 0.197 1.219 ± 0.932 0.765 ± 0.066 as described in [1]). In an additional experiment we investigate the relevance of the different input signals for the regression Tab. 2: Results of the trained networks on the hold-out set for only task. Therefore, we evaluate our trained networks using only a using a single input channel while other channels are set to zero. single channel as the input while setting the remaining chan- nels to zero and report the corresponding evaluation metrics Method Channel PCC MAE HS for each sEMG channel separately. Lastly, we report the infer- DirectNet Costmar 0.787 ± 0.224 1.400 ± 1.245 0.828 ± 0.074 DirectNet Intercostn 0.176 ± 0.269 1.560 ± 1.158 0.711 ± 0.032 ence times of our regression model: One sEMG signal patch of DirectNet Rect -0.003 ± 0.196 1.570 ± 1.166 0.707 ± 0.031 10 s is processed in approximately 6 ms on an NVIDIA RTX DirectNet Belowcost -0.035 ± 0.220 1.570 ± 1.163 0.706 ± 0.032 2070 GPU and 40 ms on an INTEL Core i7-6700K CPU. DirectNet Sterno 0.020 ± 0.207 1.566 ± 1.159 0.707 ± 0.031 HeatNet Costmar 0.812 ± 0.208 0.895 ± 0.719 0.860 ± 0.066 HeatNet Intercostn 0.000 ± 0.208 1.576 ± 1.163 0.707 ± 0.031 HeatNet Rect -0.003 ± 0.202 1.578 ± 1.164 0.703 ± 0.035 HeatNet Belowcost -0.042 ± 0.241 1.581 ± 1.163 0.699 ± 0.031 3 Results and Discussion HeatNet Sterno -0.001 ± 0.212 1.574 ± 1.158 0.705 ± 0.031 The quantitative results of the different sEMG-based predic- 𝑛 = 1) and (2) heatmap regression (output channel 𝑛 = 128), tors are provided in Table 1. For the prediction of diaphragm where the regression target is a quantized Gaussian distribu- activity, both neural network approaches clearly outperform tion with a standard deviation 𝜎 = 2 µV and the mean taken the simple costmar envelope function, that has been used in from the EAdi signal. To obtain the final single channel sig- previous publications (e.g. [1]). In comparison to the direct nal we take a Gaussian smoothed argmax from the quantized regression, the application of a heatmap regression procedure heatmap. Regressing a heatmap target rather than single val- showed an improvement by 1.5% in PCC and by 22.0% in ues has been proven advantageous in terms of robustness and MAE, while the HS is improved by 3.2 percentage points. convergence of training in many different deep learning tasks These results indicate a superior performance of the heatmap [9, 11]. We implement our regression framework in PyTorch regression over the direct regression for our given task. The [10]. All models are trained for 650 epochs with a batch size performance metrics for evaluating the neural networks on a of 26 using Adam optimization [8] with an initial learning rate single sEMG channel are given in Table 2. It turns out, that for of 0.01. L1 and L2 loss is used for the direct and heatmap re- both architectures, the costmar channel is the best predictor gression, respectively. for EAdi. Still, we achieve the best performance, when using all sEMG channels at once, which might indicate, that all of the considered electrode positions on the thorax provide some 2.4 Experiments information on the internal diaphragm activity. An exemplary segment from a patient in the hold-out set is depicted in Fig- The recordings of the 77 patients are split into a training and ure 2 – in that segment the sEMG data had a particularly low test set with 52 and 25 recordings, respectively. To evaluate signal-to-noise ratio, which is reflected by the substantial noise the performance of the regressed predictions we report results visible in the costmar envelope. Both neural network meth- for three different metrics: the Pearson correlation coefficient ods nicely capture the internal diaphragm activity. Visually, (PCC), the mean absolute error (MAE) and the Hamming sim- the HeatNet appears to better match the on-off characteristic ilarity (HS). The reference for the evaluation is again the EAdi of the EAdi signal when comparing it to the direct regression signal. The Hamming similarity was calculated as a metric of approach (see seconds 12 to 15). Both neural networks have how well the derived signals can detect the timing of inspira- a high sensitivity to crosstalk artifacts in the sEMG signals tions and expirations compared to EAdi. To this end, we used (e.g. motion artifacts), which is exemplified in Figure 3. In a simple breath detector (based on an adaptive thresholding al- that segment, both the HeatNet regression and the sEMG en- gorithm similar to [4]) and calculated the Hamming distance velope show large artifacts, that do not correspond to internal of the detected inspiratory/expiratory samples to the inspira- diaphragm activity. This problem will be addressed in further tory/expiratory samples detected in EAdi. All results are ob- studies. tained from the recordings of the hold-out set and we refer to 20 N. Bockelmann et al., Deep Learning for Prediction of Diaphragm Activity 6 might be a viable method for noninvasively assessing respira- tion and neural output of mechanically ventilated patients. Fur- thermore, the possibility of doing inference on unseen sEMG data in real-time, even on CPU, might promote the feasibility of our approach in the clinical arena, e.g. for the detection of PVA. One of the remaining challenges is the rejection of mus- cle crosstalk in the sEMG, which we will further investigate in our future work. References 0 5 10 15 20 25 30 [1] G. Bellani et al. Measurement of diaphragmatic electrical time (s) activity by surface electromyography in intubated subjects and its relationship with inspiratory effort. Respiratory Care, Fig. 2: Exemplary result on a patient from the hold-out set. The 63(11):1341–1349, 2018. DirectNet and HeatNet results are shown for three concatenated [2] L. Blanch et al. Asynchronies during mechanical ventila- patches (each approx. ten seconds long). The conventional sEMG tion are associated with mortality. Intensive Care Medicine, envelope (costmar channel) and EAdi are given as a reference. 41(4):633–641, 2015. [3] W. Dai, C. Dai, S. Qu, J. Li, and S. Das. Very deep convo- lutional neural networks for raw waveforms. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 421–425. IEEE, 2017. [4] L. Estrada, A. Torres, L. Sarlabous, and R. Jané. Onset and offset estimation of the neural inspiratory time in surface diaphragm electromyography: A pilot study in healthy sub- jects. IEEE Journal of Biomedical and Health Informatics, 22(1):67–76, 2018. [5] G. Grasselli, M. Pozzi, and G. Bellani. Monitoring Respiratory Effort by Means of the Electrical Activity of the Diaphragm, pages 299–310. Springer, Cham, 2016. 52 54 56 58 60 [6] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning time (s) for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, Fig. 3: An instance of failure for the neural network predictor at 𝑡 = 52 s: Both the envelope and the HeatNet signal contain large [7] S. Ioffe and C. Szegedy. Batch normalization: Accelerating artifacts, which can probably be prescribed to muscle crosstalk. deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015. [8] Diederik P Kingma and Jimmy Ba. Adam: A method for 4 Conclusion stochastic optimization. arXiv preprint arXiv:1412.6980, [9] A. Newell, K. Yang, and J. Deng. Stacked hourglass net- In this paper, different architectures have been evaluated for works for human pose estimation. In European Conference the task of predicting the internal diaphragm activity from on Computer Vision, pages 483–499. Springer, 2016. multiple surface EMG signals. The heatmap-based regression [10] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. De- approach outperformed the direct regression architecture, with Vito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017. a PCC of 0.830 ± 0.193, a MAE of 0.835 ± 0.700 and a [11] C. Payer, D. Štern, H. Bischof, and M. Urschler. Regressing HS of 0.888 ± 0.044 compared to EAdi. The prediction of heatmaps for multiple landmark localization using cnns. In diaphragm activity was substantially improved compared to International Conference on Medical Image Computing and the conventional single-channel sEMG envelope. Based on the Computer-Assisted Intervention, pages 230–238. Springer, learned relationship between the externally measured sEMG signals and the internal diaphragm acitvity measured by a na- [12] A. W. Thille, Pablo Rodriguez, Belen Cabello, François Lel- louche, and Laurent Brochard. Patient-ventilator asynchrony sogastric catheter we hypothesize, that the former may be used during assisted mechanical ventilation. Intensive Care as a surrogate for EAdi in clinical applications. The results fur- Medicine, 32(10):1515–1522, 2006. ther confirm the findings of Bellani et al. [1] that the sEMG sEMG sEMG EAdi envelope envelope HeatNet EAdi HeatNet DirectNet (µV) (µV) (µV) (a.u.) (µV) (a.u.) (a.u.)

Journal

Current Directions in Biomedical Engineering – de Gruyter

Published: Sep 1, 2019

Keywords: diaphragm activity; sEMG; deep learning

References

  • Very deep convo - lutional neural networks for raw waveforms In International Conference on Acoustics Speech and Signal Processing pages
    Dai