Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Locomotion Mode Recognition with Inertial Signals for Hip Joint Exoskeleton

Locomotion Mode Recognition with Inertial Signals for Hip Joint Exoskeleton Hindawi Applied Bionics and Biomechanics Volume 2021, Article ID 6673018, 11 pages https://doi.org/10.1155/2021/6673018 Research Article Locomotion Mode Recognition with Inertial Signals for Hip Joint Exoskeleton 1 2 3 4 Gang Du, Jinchen Zeng , Cheng Gong, and Enhao Zheng School of Information Engineering, China University of Geosciences, Beijing 100083, China Faculty of Electrical Engineering, Mathematics and Computer Science, Technische Universiteit Delft, Delft 2600AA, Netherlands College of Engineering, Peking University, Beijing 100871, China The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China Correspondence should be addressed to Jinchen Zeng; j.zeng-4@student.tudelft.nl and Enhao Zheng; enhao.zheng@ia.ac.cn Received 18 November 2020; Revised 15 April 2021; Accepted 11 May 2021; Published 24 May 2021 Academic Editor: Musa L. Audu Copyright © 2021 Gang Du et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Recognizing locomotion modes is a crucial step in controlling lower-limb exoskeletons/orthoses. Our study proposed a fuzzy-logic-based locomotion mode/transition recognition approach that uses the onrobot inertial sensors for a hip joint exoskeleton (active pelvic orthosis). The method outputs the recognition decisions at each extreme point of the hip joint angles purely relying on the integrated inertial sensors. Compared with the related studies, our approach enables calibrations and recognition without additional sensors on the feet. We validated the method by measuring four locomotion modes and eight locomotion transitions on three able-bodied subjects wearing an active pelvic orthosis (APO). The average recognition accuracy was 92.46% for intrasubject crossvalidation and 93.16% for intersubject crossvalidation. The average time delay during the transitions was 1897.9 ms (28.95% one gait cycle). The results were at the same level as the related studies. On the other side, the study is limited in the small sample size of the subjects, and the results are preliminary. Future efforts will be paid on more extensive evaluations in practical applications. 1. Introduction [6–13]. The assistance on the hip joint helps to stabilize the locomotion [6–9], optimize the metabolic cost [10, 11], adjust the abnormal gait patterns [12], and reduce the extra Lower-limb exoskeletons/orthoses serve as important roles in rehabilitation, industrial manufacture, and other human- loads on the spine [13], according to the design of the exoskeleton. centered areas [1]. The specifically designed mechanical One primary step in exoskeleton control is to recognize structures and the control strategies can alleviate the loads on the human body and thus increase the wearer’s absolute lower-limb motion intents accurately. It bridges the gap between the human sensorimotor system and the external strength in heavy load bearing or endurance in long-term tasks. There are various types of exoskeletons according to robotic controllers, the performance of which determines the safety and working efficiency of the whole system [4]. The the active joints, such as whole-body exoskeletons (e.g., recognition tasks include gait phase estimation/detection, BLEEX [2] and HAL [3]) and single-joint ones [4] (e.g., hip joint and ankle joint). The hip joint connects the lower locomotion mode recognition, and other joint motion param- eter estimations. Locomotion mode recognition involves the extremity and the trunk. The hip joint’s primary function is to support the weight of the body in both static (e.g., stand- ambulation modes on different terrains (e.g., level ground and stairs) and the nongait patterns (e.g., standing). The ing) and dynamic (e.g., walking) postures [5]. The develop- ment of the hip joint exoskeleton is a hot research topic in recognition system should recognize the current modes and this area. There are many groups developing hip joint mode transitions accurately on multiple subjects. The recognition approach comprises the sensing system and the exoskeletons (or active pelvic orthoses) all over the world 2 Applied Bionics and Biomechanics processing algorithms. The processing algorithms are usually 2. Experimental Setups designed based on the signal features of the sensing system. Previous studies on this area suggest that the neural- 2.1. Hip Joint Exoskeleton. In this study, we used an active mechanical signal fusion method can produce satisfactory rec- pelvic orthosis (APO) developed by the research group of ognition results (e.g., accuracies and time latency). The neural Scuola Superiore Sant’Anna (SSSA) [8]. The lightweight exo- signals are usually measured from the muscle signals (e.g., skeleton can provide assistive torque in the sagittal plane to electric activities represented as surface EMG or shape changes the hip joints (see Figure 1). The APO was designed with a represented by the noncontact capacitive sensors). The serial elastic structure based on torsional springs, and the tor- mechanical signals are measured from the inertial measure- que was transmitted to the joints with two lightweight carbon ment units (IMUs) and loadcell sensors. The sensor nodes fiber-made links (driving part in Figure 1). A C-shaped (in can be integrated into the mechanical structure of the the coronal plane) structure combined with the bandages exoskeletons. The muscle signals respond faster than the fixed the exoskeleton to the waist and pelvis of the user, keep- mechanical signals. However, they convey more noises, and ing it stable on the human body. Two orthotic shells were the mechanical signals can produce signals with high repeat- connected to the carbon fiber-made links and fixed on the ability due to the advancement of sensing technology. The thighs with bandages. The torque was applied to the human combination of the signals can compensate each other to get body through the shells. There were 3 degree-of-freedoms better performance. (DoFs) for each leg, two passive (hip adduction/abduction The target for locomotion mode recognition is to pro- and pelvic tilting), and one active (flexion/extension) [8]. duce an accuracy as high as possible with the least interven- The passive DoFs ensured the stability of the whole system tion on the human body. The muscle signals require during ambulation. The core of the actuation system of additional electrodes or front-ends on the human body, APO was the DC motors with gearboxes (80 : 1 reduction which decreases the convenience and the potential willing- ratio). The torsional spring was placed on the axis of the flex- ness of uses. Another limitation is that the recognition ion/extension of the exoskeleton, between the DC motor parameters should be calibrated for each individual, increas- (gearbox) and the carbon fiber-made link. The basic calcula- ing the time needed before use. For hip joint exoskeleton con- tion of the interaction torque was achieved by the torsional trol, many researchers purely used mechanical signals for spring constant and the relative position of the encoders human locomotion mode recognition. For instance, the [8]. The interaction torques between the human body and study [14] combined the IMU sensors on an active pelvic the exoskeleton lead to the deformation of the spring. With orthosis and the foot pressure sensors for gait mode recogni- the integrated encoders and the stiffness of the torsional tion. The designed algorithm was an event-based fuzzy-logic spring, the control system of the APO can calculate the inter- structure triggered by the foot pressure sensors. The study action torques between the human body and the DC motor. [15] identified different gait modes with the hip joint angles The control strategy of APO was hierarchical control. measured from the encoder of the hip joint exoskeleton. The low-level controller was the torque control. The interac- The designed algorithm was a multilayer perceptron neural tion torques calculated from the encoders served as the feed- network. The study [14] conducted a real-time locomotion back of the control loop. The control output determined the mode recognition with IMU signals when wearing the active applied torque on the human body. There were zero-torque pelvic orthosis (APO [8]). The machine learning-based algo- mode and assistive-torque mode. For the zero-torque mode, rithms were trained and tested onboard. The studies men- the desired interaction torques between the legs and the exo- tioned above produced accurate recognition results on skeleton were zero. In the assistive-torque mode, the control- various locomotion mode tasks. However, for hip joint exo- ler’s commanded torque was a predefined curve (in one skeleton control, burdensome calibration for different indi- stride). The high-level controller was an adaptive oscillator- viduals and additional sensor nodes on the human body (AOs-) based controller, which used a set of adaptive oscilla- still limited practical applications. For instance, the study tors to track the phase of one gait cycle continuously. The overcame the subject-dependent problems with sEMG sig- input of the AO-based controller was the encoder signals nals, but the system still required pressure insoles on feet to representing the hip joint angle information, and the output provide gait event information. The study of our group vali- of the controller was the gait phase at time t and the corre- dated the recognition method with the APO. However, sponding anticipated torque. One merit of the AO-based subject-dependent training and calibration were needed controller was the continuous estimation of gait phases with before testing procedures. robustness to different walking speeds [16]. In this study, we proposed a locomotion mode recogni- tion method based on inertial measurement unit sensors on 2.2. Sensing System. We implemented an IMU board on each the hip joint exoskeleton. The designed fuzzy-logic-based algorithm can overcome the subject-dependent parameters leg (see Figure 1). The raw signals of the IMU board included in data training, which does not require training for each sub- 3-axis accelerations and 3-axis gyroscopes. There was a ject before uses. Besides, no additional sensors are required microcontrol unit (MCU) on the board, i.e., ATMEGA328. on the human body, increasing the convenience in practical The MCU calculated the pitch angle and the roll angle (global frame of the Cartesian system) with the acceleration and applications. We preliminarily evaluated the proposed method with an APO on the locomotion mode and locomo- gyroscope signals. The board was fixed on the cuff of the exo- tion transition recognition on multiple subjects. skeleton through a connector (3D printed). The pitch angle Applied Bionics and Biomechanics 3 St LW Task 3 Task 1 Control part SA SD Control circuit Orthotic shell SA St SD St LW Task 2 Task 4 Driving part Figure 2: The tasks in the experiments. St denotes standing, LW is IMU short for level walking, SA is short for stair ascending, and SD is short for stair descending. The arrows indicate ambulation direction. The tasks are denoted with different colors. 90 gait cycles, SA with 108 gait cycles, SD with 108 gait cycles, and 9 repetitions for each locomotion transition (LW⟶ SA, SA⟶ LW, gait initiation/termination). Subject 2 performed Figure 1: The hardware of the system, including the active pelvic 5 repetitions for each task, including LW with 50 gait cycles, orthosis (APO) and the IMU for measurement. SA with 60 gait cycles, SD with 60 gait cycles, and 5 repetitions for each locomotion transition. Subject 3 performed 6 repeti- tions for each task, including LW with 60 gait cycles, SA with 72 gait cycles, SD with 72 gait cycles, and 6 repetitions for each of the IMU corresponded to the flexion/extension of the hip locomotion transition. joint. The update rate of the tilt angles was 100 Hz. The data of the IMU boards were transmitted to a control circuit on the back of APO. The control circuit synchronized 3. Recognition Method the data of APO and the IMU boards via Universal Synchro- nous Asynchronous Receiver Transmitter (USART). The 3.1. Cascaded Recognition Method. The locomotion recogni- control circuit integrated a WIFI module. The data of the tion method was designed based on the signal features IMU sensors and the states of APO were transmitted to a acquired in the IMUs of both legs. The recognition method host computer wirelessly in each 10 ms. A graphic user inter- is cascaded, which firstly (first layer) classifies the static face on the computer was designed with MATLAB R2016b to mode (St) and dynamic modes (LW, SA, SD) and secondly control the data sequence and store the data. (second layer) identifies the corresponding dynamic loco- motion modes (see Figure 3). In the second layer, we 2.3. Experimental Protocol. In this study, we recruited three designed a fuzzy-logic-based algorithm. There were two healthy subjects. They had an average age of 27.3 years, an membership function sets in the fuzzy-logic-based algo- average height of 173.7 cm, and an average weight of rithms, one for each leg. The input of the membership 67.3 kg. Each subject wore the APO, as shown in Figure 1 functions was the data pair of peak-valley values detected in the experiment. In this experiment, we recorded 5 locomo- from the thigh angles. Therefore, in the second layer, we tion modes and 8 locomotion transitions. The locomotion firstly identified the peaks and valleys and secondly calcu- modes included standing (St), level walking (LW), stairs lated the fuzzy-logic membership functions. During the ascending (SA), and stairs descending (SD). The locomotion locomotion transitions, there are different leading legs. transitions included St⟵⟶LW, LW⟵⟶SA, and The procedures of two legs worked independently in the LW⟵⟶SD. Each subject performed 4 tasks of ambula- recognition process. tion to cover all the locomotion modes and transitions (see In the cascaded recognition method, the first step is to Figure 2). For task 1, there were 2 stride cycles of LW and 7 distinguish between the static locomotion mode and the stride cycles of SA (St⟶ LW⟶ SA⟶ St). For task 2, dynamic modes. As there were no gait patterns during stand- there were 7 stride cycles of SD and 2 stride cycles of LW ing, the signal profiles were much different from that of (St⟶ SD⟶ LW⟶ St). For task 3, there were 3 stride ambulation modes. We extracted time-domain features to cycles of LW and 5 stride cycles of SD (St⟶ LW⟶ SD represent the signal profiles of standing and other locomo- ⟶ St). For task 4, there were 5 stride cycles of SA and 3 tion modes. We firstly segmented the data (pitch angles and stride cycles of LW (St⟶ SA⟶ LW⟶ St). The tasks accelerations) with a 100 ms (10 samplings) sliding window. in the experiments were shown in Figure 2. The number of We calculated the standard deviation on the windows of stride cycles was shown in Table 1. In our study, to mimic the left leg’s pitch angles stdðθ Þ and the sum of absolute the locomotion in daily activities, we allowed the subjects to values of 3-axis accelerations sumðaccÞ. Additionally, we perform the locomotion modes at their favorite paces. There- compared the angular difference of the two legs in the sagittal fore, the number of gait cycles of the subjects was different. plane, expressed as θ = jθ − θ j. θ is the right thigh’s relative L R R Three subjects performed different task repetitions. Subject pitch angle, and θ is the left thigh’s pitch angle. θ is L relative 1 performed 9 repetitions for each task, including LW with the relative pitch angle between the left thigh and right thigh. 4 Applied Bionics and Biomechanics inputted to the subsequent fuzzy-logic-based algorithm, and Table 1: The stride cycle number in each task. then the flag would be deactivated. The procedure of detecting Task 1 Task 2 Task 3 Task 4 valleys and the values of the other leg was the same. LW (stride) 2 2 3 3 3.3. Fuzzy-Logic-Based Recognition Method. We designed a SA (stride) 7 0 0 5 fuzzy-logic-based method to separate between the locomo- SD (stride) 0 7 5 0 tion modes of LW, SA, and SD. As mentioned above, the input of the fuzzy-logic algorithm was a 2-dimensional (2D) vector containing the latest detected peak and valley The first layer recognition was achieved by comparing the (one leg), represented as θp and θv, respectively. The maxi- threshold-based conditions. The logic was expressed as mum/minimum values revealed the characteristics of differ- ent locomotion modes. For instance, the θp values of SA std θ > Th or θ − θ > Th or θ − θ ðÞ jj jj L std L init θ R init were larger than that of LW and SD, as the hip joint angles ð1Þ were larger in flexion when ambulating upward. The valleys > Th or sum acc > Th and θ > θ ðÞ θ Acc relative static: of the thigh pitch angles also demonstrated similar features. During the LW and SD mode, θv values were at the same If the logic condition was satisfied, the mode would be level, while they would decrease during the SA mode because recognized as dynamic modes. Otherwise, it was classified the stair-ascending locomotion contains a kicking-back as the static mode (St). θ is the initial pitch angle for both init movement in which the hanging leg could reach the lowest thighs which is close to 0. pitch angle without the constraint of the stairs. During LW In the above logic condition, Th was the standard devi- std and SD, θv would be limited by the ground and the stairs. ation threshold for the sliding window which was selected as We visualized the characteristics in Figure 5. The distribution 0.5 . Th was the pitch angle threshold for both legs which of maximum-minimum of different locomotion modes could was selected as 8 . θ was the threshold for the static mode static be separated apart. which was selected as 10 . Th was the threshold for accel- Acc We designed multivariate membership functions to clas- eration which was chosen as 500. sify the three locomotion modes. The membership function calculates the membership value of the event-based feature 3.2. Detecting the Extreme Values. The second layer was to belonging to the target mode. The output range of member- further separate the data into corresponding dynamic loco- ship is (0,1], where 1 is the maximum membership of the motion modes (i.e., LW, SA, and SD). We designed a fuzzy- model. The membership functions were calculated in parallel logic-based algorithm to classify the locomotion modes. Each with the signals of two legs. For the signals of each leg, we input was a 2×1 vector including the peak and valley of the defined three membership functions, one for each locomo- pitch angles. We designed an algorithm find_peak() to detect tion mode. The function was expressed as the peaks and valleys of the IMU signals. The most recent true peak and valley values found by find_peak() would be 1 −1 − C −X Σ C −X i ðÞ ðÞ put in a 2×1 buffer as the input of the subsequent fuzzy- 2 i i i i i f = e , i =1,2,3, ð2Þ 1/2 logic-based algorithms. Taking finding the left leg’s peak 2πΣ jj values for example (diagram see Figure 4), we firstly prede- fined thresholds for peak value Th_ and time interval where i denoted the mode’s number, k was the scale factor, peak Th_ . θ ðtÞ was the pitch angle of the left leg at time t. interval L C = ðθ , θ Þ was the input vector including the detected i pi vi We used PðiÞ to represent the i peak value found in th peak and valley, and X = ðμ , μ Þ was the central point of pi vi (pseudo) real-time. Secondly, the past 21 samples before time the membership function. In our study, the point was repre- t were compared (i.e., θ ðtÞ, θ ðt − 1Þ, ⋯, θ ðt − 20Þ). If θ ðt L L L L sented as the mean value of the training data sets. Σ was the − 10Þ is larger than all the 10 numbers backward i covariance matrix representing the data distribution. After ( θ ðt − 20Þ, θ ðt − 19Þ, ⋯θ ðt − 11Þ and all the 10 numbers L L L calculating three membership functions of LW, SA, and SD, forward ( θ ðtÞ, θ ðt − 1Þ ⋯ θ ðt − 9Þ), we set PðiÞ = θ ðt − L L L L respectively, the algorithm proceeded to calculate the maxi- 10Þ. If the absolute value of peak value PðiÞ minus the initial mal membership of the target mode: value of pitch angle θ ð0Þ was larger than Th_ ,we would L peak consider the peak value PðiÞ as an outlier which would be dis- Target mode = arg max f : ð3Þ ðÞ carded. Otherwise, the PðiÞ would be treated as a peak candi- i date. Because of the existence of false peak values created by the noise, we compared peak candidate’s location location_ Figure 5 shows that 2-dimensional space three member- P(i) with that of the latest candidate location_P(i-1). We set ship functions created three oval shape regions, whose center the time interval threshold Th_ for two adjacent peak interval coordinates were the mean value of three membership func- values. If the time interval between these two peak values tions (X ). In our study, the parameters X and Σ were fitted i i i was smaller than Th_ , we would assume that one of interval with the training data set (described in detail below). the two candidate peak values was false. The candidate with the smaller value was aborted. We set a decision flag for the 3.4. Synchronization of the Recognition Decisions. There were algorithm. The decision flag would be activated if a true peak inertial sensors on both thighs. The fuzzy-logic-based recog- and a true valley were detected. The peak-valley pair was then nition method worked in parallel for the left leg and the right Applied Bionics and Biomechanics 5 Standing First-layer classifier Sliding windows Level walking Threshold-based Second layer Detecting the classifier Dynamic maximum/ Stair ascending Fuzzy-logic-based modes minimum algorithm hip anglesy Stair descending Figure 3: The diagram of the recognition method. The first layer was to distinguish dynamic modes from standing (St). The second layer was designed to classify the dynamic modes, and there were three dynamic modes (LW, SA, and SD). (St⟶ other modes), the first recognized transition was Definition: Th_peak, Th_interval deemed to be the results, which were expressed as Init i = 1, P (0) t = min right legs s transition time point t , i R i ð4Þ left leg s transition time t , ... P (i) = θ (t-10): θ (t-10) > θ (t) , θ (t-1) , θ (t-2) , , L L L L L ... θ (t-9)&& θ (t-10) > θ (t-20) , θ (t-19) , , θ (t-11) L L L L L where t was the timing point of the detected transition. For gait termination recognition, the last recognized transition Yes was deemed to the recognition results. The timing point t was expressed as |P (i) -θ (0)| > Th_peak t = max right leg s transition time point t , t R ð5Þ left leg s transition time t : Yes No Discard the detected P (i) For the second layer recognition, the transition timing points (t ) were the first recognized timing points between 0d the left leg’s and right leg’s results. Set P (i) as a candidate 4. Evaluation Method 4.1. Crossvalidation. We used the crossvalidation method to evaluate the performances. We evaluated the performance location_P (i)-location_P (i-1) < Th_interval with 1 : 2 intersubject crossvalidation and 1 : 1 intrasubject crossvalidation. In the 1 : 1 intrasubject crossvalidation, each subject’s data were divided into two sets with the same sizes. Yes No The first data were used for training, and the second set for testing. The procedure was repeated with the second data Discard the smaller value set for training and the first set for testing. The results of the two tests were averaged as the result of the subject. In the 1 : 2 intersubject crossvalidation, we used the data of Find true peaks and true one subject for training and the data of the other subjects locations for testing. In the training procedure, the parameters of the fuzzy-logic-based algorithms were fitted. Figure 4: The flow chart of finding a peak value from the thigh pitch 4.2. Recognition Accuracy. The first metric for evaluating the angles. performance was the recognition accuracy. In the first layer, the recognition decisions (St and other dynamic modes) were continuously calculated in each sample. The recognition accuracy (recognition accuracy 1) leg. The recognition decisions were then synchronized to was defined as minimize the errors in locomotion transitions. In our cas- caded recognition method, the first layer was to distinguish Ncorrect1 i ðÞ between St and dynamic modes. The transitions were gait recognition accuracy1ðÞ i = , ð6Þ initiation and gait termination. For gait initiation recognition Ntotal1ðÞ i 6 Applied Bionics and Biomechanics –20 LW SD –30 –40 –50 SA –60 –70 –25 –20 –15 –10 –5 0 5 10 15 20 25 Maximum values of hip angles Figure 5: The distribution of the data of LW, SA, and SD (denoted by the dots) and the calculated membership function (represented by the ellipses). The data of LW, SA, and SD were represented as the yellow dots, the red dots, and the blue dots, respectively. The data were collected from subject 1. where i was the subject’s number, Ncorrect1 was the number labeled as the middle point of the swing phase. The labeling of correctly recognized decisions, and Ntotal1 was the total method motion transitions were the same as that of number of decisions. existing-related studies [18, 19]. In the second layer, the recognition decisions were calcu- The time delay (Td_init) of gait initiations was defined as lated in each extreme point being detected (peak and valley). the difference between the recognized timing point t and the The recognition accuracy of the second layer was defined as reference transition time of gait initiation t , expressing as 0i Td_init = t − t : ð8Þ Ncorrect i ðÞ i 0i recognition accuracyðÞ i = , ð7Þ NtotalðÞ i Similarly, the timed delay (Td_terminal) of gait termina- tions was expressed as where i was the number of subjects, Ncorrect was the number of correctly recognized gait cycles, and Ntotal was the total Td_terminal = t − t , ð9Þ t 0t number of gait cycles. where t was the recognition transition time of gait termina- 4.3. Confusion Matrix. We used the confusion matrix to illus- tion, and t was the reference transition time of gait trate the recognition performance of each locomotion mode. 0t The details of the definition can be found in [17]. termination. The time delay (Td_dynamic) of the dynamic modes was 4.4. Time Delay of the Locomotion Transitions. Another met- expressed as ric for evaluating the performance was the time delay. There were three critical timing points in each transition period, i.e., Td_dynamic = t − t , ð10Þ d 0d the timing point when the data changed from St to dynamic modes (gait initiation, t ), the timing point when the data 0i where t was the recognition transition time of dynamic changed from dynamic modes to St (gait termination, t ), 0t modes, and t was the reference transition time of dynamic 0d and the timing point when the data changed from one modes. The positive value presented the delay of recognition, dynamic mode to another (t ). 0d and the negative value represented the advance of recogni- The reference transition time was determined by labels. tion, shown in Figure 6. In our cascaded recognition method, the first layer was to separate between standing and dynamic modes. We manu- 5. Results ally labeled the data as standing and dynamic modes by IMU signals. If the pitch angles exceeded a threshold com- 5.1. Recognition Accuracy. In this section, we showed the rec- pared with that of standing, the data would be labeled as ognition accuracy for both the first and second layer classi- dynamic modes, and t was defined as the reference of the fiers. The first layer recognition was designed to distinguish 0i gait initiation transition time between standing and dynamic dynamic modes from standing (St). The second layer recog- modes, while t was defined as the reference of the gait ter- nition was designed to classify three dynamic modes (LW, 0t mination transition time between dynamic and standing SA, and SD). modes. For the second layer recognition, the reference transi- As for the first layer (classification between dynamic tion time was labeled based on the gait events detected by the modes and St), the recognition accuracy for each subject foot pressure insoles. t was defined as the reference of the was 92.18%, 93.00%, and 90.45%, respectively. The average 0d transition time between two dynamic modes. t would be recognition accuracy was 91.88%. As for the second layer 0d Minimum values of hip angles Applied Bionics and Biomechanics 7 –50 –100 0 200 400 600 800 1000 1200 1400 1600 SD Td_ini SA LW Td_dynamic St Td_terminal 0 200 400 600 800 1000 1200 1400 1600 Number of samples Right leg Reference Figure 6: The pseudo real-time recognition decisions. The upper subplot is the raw pitch angles and the detected extreme points. The bottom subplot shows the recognition decisions and the reference labels. (classification between LW, SA, and SD), the fuzzy-logic- Table 2: Training/testing with whole data of a subject (1 : 2 intersubject validation). based method produced accurate recognition decisions in locomotion mode tasks (the case result see Figure 6). The rec- Subject 1 Subject 2 Subject 3 ognition accuracy was higher than 0.89 for most of the eval- Subject 1 — 90.39% 94.76% uations (intersubject shown in Table 2 and intrasubject shown in Table 3). In Tables 2 and 3, each row represented Subject 2 89.84% — 95.51% the subject number used for training, and each column repre- Subject 3 93.23% 95.20% — sented the testing data (the subject used for testing). The off- diagonal results in Table 2 were the intersubject recognition accuracies, while the diagonal results in Table 3 denoted the Table 3: Training/testing with half data of a subject (1 : 1 accuracies of the intrasubject crossvalidation. intrasubject validation). In the 1 : 2 intersubject validation (see in Table 2), the membership functions trained with subject3’s dataset had Subject 1 Subject 2 Subject 3 the best performance, with the highest accuracy, 95.51%. Subject 1 88.29% 95.42% 95.22% The average recognition accuracy for each subject trained Subject 2 87.76% 95.31% 96.70% with different datasets was 91.54%, 92.80%, and 95.14% for Subject 3 86.80% 92.07% 94.60% subject 1, subject 2, and subject 3, respectively. The lowest accuracy (89.84%) occurred in subject 1 trained with subject 2’s dataset. Subject 3’s average recognition accuracy showed 5.2. Time Delay of Locomotion Transitions. We investigated the best performance, 95.14%. In Table 3 (intrasubject crossvalidation), recognition the time delay during locomotion transitions trained/tested accuracies showed a similar pattern but slightly decreased with half the data of a subject. We calculated the time differ- ence between the recognized locomotion transitions and the compared with intersubject validation. The lowest accuracy, 86.80%, occurred in subject 1 trained with subject3’s dataset. referenced ones. We defined the time latency of gait initia- tions as Td_init, gait terminations as Td_terminal, and the The average testing set recognition accuracy of three subjects dynamic transition as Td_dynamic. The unit of the results trained with the same training dataset was 92.98%, 93.26%, and 91.16%, respectively, with subject 1, subject 2, and sub- was ms. From Table 6, we can see that the average time delay of ject3’s training dataset. The average testing set recognition accuracy for each subject trained with different training data- gait initiations for each subject was 1077.3 ms, 812.8 ms, and 268.2 ms and the average time delay of gait terminations sets was 87.62%, 94.27%, and 95.51%, respectively. was 787.5 ms, 315.5 ms, and 29.2 ms. The average time delay In Tables 4 and 5, we presented the recognition accuracy for each task. From the experiment results, the lowest recog- for all the subjects was 554.4 ms. Subject 3 has the lowest time delay for both gait initiations and gait terminations (268.2 ms nition accuracy for each subject’s results usually occurred in task 2 (St⟶ SD⟶ LW⟶ St), while the recognition and 29.2 ms). Also, we can find that large time delays occurred in task 2 and task 3 frequently. algorithm usually performed better in task 1 and task 4. Recognition decision IMU signals 8 Applied Bionics and Biomechanics Table 4: Recognition accuracy of 1 : 2 intersubject validation. Subject 1 Subject 2 Subject 3 Number of Number of Accuracy Number of Number of Accuracy Number of Number of Accuracy error steps total steps rate error steps total steps rate error steps total steps rate Task 1 —— — 2 95 98.48% 1 109 99.08% Task 2 —— — 20 85 76.47% 12 102 88.24% Subject 1 Task 3 —— — 10 74 86.49% 7 93 92.47% Task 4 —— — 0 82 100.00% 1 97 98.97% Task 1 15 174 91.38% —— — 1 109 99.08% Task 2 21 176 88.07% —— — 10 102 90.20% Subject 2 Task 3 18 175 89.71% —— — 6 93 93.55% Task 4 15 154 90.26% —— — 1 97 98.97% Task 1 18 174 89.66% 3 132 97.73% —— — Task 2 19 176 89.20% 8 85 90.59% —— — Subject 3 Task 3 12 175 93.14% 3 74 95.95% —— — Task 4 18 154 88.31% 2 82 97.56% —— — Table 5: Recognition accuracy of 1 : 1 intrasubject crossvalidation. Subject 1 Subject 2 Subject 3 Number of Number of Accuracy Number of Number of Accuracy Number of Number of Accuracy error steps total steps rate error steps total steps rate error steps total steps rate Task 1 1 79 98.73% 1 37 97.30% 0 54 100.00% Task 2 15 83 81.93% 5 32 84.38% 6 56 89.29% Subject 1 Task 3 18 78 76.92% 0 28 100.00% 3 47 93.62% Task 4 3 68 95.59% 0 33 100.00% 1 49 97.96% Task 1 9 79 88.61% 1 37 97.30% 0 54 100.00% Task 2 12 83 85.54% 4 32 87.50% 5 56 91.07% Subject 2 Task 3 10 78 87.18% 1 28 96.43% 2 47 95.74% Task 4 7 68 89.71% 0 33 100.00% 0 49 100.00% Task 1 10 79 87.34% 1 37 97.30% 0 54 100.00% Task 2 10 83 87.95% 7 32 78.13% 4 56 92.86% Subject 3 Task 3 7 78 91.03% 2 28 92.86% 2 47 95.74% Task 4 13 68 80.88% 0 33 100.00% 5 49 89.80% For Table 7, we can see that the average Td_dynamic for rent study was that we simplified the setups of the sensing each subject was 685.1 ms, 541.8 ms, and 394.0 ms. The low- approaches in both training and testing procedures. The sim- est average Td_dynamic also occurred in subject 3. Also, we plification in sensors can reduce the time needed to calibrate can see that the large Td_dynamic usually occurred in task 3. the recognition procedure in practical applications. The average gait cycle was 1897.9 ms, for which the aver- There are many studies on IMU-based locomotion mode age time delay of average Td_init, average Td_terminal, and recognition. The performances are determined by the factors, average Td_dynamic was 549.5 ms accounted for 28.95% of including the sensor setups (sensors’ number, sensing posi- a gait cycle. tions), the robotic devices (exoskeletons, prostheses), and the processing algorithms. The evaluation method also influenced the numeric recognition results. For instance, the recent stud- 6. Discussion ies on IMU-based locomotion mode recognition achieved 6.1. Recognition Performances. In this study, we designed and >95% average recognition accuracies with intrasubject cross- validation [21, 22]. The sensor setups are quite different from evaluated the fuzzy-logic-based method for locomotion ours. The study mounted an IMU board on the amputated mode/transition recognition with a hip joint exoskeleton. foot for terrain identification [21], while the study fixed the The method only relied on the inertial sensors integrated into IMU boards on the shanks, the waist, and wrists [22]. the exoskeleton, and no additional sensors were required on the human body. Compared with the previous works using In our study, the IMU boards were fixed on the thighs of the subjects. The target robot platform is the hip exoskeleton. the same exoskeleton [14, 20], one improvement of our cur- Applied Bionics and Biomechanics 9 Table 6: The initial and terminal transition time latency. Subject 1 Subject 2 Subject 3 Td_init Td_teminal Td_init Td_teminal Td_init Td_teminal Task 1 —— 836.0 376.0 -300.0 161.7 Task 2 —— 1126.0 -354.0 1145.0 -443.3 Subject 1 Task 3 —— 390.0 1626.0 -145.0 -556.0 Task 4 —— 376.0 -184.0 -156.0 -527.5 Task 1 2270.0 192.2 —— -320.0 235.0 Task 2 1207.8 -1266.7 —— 1805.0 -436.7 Subject 2 Task 3 414.4 2652.2 —— -185.0 1768.3 Task 4 1241.1 641.1 —— 301.7 31.7 Task 1 567.8 305.6 840.0 334.0 —— Task 2 904.4 764.4 1630.0 110.0 —— Subject 3 Task 3 705.6 2198.9 964.0 452.0 —— Task 4 1307.8 812.2 340.0 452.0 —— Table 7: The dynamic transition time latency. average recognition accuracies were over 98% with subject- dependent training and testing processes. The calculation Td_dynamic time of each recognition decision was less than 1 ms, but Subject 1 Subject 2 Subject 3 the time delay of locomotion transitions was not reported. LW⟶ SA — 444.0 230.0 By comparison, in our study, we produced an average recog- SD⟶ LW — 136.0 -405.0 nition accuracy of 93.16% with 1 vs. 2 intersubject crossvali- Subject 1 dations and 92.46% with 1 vs. 1 intrasubject crossvalidation. LW⟶ SD — 1696.0 1815.0 Sa⟶LW — -154.0 -1.7 6.2. Confounding Factors. One key factor that influenced the LW⟶ SA 235.6 — 230.0 recognition accuracies was the detection of the peaks and val- SD⟶ LW leys from the inertial signals. As shown in Figure 6, the loco- 5.6 — -405.0 Subject 2 motion modes and transitions were successfully recognized; LW⟶ SD 235.6 — 1690.0 although, there were misdetections in the extreme values. Sa⟶ LW 687.8 — -1.7 The output value of the membership functions was deter- mined by the detected values of the extreme points. In the LW⟶ SA 365.6 444.0 — calculated recognition results, the fuzzy-logic-based method SD⟶ LW 921.1 136.0 — could successfully tell apart the locomotion modes as long Subject 3 LW⟶ SD 2198.9 1526.0 — as the distribution of the extreme points was distinguishable Sa⟶ LW 814.4 106.0 — (as shown in Figure 5). In our study, although no signals from the feet were measured, the maximum/minimum angles still showed gait information. The maximum value The recognition performances of our study were at the same of the thigh tilt angle usually occurred at the swing phase, level as that of the previous studies with similar sensor/robot which was used to distinguish between the LW and SA. setups [14, 20]. In the previous work with the active pelvic While the minimum angle occurred near the foot-off of one orthosis [14], the authors used inertial sensors on the thigh gait cycle, the values were informative to distinguish between and the foot pressure insoles for seven locomotion mode LW and SD. The maximum hip flexion/extension angles are recognitions. The average accuracies on six healthy subjects highly correlated to the locomotion modes. For instance, the achieved over 99% with locomotion transition tasks. The maximum flexion angles of SA were significantly larger than average time delay during the locomotion transitions was that of LW and SD. The subjects can adjust the patterns intu- fixed to one step. On the other hand, the authors also claimed itively to control the exoskeletons in locomotion transition the limitations of using additional foot insoles which was not tasks. The physical significance of the features in our fuzzy- integrated on the exoskeleton. The accuracies decreased to logic-based algorithm can accelerate the training/calibration 65.7%-91.2% in different testing data sets if the centre-of- procedure for a novice subject. Another point worth being pressure (CoP) information was removed [14]. In our noted is the intersubject variability in signal profiles and rec- previous works [20], the authors designed machine- ognition performances. In addition to the difference in learning-based algorithms for locomotion mode recognition. motion patterns, the difference in relative positions of the In the study of [23], the authors designed an artificial neural IMU boards on the thigh was another important reason. network- (ANN-) based recognition algorithm with the During the experiments, the IMU boards were fixed on the inertial signals (thigh) and the foot pressure signals. The same positions at the exoskeleton. Due to the different 10 Applied Bionics and Biomechanics terrains in the laboratory environment. In future works, anthropometries of the subjects, the relative positions on the thigh were different. In practical applications, the sensors more complicated tasks, including various walking speeds, usually are fixed on the exoskeleton. Adjusting the sensor jogging, and other locomotion modes in daily life, will be investigated. Thirdly, the recognition decisions in our study position to keep the same signal profiles across subjects is also impractical. In future real-time control, we will improve were discrete in one gait cycle. In future works, we will inves- the recognition algorithms with the ability of fast calibration tigate continuous parameter changes in the locomotion tasks, to make the trained model quickly update with the new user. such as different heights of stairs and different upward loco- motion modes (ramps and stairs). The processing algorithm will also be studied to cope with more complicated problems. 6.3. Influence of the Recognition Performances on Robotic Control. In real-time control of the APO, the hierarchical control framework is usually designed. The high-level con- 7. Conclusions troller recognizes the locomotion modes and determines In this study, we designed and preliminarily validated the fea- the assistive torque curve of the recognized terrain. The sibility of a fuzzy-logic-based algorithm for the locomotion middle-level controller uses adaptive oscillators (AOs) to mode and locomotion transition recognition with an active track the desired torque curve. The low-level controller pelvic orthosis. The method purely relied on the inertial sig- drives the motors to achieve the force feedback loop. nals measured from the thigh, and the sensors were fixed on If there are recognition errors, the desired torque curve the exoskeleton. With a proper training process, the fuzzy- for the controller will be different from that needed for the based algorithm produced comparable recognition accuracies current terrain. The user will move with an inappropriate to the existing studies on the same robotic platform. The supe- assistive torque curve. The APO applies assistive torque on riority of the method was that it required no additional sensors the hip joint angle in the sagittal plane. Due to the mechani- on the human body, increasing the convenience in practical cal design of the APO (2 passive DoFs in the coronal plane applications. The inputs of the fuzzy-logic-based method were and passive compliance), it is less likely that the user will fall the detected peaks and valleys of the pitch angles of the thigh. caused by the wrong recognition of the locomotion modes. Combined with the cascaded recognition method, it produced However, in the long-time use, the mismatching between reliable recognition results as long as the detected extreme the assistive torque curve and the terrains can increase the points were distinguishable between the dynamic locomotion metabolic cost (decrease the efficacy of the exoskeleton) and modes. Future works will be focused on onboard training the risks of the fall. If the transition time delay exceeds the and real-time control of the exoskeleton, investigation of the starting timing point of the applied assistive torque of one complicated unstructured terrains, and adaptation to continu- stride, there will be a mismatch between the assistance and ous ambulation parameters. the actual locomotion mode. Otherwise, the time delay is acceptable. In our study, the average time delay during the Data Availability transitions ranges from 300 ms to 1000 ms, which can cause a mismatch between the assistive torques and current loco- The data are made available through the corresponding motion modes. The impacts on the user are the same as that authors’ emails. of the recognition errors. To quantitatively evaluate the recognition errors, further Conflicts of Interest extensive experiments combing the real-time recognition and exoskeleton controller are needed. In future studies, we The authors declare no potential conflicts of interest with will investigate the effects of the errors and time delay with respect to the research, authorship, and/or publication of this real-time recognition and control. article. 6.4. Limitations and Future Works. Our current study has Authors’ Contributions some limitations, and the following issues will be addressed in future works. Firstly, the sample size of the subjects was In this study, Gang Du and Jinchen Zeng analyzed the data small (N =3). The results were calculated with an offline and designed the experiments. Jinchen Zeng and Enhao evaluation. The generalization ability of the fuzzy-logic- Zheng designed the recognition method. Cheng Gong con- based algorithm cannot be extensively evaluated with the ducted the experiments. Enhao Zheng guided the writing of small sample size. Due to the individual difference in loco- the article. Jinchen Zeng and Enhao Zheng wrote the article. motion patterns and sensor placements, the signal profiles Gang Du and Jinchen Zeng contributed equally to this work. can vary across the subjects. The onboard training and real- time exoskeleton control have yet to be studied. In future Acknowledgments works, we will carry out an extensive study on real-time con- trol with onboard training. We will investigate the effects of Thanks are due to the China University of Geosciences false detections on control performances. We will also carry (Beijing) Information Technology Innovation Experimental out experiments on more subjects to evaluate the generaliza- Base for the support of this research work. The work is sup- tion performances. Postprocessing approaches will also be ported by the Undergraduate Education Quality Improve- designed to remove the recognition errors further. Secondly, ment Project of China University of Geosciences (Beijing) the recognition tasks in our study only involved structured (No. XNFZ202005). Applied Bionics and Biomechanics 11 [17] E. Zheng, L. Wang, K. Wei, and Q. Wang, “A noncontact References capacitive sensing system for recognizing locomotion modes [1] A. J. Young and D. P. Ferris, “State of the art and future direc- of transtibial amputees,” IEEE Transactions on Biomedical tions for lower limb robotic exoskeletons,” IEEE Transactions Engineering, vol. 61, no. 12, pp. 2911–2920, 2014. on Neural Systems and Rehabilitation Engineering, vol. 25, [18] X. Liu and Q. Wang, “Real-time locomotion mode recognition no. 2, pp. 171–182, 2017. and assistive torque control for unilateral knee exoskeleton on [2] A. B. Zoss, H. Kazerooni, and A. Chu, “Biomechanical design different terrains,” IEEE/ASME Transactions on Mechatronics, of the Berkeley lower extremity exoskeleton (BLEEX),” vol. 25, no. 6, pp. 2722–2732, 2020. IEEE/ASME Transactions on Mechatronics, vol. 11, no. 2, [19] E. Zheng and Q. Wang, “Noncontact capacitive sensing-based pp. 128–138, 2006. locomotion transition recognition for amputees with robotic [3] https://www.cyberdyne.jp/english/products/HAL/. transtibial prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 2, pp. 161–170, [4] T. Yan, M. Cempini, C. M. Oddo, and N. Vitiello, “Review of assistive strategies in powered lower-limb orthoses and exoskeletons,” Robotics and Autonomous Systems, vol. 64, [20] C. Gong, D. Xu, Z. Zhou, N. Vitiello, and Q. Wang, “Real-time pp. 120–136, 2015. on-board recognition of locomotion modes for an active pelvis orthosis,” in 2018 IEEE-RAS 18th International Conference on [5] D. A. Winter, Ed., Biomechanics and Motor Control of Human Humanoid Robots (Humanoids), pp. 346–350, Beijiing, China, Gait: Normal, Elderly and Pathological, University of Waterloo Press, 1991. [21] F. Gao, G. Liu, F. Liang, and W. H. Liao, “IMU-based locomo- [6] C. L. Lewis and D. P. Ferris, “Invariant hip moment pattern while walking with a robotic hip exoskeleton,” Journal of Bio- tion mode identification for transtibial prostheses, orthoses, and exoskeletons,” IEEE Transactions on Neural Systems and mechanics, vol. 44, no. 5, pp. 789–793, 2011. Rehabilitation Engineering, vol. 28, no. 6, pp. 1334–1343, 2020. [7] T. Zhang, M. Tran, and H. Huang, “Design and experimental verification of hip exoskeleton with balance capacities for [22] F. Sherratt, A. Plummer, and P. Iravani, “Understanding walking assistance,” IEEE/ASME Transactions on Mechatro- LSTM network behaviour of IMU-based locomotion mode nics, vol. 23, no. 1, pp. 274–285, 2018. recognition for applications in prostheses and wearables,” Sen- sors, vol. 21, no. 4, p. 1264, 2021. [8] F. Giovacchini, F. Vannetti, M. Fantozzi et al., “A light-weight active orthosis for hip movement assistance,” Robotics and [23] C. Gong, D. Xu, Z. Zhou, N. Vitiello, and Q. Wang, “BPNN- Autonomous Systems, vol. 73, pp. 123–134, 2015. based real-time recognition of locomotion modes for an active pelvis orthosis with different assistive strategies,” International [9] K. Junius, M. Degelaen, N. Lefeber, E. Swinnen, B. Vanderborght, Journal of Humanoid Robotics, vol. 17, no. 1, article 2050004, and D. Lefeber, “Bilateral, Misalignment-Compensating, Full- DOF Hip Exoskeleton: Design and Kinematic Validation,” Applied bionics and biomechanics, vol. 2017, 14 pages, 2017. [10] T. Lenzi, M. C. Carrozza, and S. K. Agrawal, “Powered hip exo- skeletons can reduce the user's hip and ankle muscle activa- tions during walking,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21, no. 6, pp. 938–948, [11] K. Seo, J. Lee, Y. Lee, T. Ha, and Y. Shim, “Fully autonomous hip exoskeleton saves metabolic cost of walking,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 4628–4635, Stockholm, Sweden, 2016. [12] S. K. Banala, S. H. Kim, S. K. Agrawal, and J. P. Scholz, “Robot assisted gait training with active leg exoskeleton (ALEX),” IEEE Transactions on Neural Systems and Rehabilitation Engi- neering, vol. 17, no. 1, pp. 2–8, 2008. [13] B. Chen, L. Grazi, F. Lanotte, N. Vitiello, and S. Crea, “A real- time lift detection strategy for a hip exoskeleton,” Frontiers in neurorobotics, vol. 12, p. 17, 2018. [14] A. Parri, K. Yuan, D. Marconi et al., “Real-Time hybrid loco- motion mode recognition for lower limb wearable robots,” IEEE/ASME Transactions on Mechatronics, vol. 22, no. 6, pp. 2480–2491, 2017. [15] X. Wu, Y. Ma, X. Yong, C. Wang, Y. He, and N. Li, “Locomotion mode identification and gait phase estimation for exoskeletons during continuous multi-locomotion tasks,” IEEE Transactions on Cognitive and Developmental Systems, vol. 13, no. 1, pp. 45– 56, 2019. [16] T. Yan, A. Parri, V. Ruiz Garate, M. Cempini, R. Ronsse, and N. Vitiello, “An oscillator-based smooth real-time estimate of gait phase for wearable robotics,” Autonomous Robots, vol. 41, no. 3, pp. 759–774, 2017. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Applied Bionics and Biomechanics Hindawi Publishing Corporation

Locomotion Mode Recognition with Inertial Signals for Hip Joint Exoskeleton

Loading next page...
 
/lp/hindawi-publishing-corporation/locomotion-mode-recognition-with-inertial-signals-for-hip-joint-FK9Hm8KBWE

References (24)

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2021 Gang Du et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
1176-2322
eISSN
1754-2103
DOI
10.1155/2021/6673018
Publisher site
See Article on Publisher Site

Abstract

Hindawi Applied Bionics and Biomechanics Volume 2021, Article ID 6673018, 11 pages https://doi.org/10.1155/2021/6673018 Research Article Locomotion Mode Recognition with Inertial Signals for Hip Joint Exoskeleton 1 2 3 4 Gang Du, Jinchen Zeng , Cheng Gong, and Enhao Zheng School of Information Engineering, China University of Geosciences, Beijing 100083, China Faculty of Electrical Engineering, Mathematics and Computer Science, Technische Universiteit Delft, Delft 2600AA, Netherlands College of Engineering, Peking University, Beijing 100871, China The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China Correspondence should be addressed to Jinchen Zeng; j.zeng-4@student.tudelft.nl and Enhao Zheng; enhao.zheng@ia.ac.cn Received 18 November 2020; Revised 15 April 2021; Accepted 11 May 2021; Published 24 May 2021 Academic Editor: Musa L. Audu Copyright © 2021 Gang Du et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Recognizing locomotion modes is a crucial step in controlling lower-limb exoskeletons/orthoses. Our study proposed a fuzzy-logic-based locomotion mode/transition recognition approach that uses the onrobot inertial sensors for a hip joint exoskeleton (active pelvic orthosis). The method outputs the recognition decisions at each extreme point of the hip joint angles purely relying on the integrated inertial sensors. Compared with the related studies, our approach enables calibrations and recognition without additional sensors on the feet. We validated the method by measuring four locomotion modes and eight locomotion transitions on three able-bodied subjects wearing an active pelvic orthosis (APO). The average recognition accuracy was 92.46% for intrasubject crossvalidation and 93.16% for intersubject crossvalidation. The average time delay during the transitions was 1897.9 ms (28.95% one gait cycle). The results were at the same level as the related studies. On the other side, the study is limited in the small sample size of the subjects, and the results are preliminary. Future efforts will be paid on more extensive evaluations in practical applications. 1. Introduction [6–13]. The assistance on the hip joint helps to stabilize the locomotion [6–9], optimize the metabolic cost [10, 11], adjust the abnormal gait patterns [12], and reduce the extra Lower-limb exoskeletons/orthoses serve as important roles in rehabilitation, industrial manufacture, and other human- loads on the spine [13], according to the design of the exoskeleton. centered areas [1]. The specifically designed mechanical One primary step in exoskeleton control is to recognize structures and the control strategies can alleviate the loads on the human body and thus increase the wearer’s absolute lower-limb motion intents accurately. It bridges the gap between the human sensorimotor system and the external strength in heavy load bearing or endurance in long-term tasks. There are various types of exoskeletons according to robotic controllers, the performance of which determines the safety and working efficiency of the whole system [4]. The the active joints, such as whole-body exoskeletons (e.g., recognition tasks include gait phase estimation/detection, BLEEX [2] and HAL [3]) and single-joint ones [4] (e.g., hip joint and ankle joint). The hip joint connects the lower locomotion mode recognition, and other joint motion param- eter estimations. Locomotion mode recognition involves the extremity and the trunk. The hip joint’s primary function is to support the weight of the body in both static (e.g., stand- ambulation modes on different terrains (e.g., level ground and stairs) and the nongait patterns (e.g., standing). The ing) and dynamic (e.g., walking) postures [5]. The develop- ment of the hip joint exoskeleton is a hot research topic in recognition system should recognize the current modes and this area. There are many groups developing hip joint mode transitions accurately on multiple subjects. The recognition approach comprises the sensing system and the exoskeletons (or active pelvic orthoses) all over the world 2 Applied Bionics and Biomechanics processing algorithms. The processing algorithms are usually 2. Experimental Setups designed based on the signal features of the sensing system. Previous studies on this area suggest that the neural- 2.1. Hip Joint Exoskeleton. In this study, we used an active mechanical signal fusion method can produce satisfactory rec- pelvic orthosis (APO) developed by the research group of ognition results (e.g., accuracies and time latency). The neural Scuola Superiore Sant’Anna (SSSA) [8]. The lightweight exo- signals are usually measured from the muscle signals (e.g., skeleton can provide assistive torque in the sagittal plane to electric activities represented as surface EMG or shape changes the hip joints (see Figure 1). The APO was designed with a represented by the noncontact capacitive sensors). The serial elastic structure based on torsional springs, and the tor- mechanical signals are measured from the inertial measure- que was transmitted to the joints with two lightweight carbon ment units (IMUs) and loadcell sensors. The sensor nodes fiber-made links (driving part in Figure 1). A C-shaped (in can be integrated into the mechanical structure of the the coronal plane) structure combined with the bandages exoskeletons. The muscle signals respond faster than the fixed the exoskeleton to the waist and pelvis of the user, keep- mechanical signals. However, they convey more noises, and ing it stable on the human body. Two orthotic shells were the mechanical signals can produce signals with high repeat- connected to the carbon fiber-made links and fixed on the ability due to the advancement of sensing technology. The thighs with bandages. The torque was applied to the human combination of the signals can compensate each other to get body through the shells. There were 3 degree-of-freedoms better performance. (DoFs) for each leg, two passive (hip adduction/abduction The target for locomotion mode recognition is to pro- and pelvic tilting), and one active (flexion/extension) [8]. duce an accuracy as high as possible with the least interven- The passive DoFs ensured the stability of the whole system tion on the human body. The muscle signals require during ambulation. The core of the actuation system of additional electrodes or front-ends on the human body, APO was the DC motors with gearboxes (80 : 1 reduction which decreases the convenience and the potential willing- ratio). The torsional spring was placed on the axis of the flex- ness of uses. Another limitation is that the recognition ion/extension of the exoskeleton, between the DC motor parameters should be calibrated for each individual, increas- (gearbox) and the carbon fiber-made link. The basic calcula- ing the time needed before use. For hip joint exoskeleton con- tion of the interaction torque was achieved by the torsional trol, many researchers purely used mechanical signals for spring constant and the relative position of the encoders human locomotion mode recognition. For instance, the [8]. The interaction torques between the human body and study [14] combined the IMU sensors on an active pelvic the exoskeleton lead to the deformation of the spring. With orthosis and the foot pressure sensors for gait mode recogni- the integrated encoders and the stiffness of the torsional tion. The designed algorithm was an event-based fuzzy-logic spring, the control system of the APO can calculate the inter- structure triggered by the foot pressure sensors. The study action torques between the human body and the DC motor. [15] identified different gait modes with the hip joint angles The control strategy of APO was hierarchical control. measured from the encoder of the hip joint exoskeleton. The low-level controller was the torque control. The interac- The designed algorithm was a multilayer perceptron neural tion torques calculated from the encoders served as the feed- network. The study [14] conducted a real-time locomotion back of the control loop. The control output determined the mode recognition with IMU signals when wearing the active applied torque on the human body. There were zero-torque pelvic orthosis (APO [8]). The machine learning-based algo- mode and assistive-torque mode. For the zero-torque mode, rithms were trained and tested onboard. The studies men- the desired interaction torques between the legs and the exo- tioned above produced accurate recognition results on skeleton were zero. In the assistive-torque mode, the control- various locomotion mode tasks. However, for hip joint exo- ler’s commanded torque was a predefined curve (in one skeleton control, burdensome calibration for different indi- stride). The high-level controller was an adaptive oscillator- viduals and additional sensor nodes on the human body (AOs-) based controller, which used a set of adaptive oscilla- still limited practical applications. For instance, the study tors to track the phase of one gait cycle continuously. The overcame the subject-dependent problems with sEMG sig- input of the AO-based controller was the encoder signals nals, but the system still required pressure insoles on feet to representing the hip joint angle information, and the output provide gait event information. The study of our group vali- of the controller was the gait phase at time t and the corre- dated the recognition method with the APO. However, sponding anticipated torque. One merit of the AO-based subject-dependent training and calibration were needed controller was the continuous estimation of gait phases with before testing procedures. robustness to different walking speeds [16]. In this study, we proposed a locomotion mode recogni- tion method based on inertial measurement unit sensors on 2.2. Sensing System. We implemented an IMU board on each the hip joint exoskeleton. The designed fuzzy-logic-based algorithm can overcome the subject-dependent parameters leg (see Figure 1). The raw signals of the IMU board included in data training, which does not require training for each sub- 3-axis accelerations and 3-axis gyroscopes. There was a ject before uses. Besides, no additional sensors are required microcontrol unit (MCU) on the board, i.e., ATMEGA328. on the human body, increasing the convenience in practical The MCU calculated the pitch angle and the roll angle (global frame of the Cartesian system) with the acceleration and applications. We preliminarily evaluated the proposed method with an APO on the locomotion mode and locomo- gyroscope signals. The board was fixed on the cuff of the exo- tion transition recognition on multiple subjects. skeleton through a connector (3D printed). The pitch angle Applied Bionics and Biomechanics 3 St LW Task 3 Task 1 Control part SA SD Control circuit Orthotic shell SA St SD St LW Task 2 Task 4 Driving part Figure 2: The tasks in the experiments. St denotes standing, LW is IMU short for level walking, SA is short for stair ascending, and SD is short for stair descending. The arrows indicate ambulation direction. The tasks are denoted with different colors. 90 gait cycles, SA with 108 gait cycles, SD with 108 gait cycles, and 9 repetitions for each locomotion transition (LW⟶ SA, SA⟶ LW, gait initiation/termination). Subject 2 performed Figure 1: The hardware of the system, including the active pelvic 5 repetitions for each task, including LW with 50 gait cycles, orthosis (APO) and the IMU for measurement. SA with 60 gait cycles, SD with 60 gait cycles, and 5 repetitions for each locomotion transition. Subject 3 performed 6 repeti- tions for each task, including LW with 60 gait cycles, SA with 72 gait cycles, SD with 72 gait cycles, and 6 repetitions for each of the IMU corresponded to the flexion/extension of the hip locomotion transition. joint. The update rate of the tilt angles was 100 Hz. The data of the IMU boards were transmitted to a control circuit on the back of APO. The control circuit synchronized 3. Recognition Method the data of APO and the IMU boards via Universal Synchro- nous Asynchronous Receiver Transmitter (USART). The 3.1. Cascaded Recognition Method. The locomotion recogni- control circuit integrated a WIFI module. The data of the tion method was designed based on the signal features IMU sensors and the states of APO were transmitted to a acquired in the IMUs of both legs. The recognition method host computer wirelessly in each 10 ms. A graphic user inter- is cascaded, which firstly (first layer) classifies the static face on the computer was designed with MATLAB R2016b to mode (St) and dynamic modes (LW, SA, SD) and secondly control the data sequence and store the data. (second layer) identifies the corresponding dynamic loco- motion modes (see Figure 3). In the second layer, we 2.3. Experimental Protocol. In this study, we recruited three designed a fuzzy-logic-based algorithm. There were two healthy subjects. They had an average age of 27.3 years, an membership function sets in the fuzzy-logic-based algo- average height of 173.7 cm, and an average weight of rithms, one for each leg. The input of the membership 67.3 kg. Each subject wore the APO, as shown in Figure 1 functions was the data pair of peak-valley values detected in the experiment. In this experiment, we recorded 5 locomo- from the thigh angles. Therefore, in the second layer, we tion modes and 8 locomotion transitions. The locomotion firstly identified the peaks and valleys and secondly calcu- modes included standing (St), level walking (LW), stairs lated the fuzzy-logic membership functions. During the ascending (SA), and stairs descending (SD). The locomotion locomotion transitions, there are different leading legs. transitions included St⟵⟶LW, LW⟵⟶SA, and The procedures of two legs worked independently in the LW⟵⟶SD. Each subject performed 4 tasks of ambula- recognition process. tion to cover all the locomotion modes and transitions (see In the cascaded recognition method, the first step is to Figure 2). For task 1, there were 2 stride cycles of LW and 7 distinguish between the static locomotion mode and the stride cycles of SA (St⟶ LW⟶ SA⟶ St). For task 2, dynamic modes. As there were no gait patterns during stand- there were 7 stride cycles of SD and 2 stride cycles of LW ing, the signal profiles were much different from that of (St⟶ SD⟶ LW⟶ St). For task 3, there were 3 stride ambulation modes. We extracted time-domain features to cycles of LW and 5 stride cycles of SD (St⟶ LW⟶ SD represent the signal profiles of standing and other locomo- ⟶ St). For task 4, there were 5 stride cycles of SA and 3 tion modes. We firstly segmented the data (pitch angles and stride cycles of LW (St⟶ SA⟶ LW⟶ St). The tasks accelerations) with a 100 ms (10 samplings) sliding window. in the experiments were shown in Figure 2. The number of We calculated the standard deviation on the windows of stride cycles was shown in Table 1. In our study, to mimic the left leg’s pitch angles stdðθ Þ and the sum of absolute the locomotion in daily activities, we allowed the subjects to values of 3-axis accelerations sumðaccÞ. Additionally, we perform the locomotion modes at their favorite paces. There- compared the angular difference of the two legs in the sagittal fore, the number of gait cycles of the subjects was different. plane, expressed as θ = jθ − θ j. θ is the right thigh’s relative L R R Three subjects performed different task repetitions. Subject pitch angle, and θ is the left thigh’s pitch angle. θ is L relative 1 performed 9 repetitions for each task, including LW with the relative pitch angle between the left thigh and right thigh. 4 Applied Bionics and Biomechanics inputted to the subsequent fuzzy-logic-based algorithm, and Table 1: The stride cycle number in each task. then the flag would be deactivated. The procedure of detecting Task 1 Task 2 Task 3 Task 4 valleys and the values of the other leg was the same. LW (stride) 2 2 3 3 3.3. Fuzzy-Logic-Based Recognition Method. We designed a SA (stride) 7 0 0 5 fuzzy-logic-based method to separate between the locomo- SD (stride) 0 7 5 0 tion modes of LW, SA, and SD. As mentioned above, the input of the fuzzy-logic algorithm was a 2-dimensional (2D) vector containing the latest detected peak and valley The first layer recognition was achieved by comparing the (one leg), represented as θp and θv, respectively. The maxi- threshold-based conditions. The logic was expressed as mum/minimum values revealed the characteristics of differ- ent locomotion modes. For instance, the θp values of SA std θ > Th or θ − θ > Th or θ − θ ðÞ jj jj L std L init θ R init were larger than that of LW and SD, as the hip joint angles ð1Þ were larger in flexion when ambulating upward. The valleys > Th or sum acc > Th and θ > θ ðÞ θ Acc relative static: of the thigh pitch angles also demonstrated similar features. During the LW and SD mode, θv values were at the same If the logic condition was satisfied, the mode would be level, while they would decrease during the SA mode because recognized as dynamic modes. Otherwise, it was classified the stair-ascending locomotion contains a kicking-back as the static mode (St). θ is the initial pitch angle for both init movement in which the hanging leg could reach the lowest thighs which is close to 0. pitch angle without the constraint of the stairs. During LW In the above logic condition, Th was the standard devi- std and SD, θv would be limited by the ground and the stairs. ation threshold for the sliding window which was selected as We visualized the characteristics in Figure 5. The distribution 0.5 . Th was the pitch angle threshold for both legs which of maximum-minimum of different locomotion modes could was selected as 8 . θ was the threshold for the static mode static be separated apart. which was selected as 10 . Th was the threshold for accel- Acc We designed multivariate membership functions to clas- eration which was chosen as 500. sify the three locomotion modes. The membership function calculates the membership value of the event-based feature 3.2. Detecting the Extreme Values. The second layer was to belonging to the target mode. The output range of member- further separate the data into corresponding dynamic loco- ship is (0,1], where 1 is the maximum membership of the motion modes (i.e., LW, SA, and SD). We designed a fuzzy- model. The membership functions were calculated in parallel logic-based algorithm to classify the locomotion modes. Each with the signals of two legs. For the signals of each leg, we input was a 2×1 vector including the peak and valley of the defined three membership functions, one for each locomo- pitch angles. We designed an algorithm find_peak() to detect tion mode. The function was expressed as the peaks and valleys of the IMU signals. The most recent true peak and valley values found by find_peak() would be 1 −1 − C −X Σ C −X i ðÞ ðÞ put in a 2×1 buffer as the input of the subsequent fuzzy- 2 i i i i i f = e , i =1,2,3, ð2Þ 1/2 logic-based algorithms. Taking finding the left leg’s peak 2πΣ jj values for example (diagram see Figure 4), we firstly prede- fined thresholds for peak value Th_ and time interval where i denoted the mode’s number, k was the scale factor, peak Th_ . θ ðtÞ was the pitch angle of the left leg at time t. interval L C = ðθ , θ Þ was the input vector including the detected i pi vi We used PðiÞ to represent the i peak value found in th peak and valley, and X = ðμ , μ Þ was the central point of pi vi (pseudo) real-time. Secondly, the past 21 samples before time the membership function. In our study, the point was repre- t were compared (i.e., θ ðtÞ, θ ðt − 1Þ, ⋯, θ ðt − 20Þ). If θ ðt L L L L sented as the mean value of the training data sets. Σ was the − 10Þ is larger than all the 10 numbers backward i covariance matrix representing the data distribution. After ( θ ðt − 20Þ, θ ðt − 19Þ, ⋯θ ðt − 11Þ and all the 10 numbers L L L calculating three membership functions of LW, SA, and SD, forward ( θ ðtÞ, θ ðt − 1Þ ⋯ θ ðt − 9Þ), we set PðiÞ = θ ðt − L L L L respectively, the algorithm proceeded to calculate the maxi- 10Þ. If the absolute value of peak value PðiÞ minus the initial mal membership of the target mode: value of pitch angle θ ð0Þ was larger than Th_ ,we would L peak consider the peak value PðiÞ as an outlier which would be dis- Target mode = arg max f : ð3Þ ðÞ carded. Otherwise, the PðiÞ would be treated as a peak candi- i date. Because of the existence of false peak values created by the noise, we compared peak candidate’s location location_ Figure 5 shows that 2-dimensional space three member- P(i) with that of the latest candidate location_P(i-1). We set ship functions created three oval shape regions, whose center the time interval threshold Th_ for two adjacent peak interval coordinates were the mean value of three membership func- values. If the time interval between these two peak values tions (X ). In our study, the parameters X and Σ were fitted i i i was smaller than Th_ , we would assume that one of interval with the training data set (described in detail below). the two candidate peak values was false. The candidate with the smaller value was aborted. We set a decision flag for the 3.4. Synchronization of the Recognition Decisions. There were algorithm. The decision flag would be activated if a true peak inertial sensors on both thighs. The fuzzy-logic-based recog- and a true valley were detected. The peak-valley pair was then nition method worked in parallel for the left leg and the right Applied Bionics and Biomechanics 5 Standing First-layer classifier Sliding windows Level walking Threshold-based Second layer Detecting the classifier Dynamic maximum/ Stair ascending Fuzzy-logic-based modes minimum algorithm hip anglesy Stair descending Figure 3: The diagram of the recognition method. The first layer was to distinguish dynamic modes from standing (St). The second layer was designed to classify the dynamic modes, and there were three dynamic modes (LW, SA, and SD). (St⟶ other modes), the first recognized transition was Definition: Th_peak, Th_interval deemed to be the results, which were expressed as Init i = 1, P (0) t = min right legs s transition time point t , i R i ð4Þ left leg s transition time t , ... P (i) = θ (t-10): θ (t-10) > θ (t) , θ (t-1) , θ (t-2) , , L L L L L ... θ (t-9)&& θ (t-10) > θ (t-20) , θ (t-19) , , θ (t-11) L L L L L where t was the timing point of the detected transition. For gait termination recognition, the last recognized transition Yes was deemed to the recognition results. The timing point t was expressed as |P (i) -θ (0)| > Th_peak t = max right leg s transition time point t , t R ð5Þ left leg s transition time t : Yes No Discard the detected P (i) For the second layer recognition, the transition timing points (t ) were the first recognized timing points between 0d the left leg’s and right leg’s results. Set P (i) as a candidate 4. Evaluation Method 4.1. Crossvalidation. We used the crossvalidation method to evaluate the performances. We evaluated the performance location_P (i)-location_P (i-1) < Th_interval with 1 : 2 intersubject crossvalidation and 1 : 1 intrasubject crossvalidation. In the 1 : 1 intrasubject crossvalidation, each subject’s data were divided into two sets with the same sizes. Yes No The first data were used for training, and the second set for testing. The procedure was repeated with the second data Discard the smaller value set for training and the first set for testing. The results of the two tests were averaged as the result of the subject. In the 1 : 2 intersubject crossvalidation, we used the data of Find true peaks and true one subject for training and the data of the other subjects locations for testing. In the training procedure, the parameters of the fuzzy-logic-based algorithms were fitted. Figure 4: The flow chart of finding a peak value from the thigh pitch 4.2. Recognition Accuracy. The first metric for evaluating the angles. performance was the recognition accuracy. In the first layer, the recognition decisions (St and other dynamic modes) were continuously calculated in each sample. The recognition accuracy (recognition accuracy 1) leg. The recognition decisions were then synchronized to was defined as minimize the errors in locomotion transitions. In our cas- caded recognition method, the first layer was to distinguish Ncorrect1 i ðÞ between St and dynamic modes. The transitions were gait recognition accuracy1ðÞ i = , ð6Þ initiation and gait termination. For gait initiation recognition Ntotal1ðÞ i 6 Applied Bionics and Biomechanics –20 LW SD –30 –40 –50 SA –60 –70 –25 –20 –15 –10 –5 0 5 10 15 20 25 Maximum values of hip angles Figure 5: The distribution of the data of LW, SA, and SD (denoted by the dots) and the calculated membership function (represented by the ellipses). The data of LW, SA, and SD were represented as the yellow dots, the red dots, and the blue dots, respectively. The data were collected from subject 1. where i was the subject’s number, Ncorrect1 was the number labeled as the middle point of the swing phase. The labeling of correctly recognized decisions, and Ntotal1 was the total method motion transitions were the same as that of number of decisions. existing-related studies [18, 19]. In the second layer, the recognition decisions were calcu- The time delay (Td_init) of gait initiations was defined as lated in each extreme point being detected (peak and valley). the difference between the recognized timing point t and the The recognition accuracy of the second layer was defined as reference transition time of gait initiation t , expressing as 0i Td_init = t − t : ð8Þ Ncorrect i ðÞ i 0i recognition accuracyðÞ i = , ð7Þ NtotalðÞ i Similarly, the timed delay (Td_terminal) of gait termina- tions was expressed as where i was the number of subjects, Ncorrect was the number of correctly recognized gait cycles, and Ntotal was the total Td_terminal = t − t , ð9Þ t 0t number of gait cycles. where t was the recognition transition time of gait termina- 4.3. Confusion Matrix. We used the confusion matrix to illus- tion, and t was the reference transition time of gait trate the recognition performance of each locomotion mode. 0t The details of the definition can be found in [17]. termination. The time delay (Td_dynamic) of the dynamic modes was 4.4. Time Delay of the Locomotion Transitions. Another met- expressed as ric for evaluating the performance was the time delay. There were three critical timing points in each transition period, i.e., Td_dynamic = t − t , ð10Þ d 0d the timing point when the data changed from St to dynamic modes (gait initiation, t ), the timing point when the data 0i where t was the recognition transition time of dynamic changed from dynamic modes to St (gait termination, t ), 0t modes, and t was the reference transition time of dynamic 0d and the timing point when the data changed from one modes. The positive value presented the delay of recognition, dynamic mode to another (t ). 0d and the negative value represented the advance of recogni- The reference transition time was determined by labels. tion, shown in Figure 6. In our cascaded recognition method, the first layer was to separate between standing and dynamic modes. We manu- 5. Results ally labeled the data as standing and dynamic modes by IMU signals. If the pitch angles exceeded a threshold com- 5.1. Recognition Accuracy. In this section, we showed the rec- pared with that of standing, the data would be labeled as ognition accuracy for both the first and second layer classi- dynamic modes, and t was defined as the reference of the fiers. The first layer recognition was designed to distinguish 0i gait initiation transition time between standing and dynamic dynamic modes from standing (St). The second layer recog- modes, while t was defined as the reference of the gait ter- nition was designed to classify three dynamic modes (LW, 0t mination transition time between dynamic and standing SA, and SD). modes. For the second layer recognition, the reference transi- As for the first layer (classification between dynamic tion time was labeled based on the gait events detected by the modes and St), the recognition accuracy for each subject foot pressure insoles. t was defined as the reference of the was 92.18%, 93.00%, and 90.45%, respectively. The average 0d transition time between two dynamic modes. t would be recognition accuracy was 91.88%. As for the second layer 0d Minimum values of hip angles Applied Bionics and Biomechanics 7 –50 –100 0 200 400 600 800 1000 1200 1400 1600 SD Td_ini SA LW Td_dynamic St Td_terminal 0 200 400 600 800 1000 1200 1400 1600 Number of samples Right leg Reference Figure 6: The pseudo real-time recognition decisions. The upper subplot is the raw pitch angles and the detected extreme points. The bottom subplot shows the recognition decisions and the reference labels. (classification between LW, SA, and SD), the fuzzy-logic- Table 2: Training/testing with whole data of a subject (1 : 2 intersubject validation). based method produced accurate recognition decisions in locomotion mode tasks (the case result see Figure 6). The rec- Subject 1 Subject 2 Subject 3 ognition accuracy was higher than 0.89 for most of the eval- Subject 1 — 90.39% 94.76% uations (intersubject shown in Table 2 and intrasubject shown in Table 3). In Tables 2 and 3, each row represented Subject 2 89.84% — 95.51% the subject number used for training, and each column repre- Subject 3 93.23% 95.20% — sented the testing data (the subject used for testing). The off- diagonal results in Table 2 were the intersubject recognition accuracies, while the diagonal results in Table 3 denoted the Table 3: Training/testing with half data of a subject (1 : 1 accuracies of the intrasubject crossvalidation. intrasubject validation). In the 1 : 2 intersubject validation (see in Table 2), the membership functions trained with subject3’s dataset had Subject 1 Subject 2 Subject 3 the best performance, with the highest accuracy, 95.51%. Subject 1 88.29% 95.42% 95.22% The average recognition accuracy for each subject trained Subject 2 87.76% 95.31% 96.70% with different datasets was 91.54%, 92.80%, and 95.14% for Subject 3 86.80% 92.07% 94.60% subject 1, subject 2, and subject 3, respectively. The lowest accuracy (89.84%) occurred in subject 1 trained with subject 2’s dataset. Subject 3’s average recognition accuracy showed 5.2. Time Delay of Locomotion Transitions. We investigated the best performance, 95.14%. In Table 3 (intrasubject crossvalidation), recognition the time delay during locomotion transitions trained/tested accuracies showed a similar pattern but slightly decreased with half the data of a subject. We calculated the time differ- ence between the recognized locomotion transitions and the compared with intersubject validation. The lowest accuracy, 86.80%, occurred in subject 1 trained with subject3’s dataset. referenced ones. We defined the time latency of gait initia- tions as Td_init, gait terminations as Td_terminal, and the The average testing set recognition accuracy of three subjects dynamic transition as Td_dynamic. The unit of the results trained with the same training dataset was 92.98%, 93.26%, and 91.16%, respectively, with subject 1, subject 2, and sub- was ms. From Table 6, we can see that the average time delay of ject3’s training dataset. The average testing set recognition accuracy for each subject trained with different training data- gait initiations for each subject was 1077.3 ms, 812.8 ms, and 268.2 ms and the average time delay of gait terminations sets was 87.62%, 94.27%, and 95.51%, respectively. was 787.5 ms, 315.5 ms, and 29.2 ms. The average time delay In Tables 4 and 5, we presented the recognition accuracy for each task. From the experiment results, the lowest recog- for all the subjects was 554.4 ms. Subject 3 has the lowest time delay for both gait initiations and gait terminations (268.2 ms nition accuracy for each subject’s results usually occurred in task 2 (St⟶ SD⟶ LW⟶ St), while the recognition and 29.2 ms). Also, we can find that large time delays occurred in task 2 and task 3 frequently. algorithm usually performed better in task 1 and task 4. Recognition decision IMU signals 8 Applied Bionics and Biomechanics Table 4: Recognition accuracy of 1 : 2 intersubject validation. Subject 1 Subject 2 Subject 3 Number of Number of Accuracy Number of Number of Accuracy Number of Number of Accuracy error steps total steps rate error steps total steps rate error steps total steps rate Task 1 —— — 2 95 98.48% 1 109 99.08% Task 2 —— — 20 85 76.47% 12 102 88.24% Subject 1 Task 3 —— — 10 74 86.49% 7 93 92.47% Task 4 —— — 0 82 100.00% 1 97 98.97% Task 1 15 174 91.38% —— — 1 109 99.08% Task 2 21 176 88.07% —— — 10 102 90.20% Subject 2 Task 3 18 175 89.71% —— — 6 93 93.55% Task 4 15 154 90.26% —— — 1 97 98.97% Task 1 18 174 89.66% 3 132 97.73% —— — Task 2 19 176 89.20% 8 85 90.59% —— — Subject 3 Task 3 12 175 93.14% 3 74 95.95% —— — Task 4 18 154 88.31% 2 82 97.56% —— — Table 5: Recognition accuracy of 1 : 1 intrasubject crossvalidation. Subject 1 Subject 2 Subject 3 Number of Number of Accuracy Number of Number of Accuracy Number of Number of Accuracy error steps total steps rate error steps total steps rate error steps total steps rate Task 1 1 79 98.73% 1 37 97.30% 0 54 100.00% Task 2 15 83 81.93% 5 32 84.38% 6 56 89.29% Subject 1 Task 3 18 78 76.92% 0 28 100.00% 3 47 93.62% Task 4 3 68 95.59% 0 33 100.00% 1 49 97.96% Task 1 9 79 88.61% 1 37 97.30% 0 54 100.00% Task 2 12 83 85.54% 4 32 87.50% 5 56 91.07% Subject 2 Task 3 10 78 87.18% 1 28 96.43% 2 47 95.74% Task 4 7 68 89.71% 0 33 100.00% 0 49 100.00% Task 1 10 79 87.34% 1 37 97.30% 0 54 100.00% Task 2 10 83 87.95% 7 32 78.13% 4 56 92.86% Subject 3 Task 3 7 78 91.03% 2 28 92.86% 2 47 95.74% Task 4 13 68 80.88% 0 33 100.00% 5 49 89.80% For Table 7, we can see that the average Td_dynamic for rent study was that we simplified the setups of the sensing each subject was 685.1 ms, 541.8 ms, and 394.0 ms. The low- approaches in both training and testing procedures. The sim- est average Td_dynamic also occurred in subject 3. Also, we plification in sensors can reduce the time needed to calibrate can see that the large Td_dynamic usually occurred in task 3. the recognition procedure in practical applications. The average gait cycle was 1897.9 ms, for which the aver- There are many studies on IMU-based locomotion mode age time delay of average Td_init, average Td_terminal, and recognition. The performances are determined by the factors, average Td_dynamic was 549.5 ms accounted for 28.95% of including the sensor setups (sensors’ number, sensing posi- a gait cycle. tions), the robotic devices (exoskeletons, prostheses), and the processing algorithms. The evaluation method also influenced the numeric recognition results. For instance, the recent stud- 6. Discussion ies on IMU-based locomotion mode recognition achieved 6.1. Recognition Performances. In this study, we designed and >95% average recognition accuracies with intrasubject cross- validation [21, 22]. The sensor setups are quite different from evaluated the fuzzy-logic-based method for locomotion ours. The study mounted an IMU board on the amputated mode/transition recognition with a hip joint exoskeleton. foot for terrain identification [21], while the study fixed the The method only relied on the inertial sensors integrated into IMU boards on the shanks, the waist, and wrists [22]. the exoskeleton, and no additional sensors were required on the human body. Compared with the previous works using In our study, the IMU boards were fixed on the thighs of the subjects. The target robot platform is the hip exoskeleton. the same exoskeleton [14, 20], one improvement of our cur- Applied Bionics and Biomechanics 9 Table 6: The initial and terminal transition time latency. Subject 1 Subject 2 Subject 3 Td_init Td_teminal Td_init Td_teminal Td_init Td_teminal Task 1 —— 836.0 376.0 -300.0 161.7 Task 2 —— 1126.0 -354.0 1145.0 -443.3 Subject 1 Task 3 —— 390.0 1626.0 -145.0 -556.0 Task 4 —— 376.0 -184.0 -156.0 -527.5 Task 1 2270.0 192.2 —— -320.0 235.0 Task 2 1207.8 -1266.7 —— 1805.0 -436.7 Subject 2 Task 3 414.4 2652.2 —— -185.0 1768.3 Task 4 1241.1 641.1 —— 301.7 31.7 Task 1 567.8 305.6 840.0 334.0 —— Task 2 904.4 764.4 1630.0 110.0 —— Subject 3 Task 3 705.6 2198.9 964.0 452.0 —— Task 4 1307.8 812.2 340.0 452.0 —— Table 7: The dynamic transition time latency. average recognition accuracies were over 98% with subject- dependent training and testing processes. The calculation Td_dynamic time of each recognition decision was less than 1 ms, but Subject 1 Subject 2 Subject 3 the time delay of locomotion transitions was not reported. LW⟶ SA — 444.0 230.0 By comparison, in our study, we produced an average recog- SD⟶ LW — 136.0 -405.0 nition accuracy of 93.16% with 1 vs. 2 intersubject crossvali- Subject 1 dations and 92.46% with 1 vs. 1 intrasubject crossvalidation. LW⟶ SD — 1696.0 1815.0 Sa⟶LW — -154.0 -1.7 6.2. Confounding Factors. One key factor that influenced the LW⟶ SA 235.6 — 230.0 recognition accuracies was the detection of the peaks and val- SD⟶ LW leys from the inertial signals. As shown in Figure 6, the loco- 5.6 — -405.0 Subject 2 motion modes and transitions were successfully recognized; LW⟶ SD 235.6 — 1690.0 although, there were misdetections in the extreme values. Sa⟶ LW 687.8 — -1.7 The output value of the membership functions was deter- mined by the detected values of the extreme points. In the LW⟶ SA 365.6 444.0 — calculated recognition results, the fuzzy-logic-based method SD⟶ LW 921.1 136.0 — could successfully tell apart the locomotion modes as long Subject 3 LW⟶ SD 2198.9 1526.0 — as the distribution of the extreme points was distinguishable Sa⟶ LW 814.4 106.0 — (as shown in Figure 5). In our study, although no signals from the feet were measured, the maximum/minimum angles still showed gait information. The maximum value The recognition performances of our study were at the same of the thigh tilt angle usually occurred at the swing phase, level as that of the previous studies with similar sensor/robot which was used to distinguish between the LW and SA. setups [14, 20]. In the previous work with the active pelvic While the minimum angle occurred near the foot-off of one orthosis [14], the authors used inertial sensors on the thigh gait cycle, the values were informative to distinguish between and the foot pressure insoles for seven locomotion mode LW and SD. The maximum hip flexion/extension angles are recognitions. The average accuracies on six healthy subjects highly correlated to the locomotion modes. For instance, the achieved over 99% with locomotion transition tasks. The maximum flexion angles of SA were significantly larger than average time delay during the locomotion transitions was that of LW and SD. The subjects can adjust the patterns intu- fixed to one step. On the other hand, the authors also claimed itively to control the exoskeletons in locomotion transition the limitations of using additional foot insoles which was not tasks. The physical significance of the features in our fuzzy- integrated on the exoskeleton. The accuracies decreased to logic-based algorithm can accelerate the training/calibration 65.7%-91.2% in different testing data sets if the centre-of- procedure for a novice subject. Another point worth being pressure (CoP) information was removed [14]. In our noted is the intersubject variability in signal profiles and rec- previous works [20], the authors designed machine- ognition performances. In addition to the difference in learning-based algorithms for locomotion mode recognition. motion patterns, the difference in relative positions of the In the study of [23], the authors designed an artificial neural IMU boards on the thigh was another important reason. network- (ANN-) based recognition algorithm with the During the experiments, the IMU boards were fixed on the inertial signals (thigh) and the foot pressure signals. The same positions at the exoskeleton. Due to the different 10 Applied Bionics and Biomechanics terrains in the laboratory environment. In future works, anthropometries of the subjects, the relative positions on the thigh were different. In practical applications, the sensors more complicated tasks, including various walking speeds, usually are fixed on the exoskeleton. Adjusting the sensor jogging, and other locomotion modes in daily life, will be investigated. Thirdly, the recognition decisions in our study position to keep the same signal profiles across subjects is also impractical. In future real-time control, we will improve were discrete in one gait cycle. In future works, we will inves- the recognition algorithms with the ability of fast calibration tigate continuous parameter changes in the locomotion tasks, to make the trained model quickly update with the new user. such as different heights of stairs and different upward loco- motion modes (ramps and stairs). The processing algorithm will also be studied to cope with more complicated problems. 6.3. Influence of the Recognition Performances on Robotic Control. In real-time control of the APO, the hierarchical control framework is usually designed. The high-level con- 7. Conclusions troller recognizes the locomotion modes and determines In this study, we designed and preliminarily validated the fea- the assistive torque curve of the recognized terrain. The sibility of a fuzzy-logic-based algorithm for the locomotion middle-level controller uses adaptive oscillators (AOs) to mode and locomotion transition recognition with an active track the desired torque curve. The low-level controller pelvic orthosis. The method purely relied on the inertial sig- drives the motors to achieve the force feedback loop. nals measured from the thigh, and the sensors were fixed on If there are recognition errors, the desired torque curve the exoskeleton. With a proper training process, the fuzzy- for the controller will be different from that needed for the based algorithm produced comparable recognition accuracies current terrain. The user will move with an inappropriate to the existing studies on the same robotic platform. The supe- assistive torque curve. The APO applies assistive torque on riority of the method was that it required no additional sensors the hip joint angle in the sagittal plane. Due to the mechani- on the human body, increasing the convenience in practical cal design of the APO (2 passive DoFs in the coronal plane applications. The inputs of the fuzzy-logic-based method were and passive compliance), it is less likely that the user will fall the detected peaks and valleys of the pitch angles of the thigh. caused by the wrong recognition of the locomotion modes. Combined with the cascaded recognition method, it produced However, in the long-time use, the mismatching between reliable recognition results as long as the detected extreme the assistive torque curve and the terrains can increase the points were distinguishable between the dynamic locomotion metabolic cost (decrease the efficacy of the exoskeleton) and modes. Future works will be focused on onboard training the risks of the fall. If the transition time delay exceeds the and real-time control of the exoskeleton, investigation of the starting timing point of the applied assistive torque of one complicated unstructured terrains, and adaptation to continu- stride, there will be a mismatch between the assistance and ous ambulation parameters. the actual locomotion mode. Otherwise, the time delay is acceptable. In our study, the average time delay during the Data Availability transitions ranges from 300 ms to 1000 ms, which can cause a mismatch between the assistive torques and current loco- The data are made available through the corresponding motion modes. The impacts on the user are the same as that authors’ emails. of the recognition errors. To quantitatively evaluate the recognition errors, further Conflicts of Interest extensive experiments combing the real-time recognition and exoskeleton controller are needed. In future studies, we The authors declare no potential conflicts of interest with will investigate the effects of the errors and time delay with respect to the research, authorship, and/or publication of this real-time recognition and control. article. 6.4. Limitations and Future Works. Our current study has Authors’ Contributions some limitations, and the following issues will be addressed in future works. Firstly, the sample size of the subjects was In this study, Gang Du and Jinchen Zeng analyzed the data small (N =3). The results were calculated with an offline and designed the experiments. Jinchen Zeng and Enhao evaluation. The generalization ability of the fuzzy-logic- Zheng designed the recognition method. Cheng Gong con- based algorithm cannot be extensively evaluated with the ducted the experiments. Enhao Zheng guided the writing of small sample size. Due to the individual difference in loco- the article. Jinchen Zeng and Enhao Zheng wrote the article. motion patterns and sensor placements, the signal profiles Gang Du and Jinchen Zeng contributed equally to this work. can vary across the subjects. The onboard training and real- time exoskeleton control have yet to be studied. In future Acknowledgments works, we will carry out an extensive study on real-time con- trol with onboard training. We will investigate the effects of Thanks are due to the China University of Geosciences false detections on control performances. We will also carry (Beijing) Information Technology Innovation Experimental out experiments on more subjects to evaluate the generaliza- Base for the support of this research work. The work is sup- tion performances. Postprocessing approaches will also be ported by the Undergraduate Education Quality Improve- designed to remove the recognition errors further. Secondly, ment Project of China University of Geosciences (Beijing) the recognition tasks in our study only involved structured (No. XNFZ202005). Applied Bionics and Biomechanics 11 [17] E. Zheng, L. Wang, K. Wei, and Q. Wang, “A noncontact References capacitive sensing system for recognizing locomotion modes [1] A. J. Young and D. P. Ferris, “State of the art and future direc- of transtibial amputees,” IEEE Transactions on Biomedical tions for lower limb robotic exoskeletons,” IEEE Transactions Engineering, vol. 61, no. 12, pp. 2911–2920, 2014. on Neural Systems and Rehabilitation Engineering, vol. 25, [18] X. Liu and Q. Wang, “Real-time locomotion mode recognition no. 2, pp. 171–182, 2017. and assistive torque control for unilateral knee exoskeleton on [2] A. B. Zoss, H. Kazerooni, and A. Chu, “Biomechanical design different terrains,” IEEE/ASME Transactions on Mechatronics, of the Berkeley lower extremity exoskeleton (BLEEX),” vol. 25, no. 6, pp. 2722–2732, 2020. IEEE/ASME Transactions on Mechatronics, vol. 11, no. 2, [19] E. Zheng and Q. Wang, “Noncontact capacitive sensing-based pp. 128–138, 2006. locomotion transition recognition for amputees with robotic [3] https://www.cyberdyne.jp/english/products/HAL/. transtibial prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 2, pp. 161–170, [4] T. Yan, M. Cempini, C. M. Oddo, and N. Vitiello, “Review of assistive strategies in powered lower-limb orthoses and exoskeletons,” Robotics and Autonomous Systems, vol. 64, [20] C. Gong, D. Xu, Z. Zhou, N. Vitiello, and Q. Wang, “Real-time pp. 120–136, 2015. on-board recognition of locomotion modes for an active pelvis orthosis,” in 2018 IEEE-RAS 18th International Conference on [5] D. A. Winter, Ed., Biomechanics and Motor Control of Human Humanoid Robots (Humanoids), pp. 346–350, Beijiing, China, Gait: Normal, Elderly and Pathological, University of Waterloo Press, 1991. [21] F. Gao, G. Liu, F. Liang, and W. H. Liao, “IMU-based locomo- [6] C. L. Lewis and D. P. Ferris, “Invariant hip moment pattern while walking with a robotic hip exoskeleton,” Journal of Bio- tion mode identification for transtibial prostheses, orthoses, and exoskeletons,” IEEE Transactions on Neural Systems and mechanics, vol. 44, no. 5, pp. 789–793, 2011. Rehabilitation Engineering, vol. 28, no. 6, pp. 1334–1343, 2020. [7] T. Zhang, M. Tran, and H. Huang, “Design and experimental verification of hip exoskeleton with balance capacities for [22] F. Sherratt, A. Plummer, and P. Iravani, “Understanding walking assistance,” IEEE/ASME Transactions on Mechatro- LSTM network behaviour of IMU-based locomotion mode nics, vol. 23, no. 1, pp. 274–285, 2018. recognition for applications in prostheses and wearables,” Sen- sors, vol. 21, no. 4, p. 1264, 2021. [8] F. Giovacchini, F. Vannetti, M. Fantozzi et al., “A light-weight active orthosis for hip movement assistance,” Robotics and [23] C. Gong, D. Xu, Z. Zhou, N. Vitiello, and Q. Wang, “BPNN- Autonomous Systems, vol. 73, pp. 123–134, 2015. based real-time recognition of locomotion modes for an active pelvis orthosis with different assistive strategies,” International [9] K. Junius, M. Degelaen, N. Lefeber, E. Swinnen, B. Vanderborght, Journal of Humanoid Robotics, vol. 17, no. 1, article 2050004, and D. Lefeber, “Bilateral, Misalignment-Compensating, Full- DOF Hip Exoskeleton: Design and Kinematic Validation,” Applied bionics and biomechanics, vol. 2017, 14 pages, 2017. [10] T. Lenzi, M. C. Carrozza, and S. K. Agrawal, “Powered hip exo- skeletons can reduce the user's hip and ankle muscle activa- tions during walking,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21, no. 6, pp. 938–948, [11] K. Seo, J. Lee, Y. Lee, T. Ha, and Y. Shim, “Fully autonomous hip exoskeleton saves metabolic cost of walking,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 4628–4635, Stockholm, Sweden, 2016. [12] S. K. Banala, S. H. Kim, S. K. Agrawal, and J. P. Scholz, “Robot assisted gait training with active leg exoskeleton (ALEX),” IEEE Transactions on Neural Systems and Rehabilitation Engi- neering, vol. 17, no. 1, pp. 2–8, 2008. [13] B. Chen, L. Grazi, F. Lanotte, N. Vitiello, and S. Crea, “A real- time lift detection strategy for a hip exoskeleton,” Frontiers in neurorobotics, vol. 12, p. 17, 2018. [14] A. Parri, K. Yuan, D. Marconi et al., “Real-Time hybrid loco- motion mode recognition for lower limb wearable robots,” IEEE/ASME Transactions on Mechatronics, vol. 22, no. 6, pp. 2480–2491, 2017. [15] X. Wu, Y. Ma, X. Yong, C. Wang, Y. He, and N. Li, “Locomotion mode identification and gait phase estimation for exoskeletons during continuous multi-locomotion tasks,” IEEE Transactions on Cognitive and Developmental Systems, vol. 13, no. 1, pp. 45– 56, 2019. [16] T. Yan, A. Parri, V. Ruiz Garate, M. Cempini, R. Ronsse, and N. Vitiello, “An oscillator-based smooth real-time estimate of gait phase for wearable robotics,” Autonomous Robots, vol. 41, no. 3, pp. 759–774, 2017.

Journal

Applied Bionics and BiomechanicsHindawi Publishing Corporation

Published: May 24, 2021

There are no references for this article.