Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Validity and Reliability of Upper Limb Functional Assessment Using the Microsoft Kinect V2 Sensor

Validity and Reliability of Upper Limb Functional Assessment Using the Microsoft Kinect V2 Sensor Hindawi Applied Bionics and Biomechanics Volume 2019, Article ID 7175240, 14 pages https://doi.org/10.1155/2019/7175240 Research Article Validity and Reliability of Upper Limb Functional Assessment Using the Microsoft Kinect V2 Sensor 1 1 2 3 Laisi Cai , Ye Ma , Shuping Xiong , and Yanxin Zhang The Research Academy of Grand Health, Faculty of Sport Science, Ningbo University, Ningbo, China Department of Industrial and Systems Engineering, College of Engineering, Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea Department of Exercise Science, Faculty of Science, The University of Auckland, New Zealand Correspondence should be addressed to Ye Ma; maye@nbu.edu.cn Received 12 August 2018; Revised 27 November 2018; Accepted 2 December 2018; Published 11 February 2019 Academic Editor: Andrea Cereatti Copyright © 2019 Laisi Cai et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Objective. To quantify the concurrent accuracy and the test-retest reliability of a Kinect V2-based upper limb functional assessment system. Approach. Ten healthy males performed a series of upper limb movements, which were measured concurrently with Kinect V2 and the Vicon motion capture system (gold standard). Each participant attended two testing sessions, seven days apart. Four tasks were performed including hand to contralateral shoulder, hand to mouth, combing hair, and hand to back pocket. Upper limb kinematics were calculated using our developed kinematic model and the UWA model for Kinect V2 and Vicon. The interdevice coefficient of multiple correlation (CMC) and the root mean squared error (RMSE) were used to evaluate the validity of the kinematic waveforms. Mean absolute bias and Pearson’s r correlation were used to evaluate the validity of the angles at the points of target achieved (PTA) and the range of motion (ROM). The intersession CMC and RMSE and the intraclass correlation coefficient (ICC) were used to assess the test-retest reliability of Kinect V2. Main Results. Both validity and reliability are found to be task-dependent and plane-dependent. Kinect V2 had good accuracy in measuring shoulder and elbow flexion/extension angular waveforms (CMC > 0 87), moderate accuracy of measuring shoulder adduction/abduction angular waveforms (CMC = 0 69-0.82), and poor accuracy of measuring shoulder internal/external angles (CMC < 0 6). We also found high test-retest reliability of Kinect V2 in most of the upper limb angular waveforms (CMC = 0 75-0.99), angles at the PTA (ICC = 0 65-0.91), and the ROM (ICC = 0 68-0.96). Significance. Kinect V2 has great potential as a low-cost, easy implemented device for assessing upper limb angular waveforms when performing functional tasks. The system is suitable for assessing relative within-person change in upper limb motions over time, such as disease progression or improvement due to intervention. 1. Introduction valid and reliable, these systems require relatively large spaces, are expensive, and require experienced technicians, Three-dimensional (3D) upper limb functional movements therefore limiting their use in the clinic, at home, in pub- such as reaching, pushing/pulling, and throwing have been lic, and so forth. In comparison, a markerless motion cap- ture system would be a possible alternative [10] for upper studied in many areas including motor control [1, 2], neuro- physiology [3], clinical assessment and rehabilitation [4–6], limb assessment. and ergonomics [7, 8]. Currently, quantitative measurements Microsoft Kinect is a low-cost markerless motion capture of upper limb functions are normally carried out using system, which estimates the 3D location of body joints based marker-based motion capture systems [9], in which the 3D on 2D images with depth information using machine learn- ing algorithms [11]. Kinect is feasible to assess gait motion data is obtained based on the passive or active markers attached on the anatomical landmarks of partici- temporal-spatial parameters and kinematics [12] and can pants. Although the marker-based systems in assessing objectively evaluate static foot posture with good accuracy upper limb kinematics [3, 5] have been confirmed to be and reliability [13]. Kinect has the potential to be used as 2 Applied Bionics and Biomechanics experimental protocol was approved by the Research an assessment tool for certain aspects of the balance perfor- mance [14]. Kinect can also assist gait rehabilitation training Academy of Grand Health’s Ethics Committee at Ningbo in clinics by providing the lateral trunk lean angle as a University. real-time feedback [15]. Some research investigated the use 2.2. Testing Procedure. This study used a concurrent validity, of Kinect in clinics [4] and confirmed that Kinect can test-retest reliability design. The study was conducted at the accurately measure gross spatial characteristics such as lower biomechanics laboratory of Ningbo University. Upper limb limb and trunk kinematics but cannot measure smaller kinematics were recorded concurrently by a Kinect V2 sys- movements such as hand clapping with the same accuracy. tem with a sampling frequency of 30 Hz and a 3D motion Researchers also investigate the use of Kinect in the work- capture system with eight infrared high-speed cameras place and found that Kinect can determine risks of musculo- (Vicon, Oxford Metrics Ltd., Oxford, UK) with a sampling skeletal injuries in the workplace [16]. frequency of 100 Hz. Prior to data collection, Kinect V2 was Attempts have been made at using Kinect in upper limb placed on a tripod at 0.8 meters above the floor. Subjects assessment [17–19]. Different scales have been used as stood at 2 meters from the camera according to the recom- outcome measures for disease progressions and medical mendation [16]. interventions, which are subjective and could vary depending Each participant attended two testing sessions, seven on different observers. Therefore, quantitative data attained days apart. For each session, reflective markers were attached by measuring kinematics is necessary for therapy practice. to the anatomical landmarks of the participants according to Chen et al. [17] developed a Kinect-based system to measure the UWA upper limb model [22]. First, a static trial is active upper limb movements as a complementary output performed during which each participant stands in the ana- measure of functional rating scales for spinal muscular atro- tomical position. Then, the elbow and wrist markers were phy. They observed no significant differences in the active removed during the following dynamic trials. Four range of motion (ROM) between the patients and the con- functional tasks were performed which represent a range of trols. They also found that the Kinect-based system is not functional activities [23]. sensitive enough to capture the minor differences or early-stage progression in the high-functioning patient group (i) Task one is hand to the contralateral shoulder, [17]. Moreira et al. [18] developed a Kinect-based system for which represents all activities near contralateral upper body function assessment in breast cancer patients. shoulder such as washing axilla or zip up a jacket. Based on the extracted upper limb kinematic features, the Subjects started with the arm in the anatomical posi- Kinect-based classification system can diagnose upper limb tion with their hand beside their body in a relaxed impairments for breast cancer patients. Kinect has also been neutral position and end up with the hand touching used to assess 3D shoulder kinematics during computer use the contralateral shoulder (see Figure 1(a)) to provide some insight on shoulder kinematics for improv- ing office ergonomics [19]. (ii) Task two is hand to mouth or drinking, which Establishing the accuracy and reliability inherent in the represents activities such as eating and reaching Microsoft Kinect system is required before using it for the face. It begins with the same starting point upper limb assessment. The accuracy of the Kinect system and ends when the hand reached the subject’s in measuring lower limb kinematics has been evaluated mouth (see Figure 1(b)) using marker-based measurements as the gold standard (iii) Task three is combing hair, which represents activi- [4, 12, 20, 21]. The reliability of the Kinect measurement ties such as reaching the (back of the) head and has also been studied in postural control assessment [21], washing hair. Subjects were instructed to move their gait analysis [12], and static foot posture evaluation [13]. hand to the back of their head (see Figure 1(c)) However, to the best of our knowledge, a thorough valid- ity and reliability study of the Kinect system on assessing (iv) Task four is hand to back pocket, which represents 3D upper limb kinematics when performing functional reaching the back and perineal care. It begins with tasks is lacking. The goal of this study was to quantify the same starting point and ends when the hand is the accuracy and test-retest reliability of a Kinect motion placed on the back pocket (see Figure 1(d)) capture system in assessing upper limb kinematics when performing functional tasks. A marker-based motion capture At least five trials were collected for each task. system (Vicon, Oxford Metrics Group, Oxford, UK) was used as the gold standard measurement. 2.3. Upper Limb Models for the Vicon System and the Kinect V2 System. The Vicon system tracked and stored the spatial trajectories of the reflective markers attached to the subjects. 2. Methodology The UWA upper limb marker set was employed in this study [24], which includes 18 markers (see Figure 2). Trunk, upper 2.1. Subjects. Ten healthy male university students (age: 24 6±2 8 years, height: 174 05 ± 4 4cm, mass: 67 2±4 2 arm, forearm, and hand segments were defined based on the anatomical landmark positions. The definition of the upper kg) with no upper limb injuries or medication use that would have influenced their upper limb functions volun- limb segment coordination system for the Vicon system is teered to participate. Participants were informed about the presented in Table 1. The calibrated anatomical systems tech- basic procedure of the experiment before the test. The nique [25] is used to establish the motion of anatomical Applied Bionics and Biomechanics 3 (a) (b) (c) (d) Figure 1: Four upper limb functional tasks performed in the study. 4 Applied Bionics and Biomechanics (a) (b) Figure 2: The upper limb models for the Vicon system and the Kinect V2 system ((a) the upper body marker set for the Vicon system; (b) the skeleton model of the Kinect V2 system). Applied Bionics and Biomechanics 5 Table 1: The upper arm and torso anatomical segment coordinate systems for the Vicon system. Name Definition Origin C7 X Unit vector defined by the Y-axis and the Z-axis to create a right-hand coordinate system Torso Y Unit vector going from T10 to C7 Z Unit vector perpendicular to the sagittal plane defined by T10, C7, and CLAV, pointing laterally Origin The elbow joint center, which was the midpoint between EL and EM X Unit vector perpendicular to the Y-axis and the Z-axis, pointing anteriorly Right upper arm Y Unit vector going from the elbow joint center to the shoulder joint center (the midpoint between ASH and PSH) Unit vector perpendicular to the plane formed by the Y-axis of the upper arm and the long-axis vector of the forearm. Note: C7: 7th cervical vertebra; CLAV: clavicular notch; EC: elbow center; EL: lateral epicondyle; EM: medial epicondyle; PSH: posterior shoulder; RS: radial styloid; STRN: sternum; T10: 10th thoracic vertebra; US: ulnar styloid. Table 2: The upper arm and torso anatomical segment coordinate systems for the Kinect V2 system. Name Definition Origin SpineShoulder X Unit vector perpendicular to two vectors (Y and the vector from ShoulderRight to ShoulderLeft) Torso Y Unit vector going from SpineMid to SpineShoulder Z Unit vector defined by the X-axis and the Y-axis to create a right-hand coordinate system Origin The elbow joint center (ElbowRight) X Unit vector perpendicular to the Y-axis and the Z-axis, pointing anteriorly Right upper arm Y Unit vector going from the elbow joint center to the shoulder joint center (ElbowRight to ShoulderRight) Unit vector perpendicular to the plane formed by the Y-axis of the upper arm and the long-axis vector of the forearm landmarks relative to the coordinate systems of the during post-processing. The joint angles from both sys- upper-arm cluster (PUA) or the forearm cluster (DUA). tems were firstly resampled to 300 Hz using the Matlab The motion of the upper limb landmarks could be recon- function “interp” and then synchronized using a structed from their constant relative positions to the cross-correlation-based shift synchronization technique. upper-arm technical coordinate system. The kinematic 2.4. Statistical Analysis. The concurrent validity of the Kinect model based on the UWA upper limb model was developed V2 system for assessing the upper limb functional movement using Vicon Bodybuilder software. The reference shoulder waveforms was carried out using the coefficient of multiple and elbow joint angles were calculated based on the mea- correlation (CMC) [26] and the root mean squared error sured position of the passive optical markers using the kine- matic model via Vicon Nexus software (Oxford Metrics (RMSE) between the waveforms calculated by the Kinect V2 and the Vicon-based system. Mean bias (Kinect-Vicon) Group, Oxford, UK). A Butterworth low-pass filter was used and Pearson’s r correlation between the two systems were with the cut-off frequency of 6 Hz for both Vicon and Kinect used to evaluate the concurrent validity of the Kinect-based V2 systems. For a more detailed description, see [22, 24]. system in assessing the joint angles at the point of target The 3D coordinates of the anatomical landmarks identi- fied from the skeletal model of the Kinect V2 system during achieved (PTA) and the range of motion (ROM). Paired Stu- dent t-tests were used to compare the results of the angles at functional tasks were also recorded. Local segment coordi- the PTA and the ROM with the significant level of 0.01. The nates including torso and upper arm were established, and concurrent validity was presented using data from session one. each of them was based on the global coordinate The CMC and RMSE between the waveforms from ses- (Table 2). Then, our customized upper limb kinematics for the Kinect V2 system calculated the three Euler angles sion one and session two measured by the Kinect V2 system were used to assess the relative and absolute test-retest reli- for shoulder rotations, which follows the flexion (+)/exten- ability. The reliability of the Kinect V2 system in assessing sion (-), adduction (+)/abduction (-), and internal (+)/exter- the joint angles at the PTA and the ROM from selected func- nal (-) rotation order (see Figure 2). The elbow flexion was tional tasks was also carried out using the intraclass correla- calculated by the position data from ShoulderRight, tion (ICC ElbowRight, and WristRight using the trigonometric ) coefficient with the absolute measure. The 3,k function. The kinematics model for Kinect V2 was developed ICC and Pearson’s r correlation as well as the descriptive sta- using Matlab 2017a. The angular waveforms between the tistics were performed using SPSS 22.0. The CMC and RMSE Kinect V2 sensor and the Vicon system were synchronized were analyzed using Matlab 2017a. 6 Applied Bionics and Biomechanics Shoulder F/E Shoulder Add/Abd 60 5 −5 −10 −15 −20 −10 −25 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 80 140 −20 0 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 3: Kinematics from hand to contralateral shoulder task. ° ° 3. Results and elbow flexion/extension angles of 7.16 and 10.43 and the highest RMSEs were identified in the combing hair ° ° 3.1. The Concurrent Validity of the Kinect V2 System for task with the RMSEs of 41.4 and 23.75 for shoulder flex- Upper Limb Functional Assessment. The kinematic wave- ion/extension elbow flexion/extension angles. For angular forms of the selected upper limb functional tasks in both ses- waveforms in the frontal plane at the shoulder joint, i.e., sions from the Kinect V2 system and the Vicon system are the shoulder adduction/abduction angle, the RMSEs presented in Figures 3-6 by means of the average segment between the Kinect V2 and Vicon systems are under six rotation angles. The validity of the Kinect V2 system in asses- degrees except for the combing hair task (RMSE = 12 31 ). sing upper limb angular waveforms, joint angles at the PTA, For shoulder angular waveforms in the transverse plane, and the ROM are presented in Tables 3 and 4. i.e., the shoulder internal/external rotation angle, the smallest High-level agreements (see Table 3) were observed for RMSE was found in the hand to mouth drinking task shoulder and elbow kinematics in the sagittal plane across (RMSE = 1 64 ) and the biggest RMSE was found in the all tasks with the CMC values of 0.81-0.94. Compared to combing hair task (RMSE = 29 38 ). the shoulder joint, elbow flexion/extension angles showed Mean (SD) values for joint angles at the PTA and the the best agreements between the two systems with the ROM estimated by the Kinect V2 and Vicon systems are CMC greater than 0.9 for all tasks except for the combing provided in Table 4. Excellent relative agreements hair task (CMC = 0 87). Shoulder adduction/abduction (r =0 73–0.97) were observed for all investigated angles angular waveforms showed moderate agreements between at PTA in all tasks except for moderate relative agreement the two systems with the CMC values of 0.69-0.82. The of the shoulder internal/external rotation in the hand to lowest CMCs between the two systems were found in the back pocket task and elbow flexion/extension in the hand transverse plane at the shoulder joint with the CMC values to contralateral shoulder task (r =0 46 and r =0 45, of lower than 0.6 except for the hand to contralateral shoul- respectively) and poor agreement of the shoulder and der task (CMC = 0 84). elbow flexion/extension angle in the combing hair task The RMSEs (see Table 3) between the angular waveforms (r = −0 20 and r =0 21, respectively). For the ROM, excellent relative agreements were observed for shoulder flexion/ex- from the two systems are also both plane-dependent and task-dependent. For angular waveforms in the sagittal plane tension and shoulder adduction/abduction angles in all tasks at shoulder and elbow joint, the lowest RMSEs were found (r =0 91-0.99) except for the combing hair task (r =0 20 and in the hand to back pocket task with the RMSEs of shoulder 0.65, respectively); excellent agreements were found for Degrees Degrees Degrees Degrees Applied Bionics and Biomechanics 7 Shoulder F/E Shoulder Add/Abd 100 −5 −10 −15 −20 −25 −20 −30 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 50 140 0 0 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 4: Kinematics from hand to mouth/drinking task. shoulder internal/external rotation angles only in the shoul- waveforms, the angles at the PTA, and the corresponding der to contralateral shoulder task and the combing hair task ROM is presented in Table 5. Good to excellent relative (r =0 74 and 0.77), respectively; poor to moderate agree- reliability (CMC = 0 75-0.99) was observed for all ments were found for elbow flexion/extension angles in all between-session angular waveforms across all selected tasks tasks (r <0 65) except for the hand to back pocket task, except for the shoulder adduction/abduction in task one which showed excellent agreement (r =0 97). (CMC = 0 7) and the shoulder internal/external rotation in There is a clear tendency that the Kinect V2 system the fourth task (CMC = 0 6). Angular waveforms in the overestimated shoulder flexion/extension angles and sagittal plane (CMC = 0 89-0.99) are more reliable than those underestimated elbow flexion/extension angles in all tasks. in the frontal plane (0.70-0.84) and transverse plane According to the mean absolute bias of the angles at the (CMC = 0 60-0.93). The RMSEs are under ten degrees except PTA between the Kinect V2 and Vicon systems (see for elbow F/E (RMSE = 8 3-11.03) in the first three tasks and Table 2, K-V), there were no significant bias only for shoulder IR/ER in task 3 (RMSE = 12 09). The worst abso- shoulder flexion/extension, shoulder internal/external rota- lute test-retest reliability was observed in the combing hair tion, and elbow flexion/extension angles in the hand to task, in which the RMSE of shoulder flexion/extension is back pocket task and shoulder adduction/abduction angle the biggest (18.91 ). in the hand to mouth/drinking task. For the absolute bias Results for the test-retest reliability of the Kinect V2 of the ROM, only elbow flexion/extension in the first and system in assessing the angles at the PTA and the ROM are last tasks, the shoulder adduction/abduction angle in the also presented in Table 5. There are no significant second task, and the shoulder internal/external rotation between-session differences in the Kinect V2 system for all in the second and fourth tasks showed no significant dif- investigated angles for all of the studied motions. All ferences. The greatest biases of the ROMs were found in between-session ICCs of the angle at the PTA from all the shoulder flexion/extension angles in the hand to motions are good to excellent for the Kinect V2 system mouth/drinking task and the combing hair task (ICC = 0 65-0.91) in all selected tasks except for shoulder (RMSE = 32 5-40.9 ). adduction/abduction in the first task (ICC = 0 59) and elbow flexion/extension in the combing hair task (ICC = 0 27). All 3.2. The Test-Retest Reliability of the Kinect V2 System for between-session ICCs of the ROM measurement from all Upper Limb Functional Assessment. The test-retest reliability studied motions are good to excellent for the Kinect V2 sys- of the Kinect V2 system in assessing the upper limb angular tem (ICC = 0 68-0.96) except for the motions in the combing Degrees Degrees Degrees Degrees 8 Applied Bionics and Biomechanics Shoulder F/E Shoulder Add/Abd 200 0 −10 −20 −30 −40 −50 −50 −60 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 60 160 −20 20 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 5: Kinematics from combing hair task. hair task (ICC = 0 35-0.70). The lowest reliability of the ROM Generally, the mean deviations between joint angles was shoulder flexion/extension in the combing hair task with assessed by the Kinect V2 system and the Vicon system are the mean ICC of 0.35. greater for tasks or planes with a larger range of motions. The shoulder and elbow angular waveforms in the sagittal plane measured by the Kinect V2 system highly agreed with 4. Discussion the reference angles, and the RMSEs of the Kinect V2 system in measuring sagittal plane kinematics are generally greater This study tested the concurrent validity and the test-retest reliability of upper limb functional assessment using Kinect or comparable than angles in the frontal plane and transverse V2. We found that both validity and reliability are plane in comparison with those from the Vicon system. Sim- ilarly, the accuracy of the Kinect V2 system in assessing task-dependent and plane-dependent. The Kinect V2 system had good accuracy in measuring angles at the PTA and the ROM are task-dependent and plane-dependent. The angles at the PTA showed good rela- shoulder and elbow flexion/extension angles, moderate accu- racy of measuring shoulder adduction/abduction angles, and tive agreement for almost all motions investigated except poor accuracy of measuring shoulder internal/external for motions in the combing hair task, which have a large range of motion. angles. We also found high test-retest reliability of the Kinect V2 system in most of the upper limb angular waveforms, Our finding agrees with the results of [19], which found that the Kinect V2 sensor had a better estimation on shoulder angles at the PTA, and the corresponding ROM. However, there are also some deviations both between the Kinect V2 flexion/extension, compared with shoulder adduction/abduc- system and the Vicon system (gold standard) and between tion and shoulder internal/external rotation during the com- puter operation tasks [19]. They also found that the shoulder two test sessions. flexion/extension angle had the lowest RMSE (under 15 ) and 4.1. Concurrent Validity of Kinect V2. The angular trajecto- the magnitude of error is proportional to the magnitude of ries from the Kinect V2 and Vicon have similar waveform the shoulder adduction/abduction angle [19]. The concur- patterns, especially for the flexion/extension angular wave- rent validity results of the upper limb motions are not as forms under all functional tasks (CMC = 0 good as those for lower limb and trunk motions. 81-0.94). Never- theless, the RMSEs between the kinematic patterns The measurement performance of the Kinect V2 system measured by the Kinect V2 system and Vicon system are is highly varied on tasks, joints, and planes of movement. not consistent with the angular waveform agreements. According to Clark et al.’s study [15, 21], the Kinect V2 Degrees Degrees Degrees Degrees Applied Bionics and Biomechanics 9 Shoulder F/E Shoulder Add/Abd 10 −5 0 −10 −10 −15 −20 −20 −30 −25 −40 −30 −50 −35 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 45 80 10 20 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 6: Kinematics from hand to back pocket task. system showed excellent concurrent validity with the Vicon and lower limb movements in comparison with those of system, with the Pearson r values >0.9 for the majority of upper limb tasks. measurements for trunk and lower limb motions during lat- eral reach, forward reach, and single leg balance. In contrast, 4.2. Test-Retest Reliability of Kinect V2. We found a high the Kinect V2 system showed lower concurrent validity in degree of reproducibility in almost all of the upper limb assessing upper limb motions in our study with the Pearson angular waveforms, angles at the PTA, and the ROM mea- r values of 0.7-0.99 for most upper limb motions at the sured by the Kinect V2 system during all functional tasks PTA and the ROM. across the two testing sessions. Our finding suggests that The abovementioned findings can be explained by the the Kinect V2 system can reliably assess 3D shoulder underlying real-time human pose recognition algorithm motions and elbow flexion/extension for individuals carry- and the nature of the single-depth camera. The Kinect system ing out such functional tasks. The repeatability of upper estimates the 3D location of body joints based on 2D images limb motions in the frontal and transverse planes was with depth information using machine learning algorithms. lower than that in the sagittal plane. However, there were The final set of confidence-weighted 3D joint proposals is still some between-session deviations in the starting point based on a global optimization algorithm using training data, and ending point of the upper limb. Greater standardiza- which represents postures in an entertainment environment tion of both the starting and ending points for the tasks [11]. Therefore, Kinect should be evaluated carefully before would be required to improve the repeatability of the employing it as a research tool. If a joint is shaded by other starting and ending points of this type of motion. The body parts, it is difficult for the Kinect system to define the test-retest reliability of the trunk motion is high corresponding anatomical landmarks of the Kinect skeletal (ICC > 0 73) during lateral reach and forward reach in model, which directly results in inaccurate joint angle predic- Clark et al.’s research [15], which is similar to results of tion. Anatomical landmarks of the trunk and lower limb the upper limb motions in our study. identified from the skeletal model of the Kinect system may The combing hair task is not an ideal upper limb func- be more accurate because the segments of the trunk and tional task when using the Kinect V2 system as the outcome lower limb are usually not shaded by other body parts during measure tool. During the combing hair task, the Kinect V2 the tasks in reach, balance test, or walking. This may be the system had the worst accuracy and reliability in assessing main reason for the higher validity in the assessment of trunk the angular waveforms, the angles at the PTA, and the Degrees Degrees Degrees Degrees 10 Applied Bionics and Biomechanics Table 3: Validity of the angular waveforms for the selected functional tasks. The mean (SD) coefficient of multiple correlation (CMC) and the root mean squared error (RMSE) between the Kinect V2-based markerless system and the Vicon-based system of the upper limb waveforms are presented. Hand to contralateral shoulder Hand to mouth/drinking Combing hair Hand to back pocket Shoulder Shoulder Shoulder Elbow Shoulder Shoulder Shoulder Elbow Shoulder Shoulder Shoulder Elbow Shoulder Shoulder Shoulder Elbow F/E Add/Abd IR/ER F/E F/E Add/Abd IR/ER F/E F/E Add/Abd IR/ER F/E F/E Add/Abd IR/ER F/E 0.88 0.82 0.84 0.94 0.82 0.58 0.92 0.81 0.78 0.54 0.87 0.93 0.69 0.55 0.90 CMC 0.80 (0.2) (0.09) (0.11) (0.07) (0.05) (0.09) (0.20) (0.04) (0.11) (0.13) (0.17) (0.05) (0.05) (0.27) (0.27) (0.13) 11.04 5.62 13.40 15.48 25.61 1.64 18.16 41.40 12.31 29.38 23.75 7.16 5.76 12.36 10.43 RMSE 6.11 (2.85) (5.16) (1.37) (3.00) (6.04) (7.56) (5.20) (4.95) (9.49) (2.65) (6.76) (4.32) (2.87) (2.76) (3.74) (6.21) Note: shoulder F/E, shoulder Add/Abd, shoulder IR/ER, and elbow F/E represent joint angles of shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation, and elbow flexion, respectively. Applied Bionics and Biomechanics 11 Table 4: Concurrent validity of the joint angles at the point of target achieved (PTA) and the ROM of the selected functional tasks. The mean (SD) peak joint angles and the ROM are presented with the Pearson correlation (r) and the discrepancy (Kinect-Vicon) between the Kinect V2-based markerless system and the Vicon-based system is presented. PTA ROM Segment rotations Kinect Vicon P Bias (K-V) r Kinect Vicon P Bias (K-V) r (K,V) (K,V) Shoulder to contralateral shoulder Shoulder F/E 52.1 (10.3) 38.1 (6.7) <0.01 14.0 0.74 51.5 (10.0) 43.7 (6.9) <0.01 7.8 0.98 Shoulder Add/Abd 5.5 (7.8) −0.4 (6.3) <0.01 5.9 0.89 26.2 (5.8) 14.3 (3.9) <0.01 7.9 0.99 Shoulder IR/ER 63.7 (6.8) 68.5 (5.7) <0.01 −4.8 0.89 38.7 (11.7) 66.7 (14.9) <0.01 −28 0.74 Elbow F/E 110.9 (8.7) 125.2 (3.5) <0.01 −14.3 0.45 91.8 (8.3) 98.2 (6.2) 0.06 −6.4 0.41 Hand to mouth/drinking Shoulder F/E 95.6 (19.4) 57.3 (10.2) <0.01 38.3 0.88 94.8 (19.4) 62.3 (12) <0.01 32.5 0.91 Shoulder Add/Abd −8.5 (4.5) −8.2 (2.5) 0.67 −0.3 0.85 20.4 (9.6) 14.4 (13.4) <0.01 6.0 0.95 Shoulder IR/ER 47.5 (11.4) 32.3 (7.3) <0.01 15.2 0.93 21.7 (9) 29.2 (10.5) 0.07 −7.5 0.33 Elbow F/E 107.1 (9.3) 129.6 (5.7) <0.01 −22.5 0.80 89.1 (10.1) 102.3 (6.6) <0.01 −13.2 0.65 Combing hair Shoulder F/E 143.5 (14.1) 102.6 (20.4) <0.01 40.9 −0.20 140 (22.7) 106 (15.6) <0.01 34.0 0.20 Shoulder Add/Abd −53.7 (7.6) −54.7 (7.2) 0.40 1.0 0.89 42.2 (6.4) 46.9 (5.6) 0.02 −4.7 0.65 Shoulder IR/ER 49.4 (11.2) 25.3 (10.2) <0.01 24.1 0.72 55.6 (16.9) 42.8 (10.9) <0.01 12.8 0.77 Elbow F/E 113.6 (3.4) 139.6 (14.1) <0.01 −26 0.21 91.1 (7.7) 110.9 (13.7) <0.01 −19.8 0.05 Hand to back pocket Shoulder F/E −41.8 (6.2) −42.9 (6.1) 0.47 1.1 0.73 43.4 (6.7) 38.7 (6.1) <0.01 4.7 0.91 Shoulder Add/Abd −33 (10.3) −25.5 (7.6) <0.01 −7.5 0.97 22.9 (9.6) 16.7 (7.6) <0.01 6.2 0.96 Shoulder IR/ER 42.3 (9.6) 37.7 (5.7) 0.02 4.6 0.46 18.6 (7.9) 23.3 (5) 0.16 −4.7 −0.10 Elbow F/E 73.7 (21.3) 81.2 (13.2) 0.03 −7.5 0.94 51.8 (25) 52.1 (18.1) 0.91 −0.3 0.97 Note: max and min are the maximum and minimum joint angles, respectively. ROM: corresponding range of motion. Shoulder F/E, shoulder Add/Abd, shoulder IR/ER, and elbow F/E represent joint angles of shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation and elbow flexion, respectively; K-V is the bias between the Kinect V2 and Vicon systems. corresponding ROM. The greatest deviations of the angular (and expensive) motion capture laboratories and would waveforms for the Kinect V2 system were identified for all allow for in situ data collection. This markerless system investigated motions in the combing hair task. The Kinect would have a significant impact in the fields of clinical practice. Unlike clinical scales, the markerless motion analy- V2 system also cannot measure shoulder and elbow flexio- n/extension angles at the PTA and the ROMs of all investi- sis technique is objective, which helps to quantify massive gated motions in the combing hair task with low test-retest kinematic parameters and turn data into a knowledge-based reliability (ICC = 0 27-0.65). During the task, the absolute data warehouse in a standard way. The whole process makes test-retest reliability of shoulder F/E is the worst with the it possible for identifying and explaining relationships between-session RMSE of 18.91 between different motion patterns or different population . Thus, it is better not to use the same functional task like combing hair in the assess- groups through data mining. ment of upper limb functions. 4.4. Limitation and Future Work. The experiment had a lim- 4.3. Clinical Implications. Our results show that the Kinect ited sample size (ten male university students). The system V2 system can reliably measure upper limb motions. Kinect has not yet been used in other populations or other environ- V2 has good relative agreements of angular waveforms and ments such as upper limb disordered patients in clinical set- can accurately measure a few of the shoulder and elbow joint tings, elderly people at home or in the clinical environment, angles during the functional tasks. Although the measure- and people working in the workplace. The finding from this ment of upper limb kinematics may be not as accurate as study may have differed if the assessment was undertaken the Vicon system, Kinect V2 is useable to track relative in a clinical population. Upper limb motions of healthy peo- within-person changes in movements over time (such as ple often have low variability. The low variability increases the worsening of movement symptoms with disease progres- heterogeneity and would potentially lead to higher reliability sion or improvement due to intervention). estimates and stronger correlations between Kinect V2 and In current clinical practice, clinical scales have been the reference. In the future, tests with larger sample size, including patients, will be conducted. widely used to assess upper limb functions, which are subjective and labour-intensive. The development of a Further work is also concerned to improve the accuracy video tracking system based on low-cost markerless cam- of the Kinect system in measuring upper limb joint angles. eras would free clinicians from the necessity of dedicated There have been several recent studies in estimating body 12 Applied Bionics and Biomechanics Table 5: Reliability of the angular waveforms, the angles at the point of target achieved (PTA), and the range of motion (ROM) for the selected functional tasks. The mean (SD) coefficient of multiple correlation (CMC) and the root mean squared error (RMSE) of the upper limb waveforms calculated by the Kinect V2 system between two sessions are presented. The intraclass correlation (ICC) of the angles at the PTA and the ROM of the selected functional tasks are also presented. ICC Segment rotation CMC RMSE PTA ROM Shoulder F/E 0.93 (0.08) 7.89 (3.88) 0.80 0.88 Shoulder Add/Abd 0.70 (0.28) 9.19 (5.29) 0.59 0.68 Hand to contralateral shoulder Shoulder IR/ER 0.93 (0.06) 5.93 (1.87) 0.88 0.96 Elbow F/E 0.97 (0.3) 11.03 (4.50) 0.86 0.80 Shoulder F/E 0.99 (0.09) 9.90 (8.67) 0.80 0.81 Shoulder Add/Abd 0.75 (0.28) 5.03 (2.98) 0.78 0.95 Hand to mouth/drinking Shoulder IR/ER 0.75 (0.28) 7.25 (5.64) 0.76 0.80 Elbow F/E 0.96 (0.05) 10.70 (5.77) 0.72 0.84 Shoulder F/E 0.97 (0.03) 18.91 (8.11) 0.65 0.35 Shoulder Add/Abd 0.84 (0.12) 7.70 (3.69) 0.65 0.43 Combing hair Shoulder IR/ER 0.90 (0.06) 12.09 (4.52) 0.83 0.67 Elbow F/E 0.95 (0.05) 10.68 (6.24) 0.27 0.70 Shoulder F/E 0.96 (0.03) 5.14 (2.51) 0.73 0.84 Shoulder Add/Abd 0.82 (0.18) 4.71 (2.50) 0.88 0.92 Hand to back pocket Shoulder IR/ER 0.60 (0.23) 7.10 (2.57) 0.91 0.82 Elbow F/E 0.89 (0.13) 8.38 (5.18) 0.85 0.88 Note: shoulder F/E, shoulder Add/Abd, shoulder IR/ER, and elbow F/E represent joint angles of shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation, and elbow flexion. landmarks and motions using the Kinect sensor [27, 28]. It is Appendix likely that using these techniques would possibly produce Appendix A: The Pearson Correlation more accurate joint kinematics. Attempts have also been Coefficient (r) of the Angles at the PTA via the made using multiple Kinect sensors concurrently to improve Vicon and Kinect Systems the accuracy of tracking movement [28]. Calibration algo- rithms are another solution to improve the joint angle pre- The scatterplot includes PTAs of shoulder flexion/extension diction accuracy of the Kinect system. Xu and colleagues (Flex/Ext), shoulder abduction/adduction (Abd/Add), shoul- used the linear regression algorithm to calibrate the shoul- der internal rotation/external rotation (IR/ER), and elbow der adduction/abduction angle, and the prediction accuracy flexion/extension (Flex/Ext) during the four upper limb func- was significantly improved [19]. Using the state-of-the-art tional tasks. Task 1 is the hand to contralateral shoulder task. machine learning algorithms, it is possible to improve the Task 2 is the hand to mouth task. Task 3 is the combing hair current joint kinematic measurement accuracy as high task. Task 4 is the hand to back pocket task. The x-axis of agreements were found for angles in the sagittal and frontal each subfigure is each individual’s PTA via the Kinect system. planes (CMC > 0 78) for most investigated motions. The y-axis of each subfigure is each individual’s correspond- ing PTA via the Vicon system. For each subfigure, the linear 5. Conclusion correlation between the x-axis value and the y-axis value are presented by a line and the corresponding Pearson correla- The Kinect V2-based upper limb functional assessment tion coefficient. system developed in this research has good test-retest reliability in assessing the upper limb angular waveforms Appendix B: The Pearson Correlation and the angles at the point of target achieved except for the combing hair task. The Kinect V2-based system can Coefficient (r) of the Range of Motion (ROM) accurately assess shoulder flexion/extension, elbow flexio- via the Vicon and Kinect Systems n/extension, and shoulder adduction/abduction in some upper limb functional tasks. The Kinect V2 sensor has The scatterplot includes ROMs of shoulder flexion/exten- great potential as a low-cost, easily implemented device sion (Flex/Ext), shoulder abduction/adduction (Abd/Add), for assessing upper limb angular waveforms when per- shoulder internal rotation/external rotation (IR/ER), and forming functional tasks. Our system is suitable for asses- elbow flexion/extension (Flex/Ext) during the four upper sing relative within-person change in upper limb motions limb functional tasks. Task 1 is the hand to contralateral over time, such as disease progression or improvement shoulder task. Task 2 is the hand to mouth task. Task 3 due to intervention. is the combing hair task. Task 4 is the hand to back Applied Bionics and Biomechanics 13 pocket task. The x-axis of each subfigure is each individ- Number: 2018A610193), Fujian Provincial Key Laboratory ual’s ROM via the Kinect system. The y-axis of each sub- of Rehabilitation and Fujian Rehabilitation Industry figure is each individual’s corresponding ROM via the Research Institute fund (Grant Number: 2015Y2001-65), Vicon system. and the National Research Foundation of Korea (Grant Number: 2017R1C1B2006811). The study was also sup- ported by the Scientific Research Foundation of Graduate Appendix C: Scatterplots of Both the School and K.C. Wong Magna Fund in Ningbo University. Between-Device CMCs (at Session One) and the Between-Session CMCs for the Kinect System References The left subfigure shows the between-device CMCs (session one). The right subfigure shows the between-session CMCs [1] N. Dounskaia, C. J. Ketcham, B. C. Leis, and G. E. Stelmach, of Kinect system. The CMCs of shoulder flexion/extension “Disruptions in joint control during drawing arm movements (Flex/Ext), shoulder abduction/adduction (Abd/Add), shoul- in Parkinson’s disease,” Experimental Brain Research, vol. 164, der internal rotation/external rotation (IR/ER), and elbow no. 3, pp. 311–322, 2005. flexion/extension (Flex/Ext) are all presented. Four upper [2] N. V. Dounskaia, S. P. Swinnen, C. B. Walter, A. J. Spaepen, limb functional tasks are included: the hand to contralateral and S. M. P. Verschueren, “Hierarchical control of different shoulder task (HCS), hand to mouth task (HM), combing elbow-wrist coordination patterns,” Experimental Brain hair task (CH), and hand to back pocket task (HBP). Research, vol. 121, no. 3, pp. 239–254, 1998. [3] S. Reid, C. Elliott, J. Alderson, D. Lloyd, and B. Elliott, “Repeat- Appendix D: The Between-Session ICCs of the ability of upper limb kinematics for children with and without cerebral palsy,” Gait & Posture, vol. 32, no. 1, pp. 10–17, 2010. Angles at the PTA and the ROM for the [4] B. Galna, G. Barry, D. Jackson, D. Mhiripiri, P. Olivier, and Kinect System L. Rochester, “Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson’s disease,” The left subfigure shows the between-session ICCs of the Gait & Posture, vol. 39, no. 4, pp. 1062–1068, 2014. angles at the PTA via the Kinect system. The right subfigure [5] E. Jaspers, H. Feys, H. Bruyninckx et al., “The reliability of shows the between-session ICCs of the ROMs via the Kinect upper limb kinematics in children with hemiplegic cerebral system. The ICCs of the angles at the PTA and the ROM in palsy,” Gait & Posture, vol. 33, no. 4, pp. 568–575, 2011. shoulder flexion/extension (Flex/Ext), shoulder abductio- [6] J. L. McGinley, R. Baker, R. Wolfe, and M. E. Morris, “The n/adduction (Abd/Add), shoulder internal rotation/external reliability of three-dimensional kinematic gait measurements: rotation (IR/ER), and elbow flexion/extension (Flex/Ext) a systematic review,” Gait & Posture, vol. 29, no. 3, pp. 360– are all presented. Four upper limb functional tasks are 369, 2009. included: the hand to contralateral shoulder task (HCS), [7] M. G. Hoy and R. F. Zernicke, “The role of intersegmental hand to mouth task (HM), combing hair task (CH), and hand dynamics during rapid limb oscillations,” Journal of Biome- to back pocket task (HBP). chanics, vol. 19, no. 10, pp. 867–877, 1986. [8] C. A. Putnam, “Sequential motions of body segments in Data Availability striking and throwing skills: descriptions and explanations,” Journal of Biomechanics, vol. 26, pp. 125–135, 1993. The data that support the findings of this study are available [9] H. Zhou and H. Hu, “Human motion tracking for rehabilita- on request from the corresponding author, Ye Ma. The data tion—a survey,” Biomedical Signal Processing and Control, are not publicly available yet due to the underdevelopment vol. 3, no. 1, pp. 1–18, 2008. of the system and the ethics of the project. [10] L. Mündermann, S. Corazza, and T. P. Andriacchi, “The evo- lution of methods for the capture of human movement leading Conflicts of Interest to markerless motion capture for biomechanical applications,” Journal of NeuroEngineering and Rehabilitation, vol. 3, no. 1, The authors declare that they have no conflicts of interest. p. 6, 2006. [11] J. Shotton, T. Sharp, A. Kipman et al., “Real-time human pose Authors’ Contributions recognition in parts from single depth images,” Communica- tions of the ACM, vol. 56, no. 1, pp. 116–124, 2013. Ye Ma and Yanxin Zhang contributed to the conception and [12] B. F. Mentiplay, L. G. Perraton, K. J. Bower et al., “Gait design, as well as the drafting of the article. Laisi Cai is assessment using the Microsoft Xbox One Kinect: concurrent responsible for the data processing and drafting. Yanxin validity and inter-day reliability of spatiotemporal and Zhang and Shuping Xiong are responsible for the overall kinematic variables,” Journal of Biomechanics, vol. 48, no. 10, content and are the guarantors. pp. 2166–2170, 2015. [13] B. F. Mentiplay, R. A. Clark, A. Mullins, A. L. Bryant, S. Bartold, and K. Paterson, “Reliability and validity of the Acknowledgments Microsoft Kinect for evaluating static foot posture,” Journal This study was supported by the Zhejiang Provincial Natural of Foot and Ankle Research, vol. 6, no. 1, p. 14, 2013. Science Foundation of China (Grant Number: [14] R. A. Clark, Y. H. Pua, C. C. Oliveira et al., “Reliability and LQ19A020001), Ningbo Natural Science Foundation (Grant concurrent validity of the Microsoft Xbox One Kinect for 14 Applied Bionics and Biomechanics assessment of standing balance and postural control,” Gait & Posture, vol. 42, no. 2, pp. 210–213, 2015. [15] R. A. Clark, Y. H. Pua, A. L. Bryant, and M. A. Hunt, “Validity of the Microsoft Kinect for providing lateral trunk lean feedback during gait retraining,” Gait & Posture, vol. 38, no. 4, pp. 1064–1066, 2013. [16] T. Dutta, “Evaluation of the Kinect™ sensor for 3-D kinematic measurement in the workplace,” Applied Ergonomics, vol. 43, no. 4, pp. 645–649, 2012. [17] X. Chen, J. Siebourg-Polster, D. Wolf et al., “Feasibility of using Microsoft Kinect to assess upper limb movement in type III spinal muscular atrophy patients,” PLoS One, vol. 12, no. 1, article e0170472, 2017. [18] R. Moreira, A. Magalhães, and H. Oliveira, “A Kinect-based system for upper-body function assessment in breast cancer patients,” Journal of Imaging, vol. 1, no. 1, pp. 134–155, 2015. [19] X. Xu, M. Robertson, K. B. Chen, J. H. Lin, and R. W. McGorry, “Using the Microsoft Kinect™ to assess 3-D shoul- der kinematics during computer use,” Applied Ergonomics, vol. 65, pp. 418–423, 2017. [20] S. Erik and S. Marjorie, “Evaluation of an inexpensive depth camera for in-home gait assessment,” Journal of Ambient Intelligence and Smart Environments, vol. 3, no. 4, pp. 349– 361, 2011. [21] R. A. Clark, Y. H. Pua, K. Fortin et al., “Validity of the Micro- soft Kinect for assessment of postural control,” Gait & Posture, vol. 36, no. 3, pp. 372–377, 2012. [22] Y. Zhang, J. Unka, and G. Liu, “Contributions of joint rotations to ball release speed during cricket bowling: a three-dimensional kinematic analysis,” Journal of Sports Sciences, vol. 29, no. 12, pp. 1293–1300, 2011. [23] C. J. van Andel, N. Wolterbeek, C. A. M. Doorenbosch, D. J. (. H. E. J.). Veeger, and J. Harlaar, “Complete 3D kine- matics of upper extremity functional tasks,” Gait & Posture, vol. 27, no. 1, pp. 120–127, 2008. [24] Y. Zhang, D. G. Lloyd, A. C. Campbell, and J. A. Alderson, “Can the effect of soft tissue artifact be eliminated in upper-arm internal-external rotation?,” Journal of Applied Biomechanics, vol. 27, no. 3, pp. 258–265, 2011. [25] A. Cappozzo, F. Catani, U. Della Croce, and A. Leardini, “Posi- tion and orientation in space of bones during movement: anatomical frame definition and determination,” Clinical Biomechanics, vol. 10, no. 4, pp. 171–178, 1995. [26] M. P. Kadaba, H. K. Ramakrishnan, M. E. Wootten, J. Gainey, G. Gorton, and G. V. B. Cochran, “Repeatability of kinematic, kinetic, and electromyographic data in normal adult gait,” Journal of Orthopaedic Research, vol. 7, no. 6, pp. 849–860, [27] L. A. Schwarz, A. Mkhitaryan, D. Mateus, and N. Navab, “Human skeleton tracking from depth data using geodesic dis- tances and optical flow,” Image and Vision Computing, vol. 30, no. 3, pp. 217–226, 2012. [28] E. E. Stone and M. Skubic, “Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing,” in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6491–6494, Boston, MA, USA, 2011. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Applied Bionics and Biomechanics Hindawi Publishing Corporation

Validity and Reliability of Upper Limb Functional Assessment Using the Microsoft Kinect V2 Sensor

Loading next page...
 
/lp/hindawi-publishing-corporation/validity-and-reliability-of-upper-limb-functional-assessment-using-the-8Q0jo0rwOn
Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2019 Laisi Cai et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
1176-2322
eISSN
1754-2103
DOI
10.1155/2019/7175240
Publisher site
See Article on Publisher Site

Abstract

Hindawi Applied Bionics and Biomechanics Volume 2019, Article ID 7175240, 14 pages https://doi.org/10.1155/2019/7175240 Research Article Validity and Reliability of Upper Limb Functional Assessment Using the Microsoft Kinect V2 Sensor 1 1 2 3 Laisi Cai , Ye Ma , Shuping Xiong , and Yanxin Zhang The Research Academy of Grand Health, Faculty of Sport Science, Ningbo University, Ningbo, China Department of Industrial and Systems Engineering, College of Engineering, Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea Department of Exercise Science, Faculty of Science, The University of Auckland, New Zealand Correspondence should be addressed to Ye Ma; maye@nbu.edu.cn Received 12 August 2018; Revised 27 November 2018; Accepted 2 December 2018; Published 11 February 2019 Academic Editor: Andrea Cereatti Copyright © 2019 Laisi Cai et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Objective. To quantify the concurrent accuracy and the test-retest reliability of a Kinect V2-based upper limb functional assessment system. Approach. Ten healthy males performed a series of upper limb movements, which were measured concurrently with Kinect V2 and the Vicon motion capture system (gold standard). Each participant attended two testing sessions, seven days apart. Four tasks were performed including hand to contralateral shoulder, hand to mouth, combing hair, and hand to back pocket. Upper limb kinematics were calculated using our developed kinematic model and the UWA model for Kinect V2 and Vicon. The interdevice coefficient of multiple correlation (CMC) and the root mean squared error (RMSE) were used to evaluate the validity of the kinematic waveforms. Mean absolute bias and Pearson’s r correlation were used to evaluate the validity of the angles at the points of target achieved (PTA) and the range of motion (ROM). The intersession CMC and RMSE and the intraclass correlation coefficient (ICC) were used to assess the test-retest reliability of Kinect V2. Main Results. Both validity and reliability are found to be task-dependent and plane-dependent. Kinect V2 had good accuracy in measuring shoulder and elbow flexion/extension angular waveforms (CMC > 0 87), moderate accuracy of measuring shoulder adduction/abduction angular waveforms (CMC = 0 69-0.82), and poor accuracy of measuring shoulder internal/external angles (CMC < 0 6). We also found high test-retest reliability of Kinect V2 in most of the upper limb angular waveforms (CMC = 0 75-0.99), angles at the PTA (ICC = 0 65-0.91), and the ROM (ICC = 0 68-0.96). Significance. Kinect V2 has great potential as a low-cost, easy implemented device for assessing upper limb angular waveforms when performing functional tasks. The system is suitable for assessing relative within-person change in upper limb motions over time, such as disease progression or improvement due to intervention. 1. Introduction valid and reliable, these systems require relatively large spaces, are expensive, and require experienced technicians, Three-dimensional (3D) upper limb functional movements therefore limiting their use in the clinic, at home, in pub- such as reaching, pushing/pulling, and throwing have been lic, and so forth. In comparison, a markerless motion cap- ture system would be a possible alternative [10] for upper studied in many areas including motor control [1, 2], neuro- physiology [3], clinical assessment and rehabilitation [4–6], limb assessment. and ergonomics [7, 8]. Currently, quantitative measurements Microsoft Kinect is a low-cost markerless motion capture of upper limb functions are normally carried out using system, which estimates the 3D location of body joints based marker-based motion capture systems [9], in which the 3D on 2D images with depth information using machine learn- ing algorithms [11]. Kinect is feasible to assess gait motion data is obtained based on the passive or active markers attached on the anatomical landmarks of partici- temporal-spatial parameters and kinematics [12] and can pants. Although the marker-based systems in assessing objectively evaluate static foot posture with good accuracy upper limb kinematics [3, 5] have been confirmed to be and reliability [13]. Kinect has the potential to be used as 2 Applied Bionics and Biomechanics experimental protocol was approved by the Research an assessment tool for certain aspects of the balance perfor- mance [14]. Kinect can also assist gait rehabilitation training Academy of Grand Health’s Ethics Committee at Ningbo in clinics by providing the lateral trunk lean angle as a University. real-time feedback [15]. Some research investigated the use 2.2. Testing Procedure. This study used a concurrent validity, of Kinect in clinics [4] and confirmed that Kinect can test-retest reliability design. The study was conducted at the accurately measure gross spatial characteristics such as lower biomechanics laboratory of Ningbo University. Upper limb limb and trunk kinematics but cannot measure smaller kinematics were recorded concurrently by a Kinect V2 sys- movements such as hand clapping with the same accuracy. tem with a sampling frequency of 30 Hz and a 3D motion Researchers also investigate the use of Kinect in the work- capture system with eight infrared high-speed cameras place and found that Kinect can determine risks of musculo- (Vicon, Oxford Metrics Ltd., Oxford, UK) with a sampling skeletal injuries in the workplace [16]. frequency of 100 Hz. Prior to data collection, Kinect V2 was Attempts have been made at using Kinect in upper limb placed on a tripod at 0.8 meters above the floor. Subjects assessment [17–19]. Different scales have been used as stood at 2 meters from the camera according to the recom- outcome measures for disease progressions and medical mendation [16]. interventions, which are subjective and could vary depending Each participant attended two testing sessions, seven on different observers. Therefore, quantitative data attained days apart. For each session, reflective markers were attached by measuring kinematics is necessary for therapy practice. to the anatomical landmarks of the participants according to Chen et al. [17] developed a Kinect-based system to measure the UWA upper limb model [22]. First, a static trial is active upper limb movements as a complementary output performed during which each participant stands in the ana- measure of functional rating scales for spinal muscular atro- tomical position. Then, the elbow and wrist markers were phy. They observed no significant differences in the active removed during the following dynamic trials. Four range of motion (ROM) between the patients and the con- functional tasks were performed which represent a range of trols. They also found that the Kinect-based system is not functional activities [23]. sensitive enough to capture the minor differences or early-stage progression in the high-functioning patient group (i) Task one is hand to the contralateral shoulder, [17]. Moreira et al. [18] developed a Kinect-based system for which represents all activities near contralateral upper body function assessment in breast cancer patients. shoulder such as washing axilla or zip up a jacket. Based on the extracted upper limb kinematic features, the Subjects started with the arm in the anatomical posi- Kinect-based classification system can diagnose upper limb tion with their hand beside their body in a relaxed impairments for breast cancer patients. Kinect has also been neutral position and end up with the hand touching used to assess 3D shoulder kinematics during computer use the contralateral shoulder (see Figure 1(a)) to provide some insight on shoulder kinematics for improv- ing office ergonomics [19]. (ii) Task two is hand to mouth or drinking, which Establishing the accuracy and reliability inherent in the represents activities such as eating and reaching Microsoft Kinect system is required before using it for the face. It begins with the same starting point upper limb assessment. The accuracy of the Kinect system and ends when the hand reached the subject’s in measuring lower limb kinematics has been evaluated mouth (see Figure 1(b)) using marker-based measurements as the gold standard (iii) Task three is combing hair, which represents activi- [4, 12, 20, 21]. The reliability of the Kinect measurement ties such as reaching the (back of the) head and has also been studied in postural control assessment [21], washing hair. Subjects were instructed to move their gait analysis [12], and static foot posture evaluation [13]. hand to the back of their head (see Figure 1(c)) However, to the best of our knowledge, a thorough valid- ity and reliability study of the Kinect system on assessing (iv) Task four is hand to back pocket, which represents 3D upper limb kinematics when performing functional reaching the back and perineal care. It begins with tasks is lacking. The goal of this study was to quantify the same starting point and ends when the hand is the accuracy and test-retest reliability of a Kinect motion placed on the back pocket (see Figure 1(d)) capture system in assessing upper limb kinematics when performing functional tasks. A marker-based motion capture At least five trials were collected for each task. system (Vicon, Oxford Metrics Group, Oxford, UK) was used as the gold standard measurement. 2.3. Upper Limb Models for the Vicon System and the Kinect V2 System. The Vicon system tracked and stored the spatial trajectories of the reflective markers attached to the subjects. 2. Methodology The UWA upper limb marker set was employed in this study [24], which includes 18 markers (see Figure 2). Trunk, upper 2.1. Subjects. Ten healthy male university students (age: 24 6±2 8 years, height: 174 05 ± 4 4cm, mass: 67 2±4 2 arm, forearm, and hand segments were defined based on the anatomical landmark positions. The definition of the upper kg) with no upper limb injuries or medication use that would have influenced their upper limb functions volun- limb segment coordination system for the Vicon system is teered to participate. Participants were informed about the presented in Table 1. The calibrated anatomical systems tech- basic procedure of the experiment before the test. The nique [25] is used to establish the motion of anatomical Applied Bionics and Biomechanics 3 (a) (b) (c) (d) Figure 1: Four upper limb functional tasks performed in the study. 4 Applied Bionics and Biomechanics (a) (b) Figure 2: The upper limb models for the Vicon system and the Kinect V2 system ((a) the upper body marker set for the Vicon system; (b) the skeleton model of the Kinect V2 system). Applied Bionics and Biomechanics 5 Table 1: The upper arm and torso anatomical segment coordinate systems for the Vicon system. Name Definition Origin C7 X Unit vector defined by the Y-axis and the Z-axis to create a right-hand coordinate system Torso Y Unit vector going from T10 to C7 Z Unit vector perpendicular to the sagittal plane defined by T10, C7, and CLAV, pointing laterally Origin The elbow joint center, which was the midpoint between EL and EM X Unit vector perpendicular to the Y-axis and the Z-axis, pointing anteriorly Right upper arm Y Unit vector going from the elbow joint center to the shoulder joint center (the midpoint between ASH and PSH) Unit vector perpendicular to the plane formed by the Y-axis of the upper arm and the long-axis vector of the forearm. Note: C7: 7th cervical vertebra; CLAV: clavicular notch; EC: elbow center; EL: lateral epicondyle; EM: medial epicondyle; PSH: posterior shoulder; RS: radial styloid; STRN: sternum; T10: 10th thoracic vertebra; US: ulnar styloid. Table 2: The upper arm and torso anatomical segment coordinate systems for the Kinect V2 system. Name Definition Origin SpineShoulder X Unit vector perpendicular to two vectors (Y and the vector from ShoulderRight to ShoulderLeft) Torso Y Unit vector going from SpineMid to SpineShoulder Z Unit vector defined by the X-axis and the Y-axis to create a right-hand coordinate system Origin The elbow joint center (ElbowRight) X Unit vector perpendicular to the Y-axis and the Z-axis, pointing anteriorly Right upper arm Y Unit vector going from the elbow joint center to the shoulder joint center (ElbowRight to ShoulderRight) Unit vector perpendicular to the plane formed by the Y-axis of the upper arm and the long-axis vector of the forearm landmarks relative to the coordinate systems of the during post-processing. The joint angles from both sys- upper-arm cluster (PUA) or the forearm cluster (DUA). tems were firstly resampled to 300 Hz using the Matlab The motion of the upper limb landmarks could be recon- function “interp” and then synchronized using a structed from their constant relative positions to the cross-correlation-based shift synchronization technique. upper-arm technical coordinate system. The kinematic 2.4. Statistical Analysis. The concurrent validity of the Kinect model based on the UWA upper limb model was developed V2 system for assessing the upper limb functional movement using Vicon Bodybuilder software. The reference shoulder waveforms was carried out using the coefficient of multiple and elbow joint angles were calculated based on the mea- correlation (CMC) [26] and the root mean squared error sured position of the passive optical markers using the kine- matic model via Vicon Nexus software (Oxford Metrics (RMSE) between the waveforms calculated by the Kinect V2 and the Vicon-based system. Mean bias (Kinect-Vicon) Group, Oxford, UK). A Butterworth low-pass filter was used and Pearson’s r correlation between the two systems were with the cut-off frequency of 6 Hz for both Vicon and Kinect used to evaluate the concurrent validity of the Kinect-based V2 systems. For a more detailed description, see [22, 24]. system in assessing the joint angles at the point of target The 3D coordinates of the anatomical landmarks identi- fied from the skeletal model of the Kinect V2 system during achieved (PTA) and the range of motion (ROM). Paired Stu- dent t-tests were used to compare the results of the angles at functional tasks were also recorded. Local segment coordi- the PTA and the ROM with the significant level of 0.01. The nates including torso and upper arm were established, and concurrent validity was presented using data from session one. each of them was based on the global coordinate The CMC and RMSE between the waveforms from ses- (Table 2). Then, our customized upper limb kinematics for the Kinect V2 system calculated the three Euler angles sion one and session two measured by the Kinect V2 system were used to assess the relative and absolute test-retest reli- for shoulder rotations, which follows the flexion (+)/exten- ability. The reliability of the Kinect V2 system in assessing sion (-), adduction (+)/abduction (-), and internal (+)/exter- the joint angles at the PTA and the ROM from selected func- nal (-) rotation order (see Figure 2). The elbow flexion was tional tasks was also carried out using the intraclass correla- calculated by the position data from ShoulderRight, tion (ICC ElbowRight, and WristRight using the trigonometric ) coefficient with the absolute measure. The 3,k function. The kinematics model for Kinect V2 was developed ICC and Pearson’s r correlation as well as the descriptive sta- using Matlab 2017a. The angular waveforms between the tistics were performed using SPSS 22.0. The CMC and RMSE Kinect V2 sensor and the Vicon system were synchronized were analyzed using Matlab 2017a. 6 Applied Bionics and Biomechanics Shoulder F/E Shoulder Add/Abd 60 5 −5 −10 −15 −20 −10 −25 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 80 140 −20 0 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 3: Kinematics from hand to contralateral shoulder task. ° ° 3. Results and elbow flexion/extension angles of 7.16 and 10.43 and the highest RMSEs were identified in the combing hair ° ° 3.1. The Concurrent Validity of the Kinect V2 System for task with the RMSEs of 41.4 and 23.75 for shoulder flex- Upper Limb Functional Assessment. The kinematic wave- ion/extension elbow flexion/extension angles. For angular forms of the selected upper limb functional tasks in both ses- waveforms in the frontal plane at the shoulder joint, i.e., sions from the Kinect V2 system and the Vicon system are the shoulder adduction/abduction angle, the RMSEs presented in Figures 3-6 by means of the average segment between the Kinect V2 and Vicon systems are under six rotation angles. The validity of the Kinect V2 system in asses- degrees except for the combing hair task (RMSE = 12 31 ). sing upper limb angular waveforms, joint angles at the PTA, For shoulder angular waveforms in the transverse plane, and the ROM are presented in Tables 3 and 4. i.e., the shoulder internal/external rotation angle, the smallest High-level agreements (see Table 3) were observed for RMSE was found in the hand to mouth drinking task shoulder and elbow kinematics in the sagittal plane across (RMSE = 1 64 ) and the biggest RMSE was found in the all tasks with the CMC values of 0.81-0.94. Compared to combing hair task (RMSE = 29 38 ). the shoulder joint, elbow flexion/extension angles showed Mean (SD) values for joint angles at the PTA and the the best agreements between the two systems with the ROM estimated by the Kinect V2 and Vicon systems are CMC greater than 0.9 for all tasks except for the combing provided in Table 4. Excellent relative agreements hair task (CMC = 0 87). Shoulder adduction/abduction (r =0 73–0.97) were observed for all investigated angles angular waveforms showed moderate agreements between at PTA in all tasks except for moderate relative agreement the two systems with the CMC values of 0.69-0.82. The of the shoulder internal/external rotation in the hand to lowest CMCs between the two systems were found in the back pocket task and elbow flexion/extension in the hand transverse plane at the shoulder joint with the CMC values to contralateral shoulder task (r =0 46 and r =0 45, of lower than 0.6 except for the hand to contralateral shoul- respectively) and poor agreement of the shoulder and der task (CMC = 0 84). elbow flexion/extension angle in the combing hair task The RMSEs (see Table 3) between the angular waveforms (r = −0 20 and r =0 21, respectively). For the ROM, excellent relative agreements were observed for shoulder flexion/ex- from the two systems are also both plane-dependent and task-dependent. For angular waveforms in the sagittal plane tension and shoulder adduction/abduction angles in all tasks at shoulder and elbow joint, the lowest RMSEs were found (r =0 91-0.99) except for the combing hair task (r =0 20 and in the hand to back pocket task with the RMSEs of shoulder 0.65, respectively); excellent agreements were found for Degrees Degrees Degrees Degrees Applied Bionics and Biomechanics 7 Shoulder F/E Shoulder Add/Abd 100 −5 −10 −15 −20 −25 −20 −30 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 50 140 0 0 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 4: Kinematics from hand to mouth/drinking task. shoulder internal/external rotation angles only in the shoul- waveforms, the angles at the PTA, and the corresponding der to contralateral shoulder task and the combing hair task ROM is presented in Table 5. Good to excellent relative (r =0 74 and 0.77), respectively; poor to moderate agree- reliability (CMC = 0 75-0.99) was observed for all ments were found for elbow flexion/extension angles in all between-session angular waveforms across all selected tasks tasks (r <0 65) except for the hand to back pocket task, except for the shoulder adduction/abduction in task one which showed excellent agreement (r =0 97). (CMC = 0 7) and the shoulder internal/external rotation in There is a clear tendency that the Kinect V2 system the fourth task (CMC = 0 6). Angular waveforms in the overestimated shoulder flexion/extension angles and sagittal plane (CMC = 0 89-0.99) are more reliable than those underestimated elbow flexion/extension angles in all tasks. in the frontal plane (0.70-0.84) and transverse plane According to the mean absolute bias of the angles at the (CMC = 0 60-0.93). The RMSEs are under ten degrees except PTA between the Kinect V2 and Vicon systems (see for elbow F/E (RMSE = 8 3-11.03) in the first three tasks and Table 2, K-V), there were no significant bias only for shoulder IR/ER in task 3 (RMSE = 12 09). The worst abso- shoulder flexion/extension, shoulder internal/external rota- lute test-retest reliability was observed in the combing hair tion, and elbow flexion/extension angles in the hand to task, in which the RMSE of shoulder flexion/extension is back pocket task and shoulder adduction/abduction angle the biggest (18.91 ). in the hand to mouth/drinking task. For the absolute bias Results for the test-retest reliability of the Kinect V2 of the ROM, only elbow flexion/extension in the first and system in assessing the angles at the PTA and the ROM are last tasks, the shoulder adduction/abduction angle in the also presented in Table 5. There are no significant second task, and the shoulder internal/external rotation between-session differences in the Kinect V2 system for all in the second and fourth tasks showed no significant dif- investigated angles for all of the studied motions. All ferences. The greatest biases of the ROMs were found in between-session ICCs of the angle at the PTA from all the shoulder flexion/extension angles in the hand to motions are good to excellent for the Kinect V2 system mouth/drinking task and the combing hair task (ICC = 0 65-0.91) in all selected tasks except for shoulder (RMSE = 32 5-40.9 ). adduction/abduction in the first task (ICC = 0 59) and elbow flexion/extension in the combing hair task (ICC = 0 27). All 3.2. The Test-Retest Reliability of the Kinect V2 System for between-session ICCs of the ROM measurement from all Upper Limb Functional Assessment. The test-retest reliability studied motions are good to excellent for the Kinect V2 sys- of the Kinect V2 system in assessing the upper limb angular tem (ICC = 0 68-0.96) except for the motions in the combing Degrees Degrees Degrees Degrees 8 Applied Bionics and Biomechanics Shoulder F/E Shoulder Add/Abd 200 0 −10 −20 −30 −40 −50 −50 −60 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 60 160 −20 20 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 5: Kinematics from combing hair task. hair task (ICC = 0 35-0.70). The lowest reliability of the ROM Generally, the mean deviations between joint angles was shoulder flexion/extension in the combing hair task with assessed by the Kinect V2 system and the Vicon system are the mean ICC of 0.35. greater for tasks or planes with a larger range of motions. The shoulder and elbow angular waveforms in the sagittal plane measured by the Kinect V2 system highly agreed with 4. Discussion the reference angles, and the RMSEs of the Kinect V2 system in measuring sagittal plane kinematics are generally greater This study tested the concurrent validity and the test-retest reliability of upper limb functional assessment using Kinect or comparable than angles in the frontal plane and transverse V2. We found that both validity and reliability are plane in comparison with those from the Vicon system. Sim- ilarly, the accuracy of the Kinect V2 system in assessing task-dependent and plane-dependent. The Kinect V2 system had good accuracy in measuring angles at the PTA and the ROM are task-dependent and plane-dependent. The angles at the PTA showed good rela- shoulder and elbow flexion/extension angles, moderate accu- racy of measuring shoulder adduction/abduction angles, and tive agreement for almost all motions investigated except poor accuracy of measuring shoulder internal/external for motions in the combing hair task, which have a large range of motion. angles. We also found high test-retest reliability of the Kinect V2 system in most of the upper limb angular waveforms, Our finding agrees with the results of [19], which found that the Kinect V2 sensor had a better estimation on shoulder angles at the PTA, and the corresponding ROM. However, there are also some deviations both between the Kinect V2 flexion/extension, compared with shoulder adduction/abduc- system and the Vicon system (gold standard) and between tion and shoulder internal/external rotation during the com- puter operation tasks [19]. They also found that the shoulder two test sessions. flexion/extension angle had the lowest RMSE (under 15 ) and 4.1. Concurrent Validity of Kinect V2. The angular trajecto- the magnitude of error is proportional to the magnitude of ries from the Kinect V2 and Vicon have similar waveform the shoulder adduction/abduction angle [19]. The concur- patterns, especially for the flexion/extension angular wave- rent validity results of the upper limb motions are not as forms under all functional tasks (CMC = 0 good as those for lower limb and trunk motions. 81-0.94). Never- theless, the RMSEs between the kinematic patterns The measurement performance of the Kinect V2 system measured by the Kinect V2 system and Vicon system are is highly varied on tasks, joints, and planes of movement. not consistent with the angular waveform agreements. According to Clark et al.’s study [15, 21], the Kinect V2 Degrees Degrees Degrees Degrees Applied Bionics and Biomechanics 9 Shoulder F/E Shoulder Add/Abd 10 −5 0 −10 −10 −15 −20 −20 −30 −25 −40 −30 −50 −35 0 20 40 60 80 100 0 20 40 60 80 100 Shoulder IR/ER Elbow F/E 45 80 10 20 0 20 40 60 80 100 0 20 40 60 80 100 Kinect session 1 Kinect session 2 Vicon session 1 Vicon session 2 Figure 6: Kinematics from hand to back pocket task. system showed excellent concurrent validity with the Vicon and lower limb movements in comparison with those of system, with the Pearson r values >0.9 for the majority of upper limb tasks. measurements for trunk and lower limb motions during lat- eral reach, forward reach, and single leg balance. In contrast, 4.2. Test-Retest Reliability of Kinect V2. We found a high the Kinect V2 system showed lower concurrent validity in degree of reproducibility in almost all of the upper limb assessing upper limb motions in our study with the Pearson angular waveforms, angles at the PTA, and the ROM mea- r values of 0.7-0.99 for most upper limb motions at the sured by the Kinect V2 system during all functional tasks PTA and the ROM. across the two testing sessions. Our finding suggests that The abovementioned findings can be explained by the the Kinect V2 system can reliably assess 3D shoulder underlying real-time human pose recognition algorithm motions and elbow flexion/extension for individuals carry- and the nature of the single-depth camera. The Kinect system ing out such functional tasks. The repeatability of upper estimates the 3D location of body joints based on 2D images limb motions in the frontal and transverse planes was with depth information using machine learning algorithms. lower than that in the sagittal plane. However, there were The final set of confidence-weighted 3D joint proposals is still some between-session deviations in the starting point based on a global optimization algorithm using training data, and ending point of the upper limb. Greater standardiza- which represents postures in an entertainment environment tion of both the starting and ending points for the tasks [11]. Therefore, Kinect should be evaluated carefully before would be required to improve the repeatability of the employing it as a research tool. If a joint is shaded by other starting and ending points of this type of motion. The body parts, it is difficult for the Kinect system to define the test-retest reliability of the trunk motion is high corresponding anatomical landmarks of the Kinect skeletal (ICC > 0 73) during lateral reach and forward reach in model, which directly results in inaccurate joint angle predic- Clark et al.’s research [15], which is similar to results of tion. Anatomical landmarks of the trunk and lower limb the upper limb motions in our study. identified from the skeletal model of the Kinect system may The combing hair task is not an ideal upper limb func- be more accurate because the segments of the trunk and tional task when using the Kinect V2 system as the outcome lower limb are usually not shaded by other body parts during measure tool. During the combing hair task, the Kinect V2 the tasks in reach, balance test, or walking. This may be the system had the worst accuracy and reliability in assessing main reason for the higher validity in the assessment of trunk the angular waveforms, the angles at the PTA, and the Degrees Degrees Degrees Degrees 10 Applied Bionics and Biomechanics Table 3: Validity of the angular waveforms for the selected functional tasks. The mean (SD) coefficient of multiple correlation (CMC) and the root mean squared error (RMSE) between the Kinect V2-based markerless system and the Vicon-based system of the upper limb waveforms are presented. Hand to contralateral shoulder Hand to mouth/drinking Combing hair Hand to back pocket Shoulder Shoulder Shoulder Elbow Shoulder Shoulder Shoulder Elbow Shoulder Shoulder Shoulder Elbow Shoulder Shoulder Shoulder Elbow F/E Add/Abd IR/ER F/E F/E Add/Abd IR/ER F/E F/E Add/Abd IR/ER F/E F/E Add/Abd IR/ER F/E 0.88 0.82 0.84 0.94 0.82 0.58 0.92 0.81 0.78 0.54 0.87 0.93 0.69 0.55 0.90 CMC 0.80 (0.2) (0.09) (0.11) (0.07) (0.05) (0.09) (0.20) (0.04) (0.11) (0.13) (0.17) (0.05) (0.05) (0.27) (0.27) (0.13) 11.04 5.62 13.40 15.48 25.61 1.64 18.16 41.40 12.31 29.38 23.75 7.16 5.76 12.36 10.43 RMSE 6.11 (2.85) (5.16) (1.37) (3.00) (6.04) (7.56) (5.20) (4.95) (9.49) (2.65) (6.76) (4.32) (2.87) (2.76) (3.74) (6.21) Note: shoulder F/E, shoulder Add/Abd, shoulder IR/ER, and elbow F/E represent joint angles of shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation, and elbow flexion, respectively. Applied Bionics and Biomechanics 11 Table 4: Concurrent validity of the joint angles at the point of target achieved (PTA) and the ROM of the selected functional tasks. The mean (SD) peak joint angles and the ROM are presented with the Pearson correlation (r) and the discrepancy (Kinect-Vicon) between the Kinect V2-based markerless system and the Vicon-based system is presented. PTA ROM Segment rotations Kinect Vicon P Bias (K-V) r Kinect Vicon P Bias (K-V) r (K,V) (K,V) Shoulder to contralateral shoulder Shoulder F/E 52.1 (10.3) 38.1 (6.7) <0.01 14.0 0.74 51.5 (10.0) 43.7 (6.9) <0.01 7.8 0.98 Shoulder Add/Abd 5.5 (7.8) −0.4 (6.3) <0.01 5.9 0.89 26.2 (5.8) 14.3 (3.9) <0.01 7.9 0.99 Shoulder IR/ER 63.7 (6.8) 68.5 (5.7) <0.01 −4.8 0.89 38.7 (11.7) 66.7 (14.9) <0.01 −28 0.74 Elbow F/E 110.9 (8.7) 125.2 (3.5) <0.01 −14.3 0.45 91.8 (8.3) 98.2 (6.2) 0.06 −6.4 0.41 Hand to mouth/drinking Shoulder F/E 95.6 (19.4) 57.3 (10.2) <0.01 38.3 0.88 94.8 (19.4) 62.3 (12) <0.01 32.5 0.91 Shoulder Add/Abd −8.5 (4.5) −8.2 (2.5) 0.67 −0.3 0.85 20.4 (9.6) 14.4 (13.4) <0.01 6.0 0.95 Shoulder IR/ER 47.5 (11.4) 32.3 (7.3) <0.01 15.2 0.93 21.7 (9) 29.2 (10.5) 0.07 −7.5 0.33 Elbow F/E 107.1 (9.3) 129.6 (5.7) <0.01 −22.5 0.80 89.1 (10.1) 102.3 (6.6) <0.01 −13.2 0.65 Combing hair Shoulder F/E 143.5 (14.1) 102.6 (20.4) <0.01 40.9 −0.20 140 (22.7) 106 (15.6) <0.01 34.0 0.20 Shoulder Add/Abd −53.7 (7.6) −54.7 (7.2) 0.40 1.0 0.89 42.2 (6.4) 46.9 (5.6) 0.02 −4.7 0.65 Shoulder IR/ER 49.4 (11.2) 25.3 (10.2) <0.01 24.1 0.72 55.6 (16.9) 42.8 (10.9) <0.01 12.8 0.77 Elbow F/E 113.6 (3.4) 139.6 (14.1) <0.01 −26 0.21 91.1 (7.7) 110.9 (13.7) <0.01 −19.8 0.05 Hand to back pocket Shoulder F/E −41.8 (6.2) −42.9 (6.1) 0.47 1.1 0.73 43.4 (6.7) 38.7 (6.1) <0.01 4.7 0.91 Shoulder Add/Abd −33 (10.3) −25.5 (7.6) <0.01 −7.5 0.97 22.9 (9.6) 16.7 (7.6) <0.01 6.2 0.96 Shoulder IR/ER 42.3 (9.6) 37.7 (5.7) 0.02 4.6 0.46 18.6 (7.9) 23.3 (5) 0.16 −4.7 −0.10 Elbow F/E 73.7 (21.3) 81.2 (13.2) 0.03 −7.5 0.94 51.8 (25) 52.1 (18.1) 0.91 −0.3 0.97 Note: max and min are the maximum and minimum joint angles, respectively. ROM: corresponding range of motion. Shoulder F/E, shoulder Add/Abd, shoulder IR/ER, and elbow F/E represent joint angles of shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation and elbow flexion, respectively; K-V is the bias between the Kinect V2 and Vicon systems. corresponding ROM. The greatest deviations of the angular (and expensive) motion capture laboratories and would waveforms for the Kinect V2 system were identified for all allow for in situ data collection. This markerless system investigated motions in the combing hair task. The Kinect would have a significant impact in the fields of clinical practice. Unlike clinical scales, the markerless motion analy- V2 system also cannot measure shoulder and elbow flexio- n/extension angles at the PTA and the ROMs of all investi- sis technique is objective, which helps to quantify massive gated motions in the combing hair task with low test-retest kinematic parameters and turn data into a knowledge-based reliability (ICC = 0 27-0.65). During the task, the absolute data warehouse in a standard way. The whole process makes test-retest reliability of shoulder F/E is the worst with the it possible for identifying and explaining relationships between-session RMSE of 18.91 between different motion patterns or different population . Thus, it is better not to use the same functional task like combing hair in the assess- groups through data mining. ment of upper limb functions. 4.4. Limitation and Future Work. The experiment had a lim- 4.3. Clinical Implications. Our results show that the Kinect ited sample size (ten male university students). The system V2 system can reliably measure upper limb motions. Kinect has not yet been used in other populations or other environ- V2 has good relative agreements of angular waveforms and ments such as upper limb disordered patients in clinical set- can accurately measure a few of the shoulder and elbow joint tings, elderly people at home or in the clinical environment, angles during the functional tasks. Although the measure- and people working in the workplace. The finding from this ment of upper limb kinematics may be not as accurate as study may have differed if the assessment was undertaken the Vicon system, Kinect V2 is useable to track relative in a clinical population. Upper limb motions of healthy peo- within-person changes in movements over time (such as ple often have low variability. The low variability increases the worsening of movement symptoms with disease progres- heterogeneity and would potentially lead to higher reliability sion or improvement due to intervention). estimates and stronger correlations between Kinect V2 and In current clinical practice, clinical scales have been the reference. In the future, tests with larger sample size, including patients, will be conducted. widely used to assess upper limb functions, which are subjective and labour-intensive. The development of a Further work is also concerned to improve the accuracy video tracking system based on low-cost markerless cam- of the Kinect system in measuring upper limb joint angles. eras would free clinicians from the necessity of dedicated There have been several recent studies in estimating body 12 Applied Bionics and Biomechanics Table 5: Reliability of the angular waveforms, the angles at the point of target achieved (PTA), and the range of motion (ROM) for the selected functional tasks. The mean (SD) coefficient of multiple correlation (CMC) and the root mean squared error (RMSE) of the upper limb waveforms calculated by the Kinect V2 system between two sessions are presented. The intraclass correlation (ICC) of the angles at the PTA and the ROM of the selected functional tasks are also presented. ICC Segment rotation CMC RMSE PTA ROM Shoulder F/E 0.93 (0.08) 7.89 (3.88) 0.80 0.88 Shoulder Add/Abd 0.70 (0.28) 9.19 (5.29) 0.59 0.68 Hand to contralateral shoulder Shoulder IR/ER 0.93 (0.06) 5.93 (1.87) 0.88 0.96 Elbow F/E 0.97 (0.3) 11.03 (4.50) 0.86 0.80 Shoulder F/E 0.99 (0.09) 9.90 (8.67) 0.80 0.81 Shoulder Add/Abd 0.75 (0.28) 5.03 (2.98) 0.78 0.95 Hand to mouth/drinking Shoulder IR/ER 0.75 (0.28) 7.25 (5.64) 0.76 0.80 Elbow F/E 0.96 (0.05) 10.70 (5.77) 0.72 0.84 Shoulder F/E 0.97 (0.03) 18.91 (8.11) 0.65 0.35 Shoulder Add/Abd 0.84 (0.12) 7.70 (3.69) 0.65 0.43 Combing hair Shoulder IR/ER 0.90 (0.06) 12.09 (4.52) 0.83 0.67 Elbow F/E 0.95 (0.05) 10.68 (6.24) 0.27 0.70 Shoulder F/E 0.96 (0.03) 5.14 (2.51) 0.73 0.84 Shoulder Add/Abd 0.82 (0.18) 4.71 (2.50) 0.88 0.92 Hand to back pocket Shoulder IR/ER 0.60 (0.23) 7.10 (2.57) 0.91 0.82 Elbow F/E 0.89 (0.13) 8.38 (5.18) 0.85 0.88 Note: shoulder F/E, shoulder Add/Abd, shoulder IR/ER, and elbow F/E represent joint angles of shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation, and elbow flexion. landmarks and motions using the Kinect sensor [27, 28]. It is Appendix likely that using these techniques would possibly produce Appendix A: The Pearson Correlation more accurate joint kinematics. Attempts have also been Coefficient (r) of the Angles at the PTA via the made using multiple Kinect sensors concurrently to improve Vicon and Kinect Systems the accuracy of tracking movement [28]. Calibration algo- rithms are another solution to improve the joint angle pre- The scatterplot includes PTAs of shoulder flexion/extension diction accuracy of the Kinect system. Xu and colleagues (Flex/Ext), shoulder abduction/adduction (Abd/Add), shoul- used the linear regression algorithm to calibrate the shoul- der internal rotation/external rotation (IR/ER), and elbow der adduction/abduction angle, and the prediction accuracy flexion/extension (Flex/Ext) during the four upper limb func- was significantly improved [19]. Using the state-of-the-art tional tasks. Task 1 is the hand to contralateral shoulder task. machine learning algorithms, it is possible to improve the Task 2 is the hand to mouth task. Task 3 is the combing hair current joint kinematic measurement accuracy as high task. Task 4 is the hand to back pocket task. The x-axis of agreements were found for angles in the sagittal and frontal each subfigure is each individual’s PTA via the Kinect system. planes (CMC > 0 78) for most investigated motions. The y-axis of each subfigure is each individual’s correspond- ing PTA via the Vicon system. For each subfigure, the linear 5. Conclusion correlation between the x-axis value and the y-axis value are presented by a line and the corresponding Pearson correla- The Kinect V2-based upper limb functional assessment tion coefficient. system developed in this research has good test-retest reliability in assessing the upper limb angular waveforms Appendix B: The Pearson Correlation and the angles at the point of target achieved except for the combing hair task. The Kinect V2-based system can Coefficient (r) of the Range of Motion (ROM) accurately assess shoulder flexion/extension, elbow flexio- via the Vicon and Kinect Systems n/extension, and shoulder adduction/abduction in some upper limb functional tasks. The Kinect V2 sensor has The scatterplot includes ROMs of shoulder flexion/exten- great potential as a low-cost, easily implemented device sion (Flex/Ext), shoulder abduction/adduction (Abd/Add), for assessing upper limb angular waveforms when per- shoulder internal rotation/external rotation (IR/ER), and forming functional tasks. Our system is suitable for asses- elbow flexion/extension (Flex/Ext) during the four upper sing relative within-person change in upper limb motions limb functional tasks. Task 1 is the hand to contralateral over time, such as disease progression or improvement shoulder task. Task 2 is the hand to mouth task. Task 3 due to intervention. is the combing hair task. Task 4 is the hand to back Applied Bionics and Biomechanics 13 pocket task. The x-axis of each subfigure is each individ- Number: 2018A610193), Fujian Provincial Key Laboratory ual’s ROM via the Kinect system. The y-axis of each sub- of Rehabilitation and Fujian Rehabilitation Industry figure is each individual’s corresponding ROM via the Research Institute fund (Grant Number: 2015Y2001-65), Vicon system. and the National Research Foundation of Korea (Grant Number: 2017R1C1B2006811). The study was also sup- ported by the Scientific Research Foundation of Graduate Appendix C: Scatterplots of Both the School and K.C. Wong Magna Fund in Ningbo University. Between-Device CMCs (at Session One) and the Between-Session CMCs for the Kinect System References The left subfigure shows the between-device CMCs (session one). The right subfigure shows the between-session CMCs [1] N. Dounskaia, C. J. Ketcham, B. C. Leis, and G. E. Stelmach, of Kinect system. The CMCs of shoulder flexion/extension “Disruptions in joint control during drawing arm movements (Flex/Ext), shoulder abduction/adduction (Abd/Add), shoul- in Parkinson’s disease,” Experimental Brain Research, vol. 164, der internal rotation/external rotation (IR/ER), and elbow no. 3, pp. 311–322, 2005. flexion/extension (Flex/Ext) are all presented. Four upper [2] N. V. Dounskaia, S. P. Swinnen, C. B. Walter, A. J. Spaepen, limb functional tasks are included: the hand to contralateral and S. M. P. Verschueren, “Hierarchical control of different shoulder task (HCS), hand to mouth task (HM), combing elbow-wrist coordination patterns,” Experimental Brain hair task (CH), and hand to back pocket task (HBP). Research, vol. 121, no. 3, pp. 239–254, 1998. [3] S. Reid, C. Elliott, J. Alderson, D. Lloyd, and B. Elliott, “Repeat- Appendix D: The Between-Session ICCs of the ability of upper limb kinematics for children with and without cerebral palsy,” Gait & Posture, vol. 32, no. 1, pp. 10–17, 2010. Angles at the PTA and the ROM for the [4] B. Galna, G. Barry, D. Jackson, D. Mhiripiri, P. Olivier, and Kinect System L. Rochester, “Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson’s disease,” The left subfigure shows the between-session ICCs of the Gait & Posture, vol. 39, no. 4, pp. 1062–1068, 2014. angles at the PTA via the Kinect system. The right subfigure [5] E. Jaspers, H. Feys, H. Bruyninckx et al., “The reliability of shows the between-session ICCs of the ROMs via the Kinect upper limb kinematics in children with hemiplegic cerebral system. The ICCs of the angles at the PTA and the ROM in palsy,” Gait & Posture, vol. 33, no. 4, pp. 568–575, 2011. shoulder flexion/extension (Flex/Ext), shoulder abductio- [6] J. L. McGinley, R. Baker, R. Wolfe, and M. E. Morris, “The n/adduction (Abd/Add), shoulder internal rotation/external reliability of three-dimensional kinematic gait measurements: rotation (IR/ER), and elbow flexion/extension (Flex/Ext) a systematic review,” Gait & Posture, vol. 29, no. 3, pp. 360– are all presented. Four upper limb functional tasks are 369, 2009. included: the hand to contralateral shoulder task (HCS), [7] M. G. Hoy and R. F. Zernicke, “The role of intersegmental hand to mouth task (HM), combing hair task (CH), and hand dynamics during rapid limb oscillations,” Journal of Biome- to back pocket task (HBP). chanics, vol. 19, no. 10, pp. 867–877, 1986. [8] C. A. Putnam, “Sequential motions of body segments in Data Availability striking and throwing skills: descriptions and explanations,” Journal of Biomechanics, vol. 26, pp. 125–135, 1993. The data that support the findings of this study are available [9] H. Zhou and H. Hu, “Human motion tracking for rehabilita- on request from the corresponding author, Ye Ma. The data tion—a survey,” Biomedical Signal Processing and Control, are not publicly available yet due to the underdevelopment vol. 3, no. 1, pp. 1–18, 2008. of the system and the ethics of the project. [10] L. Mündermann, S. Corazza, and T. P. Andriacchi, “The evo- lution of methods for the capture of human movement leading Conflicts of Interest to markerless motion capture for biomechanical applications,” Journal of NeuroEngineering and Rehabilitation, vol. 3, no. 1, The authors declare that they have no conflicts of interest. p. 6, 2006. [11] J. Shotton, T. Sharp, A. Kipman et al., “Real-time human pose Authors’ Contributions recognition in parts from single depth images,” Communica- tions of the ACM, vol. 56, no. 1, pp. 116–124, 2013. Ye Ma and Yanxin Zhang contributed to the conception and [12] B. F. Mentiplay, L. G. Perraton, K. J. Bower et al., “Gait design, as well as the drafting of the article. Laisi Cai is assessment using the Microsoft Xbox One Kinect: concurrent responsible for the data processing and drafting. Yanxin validity and inter-day reliability of spatiotemporal and Zhang and Shuping Xiong are responsible for the overall kinematic variables,” Journal of Biomechanics, vol. 48, no. 10, content and are the guarantors. pp. 2166–2170, 2015. [13] B. F. Mentiplay, R. A. Clark, A. Mullins, A. L. Bryant, S. Bartold, and K. Paterson, “Reliability and validity of the Acknowledgments Microsoft Kinect for evaluating static foot posture,” Journal This study was supported by the Zhejiang Provincial Natural of Foot and Ankle Research, vol. 6, no. 1, p. 14, 2013. Science Foundation of China (Grant Number: [14] R. A. Clark, Y. H. Pua, C. C. Oliveira et al., “Reliability and LQ19A020001), Ningbo Natural Science Foundation (Grant concurrent validity of the Microsoft Xbox One Kinect for 14 Applied Bionics and Biomechanics assessment of standing balance and postural control,” Gait & Posture, vol. 42, no. 2, pp. 210–213, 2015. [15] R. A. Clark, Y. H. Pua, A. L. Bryant, and M. A. Hunt, “Validity of the Microsoft Kinect for providing lateral trunk lean feedback during gait retraining,” Gait & Posture, vol. 38, no. 4, pp. 1064–1066, 2013. [16] T. Dutta, “Evaluation of the Kinect™ sensor for 3-D kinematic measurement in the workplace,” Applied Ergonomics, vol. 43, no. 4, pp. 645–649, 2012. [17] X. Chen, J. Siebourg-Polster, D. Wolf et al., “Feasibility of using Microsoft Kinect to assess upper limb movement in type III spinal muscular atrophy patients,” PLoS One, vol. 12, no. 1, article e0170472, 2017. [18] R. Moreira, A. Magalhães, and H. Oliveira, “A Kinect-based system for upper-body function assessment in breast cancer patients,” Journal of Imaging, vol. 1, no. 1, pp. 134–155, 2015. [19] X. Xu, M. Robertson, K. B. Chen, J. H. Lin, and R. W. McGorry, “Using the Microsoft Kinect™ to assess 3-D shoul- der kinematics during computer use,” Applied Ergonomics, vol. 65, pp. 418–423, 2017. [20] S. Erik and S. Marjorie, “Evaluation of an inexpensive depth camera for in-home gait assessment,” Journal of Ambient Intelligence and Smart Environments, vol. 3, no. 4, pp. 349– 361, 2011. [21] R. A. Clark, Y. H. Pua, K. Fortin et al., “Validity of the Micro- soft Kinect for assessment of postural control,” Gait & Posture, vol. 36, no. 3, pp. 372–377, 2012. [22] Y. Zhang, J. Unka, and G. Liu, “Contributions of joint rotations to ball release speed during cricket bowling: a three-dimensional kinematic analysis,” Journal of Sports Sciences, vol. 29, no. 12, pp. 1293–1300, 2011. [23] C. J. van Andel, N. Wolterbeek, C. A. M. Doorenbosch, D. J. (. H. E. J.). Veeger, and J. Harlaar, “Complete 3D kine- matics of upper extremity functional tasks,” Gait & Posture, vol. 27, no. 1, pp. 120–127, 2008. [24] Y. Zhang, D. G. Lloyd, A. C. Campbell, and J. A. Alderson, “Can the effect of soft tissue artifact be eliminated in upper-arm internal-external rotation?,” Journal of Applied Biomechanics, vol. 27, no. 3, pp. 258–265, 2011. [25] A. Cappozzo, F. Catani, U. Della Croce, and A. Leardini, “Posi- tion and orientation in space of bones during movement: anatomical frame definition and determination,” Clinical Biomechanics, vol. 10, no. 4, pp. 171–178, 1995. [26] M. P. Kadaba, H. K. Ramakrishnan, M. E. Wootten, J. Gainey, G. Gorton, and G. V. B. Cochran, “Repeatability of kinematic, kinetic, and electromyographic data in normal adult gait,” Journal of Orthopaedic Research, vol. 7, no. 6, pp. 849–860, [27] L. A. Schwarz, A. Mkhitaryan, D. Mateus, and N. Navab, “Human skeleton tracking from depth data using geodesic dis- tances and optical flow,” Image and Vision Computing, vol. 30, no. 3, pp. 217–226, 2012. [28] E. E. Stone and M. Skubic, “Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing,” in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6491–6494, Boston, MA, USA, 2011. International Journal of Advances in Rotating Machinery Multimedia Journal of The Scientific Journal of Engineering World Journal Sensors Hindawi Hindawi Publishing Corporation Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Hindawi Hindawi Hindawi Volume 2018 Volume 2018 Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com www.hindawi.com www.hindawi.com Volume 2018 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

Journal

Applied Bionics and BiomechanicsHindawi Publishing Corporation

Published: Feb 11, 2019

References