GBDT-Based Fall Detection with Comprehensive Data from Posture Sensor and Human Skeleton Extraction
GBDT-Based Fall Detection with Comprehensive Data from Posture Sensor and Human Skeleton Extraction
Cai, Wen-Yu;Guo, Jia-Hao;Zhang, Mei-Yan;Ruan, Zhi-Xiang;Zheng, Xue-Chen;Lv, Shuai-Shuai
2020-06-25 00:00:00
Hindawi Journal of Healthcare Engineering Volume 2020, Article ID 8887340, 15 pages https://doi.org/10.1155/2020/8887340 Research Article GBDT-Based Fall Detection with Comprehensive Data from Posture Sensor and Human Skeleton Extraction 1,2 1 3 1 1 Wen-Yu Cai , Jia-Hao Guo, Mei-Yan Zhang , Zhi-Xiang Ruan, Xue-Chen Zheng, and Shuai-Shuai Lv College of Electronics and Information, Hangzhou Dianzi University, Hangzhou 310018, China Zhejiang Provincial Key Lab of Equipment Electronics, Hangzhou Dianzi University, Hangzhou 310018, China School of Electrical Engineering, Zhejiang University of Water Resources and Electric Power, Hangzhou 310018, China Correspondence should be addressed to Mei-Yan Zhang; meiyan19831109@163.com Received 11 March 2020; Revised 26 May 2020; Accepted 2 June 2020; Published 25 June 2020 Academic Editor: Fabrizio Taffoni Copyright © 2020 Wen-Yu Cai et al. .is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Since fall is happening with increasing frequency, it has been a major public health problem in an aging society. .ere are considerable demands to distinguish fall down events of seniors with the characteristics of accurate detection and real-time alarm. However, some daily activities are erroneously signaled as falls and there are too many false alarms in actual application. In order to resolve this problem, this paper designs and implements a comprehensive fall detection framework on the basis of inertial posture sensors and surveillance cameras. In the proposed system framework, data sources representing behavior characteristics to indicate potential fall are derived from wearable triaxial accelerometers and monitoring videos of surveillance cameras. Moreover, the NB-IoT based communication mode is adopted to transmit wearable sensory data to the Internet for subsequent analysis. Furthermore, a Gradient Boosting Decision Tree (GBDT) classifier-based fall detection algorithm (GBDT-FD in short) with comprehensive data fusion of posture sensor and human video skeleton is proposed to improve detection accuracy. Experimental results verify the good performance of the proposed GBDT-FD algorithm compared to six kinds of existing fall detection algorithms, including SVM-based fall detection, NN-based fall detection, etc. Finally, we implement the proposed integrated systems including wearable posture sensors and monitoring software on the Cloud Server. systems on activity monitoring of humans based on wearable 1.Introduction sensors and issues to be addressed to tackle the challenges. As An increased aging population in the world is forcing rapid far as we know, there are three main categories of fall detection rises in healthcare requirements [1]. Everyone knows that older technologies: fall detection using wearable sensors [4, 5], fall people have poor balance ability and slow response ability. Falls detection using environmental sensors [6, 7], and video-based are a major cause of injury for the elderly and a huge obstacle in fall detection [8, 9]. Although there are some other methods the independent living of the seniors. Once the elderly falls such as radar-based fall detection [10], they are more com- plicated compared to the above three methods. down alone at home without help, the injured elderly may be lying on the ground for several hours or even days. More Wearable sensor based fall detection methods mainly seriously, it is very likely to extended injury and be life- depend on sensory data gathered from wearable acceler- threatening if he did not get treatment timely. .erefore, timely ometer and gyroscope. It is generally agreed that the use of fall incident detection and medical assistance for the elderly are wearable sensors has played a quite important role in intuitively important. However, due to different application monitoring the physiological parameters of a person to scenarios and various body activities, satisfactory and reliable minimize any malfunctioning happening in the body. In fall detection results are too hard to guarantee [2]. recent years, the advancement of sensing technologies, Some related fall detection algorithms have been proposed embedded systems, wireless communication technologies, in the literature. Broadley et al. [3] review the latest reported nanotechnologies, and miniaturization makes it possible to 2 Journal of Healthcare Engineering Kinect camera and a device consisting of an accelerometer develop smart wearable sensors to monitor activities of human beings continuously. Nag et al. [11] provide a review and a gyroscope, and then a fuzzy inference system is used to separate fall from daily activities. Hondori et al. [20] present on some of the significant research work done on wearable flexible sensors. Chen et al. [12] propose a novel intelligent a detection system that helps monitor various dining ac- fall detection method, named as ESAEs-OCCCH, which uses tivities of poststroke patients using a Kinect camera and an acceleration data from a wrist-worn smart watch. ESAEs- accelerometer. Nizam et al. [21] propose a novel approach OCCCH is first adopted for unsupervised feature extraction that uses a depth sensor and employs a unique procedure to overcome the disadvantages of artificial feature extraction. that identifies the fall risk levels to adapt the algorithm for Yacchirema et al. [13] propose an innovative IoT (Internet of different people with their physical strength to withstand falls. Bogdan et al. [22] present a low-cost system for reliable .ing) based online system for detecting falls of the aged. Sensory readings are processed and analyzed using a deci- fall detection with a very low false alarm ratio on the basis of accelerometric data and depth maps. .e single drawback of sion tree based Big Data model running on a Smart IoT Gateway [14]. Although these wearable sensors have high the above methods is that Kinect camera is not cheap since a considerable computational power is needed to execute sensitivity and good real-time characteristics, higher de- tection accuracy cannot typically be achieved due to the image processing algorithms. He et al. [23] propose a interference from diverse activities of hand or wrist. Hence, method to integrate the information of video images, sound, it is easy to cause misjudgment and missed detection of fall infrared, pulse, and other information into the elderly care actions relying only on wearable sensory data. system. However, it is not very realistic to detect fall acci- Secondly, a few scholars apply environmental sensors to dents with so many sensors. detect falls. Li et al. [15] propose a phase transform (SRP- To overcome these shortcomings, we use an ordinary camera and accelerometer as a data source in this paper so as PHAT) method which can locate the original source of a certain voice. In terms of sound classification phase, they to improve the practicability of the detection system. Fur- thermore, a novel data fusion based fall detection online apply the Mel-Frequency Cepstral Coefficients (MFCC) features with a Nearest Neighbor (NN) approach to improve system and one GBDT based detection algorithm is provided in detail. Data fusion of human activity features obtained by fall detection performance. However, expensive acoustic devices have high requirements on the environment, and it is posture sensor and surveillance cameras plays a significant not feasible to promote accurate detection with certain role in the recognition of abnormal activities. In addition, ambient noise. Adnan et al. [16] adopt acoustic Local Ter- the proposed platform uses NB-IoT communication and nary Patterns (acoustic-LTPs) to detect fall events by ana- Ethernet to transmit manifold data to the Cloud Server for lyzing environmental noise. Acoustic features are extracted further analysis. .e platform is able to effectively monitor from the separated source components using the proposed the daily life of the elderly. When an unexpected fall incident occurs, the proposed system will send an alarm signal to acoustic-LTPs scheme. Subsequently, fall events would be identified with SVM based classifier. However, it will cause inform the family relatives or other related guardians. In conclusion, the proposed system would meet the require- noise during the audio signal acquisition, which could lead to low accuracy and frequent false alarms. ments of high sensitivity and precision. As a result, necessary .irdly, vision-based fall detection typically uses image assistance could be provided in times with high coverage processing techniques to construct a human body model to communication technology, so it is suitable for application detect fall. In general, video-based fall detection systems in the elderly care system. have shown some potential and reliability in detecting falls .e rest of the paper is organized as follows. Section 2 in public places. Due to the popularity of video surveillance, describes the whole online fall detection platform framework vision-based fall detection methods have already become and comprehensive data source. In Section 3, GBDT based one research hotspot. .e boundary extraction method is fall detection algorithm using comprehensive data from an accelerometer based posture sensor and human skeleton used to obtain the aspect ratio of the human body and then to judge falls. Sase and Bhandari [17] used contour-based extraction is presented in detail. Comparable experimental results and actual operating interface are described in template matching to distinguish human and nonhuman and then judged human fall according to the distance be- Section 4. Finally, a conclusion is drawn in Section 5. tween the top and center of the external rectangle of the human body. Shen et al. [18] propose a fall detection method 2.System Framework and Comprehensive using the Deeper Cut model to exact human key points, and Data Source it is implemented using Raspberry Pi platform. Vision-based fall detection can use relatively cheap cameras to quantify Our complete framework of the fall detection system for and judge various activities; nevertheless, it requires com- seniors is illustrated in Figure 1. Each user is equipped with a plex handling methods to construct a human body model kind of self-made MEMS (Micro Electro Mechanical Sys- and it is unsuitable for real-time detection mode. tems) based wearable sensor with hardware block diagram in In addition, some results have suggested that a single Figure 2, which uses triaxial acceleration and angular ve- detection model from individual DataSet could easily lead to locity sensor to capture the body posture. Besides, NB-IoT false detection. Recently, there are already a few methods on (Narrow Band Internet of .ings) [24] communication the basis of wearable sensors and surveillance cameras to mode is used to transmit sensory data to the Cloud Server. classify body activities. Kepski and Kwolek [19] apply a With the development of Internet of .ings technology, the Journal of Healthcare Engineering 3 User_1 Camera 4G LTE Guardian _1 User_2 Internet Cloud server Guardian _M User_3 Camera Camera MEMS sensor User_N Figure 1: Framework of the proposed platform. one ensemble learning method named Gradient Boost Alarm button Decision Tree (GBDT) [27] is applied for self-learning with MDS, so as to classify fall and other normal activities in a robust way. 3-axis Low-power NB-IoT Since human activity frequency generally does not ex- MEMS MCU communication sensor processor module ceed 20 Hz [28], the acceleration acquisition frequency is set to 30 Hz so as to process data more accurately, and video acquisition frequency is adjusted to 30 fps (frames per 2 AAA battery second) after software processing. .e entire self-made DataSet comprises six kinds of human activities involving Figure 2: .e block diagram of self-made posture sensor. fall, walk, sit, squat, lie down, and jump. Due to the diversity and complexity of fall accidents, it is hard to identify the way and direction of fall events. To health care field is also affected deeply. As we know, NB-IoT overcome this shortcoming, we use the sum vector of is an emerging technology with many good features such as triaxial acceleration value to measure the human activity. wide coverage, multiple connections, low speed, low power Let a , a , a denote acceleration value in three dimen- x y z consumption, etc. In our IoT based health monitoring sions, respectively; A represent the actual value of 3-axis system, various detection devices are connected together for triaxial acceleration, which can be calculated with the data exchange, so as to deliver warnings to medical staff or following equation: guardians in time when the elderly fall. Moreover, it is also ���������� 2 2 2 supposed that each user is covered by at least one surveil- A � a + a + a . (1) 3−axis x y z lance camera and so we can monitor and record each user’s activity. Today, almost all surveillance cameras have the .e measured sensory data are illustrated in Figure 4 to ability to transmit video sequences to the Internet through compare the acceleration variation curve of falls and that of Ethernet or wireless networks. .erefore, the Cloud Server other normal activities, including squat, lie down, jump, could obtain both attitude data and video data and store it at walk, and sit down. the local database for further analysis. .e fall detection .e other fall characteristics are key skeleton coordinates algorithm is running on the Cloud Server with high per- of the human body as shown in Figure 5. .e rectangular formance. Once fall events are detected, the Cloud Server coordinate system is established in Figure 5, and the hori- will send an alarm signal to the specific guardian through 4G zontal and vertical coordinates of N � 18 key points would be LTE (Long Term Evolution) communication technology. As obtained, respectively. (X , Y ) (i � 1, 2, . . ., N) denotes sk−i sk−i a result, each user can get instant help and timely treatment the respective coordinates of each key point. Hence, the in case of any abnormality with our proposed framework. collection of key skeleton coordinates is as follows: .e principle of the fall detection process is demon- strated in Figure 3. In this system framework, real-time X , Y �