1Department of Engineering, Faculty of Engineering and Computer Science, Qaiwan International University, Sulaymaniyah, Iraq; 2Department of Computer Engineering and Information Technology, Faculty of Engineering, Razi University, Kermanshah, Iran; 3Department of Computer Science, College of Science, University of Halabja, Halabja, Iraq; 4Department of Computer Network, Technical College of Informatics, Sulaimani Polytechnic University, Sulaymaniyah, Iraq; 5Department of Information Technology, Lebanese French University, Erbil, Kurdistan Region, Iraq
Fall event has become a critical health problem among elderly people. We propose a fall detection system that analyzes real-time streaming data from the Internet of Things (IoT) to detect irregular patterns related to fall. We train a deep neural network model using accelerometer data from an online physical activity monitoring dataset named, MobiAct. An IBM Cloud-based IoT data processing framework is used to manage streaming data. About 96.71% of accuracy is achieved in assessing the performance of the proposed model.
Index Terms: Fall Detection, Internet of Things, Artificial Neural Networks, Machine Learning
These days, we can see a quick increment in people using wearable devices for more different purposes and reasons in their lives. According to the previous studies, the employment of connected wearable detector devices is expected to extend from 325 million in 2016 to 1105 million in 2022 . In this way, where an unmeasurable number of wearable devices is connected together will produce a huge amount of information consistently, at that time, we face or interaction with a critical test of storing, handling, and processing that information’s to produce usable information and to make a keen world. Distinguishing regularities and abnormalities or inconsistencies in gushing information from that measure of usable information, we previously produced subsequent to putting away and preparing it can give us an encounter and can possibly give experiences, and is useful in human services, money, security, web-based social networking, and numerous applications , .
Over the most recent few years, totally different types of methodologies and techniques have been proposed for identifying regularities and irregularities pattern in streaming data for fall detection, which may be a wearable tool based, ambiance sensor-based, and vision-based . Above all, wearable devices typically take some different benefit of embedded sensors to observe the movement and placement of the body, such as measuring system, accelerometer, magnetometer, and gyroscope , . As well as the value of wearable tool based any methodologies or techniques are very low, also because the installation and operation are not complex, and the task is not difficult for the elderly , . If from now the physician could not specifically monitoring and identify falls, then there is no chance to save or protect you in future to preventing an accident. For such reasons, fall identification and anticipation have turned into a significant issue then must be find and propose a good way to solve in the method of detecting regularities and irregularities in streaming data for fall detection , .
Fall in the human daily life is one of the main health risks and dangerous, mostly for the older community in our today’s society, because of the rose growing in mortality, morbidity, incapacity, disability, and frailty . Nearly for fall initiate injuries as collected and represented, over 80% of all damage related clinic confirmations among peoples for more than 65 years , . According to the reasons as we mentioned before, falls affect a huge number of the elderly all through the world. For instance, falls some of the aged cost the National Health Service more than £4.6 million every day as indicated by a report by the Centre for Social Justice UK .
The notable studies and researches in this filed are detecting fall or anomaly in real streaming data - and outliers , . We will probably investigate the circumstance of unpredictable human development identification, for example, fall, by utilizing continuous sensor information. At that point, we map that issue as sporadic example location issue, thinking about a fall as a capricious action with respect to standard human action and attempt to perceive the exceptional fall situations from ordinary development or human activities, for example, walking, sitting, lying (LYI), sleeping, standing, and every other movement such as playing, cooking, and running.
In general, Artificial Neural Networks (ANN) have systematically achieved higher results in the detection falls from physical activity observation knowledge. Ozdemir and Barshan have used a pair of 2520 trials to make a huge amount of dataset . Their fall detection system achieved 95% accuracy by employing a multi-layer perceptron (MLP) for binary classification between activities of daily living (ADL) and fall. Kerdegari et al. recorded 1000 movement acceleration data collection using a waist-worn measuring system and obtained 91.6% accuracy for binary classification for ADL against fall using MLP . Nukala et al. collected knowledge from 322 tests achieved 98.7% accuracy with MLP victimization scaled conjugate graduate learning .
Theodoridis et al.  developed two long short-term memory (LSTM) models, one with easy measuring system data collection associated another with accelerometer data revolved at an angle, using a published dataset referred to as UR Fall Detection. The LSTM model with rotation obtained the most effective results with 98.57% accuracy.
The work by class Musci et al. used one public dataset (SisFall) dataset implementing Recurrent Neural Network with underlying LSTM blocks for developing online fall detection system . They achieved 97.16% accuracy victimization associate best window breadth of w = 256. Ajerla et al.  slightly changed the preprocessing done by Vavoulas et al. on other public dataset (MobiAct) dataset and achieved quite 90% accuracies on most of their experiments victimization MLP and LSTM , . They also obtained 99% accuracy victimization LSTM in two of their experiments. Table 1 shows the compares performance of the ANN techniques.
TABLE 1 Comparison of the ANN techniques
Nowadays, there are a lot of wearable sensors devices exist which may observe falls automatically and send a notification to the caregivers, machine services, or ambulance offerings. However, most of them are expensive or need a subscription of month-to-month service. A large number of studies and research has been done on fall detection the usage of sensors such as accelerometers and gyroscopes, because of low cost and incorporated into a large number of cell phones accessible todays in the world such as smartphones and tablets. Rather than fall identification, there is a dire need for prediction and prevention systems , .
In this paper, we propose a system for detecting abnormal pattern of falling behavior in real-time streaming data in internet of things (IoT). The data were obtained from wearable sensor systems used for human health and activity tracking and control. In our research, we focus on how to train the proposed system to recognize and distinguish irregular activity pattern related to specific kinds of fall according to three different annotated published datasets: MobiAct, using Vavoulas et al. in 2016 , SisFall, using Sucerquia et al. in 2017 , and UMAFALL by way of Eduardo et al. in 2018  for implementing our framework from streaming data for fall detection, we used an ANN model for fall recognition giving (96.71%) accuracy. At that point, we integrate our system for that huge amount of data recordings from streaming sensor (online) into free IBM Streams to fabricate an IBM Cloud-based IoT information preparing structure.
We proposed a profound learning model to recognize fall and in this manner, another structure for utilizing the model with steaming information from wearable sensor frameworks is proposed. Simulated fall detection is used due to the scarcity of actual fall data, as fall is a dangerous event. We consider a framework that includes ANN as one of the evaluating procedure.
ANN has ceaselessly achieved better outcomes in fall recognition from physical checking spilling information. The architecture comprises three essential layers (I) gathering stream for information ingestion, (ii) streaming extract, transform, and load engine for real-time query processing, and (iii) Online Analytical Processing backend for taking care of long-running inquiries. The means of the proposed framework are as per the following data collection
All open datasets or we can say public data sets for wearable fall detection frameworks are distinguished that the selection criteria for any datasets should give priority to the experimental subjects, the quantity of tests and sorts of ADL and falls included in the study .
Based on that, we have chosen three different datasets, MobiAct, SisFall, and UMAFall, to evaluate our framework dependent on the number of subjects and number of exercises secured by each dataset.
It is possible to collect data from one or several sources. For instance, in an IoT workload, data are ingested simultaneously from thousands of separate data sources. Each source submits fresh tuples to a stream (likely to send them through a socket), which a data collector mechanism then receives. This data collector mainly act as a queue for messaging .
As we explained in Table 2, MobiAct dataset contains labeled data for four differing kinds of falls and nine completely different ADLs were collected from 70 subjects (Male/Female) and quite a pair of 2500 trials buy using so many different kind of a smartphone. The activities are depicted employing a time stamp, raw measuring system values, raw rotating mechanism values, and orientation information. Table 3 shows the activities covered within the MobiAct dataset. In the MobiAct dataset is that it does not embrace fall data from any old people . SisFall remedies this situation by mixing with information from 15 old people aged between 60 and 75 years. Still, in these two datasets real fall data it does not available or do not included.
TABLE 2 Activites covered in datasets
TABLE 3 Datasets used for training
SisFall contains annotated data for 15 differing kinds of falls and 19 different types of ADLs were collected from 38 subjects and quite 4500 trials using a custom measuring instrument containing two completely different models of 3D accelerometers and a rotating mechanism positioned on a buckle. Furthermore, UMAFall datasets include 12 different types of fall and 15 types of ADLs were collected from 2600 trials by using so many kinds of smartphone and remote accelerators. Like first dataset as I mention above these activities are also depicted employing a time stamp, raw measuring system values, raw rotating mechanism values, and orientation information
Using the same procedure given by Aziz et al. , the raw data are segmented into 200 blocks and then the feature sets for each block are generated.
List of features A: A sum of 54 features were created in include setA . For every pivot (x, y, and z) of the quickening, 21 highlights were determined from the mean, middle, Standard Deviation (STD), slant, kurtosis, least, and most extreme. Utilizing the outright estimations of every hub (x, y, and z) of the quickening, another 21 highlights were determined from the mean, middle, STD, slant, kurtosis, least, and most extreme. Incline, another component, was determined utilizing Eq. 1, one for the given tomahawks esteems (x, y, and z) and another for the supreme qualities (|x|, |y|, |z|).
Four different features were determined utilizing mean, STD, slant, and kurtosis of the tilt edge (TAi) between the gravitational vector and the Y-axis utilizing Eq. 2.
Where i denotes the sequence of the sample.
Utilizing the magnitude of the acceleration vector, six features were resolved from the mean, STD, least, greatest, contrast among most extreme and least, and zero-intersection rate. The size was determined utilizing Eq. 3.
Where i denotes the sequence of samples.
For each of the three axes (x, y, and z), the average absolute difference was calculated . Furthermore, the average resultant acceleration of all the three axes was generated using Eq. 4.
Where i denotes the sequence of samples.
Combined Feature Set. The feature sets A, B, and C are merged to generate the dataset with the combined features. We had 7670 samples in the dataset. After feature extraction, each sample had one of six classification values and 58 extracted features. Class values include four kinds of falls and two kinds of non-falls as defined in the MobiAct dataset. The two non-fall classes denote Standing (STD) and LYI positions.
The extracted features were normalized using Matlab R2017b with the min-max scaling formula given in Eq. 5.
The obtained dataset after normalization was very unbalanced containing 5830 non-fall and 1840 fall data. We made data balanced for our dataset using Matlab R2017b as follows: The various fall data categories were merged into a single fall data classification while the two non-fall data categories were merged into a single non-fall data classification. Then, the fall data were oversampled to create 2000 samples and the non-fall data were under-sampled to create 2000 samples to create a combined balanced dataset containing 4000 samples.
Selection of discriminable features affects the performance of the classification in terms of accuracy and complexity. Among different feature selection methods, the relief-F method is used for feature selection, the relief-F method in python is used with number of neighbor set to two and number of features to be kept set to four. The relief-F compared to the relief method is more robust and can deal with incomplete and noisy data .
We created a simple predictive Deep Neural Network (DNN) model consisting of five layers including three hidden layers to detect fall based on data stream sensor.
The structure of DNN is displayed in Fig. 1. The model was constructed and designed using offline datasets and then was implemented in our streaming data processing system.
Fig. 1. Deep learning model utilized for the proposed framework.
We are building a streaming data processing system with IBM tools, and then we tested and validated our prediction model with a static fall data.
After finalizing, testing and approving our proposed model using specific static data for fall detection, we build up a streaming data processing and handling structure with IBM devices. The framework structure is appeared in Fig. 2.
Fig. 2. The framework of the data stream processing system to detect irregular patterns for internet of things streaming data.
• Data source: Sensor information is recovered from the cell phone or any wearable devices must be fixed in the patient body.
• Data ingestion system: To receiving the sensor data from a smartphone or a wearable sensor and go about as the Message Queuing Telemetry Transport  message broker, we used IBM Watson IoT Platform. And also Apache Kafka  is used here as an open source alternative, which uses depends on its own protocol.
• Data stream processing: To start with we utilized Jupyter Notebook with IBM Cloud  as appeared in Fig. 3, for offline information preparing to run the element extraction and to execute the AI model utilizing the sample data. The Python code utilized in Jupyter Notebook and the middle information records have been posted on Github. We had an arrangement to utilize the IBM Streaming Analytics , an organization for IBM Streams on IBM cloud, to continually screen the sensor information from mobile phone or wearable sensors and send an admonition to the watching application to alert human services suppliers about the crisis care required by patients in the unexpectedly occasion or some other of a fall.
• Data store: We have the plan to use IBM Cloud  as the NoSQL database to store the sensor information from the Watson IoT platform and classification of that streaming data which are obtained from Watson Studio.
• Data sink: The framework will have different output channels:
• Knowledge base.
• IBM cloud.
1. Implementation: The tests were run on a framework with the accompanying least equipment and programming prerequisites:
• Platform: Keras with tensor flow backend
• CPU: Intel Core i5-2467M, 1.6 GHz
• RAM: 10 GB
• Hard disk: 128 GB
• OS: Windows 10 64 bit .
To actualize the framework, we utilized a free IBM cloud account and the relevant services. We included the following accompanying assets.
The machine learning model for the complete feature set was first tried on MobiAct as explained in Section 2.1. This resulted in our model in an accuracy of (96.71%) for binary classification, fall and ADL (non-fall) using Eq 6. The model was lept running for 200 epochs.
ACC = (True positive+True negative)/ Number of all samples (6)
The normalized and un-normalized confusion matrices were implemented for classifying fall and non-fall (ADL) cases out of 7670 samples are appeared in Fig. 4. This shows the created model performs incredible and can be utilized to develop the proposed framework.
Fig. 4. Normalized confusion matrix for the trained model.
In this paper, we addressed the problem of irregularity pattern detection from online streaming data. Here, we especially focused on detecting fall as an irregular human activity which is common amongst elderly people. We implemented a DNN model that can classify fall from non-fall activities based on the dataset MobiAct and two more datasets with an accuracy of 96.71%. For future, we plan to compare the proposed system with other systems that use open source streaming data analytics tools to evaluate the functionality and performance of the IBM tools used in our framework.
. H. Tankovska. “Statistic“, 2020. Available from:https://www.statista.com/statistics/487291/global-connected-wearable-devices.
. Mahfuz. “Detecting Detecting Irregular Patterns in IoT Streaming Data for Fall Detection Irregular Patterns in IoT Streaming Data for Fall Detection“. 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), 2018.
. E. Bahmani, M. Jamshidi and A. Shaltooki. “Breast cancer prediction using a hybrid data mining model“.
. J. Liu. “Development and evaluation of a prior-to-impact fall event detection algorithm“.
. P. Pierleoni. “A high reliability wearable device for elderly fall detection“.
. R. Freitas. “
. X. Zhuang. “
. Y. Li. “Efﬁcient source separation algorithms for acoustic fall detection using a microsoft kinect“.
. R. Hamedanizad, E. Bahmani, M. Jamshidi and A. M. Darwesh. “Employing data mining techniques for predicting opioid withdrawal in applicants of health centers“.
. A. Shaltooki and M. Jamshidi. “The use of data mining techniques in predicting the noise emitted by the trailing edge of aerodynamic objects“.
. A. S. Cook. “Falls in the Medicare population:incidence, associated factors, and impact on health care“.
. P. Kannus. “Fall-induced injuries and deaths among older adults“.
. Y. Cheng. “
. O. Ojetola, E. I. Gaura and J. Brusey. “
. S. M. S. Forbes. “Fall prediction using behavioural modelling from sensor data insmarthomes“.
. S. K. Gharghan. “Accurate fall detection and localization for elderly people based on neural network and energy-efﬁcient wireless sensor network“.
. D. Yacchirema. “Fall detection system for elderly people using IoT and big data“.
. C. C. H. Hsu. “
. Ahmad S. “
. S. C. Tan. “
. M. Gupta. “Outlier detection for temporal data:A survey“.
. S. Sadik and L. Gruenwald. “
. T. Özdemir and B. Barshan. “Detecting falls with wearable sensors using machine learning techniques“.
. H. Kerdegari, K. Samsudin, A. R. Ramli and S. Mokaram. “Development of wearable human fall detection system using multilayer perceptron neural network“.
. T. Nukala, N. Shibuya, A. Rodriguez and J. Tsay. “An efficient and robust fall detection system using wireless gait analysis sensor with artificial neural network (ANN) and support vector machine (SVM) algorithms“.
. T. Theodoridis, V. Solachidis, N. Vretos and P. Daras. “Human fall detection from acceleration measurements using a recurrent neural network“. In:
. M. Musci, D. De Martini, N. Blago, T. Facchinetti and M. Piastra. “
. G. Vavoulas, C. Chatzaki, T. Malliotakis, M. Pediaditis and M. Tsiknakis. “
. D. Dharmitha, M. Sazia and Z. Farhana. “Fall Detection from Physical Activity Monitoring Data“. Presented at the International SIGKDD workshop on Big Data, Streams and Heterogeneous Source Mining:Algorithms, Systems, Programming Models and Applications (BigMine), London, 2018.
. D. Ajerla, S. Mahfuz and F. H. Zulkernine. “A Real-Time Patient Monitoring Framework for Fall Detection“. Queen's University, Kingston, Ontario, Canada, 2018.
. D. Ajerla. “
. T. Xu, Y. Zhou and J. Zhu. “New advances and challenges of fall detection systems:A survey“.
. H. Chen, (Eds.). “
. E. C. J. Santoyo-Ramón. “
. S. Ahmad, A. Lavin, S. Purdy and Z. Agha. “Unsupervised real-time anomaly detection for streaming data“.
. M. Ahmed, A. N. Mahmood and M. R. Islam. “A survey of anomaly detection techniques in financial domain“.
. S. Ahmad and S. Purdy. “
. M. Mohammadi, A. Al-Fuqaha, S. Sorour, and M. Guizani, “Deep Learning for IoT Big Data and Streaming Analytics:A Survey,“
. J. Meehan, C. Aslantas, S. Zdonik, N. Tatbul and J. Du. “Data
. S. Sadik and L. Gruenwald. “Research issues in outlier detection for data streams“.
. M. Gupta, J. Gao, C. C. Aggarwal and J. Han. “Outlier detection for temporal data:A survey“.
. E. Casilari, J. A. Santoyo-Ramón and J. M. Cano-García. “Analysis of public datasets for wearable fall detection systems“.
. J. Meehan. “
. M. P. G. Vavoulas. “The mobiFall dataset:Fall detection and classification with a smartphone“.
. O. Aziz, M. Musngi, E. J. Park, G. Mori and S. N. Robinovitch. “A comparison of accuracy of fall detection algorithms (threshold-based vs. machine learning) using waist-mounted tri-axial accelerometer signals from a comprehensive set of falls and non-fall trials“.
. G. Vavoulas1. “
. J. R. Kwapisz, G. M. Weiss and S. A. Moore. “Activity recognition using cell phone accelerometers“.
. Y. Z. Z. Wang. “
. MQTT. Available from:https://www.mqtt.org/getting-started. [Last accessed on 2020 Jan 01].
. A. Kafka. Available from:https://www.kafka.apache.org/intro. [Last accessed on 2020 Jan 05].
. IBM. “IBM Watson Studio IBM Watson and Cloud Platform Learning Center“, 2020. Available from:https://www.developer.ibm.com/technologies/data-science.
. IBM. “
. IBM. “IBM Cloudant-IBM Watson and Cloud Platform Learning Center“, 2020. Available from:https://www.developer.ibm.com/components/cloud-ibm.