Multi-Sensor Fall Detection for Smartphones

both Before detailing of proposed application, we provide an analysis literature about fall detection. are several studies discussing fall Fall ABSTRACT Elderly people fall detection is a very relevant and widely studied problem. It is associated with the need to detect fall events using pervasive and largely accepted technologies and the need to suddenly manage the first aid phase after a fall occurs. This research work aims to provide a non-invasive fall detection system, widely accepted by users, limiting data exchange only to the first aid request and help phase in order to reduce privacy issues. This paper proposes a fall detection methodology and a mobile application built on sensors available on smartphones, such as accelerometer, gyroscope, proximity sensor, microphone and GPS. The proposed fall detection method combines data from the five sensors using a threshold-based algorithm; the data processing allows fall detection and enables a first aid request sending a message to rescuers.


Introduction
Among unintentional injuries, the World Health Organization [1] identified falls as part of the leading causes of death and, in less dangerous cases, provoking immobility and premature nursing home placement. About 37.3 million falls occur every year that are serious enough to require medical support and around 646,000 people die from fall injuries. This situation is even more dangerous among elderly people. Frequently, elderly people are alone when a fall occurs, either at home or in other situations. Fall detection may allow decreasing the time until first aid is performed, reducing risks such as hypothermia, bronchopneumonia, dehydration, pressure sores and post-fall anxiety syndromes, which could compromise any Activity of Daily Living (ADL). Experimental data about using sensors to detect falls are encouraging. Sensors gather data to effectively detect and confirm falls, and, with a network connection, they can easily activate first aid and rescue procedures.
Fall detection is being widely researched, and numerous fall detection systems have been created, but no specific method has been standardized or accepted globally [2]. This paper proposes a fall detection methodology and an application based on the use of five sensors: accelerometer, gyroscope, proximity sensor, microphone and GPS embedded in smartphones. This research targets all elderly people, not only those with limited mobility, but also those who want to use the application for prevention purposes, even if they are not currently concerned with fall events and have an independent lifestyle that includes outdoor activities.
For this reason, the proposed application does not include any technology installed in a specific location (i.e., ambient sensors) or wearable sensors. Indeed, elderly people, who does not have mobility problems, usually carry out activities alone and outdoor (for example going out to the grocery store). An elderly person rejects typically the idea of living by always wearing special devices that monitor her or his activities 24 hours a day; the smartphone is the type of device that could be mostly accepted because it is already part of people's daily lives. Back to the example, an elderly person could pick up her or his phone and go out. On the way to the grocery store, he could face an emergency, such as tripping over a manhole, and could have difficulty asking for help because she or he could be alone or unconscious. Furthermore, we had the intention of pursuing a limited data exchange between the device and the network both to make the system more efficient and to avoid privacy problems. Before detailing the description of the proposed application, we provide an analysis of the literature about fall detection. There are several studies discussing fall detection, as mentioned before. Fall detection systems can be grouped into the following categories: ambient-based, wearable and devices sensorbased, and hybrid that use a combination of both sensors (ambient and wearable) [3][4][5].
or resistive) or a sensor pad/mat [6]. D Litvak, et al. [7] developed a fall detection system based on pattern recognition techniques using an accelerometer and microphones placed on the floor. The system could distinguish between a human or an object falling and Activity of Daily Living (ADL). Popescu, et al. [8] used a set of microphones consisting of a linear array of electret condenser acoustic sensors installed vertically to capture sound height information. Yun Li, et al. [9] proposed an acoustic system based on a circular microphone array and a data processing software that recognize the sound source and classify the sound as Fall or non-fall. Popescu, et al. [10] studied a system composed of four sets of two Passive InfraRed (PIR) sensors on vertical support. It takes advantage of measuring infrared light wavelengths: the human body has a specific measure of reflection, and a fall can be recognized by its specific pattern using a hidden Markov model. Ariani, et al. [11] used a wireless sensor network that emulates a Dual-Technology Sensor (DTS) motion detector, a hybrid sensor that combines PIR and microwave motion detectors and pressure mats.
Thome, et al. [12] used a camera approach, providing a theoretical analysis to define the optimal camera placement for detecting people falling in unspecified situations; they proved that two cameras are sufficient in practice. Fern'dez-Caballero, et al. [13] based their project on actuators and sensors (like an accelerometer and IR -InfraRed sensors) combined with image processing. Yu, et al. [14] proposed a detection system based on posture recognition using a single camera combined with a directed acyclic graph support vector machine for posture classification. Liang Liu, et al. [15] [16] presented eCAALYX, which is a wearable sensor system composed of health and mobility sensors; they use a one-axis accelerometer to detect a fall. Niazmand, et al. [17] presented a garment, a pullover "with integrated acceleration sensors, evaluation and control electronics." The system measures the acceleration of the torso and the arms. Sim, et al. [18] proposed an alternative position for the accelerometer: shoes. They calculate the change in acceleration values to recognize falls and ADLs.
Narasimhan, et al. [19] developed an adhesive sensor system composed of a tri-axial accelerometer, a microcontroller and a Bluetooth Low Energy transceiver, worn on a subject's torso. Park, et al. [20,21] introduced a system composed of a 3-axis accelerometer, a 2-axis gyroscope, digital compasses or clinometer.
Tolkiehn, et al. [22] proposed a waist-worn sensor consisting of a 3-axis accelerometer and a barometric pressure sensor to detect a fall and its direction. Many research works used sensors already embedded in smartphones. He, et al. [23] proposed a solution with a waist-mounted smartphone that used the built-in accelerometer to detect falls. Lee, et al. [24] proposed a system using a tri-axial accelerometer embedded in a smartphone to distinguish fall events from ADLs considering the four directions of the falls (lateral, left and right, frontal, backwards). The measurements of variables returned by the sensors (usually one or two) indicate whether a fall event has happened or not. The results described in the cited works are obtained through experiments set in a laboratory. Ambient sensors are set up within a specific place/ambient, while the wearable/smartphone sensors collect and return data related to a specific person [25]. Another difference lies in the fact that ambient sensors are generally used indoors and are mainly for people that live in a controlled environment. In contrast, wearable/smartphone sensors are applicable both indoors and outdoors and can be used by people that have a more independent life. Wearable sensors have better results than sensors embedded into smartphones. However, people perceive skin and wearable sensors as foreign objects.
Moreover, Kosse, et al. [26] observed that acceptance for fall detection sensor systems is not universal, using an analysis of some studies reporting positive cases. Other studies report somewhat mixed results in terms of incorporating sensor systems in care [27].
These devices are widely used and can be easily applied to detect falls. They can be used out of any clinical or wired environment, in any situation that is part of daily life and without any added equipment. Wearable sensors, in particular accelerometers (the most frequently used), applied to the skin usually perform better than sensors embedded in smartphones. Still, they are not practical and accepted because they are not integrated into the human body. The latest generation of smartphones is generally equipped with a variety of sensors such as an accelerometer, a gyroscope, a microphone, GPS and a proximity sensor. Hawley-Hague et al. suggest that acceptance of using sensors should be improved by making the usefulness of their adoption evident in terms of "…potential benefits such as independence, increased safety, convenience, increased social opportunities" [29]. The study underlines the users' need to maintain control over the use of technology, especially in regard to ambient sensors, which can cause more serious privacy issues. With these considerations and reflecting on the fact that most seniors lead an active life, this paper (as already explained before) focuses mainly on using smartphone sensors to collect data that indoor sensors cannot. Moreover, this research targets all elderly people who want to use the application for preventive purposes, even if they are not currently concerned with fall events. This paper starts with the idea of holistically using data collected by five sensors to detect and evaluate fall events and consequently activating first aid and rescue procedures. Indeed, it proposes the combined use of three sensors (accelerometer, gyroscope and microphone) to detect falls indoors and outdoors, the use of the GPS to determine the position of the subject in outdoor/indoor applications and the use of the proximity sensor to determine the state of the fallen person in order to organize first aid activities.
There are two approaches to elaborate data gathered from the wearable/smartphone sensors for fall detection: threshold-based systems and machine learning-based systems [30,31]. Thresholdbased algorithms use a predefined fixed value to decide on a specific event; they require less computational power and are also less complex than other sophisticated algorithms [32]. The performance and accuracy of this kind of approach largely depend on threshold value tuning. Machine learning-based technologies produce results which can be considered similar to outcomes provided by threshold-based approaches [33,34]. Nevertheless, threshold-based algorithms have been popular because of their low computational overhead and complexity, while the machine learning approaches require high consumption of resources.
This paper describes the model and the software application for fall detection as it is defined and built. It takes advantage of five different sensors embedded in smartphones, removing the need to install ad-hoc sensors on the human body. This will result in wider acceptance of the application, as the target users already use the smartphone. In particular, the five smartphone sensors' data provide information that is useful for the detection and evaluation of probable fall events and, consequently, for activating the first aid The previously cited disadvantages of using smartphones rather than wearable sensors have been mitigated by combining the inputs from five sensors embedded in a smartphone (fewer wearable sensors are generally used). The paper has the following organization. The next section will give the materials and methods used for detecting falls. It will describe the rescue process, including the system (based on PLAKSS -PLAtform for Knowledge and Services Sharing) components that is activated in case of falls.
Section III describes the tests and their results; section IV provides a discussion by mapping experimental data and the three sensors' thresholds (accelerometer, gyroscope and microphone). Finally, section V concludes the paper.

Materials and Methods
As explained in the previous section, some literature studies describe systems that collect data from smartphone sensors, such as accelerometer or gyroscope. Aiming to mitigate the smartphone sensors performances weaknesses, this paper presents the results obtained using the fall detection process that we propose by combining data from smartphone sensors, in order to maximise the detection of fall events and, at the same time, reduce the problem of overfitting (i.e., filtering out false-positive), which could unnecessarily trigger phone calls and the rescue processes.

Method for Detecting Falls
The process of detecting falls consists of two phases: detection and rescue, summarised in Figure 1. The detection phase has, as input, a real-time data flow coming from the smartphone's built-in sensors, i.e., the accelerometer, the gyroscope and the microphone.
The algorithm analyses data flows to detect combinations of the three sensors' thresholds by comparing the fall matrix (see Section IV). If the comparison detects a fall, the rescue phase is activated.
In this phase, the system sends an emergency message to rescuers; this message also contains the data from the other two sensors, the GPS and the proximity sensor, to enrich the knowledge of the fall situation. The designed algorithm has been implemented in an Android app. The following subsections provide a detailed description of how the three sensors gather and process data flows. to take gravity into account and measure acceleration correctly.
This is the value provided by the sensor when the smartphone is motionless. Subsequently, the algorithm, developed on a smartphone app, calculates the Signal Magnitude Vector (SMV), also referred to as the Sum Vector (SV). In this way, the movement intensity is computed using this physical magnitude, depicted by equation (1).

Signal Magnitude Vector=
A x , A y and A z represent signals according to the x, y and z components, respectively. We use the SMV to identify a probable fall event by deriving acceleration peak thresholds. Therefore, if a device is placed on a table with the screen facing upwards, it is possible to infer that: When a fall occurs, it is possible to observe that the acceleration suddenly decreases and then increases again, with a fast sequence of a peak close to zero (lower peak) and an upper peak in rapid succession (see Figure 2). Several experiments have been done to identify the optimal threshold values: Figure 2 represents the average evolution of the SMV value, obtained during several tests. As already explained, a fall event is characterized by a rapid fluctuation of the SMV around the value of 9.8 m/s 2 . The SMV suddenly decreases towards a minimum value and then increases towards a maximum value. Through peaks values analysis, we observed that the minimum value was in a range from 2 to 4 m/s2 ", while the maximum value was in a range from 14 to 17 m/s 2 . For this reason, we decided to use these experimentally obtained threshold values as input for the fall detection algorithm.   Therefore, if the condition of an acceleration value greater than the minimum threshold occurs at the end of the post-fall time window, the state "Sending fall" is enabled. This means that the collected data will be sent to a server that will carry out further evaluations to confirm (or not) the Fall. To reduce the number of false alarms without increasing the linear computational effort, we decided to evaluate not only the accelerometer data but also data streams originated by the gyroscope and the microphone.

Gyroscope:
The gyroscope sensor produces angular speed data along the three axes x, y, z. These data are used in equation (3) to get the speed of rotation of a body: ( ) getMaxAmplitude() method.

Proximity Sensor:
The proximity sensor collects samples of the proximity before, during, and after detecting the fall event.
In the experimental tests, smartphones with binary (near, far) proximity sensors have been used.

GPS:
The values of the coordinates are sent to the parents and first aid rescuers in order to locate the patient and allow assistance to be provided.

The Rescue Processes
In the case of confirmation of a fall, the system sends an alarm to family members and/or health assistants, and sends the patient's data to be rescued. GPS also sends the location of the fall event.
The message also contains the state change of the smartphone proximity sensor, through which it is possible to determine whether the smartphone is close to the patient or not. Indeed, in the event of a fall, the proximity sensor can detect if the smartphone screen is in contact with some object; for example, a device could be in the person's pocket or bag (state = 1), and as a consequence of the Fall the smartphone could come out and fall down. In this case, the display, at least for a short time, could change the state (from 1 to 0); this indicates that the patient may be far from the device after the Fall. Rescuers receive the emergency alert. They can call the patient to check her or his condition; if the person answers, she or he can directly provide information about the situation. Otherwise, it is assumed that the person is unable to answer, and this can imply that the person is in a state of unconsciousness or far from the device. The rescue process has the objective to provide rapid assistance to the fallen person. The network of a person's caregivers usually involves people (like relatives and friends) who live in close proximity to the patient who can provide first aid and assess the severity of the injuries.
The implementation of the rescue procedure required the configuration of a web platform with specific services. The rescue procedure is based on three main services: a. The User-Handler is the service called for registration on the platform and for the login access to activate the service. Each person can disable the service based on her or his preferences.
b. The Emergency-Handler is the service called when 30 seconds have passed, after a fall has been detected, without any input from the user (see Figure 4) or when the user presses the SOS button (see Figure 2). This service sends a message to a relative's/friend's or a caregiver's smartphone through the Google Messaging Cloud (GMC) system containing the information related to the patient and type of emergency.

The System in Action
On the patient side, i.e., on the smartphone, an app continuously runs in the background after a login procedure, gaining data from the sensors.   c. The person has not fallen, but the device detects a fall event (false positive). In this case, the patient can press the green button to disable the emergency procedure.
d. The person has fallen, but she or he does not need any kind of assistance. In this case, the patient can press the green button to disable the emergency procedure. e. Depending on the user's actions (or if there is no action), the system will change the state from Alarm to Listening or Management.

Results
The application acquires and analyses three streams of data on the smartphones for assessing a fall: the streams of the accelerometer, gyroscope and microphone (GPS and proximity sensor are used to support the first aid and rescue phase). If a fall is detected, a rescue procedure is activated. The first aid request is the only case of data transmission to the server. Indeed, the proposed methodology avoids to send all the data stream in a continuous cycle: this choice has been made to limit battery consumption and privacy issues. It has been demonstrated that "a single accelerometer sensor at 200 Hz generates about 2.3 GB of data per day, the more the sensors or monitoring metrics are added, the more the data are generated by the sensors" [35]. If all these data have to be uploaded to the cloud for analytics or sent to a server as an input for machine learning approaches, it would result in a wastage of network bandwidth, high consumption of the smartphone battery, a decrease in response time and efficiency and a continuous activity monitoring that lead to user's privacy deprivation.

Experimental Data
The tests addressed the necessity of identifying optimal settings of the data detected by the sensors for different smartphones, with the aim of maximizing sensitivity, specificity and accuracy. The data collected concern all five sensors, but a specific analysis has been done on the accelerometer, the gyroscope and the microphone to set the best operative parameters related to the fall detection phase.
This enabled us to establish the most suitable trade-off between minimum and maximum threshold variation. This subsection illustrates the data collected from the accelerometer, gyroscope    Tables   2 & 3 summarize the results obtained using different threshold values for the accelerometer. The data show that lowering the minimum threshold produces a loss of sensitivity, i.e., fewer falls are detected (false negatives are present). When the threshold is raised, more falls are detected, but many false positives are generated.
On the other hand, high values of the maximum threshold cause a decrease in sensitivity and an increase in specificity. Decreasing the maximum threshold value causes the sensitivity to increase, i.e., more falls are detected. At the same time, however, specificity decreases, i.e., more false positives are generated.    these threshold values, the system cuts off real falls. Tables 6 & 7 show the results obtained using different threshold values for the microphone. The threshold values considered in the experiment are those in which the specificity is over 40%, and the sensitivity is around 90%.

Discussion
This section provides a discussion by mapping experimental data and the accelerometer, gyroscope, and microphone thresholds.
The thresholds considered in the accelerometer, gyroscope, and  Table 8 has been used as an input for the fall detection system, and with several tests, in the same conditions described above, we obtained a sensitivity value of 89.28%.  The way that we have chosen the combinations in this paper is only the first step. Wider experimentation on the field, with a large number of users will enable to refine finding the optimal combinations. It is important to set up combinations of threshold values representing the best solution for sensitivity and specificity.
Another goal is to create different settings of the cloud of thresholds based on users' features such as weight, age and gender. Then, the cloud of values could be adapted based on individually collected data. Table 9 compares performances of our methodology with [21][22][23]25,33]. As shown in the following table, not all the cited works show measurable results. As compared to [22], our approach proposes a more practical system, avoiding additional sensors to be worn by the user, which only consists in a smartphone, achieving better results in terms of specificity and sensitivity. In the same way in [33], the system consists in a wearable device to be worn at the waist which detects Fall through the use of a gyroscope and an accelerometer: analyzing its results, it can be observed that our approach is pretty similar in terms of sensitivity, however it can achieve better results in detecting true negatives (with a variation of about 3.5% in specificity).

Conclusion
In this study, we developed a fall detection system for elderly people using smartphones. Our initial hypothesis was to design a fall detection system that is not perceived as invasive, widely accepted by people, and based on already existing and used technologies. Therefore, we decided to design the application based on smartphone sensors to make it available to as many people as possible. Indeed, using a smartphone, people do not have to wear other devices or sensors; this is a strength for the acceptance by the elderly. We developed a fall detection algorithm that uses data streams from five sensors (accelerometer, gyroscope, microphone, proximity and GPS) embedded in a smartphone. The algorithm works with thresholds and time windows. We demonstrated that our algorithm can work efficiently using ranges of thresholds experimentally defined; it recognizes falls through the combination of these ranges of values and streams from gyroscope and accelerometer, with further refinements done by microphone and proximity sensor. Our system can be used during outdoor activities by elderly people, for example. The smartphone, which runs the proposed system, monitors user movements through its sensors.
If a fall occurs, the system recognizes specific conditions, using thresholds and time windows, and sends an alarm to the server which forwards the first aid request to pre-selected rescuers. As already explained, an essential hypothesis that we assumed was to respond to the privacy issues and to build a non-invasive system. To achieve these objectives, we limited data exchange, using a server only for the rescue management procedure; we investigated on how to reduce the false positives to avoid many false alarms which could induce users to stop using the system. The algorithm has been proven to have good performance in terms of sensitivity, specificity, accuracy and computational complexity. The algorithm takes about 6.5 seconds to detect the Fall and collect the necessary data to be sent remotely. The entire system takes, on average, less than 1 minute to ensure that the emergency notification arrives on the smartphone of the person responsible for the rescue; these values can vary depending on the traffic conditions in the communication network.
For future work, many issues need to be further investigated.
Our algorithm is going to be tested through realistic experiments, in a similar way described in [24]: we planned to distribute the application on a wider scale to verify its efficacy in everyday life activities. This large scale and testing phase will have a duration of two years. We planned to test the entire system in real operating conditions involving many people, including people whose age is greater than the ones we tested on to produce this work. The low cost facilitates the activation of the experimentation. The experiments will use devices (smartphones) that are already available for the population. The testing phase in the field will enable both tunings of optimal parameters and the extension of the trial over the devices considered during the test phase.
Finally, regarding emergency management, the combination of IoT (Internet of Things) and communication channels can enlarge the perspective of safety, and healthcare in the future toward a social paradigm, in which smart objects participate as active agents [36][37][38] in collaborative social networks with people (patients, relatives, friends and caregivers).This is particularly relevant to senior citizens living in their homes [39] or that have an independent life. It allows early detection of a fall, a fast verification of the real conditions and integrated management of potential emergencies based on proximity (involving relatives and friends), competence and knowledge [40].