WO2018168604A1

WO2018168604A1 - Method, system, storage medium and computer system for determining fall response of subject

Info

Publication number: WO2018168604A1
Application number: PCT/JP2018/008753
Authority: WO
Inventors: Shu Feng; Seng Khoon TEH
Original assignee: Nec Corporation
Priority date: 2017-03-15
Filing date: 2018-03-07
Publication date: 2018-09-20
Also published as: SG10201702112XA; JP2020512615A; JP7081606B2

Abstract

To provide a method etc. for determining a fall response thereby detecting a fall down event with reduced false alert. A method according to an aspect of the present invention includes: receiving a signal relating to an activity of a subject from a sensor attached to subject, a state of the signal relating to one of three or more activities of the subject; in response to detecting a fall of the subject based on the state, comparing a first state of a signal at a time when a first predetermined time has elapsed after the fall and a second state a signal at a time when a second predetermined time has elapsed after the fall; and determining a fall response of the subject based on comparison of the first state and the second state.

Description

METHOD, SYSTEM, STORAGE MEDIUM AND COMPUTER SYSTEM FOR DETERMINING FALL RESPONSE OF SUBJECT

The present invention relates broadly, but not exclusively, to a method, a system and a storage medium for determining a fall response of a subject.

A fall is one of the major sources of injuries among elderly people, especially for the elderly people who live alone. Also, injuries for elderly people who live alone tend to be critical because it is difficult for the elderly people who live alone to call for help. Accordingly, monitoring activities of the elderly people and detecting a fall down event are important for attending such an event in a prompt manner.

Currently, there are several methods for detecting fall down events. However, conventional methods for detecting fall down events are not able to avoid false alerts, in which there are no fall events but caring service providers are alerted. Such false alerts cause disturbances for both users and related caring service providers.

A need therefore exists to provide a method and a system etc. for determining a fall response thereby detecting a fall down event with reduced false alert in order to address the above-mentioned problems.

A method according to an aspect of the present invention includes: receiving a signal relating to an activity of a subject from a sensor attached to subject, a state of the signal relating to one of three or more activities of the subject; in response to detecting a fall of the subject based on the state, comparing a first state of a signal at a time when a first predetermined time has elapsed after the fall and a second state a signal at a time when a second predetermined time has elapsed after the fall; and determining a fall response of the subject based on comparison of the first state and the second state.

A system according to an aspect of the present invention includes: receiver means for receiving a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject; comparing means for comparing a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall, in response to detecting a fall of the subject; and determining means for determining a fall response of the subject based on comparison of the first state and the second state.

A computer readable medium according to an aspect of the present invention stores a computer program causing a computer to execute: reception processing of receiving a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject; in response to detecting a fall of the subject based on the signal, comparison processing of comparing a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall; and determination processing of determining a fall response of the subject based on comparison of the first state and the second state.

A computer system according to an aspect of the present invention includes: a memory device; and at least one processor coupled to the memory device and being configured to: receive a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject; in response to detecting a fall of the subject based on the signal, compare a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall; and determine a fall response of the subject based on comparison of the first state and the second state.

Embodiments of the present invention will be better understood and readily apparent to one of the ordinary skilled in the art from the following written description, which provides examples only, and in conjunction with the drawings in which:
Figure 1 shows a flow chart illustrating a method for determining a fall response of a subject according to a first example embodiment of the present embodiment; Figure 2 shows a schematic diagram illustrating a system for determining a fall response of a subject according to a first example embodiment of the present embodiment; Figure 3 shows an exemplary system for determining a fall response of a subject according to present embodiment; Figure 4 shows a subject having a sensor according to present embodiment; Figure 5 shows a schematic diagram of a system for determining a fall response of a subject, according to present embodiment; Figure 6 shows a schematic diagram illustrating transactions between FD engine and ADL engine in a system according to present embodiment; Figure 7 shows a schematic diagram illustrating detection of daily life activity based on a binary tree structured approach according to present embodiment; Figure 8 shows a schematic diagram illustrating tuning parameters in fall detection algorithm utilizing activities of daily life according to present embodiment; Figure 9 shows a schematic diagram of a system for determining a fall response of a subject according to present embodiment; and Figure 10 shows an exemplary computing device suitable for executing the method for determining a fall response of a subject according to present embodiment.

Unless context dictates otherwise, the following terms will be given the meaning provided here.
The term "subject" indicates a person, an animal, a robot or any other things on which a sensor is attached to detect a fall response, thereby confirming a fall down event;
The terms "fall", "fall event" and "fall down event" indicate events in which the status of a subject is confirmed as fall down. In other words, it is confirmed that the subject has fallen down.
The terms "possible fall down" and "suspicious fall down" mean that the status is pending and further input is required to determine whether or not the status of the subject is confirmed as fall down. In other words, it is not possible to confirm that the subject has fallen down.
The term "a fall response" indicates status of a subject after a fall down event. The fall response is one that is detected by a sensor, after it has been confirmed that the subject has fallen down.
The term "alert" includes any type of action (e.g., message, notification, signal) which informs of a targeted person (e.g., user of a system) that the subject has fallen down. It is to be understood that the alert (which may be a visual alert or an audio alert) may be sent via any type of suitable communication means. The "alert" may be sent out together with a likelihood of a fall down event such as 95% etc. The likelihood can be estimated based on past data for a fall down event detection.
The terms "care giver", "care service provider" and "user" indicate a person or an organization who takes care of the subject and are supposed to receive an emergency call (or an alert) when a fall down event of the subject is confirmed.
The term "one or more databases" refers to any database or databases located within a computing system or remote server such as a computer in hospital or a cloud server. The database or databases may be a cloud database running on a cloud computing platform.

Example embodiments of the present invention will be described, by way of example only, with reference to the drawings. Like reference signs in the drawings refer to like elements or equivalents.

Some portions of the description which follows are explicitly or implicitly presented in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to convey most effectively the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities, such as electrical, magnetic or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise, and as apparent from the following, it will be appreciated that throughout the present specification, discussions utilizing terms, such as "receiving", "detecting", "applying", "training", "determining", "comparing", "extracting", "identifying" or the like, refer to the action and processes of a computer system, or similar electronic device, which manipulates and transforms data represented as physical quantities within the computer system into other data similarly represented as physical quantities within the computer system or other information storage, transmission or display devices.

The present specification also discloses apparatus for performing the operations of the methods. Such apparatus may be specially constructed for the required purposes, or may include a computer or other device selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various machines may be used with programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate. The structure of a computer will appear from the description below.

In addition, the present specification also implicitly discloses a computer program, in which it would be apparent to the person skilled in the art that the individual steps of the method described herein may be put into effect by computer codes. The computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the spirit or scope of the invention.

Furthermore, one or more of the steps of the computer program may be performed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a computer. The computer readable medium may also include a hard-wired medium such as exemplified in the Internet system, or wireless medium such as exemplified in the global system for mobile communications (GSM) mobile telephone system. The computer program when loaded and executed on such a computer effectively results in an apparatus that implements the steps of the preferred method.

Figure 1 shows a flow chart illustrating a method 100 for determining a fall response of a subject, according to an example embodiment of the present invention. The method 100 may be performed by a computer coupled to one or more databases. Furthermore, the method 100 may be performed by a computing device which may be a server system, a mobile device (e.g. a smart phone or tablet computer) or a personal computer. Further details on the computer and databases will be provided below with reference to Figures 9 and 10.

The method 100 broadly includes:
Step 102: receiving, at the processor, a signal representing to an activity of a subject, a state of the signal representing one of three or more activities;
Step 104: in response to a detection of a fall of the subject based on the state of the received signal, comparing the state of the signal at the time when a first predetermined time has elapsed after the fall and the state of the signal at the time when a second predetermined time has elapsed after the fall; and
Step 106: determining, at the processor, a fall response of the subject in response to the comparison.

Step 102 involves receiving a signal relating to an activity of the subject. In an example, the subject may be an elderly person, especially an elderly person who lives alone. Additionally or alternatively, the subject may be a toddler, a patient in hospital or any other people who requires fall detection monitoring. Also, additionally or alternatively, the subject may be a pet dog, a pet cat or an animal of one of any other types. Furthermore, the subject may include a two legged robot, a four legged robot or a legged robot of one of any other types.

A state of the signal that has been received relates to one of three or more activities of the subject. In an example, the activity of the subject may be sitting, standing, lying, walking, running or jumping. In other words, the state of the signal that is received relates to one or these three or more activities of the subject. It is to be understood that the three or more activities of the subjects are so identified to increase accuracy of determining a fall response. That is, any other type of activities, which are not listed above, may also be included in the activities of the subject that is a target. The activities of the subject may depend on the subject. For example, if the subject is a young baby or an elderly person who is not capable of jumping, jumping may be excluded from the subject’s activities.

A sensor, e.g. an accelerometer sensor, attached to the subject collects information on an activity of the subject and sends out a signal relating to the activity of the subject. An exemplary position of the sensor on the subject is shown in Figure 4. A receiver coupled to a processor may receive the signal relating to the activity of the subject. The signal may be received in a continuous manner over a predetermined period of time. Additionally or alternatively, the signal may be received whenever an activity is detected. The signal may be sent to the receiver via Bluetooth, WiFi, near field communication (NFC) or any other type of wireless communication. The receiver may be a receiving module in a tablet computer, a smart phone or a personal computer.

Step 104 may include, in response to a detection of a fall (e.g. a fall down event) of the subject, comparing the state of the signal at the time when a first predetermined time has elapsed after the fall and the signal at the time when a second predetermined time has elapsed after the fall. In an example, the first predetermined time is 10 seconds and the second predetermined time is 30 seconds. In an example, an accelerometer sensor attached to an elderly person may detect a sudden acceleration which exceeds a predetermined threshold. Conventionally, an alert is sent to a care giver or a family member of the subject in response to the detection of such a sudden acceleration, which is at the time of a suspected fall. However, in this example embodiment, additional data are used to confirm whether or not the subject has fallen down so as to reduce sending a false alert. In an example, the additional data is collected from the accelerometer sensor twice, e.g., at the time when 10 seconds have elapsed after the suspected fall and at the time when 30 seconds have elapsed after the suspected fall. By comparing data collected at the time when 10 seconds have elapsed after the suspected fall and at the time when 30 seconds have elapsed after the suspected fall, it will be determined whether or not an alert is required to be sent out. In an example, the additional data relate to an activity of the subject at the time when 10 seconds have elapsed after the suspected fall and at the time when 30 seconds have elapsed after the suspected fall. The first predetermined time and the second predetermined time may be appropriately selected to reduce sending a false alert. Instead of checking a state of signal at a single timing, the state of signal is monitored during a predetermined time range including a first predetermined timing and a second predetermined timing to enhance the accuracy of the fall alert.

Step 106 may include determining, at the processor, a fall response of the subject in response to the comparison in step 104. The results of the comparison in step 104 are processed and a fall response of the subject is determined. In an example, features in the signal are extracted and the state of the signal is determined to identify the one of three or more activities of the subject on the basis of the extracted features. In an example, a classification algorithm is applied to the extracted features to identify the one of three or more activities of the subject.

In an example, the classification algorithm includes a binary tree approach, which will be explained together with Figure 7 later. Further, in an example, an activity represented by the state of the signal at the time when a first predetermined time has elapsed after the fall and another activity represented the signal at the time when a second predetermined time has elapsed after the fall are compared to a sequence of activities registered for a fall response.

In one example, whether an alarm signal is to be sent is determined when a fall response is determined. The alarm may be sent to a family member of the subject who lives near the subject or care giver who is taking care of the subject. If the subject is industrial robot, the alarm may be sent to a factory manager. The alarm may be a phone call, a message via a short message service (SMS) or an email. If the subject is hospitalized, the alarm may be a call to a nurse center.

Figure 2 shows a schematic diagram illustrating a system 200 for determining a fall response of a subject according to an example embodiment. In one example, the system 200 may include a fall down detection (FD) engine 202, an activity of daily living (ADL) engine 204 and an accelerometer sensor 206. The accelerometer sensor 206 may be an attachable type such as a patch type and is configured to sense acceleration data in three dimensional manner, e.g. x, y and z directions. The acceleration data 208 sensed by the accelerometer sensor 206 may be continuously sent to an external device such as a computer server, a tablet and a mobile phone by wireless communication such as Bluetooth communication, which will be explained together with Figure 3. The patch type acceleration sensor will be explained together with Figure 4.

In one example, the acceleration data 208 may be sent to a backend device 216, such as a computer server on which the ADL engine 204 runs. Based on the acceleration data 208, the ADL engine 204 predicts a live status e.g. an activity of a subject whose acceleration data 208 is sensed by the accelerometer sensor 206. The predicted live status 210 of the subject may be sent to the FD engine 202 which runs on a frontend device 214 such as a smartphone. The predicted live status 210 of the subject may be used to tune a threshold for fall down event detection in the FD engine 202. When the frontend device 214 receives the acceleration data 208, a fall down event is determined by comparing the received acceleration data 208 with the threshold tuned by using the predicted live status 210 of the subject.

Upon receiving the predicted live status 210 from the ADL engine 204, the FD engine 202 may integrate the predicted live status 210 with the acceleration data 208 received from the sensor 206 to estimate the possible fall event 212. The post-estimation mechanism may be triggered once a possible fall event is alerted. The FD engine 202 will estimate the scenario based on the live status 210 predicted by the ADL engine 204 and detect whether the fall alert is a true alert. The post-estimation mechanism may enable to reduce the false alert of fall detection. Detailed functions of the ADL engine 204 and the FD engine 202 will be discussed together with Figure 5 and Figure 6.

Figure 3 shows an exemplary system 300 for determining a fall response of a subject according to an example embodiment. The system 300 may include a frontend device 302, a backend device 304 and an accelerometer sensor 306. The accelerometer sensor 306 sends an acceleration data 308 sensed by the accelerometer sensor 306 to the frontend device 302 and the backend device 304. The backend device 304, such as a computer server, may predict the live status of the subject. The predicted live status 310 may be sent to the frontend device 302, such as a smart phone. Therefore, the frontend device 302 receives information from the accelerometer sensor 306 and from the backend device 304.

Figure 4 shows a subject 400 having a sensor 402 according to an example embodiment. The sensor 402 may be a patch type sensor attached to a middle part e.g. a torso of the subject. Preferably, a single sensor instead of multiple sensors may be used for detecting fall because wearing two or more devices may cause the subject to be uncomfortable.

A location of the sensor 402 on the subject 400 may be determined based on an accuracy of the acceleration data sensed at the location of the sensor 402. The accuracy may be determined based on how the sensed acceleration data reflects movements of the subject. In an example, the sensor 402 is attached on the waist part of the subject 400.

Figure 5 shows a schematic diagram of the ADL engine 502 and the FD engine 504 in a system 500 according to an example embodiment. In an example, the ADL engine 502 classifies activity of the subject into one of specific activities in an activity classification module 506 of the ADL engine 502. A classified activity 516 (i.e. the specific activity into which the activity of the subject is classified) may be sent to the FD engine 504 for setting parameters of fall detection under the specific activity (i.e. the classified activity 516) as shown in module 508. In other words, parameters of fall detection may be based on the activity classified in the ADL engine. Therefore, different parameters may be applied to fall detections for the different activities.

Threshold based approach is adopted by the FD engine 504. A suspected fall can be detected once the threshold is violated in module 510. Tuning thresholds by activities means that the FD engine 504 will dynamically set different thresholds based on current activities, in which the falling down happens. In an example, the thresholds of falling down in running, sitting and standing are different. This method could differentiate various falling down scenarios such as falling down when a person is walking, and sliding down from the chair when an individual is sitting. Comparing with using one unified threshold without considering the activities, turning parameters set based on the classified activity 516 sent by the ADL engine 502 could reduce false alert in fall detection in module 510.

In order to determine the thresholds for the measured acceleration values in x, y, and z directions, a set of IF THEN rules are predefined in the knowledge base, which will be explained together with Figure 8. The FD engine 504 needs to obtain information on the current activity of daily life such as sitting, standing, lying, etc. from the ADL engine 502, and select the right threshold accordingly.

Similarly, a classified activity 518 (that is the same as the classified activity 516) may be sent from the ADL engine 502 to the FD engine 504 for a post-fall knowledge inference in module 512. After detection of a fall, the classified activity 518 is further used to reduce a false detection of fall. The post-fall knowledge inference will be triggered once a suspected fall has been detected. If the module 512 infers that the suspected fall has been recovered or it is not a fall at all, the system will not send the alert. Otherwise, the system will send the alert to a predetermined contact person such as a family member of the subject or a care giver who takes care of the subject. Accordingly, this method contributes in reducing the false alert of fall detection.

In this process, a set of human knowledge can be predefined based on how long the subject is required to be monitored and what activities shall happen simultaneously with the falling down. To complete the knowledge inference, the FD engine 504 also needs to obtain information on the current ADL information (i.e. the classified activity 518), which is detected by the ADL engine 502.

In an example, after a suspected fall is detected, the system will confirm the fall event by continuous activity monitoring and knowledge inference. An exemplary knowledge inference rules are shown in below table 1.

In an example, if the system gets "possible recovered" as a fall detection result, the system continuously monitors the activity during a predetermined threshold period (e.g. in the following 1 minute). The subject could be regarded as (i) recovered if he/she keeps active status such as walking status and running/jumping status in the following 1 minute, or (ii) unrecovered if he/she keeps still status such as sitting/standing status and lying status.

Once the subject is confirmed as fallen down, an alert is sent to a predetermined person in module 514 so that the predetermined person can take an immediate action.

Figure 6 shows a schematic diagram illustrating transactions between an FD engine 632 and an ADL engine 634 in a system 600 according to an example embodiment. The two

engines

632 and 634 run in parallel mode in the frontend device and the backend device, respectively. In an example, a sensor 602 sends a signal of the accelerometer data of the subject to both of the FD engine 632 and the ADL engine 634.

In the ADL engine 634, the acceleration data in x, y, and z directions from the sensor 602 is obtained as a time series data in step 604 and is stored in a memory 614. And then, a window size of the time series data is set and a window of data is captured for analyzing in step 606.

Based on the time series data, features of the signal is extracted in step 608. Further, the extracted features are used as an input for activity classification model to identify the activity of daily life status such as sitting, standing, lying, walking, running or jumping in step 610. Once the activity of daily life is identified, in step 612, the identified activity status is stored in the memory 614. The identified activity status is used in the FD engine 632.

In the FD engine 632, the acceleration data in x, y, and z directions from the sensor 602 is obtained from the memory 614. In an example, the obtained acceleration data in x, y and z directions is filtered by applying high-pass filters in step 616.

In step 618, a current activity of daily life status such as sitting, standing, lying, walking, running and jumping are read out from the memory 614. Also, threshold parameters are tuned based on the predefined knowledge system.

Once the acceleration data in x, y and z directions satisfies the fall down conditions which are tuned with the current activity of daily life status in step 618, a suspected fall is detected in step 620. If there is no suspected fall, the FD engine 636 continues to get new acceleration data in step 628.

In step 624, the post-fall detection is performed by continuous activity monitoring and knowledge inference as shown in table 1. This step detects whether a real fall happens or if the suspected fall has been recovered. Once a fall event is confirmed in step 626, an alert is sent out in step 636. If there is no fall confirmed or if the fall is recovered, the monitoring is terminated and the new acceleration data is sensed in step 630. As a future reference, the FD engine 632 sends information on the present process to the memory 614.

Figure 7 shows an exemplary binary tree structured approach for detecting activity of daily life in accordance with an example embodiment. In an example, the ADL engine performs a binary tree structured approach to differentiate various activities of daily life (ADL) based on a machine learning algorithm. In this exemplary binary tree 700 includes five

classification nodes

702, 704, 706, 708, and 710, and each of the nodes needs to be trained by a classification algorithm. Once the ADL engine senses acceleration data, the binary tree could calculate and predict the corresponding ADL status such as sitting standing, lying, walking, etc. In this example, a binary-tree based activity classification approach is used. However, different design of classification algorithm is used for different concerns.

In an example, an Ensemble Support Vector Machine (SVM) method is applied for binary classification in each of the nodes of the tree. When sensor data is newly received, this approach conducts a pre-processing, i.e. extracting the statistical features for an Ensemble SVM model. The statistical features may include a mean, a standard deviation, sum values, percentiles, a skewness, a kurtosis, correlations, coefficients of a variation, peak-to-peak amplitude, zero crossings, etc.

In an example, when new sensor data is received, this approach firstly determines whether the current activity is an active activity or a still activity in the node "Still vs Active" 702. If the current activity is determined to be a still activity, whether the current activity is a still/standing activity or a lying activity is determined in the node "Still + Standing vs Lying" 704. If the current activity is not determined to be a lying activity, whether the current activity is a sitting activity or a standing activity is determined in the node "Sitting vs Standing" 708.

Similarly, if the current activity is determined to be an active activity in the node 702, whether the current activity is a walking activity or a running/jumping activity is determined in the node "Walking vs Running + Jumping" 706. If the current activity is not determined to be a walking activity, whether the current activity is a running activity or a jumping activity is determined in the node "Running vs Jumping" 710.

The performance of a single classifier model in distinguishing two classes is always better than that in distinguishing multi-classes. Thus, the binary tree structure contributes in enhancing the overall prediction ability of activities by integrating a plurality of binary classification models. This binary tree structure is extendable if there is a requirement to extend the scope of activities. For example, to distinguish walking with different speeds. More binary nodes may be created as descendants of the node "Sitting vs Standing" 708. This approach is shown for explanation. Other approach such as Support-Vector Networks (SVN), k-nearest neighbors algorithm (KNN), and neural networks may be applicable to predict the ADL status.

Figure 8 shows a schematic diagram illustrating tuning parameters in fall detection algorithm utilizing activities of daily life according to an example embodiment. To detect a fall event precisely, different parameters of fall detection is set for different activities. If current activity is ACTIVITY_1 (YES in step 802), the threshold is set to be PARAMETER_1 (step 808). If the current activity i.e. ACTIVITY_1 is Running activity, there is an appropriate threshold of fall down in Running activity. Based on such knowledge, each activity in

steps

802, 804, and 806 is linked to an appropriate threshold in

steps

808, 810, and 812. By selecting an appropriate threshold, a fall event during each of these activities are precisely detected.

Figure 9 shows a schematic of a network-based system 900 for determining a fall response of a subject according to an example embodiment of the invention. The system 900 includes a computer 902, one or more databases 9041…904n, a user input module 906 and a user output module 908. Each of the one or more databases 9041…904n are communicably coupled with the computer 902. The user input module 906 and a user output module 908 may be separate and distinct modules communicably coupled with the computer 902. Alternatively, the user input module 906 and a user output module 908 may be integrated within a single mobile electronic device (e.g. a mobile phone, a tablet computer, etc.). The mobile electronic device may have appropriate communication modules for wireless communication with the computer 902 via existing communication protocols.

The computer 902 may include: at least one processor; and at least one memory storing computer program code, wherein the at least one memory and the computer program code configured to, with at least one processor, cause the computer at least to: (A) receive a signal relating to the subject’s activity, a state of the signal relating to one of three or more activities of the subject; (B) in response to detecting a fall of the subject, compare the state of the signal at the time when a first predetermined time has elapsed after the fall and the signal at the time when a second predetermined time has elapsed after the fall; and (C) determine a fall response of the subject in response to the comparison.

The various types of data, e.g. activity status, acceleration data in x, y and z direction, knowledge inference rules for post fall knowledge inference can be stored in a single database (e.g. 9041), or stored in multiple databases (e.g. acceleration data in x, y and z directions are stored on database 9041, knowledge inference rules are stored on database 904n, etc.). The databases 9041…904n may be achieved using cloud computing storage modules and/or dedicated servers communicably coupled with the computer 902.

Figure 10 depicts an exemplary computer / computing device 1000, hereinafter interchangeably referred to as a computer system 1000, where one or more such computing devices 1000 may be used to facilitate execution of the above-described method for determining a fall response of a subject. In addition, one or more components of the computer system 1000 may be used to achieve the computer 902. The following description of the computing device 1000 is provided by way of example only and is not intended to be limiting.

As shown in Figure 10, the example computing device 1000 includes a processor 1004 for executing software routines. Although a single processor is shown for the sake of clarity, the computing device 1000 may also include a multi-processor system. The processor 1004 is connected to a communication infrastructure 1006 for communication with other components of the computing device 1000. The communication infrastructure 1006 may include, for example, a communications bus, a cross-bar, or a network.

The computing device 1000 further includes a main memory 1008, such as a random access memory (RAM), and a secondary memory 1010. The secondary memory 1010 may include, for example, a storage drive 1012, which may be a hard disk drive, a solid state drive or a hybrid drive and/or a removable storage drive 1014, which may be a magnetic tape drive, an optical disk drive, a solid state storage drive (such as a Universal Serial Bus (USB) flash drive, a flash memory device, a solid state drive or a memory card), or the like. The removable storage drive 1014 performs reading from and/or writing to a removable storage medium 1044 in a well-known manner. The removable storage medium 1044 may be a magnetic tape, an optical disc, a non-volatile memory storage medium, or the like, which is read by and written to by the removable storage drive 1014. As will be appreciated by persons skilled in the relevant art(s), the removable storage medium 1044 includes a computer readable storage medium having stored therein computer executable program code instructions and/or data.

In an alternative implementation, the secondary memory 1010 may additionally or alternatively include other similar means for allowing computer programs or other instructions to be loaded into the computing device 1000. Such means can include, for example, a removable storage unit 1022 and an interface 1040. Examples of the removable storage unit 1022 and the interface 1040 include a program cartridge and cartridge interface (such as that found in video game console devices), a removable memory chip (such as an erasable programmable read only memory (EPROM) or programmable read only memory (PROM)) and associated socket, a removable solid state storage drive (such as a USB flash drive, a flash memory device, a solid state drive or a memory card), and other removable storage units 1022 and interfaces 1040 which allow software and data to be transferred from the removable storage unit 1022 to the computer system 1000.

The computing device 1000 also includes at least one communication interface 1024. The communication interface 1024 allows software and data to be transferred between computing device 1000 and external devices via a communication path 1026. In various embodiments of the inventions, the communication interface 1024 permits data to be transferred between the computing device 1000 and a data communication network, such as a public data or private data communication network. The communication interface 1024 may be used to exchange data between different computing devices 1000 so that such computing devices 1000 form a part of an interconnected computer network. Examples of a communication interface 1024 can include a modem, a network interface (such as an Ethernet card), a communication port (such as a serial bus, a parallel bus, a printer bus, a general purpose interface bus (GPIB), Institute of Electrical and Electronics Engineers (IEEE) 1394, registered jack (RJ)-45, USB), an antenna with associated circuitry and the like. The communication interface 1024 may be wired or may be wireless. Software and data transferred via the communication interface 1024 are in the form of signals which can be electronic, electromagnetic, optical or other signals capable of being received by communication interface 1024. These signals are provided to the communication interface 1024 via the communication path 1026.

As shown in Figure 10, the computing device 1000 further includes a display interface 1002 which performs operations for rendering images to an associated display 1030 and an audio interface 1032 for performing operations for playing audio content via associated speaker(s) 1034.

As used herein, the term "computer program product" may refer, in part, to a removable storage medium 1044, a removable storage unit 1022, a hard disk installed in storage drive 1012, or a carrier wave carrying software over communication path 1026 (wireless link or cable) to communication interface 1024. The computer readable storage medium refers to any non-transitory, non-volatile tangible storage medium that provides recorded instructions and/or data to the computing device 1000 for execution and/or processing. Examples of such a storage medium include a magnetic tape, a compact disc read only memory (CD-ROM), a DVD, a Blu-ray Disc, a hard disk drive, a ROM or integrated circuit, a solid state storage drive (such as a USB flash drive, a flash memory device, a solid state drive or a memory card), a hybrid drive, a magneto-optical disc, or a computer readable card such as a secure digital (SD) card and the like, whether or not such devices are internal or external of the computing device 1000. Examples of a transitory or non-tangible computer readable transmission medium that may also participate in the provision of software, application programs, instructions and/or data to the computing device 1000 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like.

The computer programs (also called computer program code) are stored in the main memory 1008 and/or the secondary memory 1010. The computer programs can also be received via the communication interface 1024. Such computer programs, when executed, cause the computing device 1000 to perform one or more features of the example embodiments discussed herein. In an example embodiment, the computer programs, when executed, cause the processor 1004 to perform features of the above-described example embodiments. Accordingly, such computer programs represent controllers of the computer system 1000.

The software may be stored in a computer program product and loaded into the computing device 1000 using the removable storage drive 1014, the storage drive 1012, or the interface 1040. Alternatively, the computer program product may be downloaded to the computer system 1000 over the communications path 1026. The software, when executed by the processor 1004, causes the computing device 1000 to perform functions of the example embodiments described herein.

It is to be understood that the example embodiment of Figure 10 is presented merely by way of example. Therefore, in an example embodiment, one or more features of the computing device 1000 may be omitted. Also, in an example embodiment, one or more features of the computing device 1000 may be combined together. Additionally, in an example embodiment, one or more features of the computing device 1000 may be split into one or more component parts.

Although a part or the whole of the above described example embodiments described above can be represented by Supplementary Notes described below, a part or the whole of the above described example embodiments described above is not limited to the following descriptions.

(Supplementary Note 1)
A method comprising:
receiving a signal relating to an activity of a subject from a sensor attached to subject, a state of the signal relating to one of three or more activities of the subject;
in response to detecting a fall of the subject based on the state, comparing a first state of a signal at a time when a first predetermined time has elapsed after the fall and a second state of a signal at a time when a second predetermined time has elapsed after the fall; and
determining a fall response of the subject based on comparison of the first state and the second state.
(Supplementary Note 2)
The method according to Supplementary Note 1, further comprising:
extracting a feature in the signal; and
determining the state of the signal among the three or more activities based on the extracted feature.
(Supplementary Note 3)
The method according to Supplementary Note 2, wherein
the determining the state of the signal includes applying a classification algorithm to the extracted features, thereby identifying one of the three or more activities of the subject.
(Supplementary Note 4)
The method according to Supplementary Note 3, further comprising
training the classification algorithm based on the extracted feature.
(Supplementary Note 5)
The method according to Supplementary Note 3 or 4, wherein
the classification algorithm includes a binary tree approach.
(Supplementary Note 6)
The method according to any one of Supplementary Notes 1 to 5, wherein
the determining the fall response includes comparing a first activity and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
(Supplementary Note 7)
The method according to any one of Supplementary Notes 1 to 6, further comprising
determining if an alarm signal is to be sent when a fall response is determined.
(Supplementary Note 8)
The method according to any one of Supplementary Notes 1 to 7, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.
(Supplementary Note 9)
A system comprising:
receiver means for receiving a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject;
comparing means for comparing a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall, in response to detecting a fall of the subject; and
determining means for determining a fall response of the subject based on comparison of the first state and the second state.
(Supplementary Note 10)
The system according to Supplementary Note 9, further comprising
feature extractor means for extracting a feature in the signal,
wherein the determining means determines the state of signal, thereby identifying one of three or more activities of the subject based on the extracted feature.
(Supplementary Note 11)
The system according to Supplementary Note 10, wherein
the determining means applies a classification algorithm to the extracted feature, thereby identifying one of the three or more activities of the subject.
(Supplementary Note 12)
The system according to Supplementary Note 11, further comprising
a trainer means for training the classification algorithm based on the extracted feature.
(Supplementary Note 13)
The system according to Supplementary Note 11 or 12, wherein
the classification algorithm includes a binary tree approach.
(Supplementary Note 14)
The system according to any one of Supplementary Notes 9 to 13, wherein
the comparing means further compares a first activity represented by the first state of the signal and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
(Supplementary Note 15)
The system according to any one of Supplementary Notes 9 to 14, wherein
the determining means further determines if an alarm signal is to be sent when a fall response is determined.
(Supplementary Note 16)
The system according to any one of Supplementary Notes 9 to 15, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.
(Supplementary Note 17)
A computer readable medium storing a computer program causing a computer to execute:
reception processing of receiving a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject;
in response to detecting a fall of the subject based on the signal, comparison processing of comparing a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall; and
determination processing of determining a fall response of the subject based on comparison of the first state and the second state.
(Supplementary Note 18)
The medium according to Supplementary Note 17, the computer program further causing a computer to execute:
extraction processing of extracting a feature in the signal; and
determination processing of determining the state of the signal among the three or more activities based on the extracted feature.
(Supplementary Note 19)
The medium according to Supplementary Note 18, wherein
the determination processing determines the state of the signal includes applying a classification algorithm to the extracted features, thereby identifying one of the three or more activities of the subject.
(Supplementary Note 20)
The medium according to Supplementary Note 19, the computer program further causing a computer to execute
training processing of training the classification algorithm based on the extracted feature.
(Supplementary Note 21)
The medium according to Supplementary Note 19 or 20, wherein
the classification algorithm includes a binary tree approach.
(Supplementary Note22)
The medium according to any one of Supplementary Notes 17 to 21, wherein
the determination processing determines the fall response includes comparing a first activity and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
(Supplementary Note 23)
The medium according to any one of Supplementary Notes 17 to 22, the computer program further causing a computer to execute
second determination processing of determining if an alarm signal is to be sent when a fall response is determined.
(Supplementary Note 24)
The medium according to any one of Supplementary Notes 17 to 23, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.
(Supplementary Note 25)
A computer system comprising:
a memory device; and
at least one processor coupled to the memory device and being configured to:
receive a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject;
in response to detecting a fall of the subject based on the signal, compare a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall; and
determine a fall response of the subject based on comparison of the first state and the second state.
(Supplementary Note 26)
The computer system according to Supplementary Note 25, wherein
the at least one processor is further configured to:
extract a feature in the signal; and
determine the state of the signal among the three or more activities based on the extracted feature.
(Supplementary Note 27)
The computer system according to Supplementary Note 26 wherein
the at least one processor is further configured to
apply a classification algorithm to the extracted features, thereby identifying one of the three or more activities of the subject.
(Supplementary Note 28)
The computer system according to Supplementary Note 27, wherein
the at least one processor is further configured to
train the classification algorithm based on the extracted feature.
(Supplementary Note 29)
The computer system according to Supplementary Note 27 or 28, wherein
the classification algorithm includes a binary tree approach.
(Supplementary Note 30)
The computer system according to any one of Supplementary Notes 25 to 29, wherein
the at least one processor is further configured to
compare a first activity and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
(Supplementary Note 31)
The computer system according to any one of Supplementary Notes 25 to 30, wherein
the at least one processor is further configured to
determine if an alarm signal is to be sent when a fall response is determined.
(Supplementary Note 32)
The computer system according to any one of Supplementary Notes 25 to 31, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.

It is to be understood by a person skilled in the art that numerous variations and/or modifications may be made to the present invention as shown in the specific example embodiments without departing from the spirit or scope of the invention as broadly described. The present example embodiments are, therefore, to be considered in all respects to be illustrative and not restrictive.

This application is based upon and claims the benefit of priority from Singapore patent application No. 10201702112X, filed on May 15, 2017, the disclosure of which is incorporated herein in its entirety by reference.

100 Method
200 System
202 Fall down detection (FD) engine
204 Activity of daily living (ADL) engine
206 Accelerometer sensor
208 Acceleration data
210 Predicted live status
212 Possible fall event
214 Frontend device
216 Backend device
300 System
302 Frontend device
304 Backend device
306 Accelerometer sensor
308 Acceleration data
310 Predicted live status
400 Subject
402 Sensor
500 System
502 ADL engine
504 FD engine
506 Module
508 Module
510 Module
512 Module
514 Module
516 Classified activity
518 Classified activity
600 System
602 Sensor
614 Memory
632 FD engine
634 ADL engine
700 Binary tree
702 Node
704 Node
706 Node
708 Node
710 Node
900 System
902 Computer
9041 Database
904n Database
906 User input module
908 User output module
1000 System
1002 Display interface
1004 Processor
1006 Communication infrastructure
1008 Main memory
1010 Secondary memory
1012 Storage drive
1014 Removable storage drive
1022 Removable storage unit
1024 Communication interface
1026 Communication path
1030 Display
1032 Audio interface
1034 Speaker(s)
1040 Interface
1044 Removable storage medium

Claims

A method comprising:
receiving a signal relating to an activity of a subject from a sensor attached to subject, a state of the signal relating to one of three or more activities of the subject;
in response to detecting a fall of the subject based on the state, comparing a first state of a signal at a time when a first predetermined time has elapsed after the fall and a second state of a signal at a time when a second predetermined time has elapsed after the fall; and
determining a fall response of the subject based on comparison of the first state and the second state.
The method according to claim 1, further comprising:
extracting a feature in the signal; and
determining the state of the signal among the three or more activities based on the extracted feature.
The method according to claim 2, wherein
the determining the state of the signal includes applying a classification algorithm to the extracted features, thereby identifying one of the three or more activities of the subject.
The method according to claim 3, further comprising
training the classification algorithm based on the extracted feature.
The method according to claim 3 or 4, wherein
the classification algorithm includes a binary tree approach.
The method according to any one of claims 1 to 5, wherein
the determining the fall response includes comparing a first activity and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
The method according to any one of claims 1 to 6, further comprising
determining if an alarm signal is to be sent when a fall response is determined.
The method according to any one of claims 1 to 7, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.
A system comprising:
receiver means for receiving a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject;
comparing means for comparing a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall, in response to detecting a fall of the subject; and
determining means for determining a fall response of the subject based on comparison of the first state and the second state.
The system according to claim 9, further comprising
feature extractor means for extracting a feature in the signal,
wherein the determining means determines the state of signal, thereby identifying one of three or more activities of the subject based on the extracted feature.
The system according to claim 10, wherein
the determining means applies a classification algorithm to the extracted feature, thereby identifying one of the three or more activities of the subject.
The system according to claim 11, further comprising
a trainer means for training the classification algorithm based on the extracted feature.
The system according to claim 11 or 12, wherein
the classification algorithm includes a binary tree approach.
The system according to any one of claims 9 to 13, wherein
the comparing means further compares a first activity represented by the first state of the signal and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
The system according to any one of claims 9 to 14, wherein
the determining means further determines if an alarm signal is to be sent when a fall response is determined.
The system according to any one of claims 9 to 15, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.
A computer readable medium storing a computer program causing a computer to execute:
reception processing of receiving a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject;
in response to detecting a fall of the subject based on the signal, comparison processing of comparing a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall; and
determination processing of determining a fall response of the subject based on comparison of the first state and the second state.
The medium according to claim 17, the computer program further causing a computer to execute:
extraction processing of extracting a feature in the signal; and
determination processing of determining the state of the signal among the three or more activities based on the extracted feature.
The medium according to claim 18, wherein
the determination processing determines the state of the signal includes applying a classification algorithm to the extracted features, thereby identifying one of the three or more activities of the subject.
The medium according to claim 19, the computer program further causing a computer to execute
training processing of training the classification algorithm based on the extracted feature.
The medium according to claim 19 or 20, wherein
the classification algorithm includes a binary tree approach.
The medium according to any one of claims 17 to 21, wherein
the determination processing determines the fall response includes comparing a first activity and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
The medium according to any one of claims 17 to 22, the computer program further causing a computer to execute
second determination processing of determining if an alarm signal is to be sent when a fall response is determined.
The medium according to any one of claims 17 to 23, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.
A computer system comprising:
a memory device; and
at least one processor coupled to the memory device and being configured to:
receive a signal relating to an activity of a subject from a sensor attached to the subject, a state of the signal relating to one of three or more activities of the subject;
in response to detecting a fall of the subject based on the signal, compare a first state of the signal at a time when a first predetermined time has elapsed after the fall and a second state of the signal at a time when a second predetermined time has elapsed after the fall; and
determine a fall response of the subject based on comparison of the first state and the second state.
The computer system according to claim 25, wherein
the at least one processor is further configured to:
extract a feature in the signal; and
determine the state of the signal among the three or more activities based on the extracted feature.
The computer system according to claim 26 wherein
the at least one processor is further configured to
apply a classification algorithm to the extracted features, thereby identifying one of the three or more activities of the subject.
The computer system according to claim 27, wherein
the at least one processor is further configured to
train the classification algorithm based on the extracted feature.
The computer system according to claim 27 or 28, wherein
the classification algorithm includes a binary tree approach.
The computer system according to any one of claims 25 to 29, wherein
the at least one processor is further configured to
compare a first activity and a second activity with a sequence of activities registered for a fall response, the first activity being represented by the first state, the second activity being represented by the second state.
The computer system according to any one of claims 25 to 30, wherein
the at least one processor is further configured to
determine if an alarm signal is to be sent when a fall response is determined.
The computer system according to any one of claims 25 to 31, wherein
the three or more activities include at least one of walking, lying, standing, sitting, running and jumping.