CN113936420B

CN113936420B - Detecting falls using a mobile device

Info

Publication number: CN113936420B
Application number: CN202110792510.7A
Authority: CN
Inventors: H·A·范姆; S·P·杰克逊; V·R·马伊吉; K·加亚拉曼·拉格胡拉姆; A·V·尤拉尔; Y·J·J·雷纳德; T·E·弗格提三世
Original assignee: Apple Inc
Current assignee: Apple Inc
Priority date: 2020-07-14
Filing date: 2021-07-14
Publication date: 2023-06-16
Anticipated expiration: 2041-07-14
Also published as: CN113936420A

Abstract

The invention relates to a method for detecting falling by using a mobile device. "in an example method, a mobile device obtains sample data generated by one or more sensors over a period of time, wherein the one or more sensors are worn by a user. The mobile device determines that the user has fallen based on the sample data and determines the severity of the injury suffered by the user based on the sample data. The mobile device generates one or more notifications based on the determination that the user has fallen and the determined severity of the injury.

Description

Detecting falls using a mobile device

Cross Reference to Related Applications

This application is a continuation-in-part application of U.S. patent application Ser. No.16/852,370, filed on even 17, 4, 2020, which is a continuation-in-part application of U.S. application Ser. No.16/128,464, filed on even 11, 9, 2018, which claims priority from U.S. provisional application Ser. No.62/565,988, filed on 29, 9, 2017, each of which is incorporated herein by reference in its entirety.

Technical Field

The present disclosure relates to techniques for using a mobile device to determine whether a user has fallen.

Background

A motion sensor is a device that measures the motion experienced by an object (e.g., the speed or acceleration of an object with respect to time, the orientation or change in orientation of an object with respect to time, etc.). In some cases, a mobile device (e.g., a cellular phone, a smart phone, a tablet, a wearable electronic device such as a smart watch, etc.) may include one or more motion sensors that determine motion experienced by the mobile device over a period of time. If the mobile device is worn by the user, measurements taken by the motion sensor may be used to determine the motion experienced by the user over a period of time.

Disclosure of Invention

Disclosed herein are systems, methods, devices, and non-transitory computer-readable media for electronically determining whether a user has fallen using a mobile device.

In one aspect, a method includes obtaining, by a mobile device, motion data indicative of motion measured by a motion sensor over a period of time. The motion sensor is worn or carried by the user. The method also includes determining, by the mobile device, an impact experienced by the user based on the motion data, the impact occurring during a first interval of the time period. The method also includes determining, by the mobile device, one or more first motion characteristics of the user during a second interval of the time period based on the motion data. The second interval occurs before the first interval. The method also includes determining, by the mobile device, one or more second motion characteristics of the user during a third interval of the time period based on the motion data. The third interval occurs after the first interval. The method further comprises the steps of: the method includes determining, by the mobile device, that the user has fallen based on the impact, the one or more first motion characteristics of the user, and the one or more second motion characteristics of the user, and in response to determining that the user has fallen, generating, by the mobile device, a notification indicating that the user has fallen.

Implementations of this aspect can include one or more of the following features.

In some implementations, determining the one or more first motion characteristics may include determining that the user is walking during the second interval based on the motion data.

In some implementations, determining the one or more first motion characteristics may include determining that the user is ascending or descending stairs during the second interval based on the motion data.

In some implementations, determining the one or more first motion characteristics may include determining, based on the motion data, that the user is moving the body part according to a swing motion or a support motion during the second interval.

In some implementations, determining the one or more second motion characteristics may include determining that the user is walking during the third interval based on the motion data.

In some implementations, determining the one or more second motion characteristics may include determining that the user is standing during the third interval based on the motion data.

In some implementations, determining the one or more second motion characteristics may include determining that the orientation of the body part of the user has changed N or more times during the third interval based on the motion data.

In some implementations, determining that the user has fallen may include determining that the impact is greater than a first threshold based on the motion data, and determining that the user's motion is impaired during a third interval based on the motion data.

In some implementations, determining that the user has fallen may include determining, based on the motion data, that the impact is less than a first threshold and greater than a second threshold; determining, based on the motion data, whether the user walks during the second interval, steps up during the second interval, or steps down during the second interval; determining, based on the motion data, that the user is moving the body part according to a swing motion or a support motion during the second interval; and determining, based on the motion data, that the motion of the user is impaired during the third interval.

In some implementations, generating the notification can include presenting an indication on at least one of a display device or an audio device of the mobile device that the user has fallen.

In some implementations, generating the notification can include transmitting data to a communication device remote from the mobile device, the data including an indication that the user has fallen.

In some implementations, determining that the user has fallen may include generating a statistical model based on the one or more sampled impacts, the one or more sampled first motion characteristics, and the one or more sampled second motion characteristics. One or more sampled impacts, one or more sampled first motion characteristics, and one or more sampled second motion characteristics may be determined based on the sample motion data. The sample motion data may be indicative of motion measured by one or more additional sensors over one or more additional time periods, wherein each additional motion sensor is worn by a respective additional user.

In some implementations, the statistical model may be a bayesian statistical model.

In some implementations, the one or more sampled first motion characteristics may include an indication of a type of activity performed by the particular additional user with respect to the sample motion data, an indication of a level of activity of the particular additional user with respect to the sample motion data, and/or an indication of a speed of travel of the particular additional user with respect to the sample motion data.

In some implementations, the method may be performed by a coprocessor of a mobile device. The co-processor may be configured to receive motion data acquired from the one or more motion sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In some implementations, the mobile device may include a motion sensor.

In some implementations, the mobile device may be worn on the user's arm or wrist when the sensor measures motion.

In some implementations, the mobile device may be a wearable mobile device.

In another aspect, a method includes: a first signal indicative of acceleration measured by an accelerometer over a time period is acquired by a mobile device, and a second signal indicative of orientation measured by an orientation sensor over the time period, wherein the accelerometer and the orientation sensor are physically coupled to a user. The method further comprises the steps of: rotation data is determined by the mobile device that indicates an amount of rotation the user has experienced during the time period, the user has been flipped by the mobile device is determined based on the rotation data, and in response to determining that the user has flipped, a notification is generated by the mobile device that indicates that the user has flipped.

In some implementations, the rotation data can include a third signal indicative of a rate of rotation experienced by the user during the period of time.

In some implementations, the rotation data can include an indication of one or more axes of rotation of the user in the reference frame during the period of time.

In some implementations, the rotation data can include an indication of an average axis of rotation of the user during the period of time.

In some implementations, determining that the user has rolled over may include determining a change between one or more axes of rotation of the user during the time period and an average axis of rotation of the user during the time period.

In some implementations, determining that the user has flipped may include determining that the change is less than a first threshold, and in response to determining that the change is less than the first threshold, determining a fourth signal corresponding to an angular displacement of the user during the time period based on the third signal.

In some implementations, determining the fourth signal may include integrating the third signal with respect to the time period.

In some implementations, determining that the user has rolled over may include determining that the angular displacement of the user during the period of time is greater than a second threshold; determining that at least one of the one or more axes of rotation of the user during the time period is greater than a third threshold; and determining that the user has fallen in response to determining that the angular displacement experienced by the user during the time period is greater than a second threshold and determining that at least one of the one or more axes of rotation of the user during the time period is greater than a third threshold.

In some implementations, generating the notification can include presenting an indication on at least one of a display device or an audio device of the mobile device that the user has flipped over.

In some implementations, the method may be performed by a coprocessor of a mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In another aspect, a method includes obtaining, by a mobile device, motion data indicative of motion measured by one or more motion sensors over a first period of time. One or more motion sensors are worn by a user. The method also includes determining, by the mobile device, that the user has fallen based on the motion data, and in response to determining that the user has fallen, generating, by the mobile device, one or more notifications indicating that the user has fallen.

In some implementations, generating the one or more notifications can include presenting a first notification to the user indicating that the user has fallen.

In some implementations, the first notification can include at least one of a visual message, an audio message, or a haptic message.

In some implementations, generating the one or more notifications can include receiving, by the mobile device, input from the user in response to the first notification. The input may indicate a request for assistance by the user. Additionally, generating the one or more notifications may include, in response to receiving the input, transmitting a second notification to a communication device remote from the mobile device indicating a request for assistance.

In some implementations, the communication device may be an emergency response system.

In some implementations, the second notification can indicate a location of the mobile device.

In some implementations, generating the one or more notifications can include determining, by the mobile device, that the user has not moved during a second period of time after the user falls, and transmitting, to the communication device remote from the mobile device, a second notification indicating a request for assistance in response to determining that the user has not moved during the second period of time.

In some implementations, generating the one or more notifications can include determining, by the mobile device, that the user has moved during a second period of time after the user falls, and refraining from transmitting, to the communication device remote from the mobile device, a second notification indicating a request for assistance in response to determining that the user has moved during the second period of time.

In some implementations, one or more notifications can be generated according to a state machine.

In some implementations, the one or more motion sensors may include at least one of an accelerometer or a gyroscope.

In some implementations, the mobile device may be a wearable mobile device.

In some implementations, determining that the user has fallen may include determining that the user experiences an impact based on the motion data.

In some implementations, determining that the user has fallen may include determining a behavior of the user during the first period of time.

In another aspect, a method includes obtaining, by a mobile device, sample data generated by a plurality of sensors over a period of time. The plurality of sensors is worn by a user. The sample data includes motion data acquired from one or more motion sensors of the plurality of sensors indicative of motion of the user, and at least one of: position data indicative of a position of the mobile device acquired from one or more of the plurality of sensors, altitude data indicative of an altitude of the mobile device acquired from one or more of the plurality of sensors, or heart rate data indicative of a heart rate of the user acquired from one or more of the plurality of heart rate sensors. The method also includes determining, by the mobile device, that the user has fallen based on the sample data, and in response to determining that the user has fallen, generating, by the mobile device, one or more notifications indicating that the user has fallen.

In some implementations, acquiring the motion data may include using an accelerometer to acquire acceleration data during a first time interval during the time period. The gyroscope may be disabled during the first time interval. Acquiring motion data may also include determining that movement of the user exceeds a threshold level during a first time interval based on the acceleration data acquired during the first time interval, and in response to determining that movement of the user exceeds the threshold level during the first time interval, acquiring acceleration data using the accelerometer and acquiring gyroscope data using the gyroscope during a second time interval subsequent to the first time interval.

In some implementations, the one or more height sensors may include at least one of an altimeter or a barometer.

In some implementations, the one or more position sensors may include at least one of a wireless transceiver or a global navigation satellite system receiver.

In some implementations, determining that the user has fallen may include determining a change in orientation of the mobile device during the period of time based on the motion data, and determining that the user has fallen based on the change in orientation.

In some implementations, determining that the user has fallen may include determining, based on the motion data, that the user has experienced an impact during the period of time, and determining, based on the impact, that the user has fallen.

In some implementations, determining that the user has fallen may include determining a change in the height of the mobile device during the period of time based on the height data, and determining that the user has fallen based on the change in the height.

In some implementations, determining that the user has fallen may include determining a change in the user's heart rate during the period based on the heart rate data, and determining that the user has fallen based on the change in the heart rate.

In some implementations, determining the change in the heart rate of the user during the time period may include determining a decay rate of the heart rate of the user during the time period.

In some implementations, determining that the user has fallen may include determining an environmental condition at the location of the mobile device based on the location data, and determining that the user has fallen based on the environmental condition.

In some implementations, the environmental condition may be weather.

In some implementations, the mobile device can determine that the user has fallen based on the motion data, the location data, the altitude data, and the heart rate data.

In some implementations, the mobile device may be a wearable mobile device.

In some implementations, generating the one or more notifications can include transmitting the notification to a communication device remote from the mobile device.

In another aspect, a method includes: receiving, by the mobile device, motion data acquired by one or more sensors over a period of time, wherein the one or more sensors are worn by a user; determining, by the mobile device, an impact experienced by the user during the time period based on the motion data; determining, by the mobile device, one or more of the characteristics of the user; determining, by the mobile device, a likelihood that the user needs assistance after the impact based on the motion data and the one or more characteristics of the user; and generating, by the mobile device, one or more notifications based on the likelihood.

In some implementations, the one or more characteristics of the user can include an age of the user. The likelihood may increase as the age of the user increases.

In some implementations, the one or more characteristics of the user can include a gender of the user. The likelihood may depend on the gender of the user.

In some implementations, the one or more characteristics of the user can include a historical physical activity level of the user. The likelihood may increase as the user's historical physical activity level decreases. In some implementations, the historical physical activity level may indicate a frequency of movement of the user prior to the impact. In some implementations, the historical physical activity level may indicate the strength of movement of the user prior to the impact.

In some implementations, the one or more characteristics of the user can include vascular health of the user. The likelihood may increase as the vascular health of the user decreases. In some implementations, the maximum oxygen uptake (VO) of the user may be based ₂ max) determines the vascular health of the user.

In some implementations, the one or more characteristics of the user may include a historical walking speed of the user. The likelihood may increase as the user's historical walking speed decreases.

In some implementations, generating the one or more notifications may include determining that the likelihood exceeds a threshold level, and generating the one or more notifications in response to determining that the likelihood exceeds the threshold level.

In some implementations, generating the one or more notifications can include transmitting a first notification to a communication device remote from the mobile device, the first notification including an indication that the user has fallen. The communication device may be an emergency response system.

In some implementations, the mobile device may be a wearable mobile device.

In some implementations, at least some of the one or more sensors may be disposed on or in the mobile device.

In some implementations, at least some of the one or more sensors may be remote from the mobile device.

In another aspect, a method includes: a database comprising a plurality of data records is obtained by a mobile device. Each data record includes an indication of a respective impact previously experienced by a user of the mobile device, and sensor data generated by one or more first sensors worn by the user during the impact. The method also includes obtaining, by the mobile device, additional sensor data generated by one or more second sensors worn by the user over a period of time; determining, by the mobile device, whether the user has fallen during the period of time based on the database and the additional sensor data; and generating, by the mobile device, one or more notifications based on determining whether the user has fallen during the period of time.

In some implementations, the method may include generating additional data records based on the additional sensor data; and including the additional data record in a database.

In some implementations, the database can be stored on a storage device of the mobile device.

In some implementations, the method can include generating, by the mobile device, one or more clusters in the plurality of data records based on similarities between the plurality of data records. One or more clusters may be generated using K-means clustering.

In some implementations, the one or more first sensors and the one or more second sensors may include at least one of an accelerometer or an orientation sensor. For each data record, the sensor data may include one or more first signals indicative of acceleration measured by the accelerometer during an impact associated with the data record, and one or more second signals indicative of orientation measured by the orientation sensor during the impact associated with the data record.

In some implementations, the additional sensor data may include one or more additional first signals indicative of acceleration measured by the accelerometer during the time period, and one or more additional second signals indicative of orientation measured by the orientation sensor during the time period. Each data record may include metadata regarding an impact associated with the data record. The metadata may include at least one of: an indication of a respective time of impact associated with the data record or an indication of a respective day of the week of impact associated with the data record.

In some implementations, determining whether the user has fallen may include determining that the user has experienced an impact based on additional sensor data.

In some implementations, determining whether the user has fallen may include determining, based on the additional sensor data, that the impact corresponds to a likelihood that the user has fallen in response to determining that the user has experienced the impact.

In some implementations, determining whether the user has fallen may include determining that the user has fallen based on the determined likelihood, and determining a similarity measure indicative of similarity between the additional sensor data and the sensor data of one or more clusters in the plurality of data records in response to determining that the user has fallen.

In some implementations, generating the one or more notifications can include determining that the similarity measure is less than a threshold level, and generating a first notification to the user to confirm whether the user has fallen in response to determining that the similarity measure is less than the threshold level.

In some implementations, generating the one or more notifications can include receiving input from the user indicating that the user has fallen; and in response to receiving the input, transmitting a second notification to a communication device remote from the mobile device. The second notification may include an indication that the user has fallen. The communication device may be an emergency response system.

In some implementations, determining whether the user has fallen during the time period may include determining that the user has experienced multiple impacts during the time period based on additional sensor data; determining that the multiple impacts are similar to each other based on the additional sensor data; and in response to determining that the user has experienced multiple impacts during the time period and that the multiple impacts are similar to each other, determining that the user is unlikely to have fallen during the time period.

In some implementations, determining whether the user has fallen during the time period may include determining, based on the additional sensor data, that the user has experienced a periodic sequence of multiple impacts during the time period; and in response to determining that the user has experienced multiple impacts in a periodic sequence during the period of time, determining that the user is unlikely to have fallen during the period of time.

In some implementations, determining whether the user has fallen during the time period can include determining smoothness of movement of the user during the time period based on the additional sensor data; and determining whether the user has fallen during the period of time based on the smoothness of the user's movement.

In some implementations, determining whether the user has fallen during the time period may include determining acceleration of the user during the time period relative to a first direction and a second direction orthogonal to the first direction based on additional sensor data; determining that the acceleration in the first direction is greater than a first threshold and the acceleration in the second direction is less than a second threshold; in response to determining that the acceleration in the first direction is greater than a first threshold and the acceleration in the second direction is less than a second threshold, it is determined that the user is unlikely to have fallen during the period of time. The first direction may be orthogonal to the direction of gravity. The second direction may be parallel to the direction of gravity.

In some implementations, at least some of the one or more first sensors or the one or more second sensors may be disposed on or in the mobile device.

In some implementations, at least some of the one or more first sensors or the one or more second sensors may be remote from the mobile device.

In some implementations, at least some of the one or more first sensors may be the same as at least some of the one or more second sensors.

In another aspect, a method includes: receiving, by the mobile device, motion data acquired by one or more sensors, wherein the one or more sensors are worn by a user; determining, by the mobile device, that the user has fallen at a first time based on the motion data; determining, by the mobile device, whether the user is moving between a second time and a third time after the first time based on the motion data; and upon determining that the user has not moved between the second time and the third time, initiating a communication at a fourth time subsequent to the third time requesting an emergency response service, the communication including an indication that the user has fallen and a location of the user.

In some implementations, the one or more sensors include one or more accelerometers. The motion data may include one or more acceleration signals acquired by one or more accelerometers. Determining whether the user has moved between the first time and the second time may include determining a change in one or more acceleration signals between the first time and the second time.

In some implementations, the one or more sensors include one or more orientation sensors. The motion data may include one or more orientation signals acquired by the one or more orientation sensors. Determining whether the user has moved between the first time and the second time may include determining a change in one or more orientation signals between the first time and the second time.

In some implementations, determining whether the user has moved between the second time and the third time may include determining whether the user is walking between the second time and the third time.

In some implementations, determining whether the user has moved between the second time and the third time may include determining whether the user is standing between the second time and the third time.

In some implementations, the method can include refraining from initiating the communication request for the emergency response service upon determining that the user has moved between the second time and the third time.

In some implementations, the method can include receiving, by the mobile device, a command from the user to terminate the communication after initiating the communication request emergency response service; and terminating the communication in response to receiving a command to terminate the communication.

In some implementations, the mobile device may be a wearable mobile device.

In another aspect, a method includes: obtaining, by a mobile device, sample data generated by one or more sensors over a period of time, wherein the one or more sensors are worn by a user; determining, by the mobile device, that the user has fallen based on the sample data; determining, by the mobile device, a severity of injury suffered by the user based on the sample data; and generating, by the mobile device, one or more notifications based on the determination that the user has fallen and the determined severity of the injury.

In some implementations, the one or more sensors may include at least one of an accelerometer, an orientation sensor, or an altimeter.

In some implementations, the sample data can include movement data indicative of movement of the user over a period of time. Determining that the user has fallen may include determining, based on the motion data, a first impact experienced by the user during the period of time, and determining, based on the motion data, a change in orientation of a portion of the user's body during the period of time.

In some implementations, the portion of the user's body can include the user's wrist.

In some implementations, the motion data may include a first signal indicative of acceleration measured by the accelerometer over a period of time, and a second signal indicative of orientation measured by the orientation sensor over the period of time.

In some implementations, determining the severity of the injury suffered by the user may include determining the severity of the first impact experienced by the user based on the first signal and the second signal.

In some implementations, determining the severity of the injury suffered by the user may include determining that the user has experienced a plurality of impacts including a first impact during the period of time based on the first signal and the second signal.

In some implementations, determining the severity of the injury suffered by the user may include determining a first set of characteristics associated with the first impact based on the first signal and the second signal; determining a second set of characteristics associated with a second impact experienced by the user during the time period based on the first signal and the second signal; and determining a similarity between the first set of characteristics and the second set of characteristics.

In some implementations, the motion data can include a third signal indicative of an altitude measured by the altimeter over a period of time. Determining the severity of the injury suffered by the user may comprise determining a distance over which the user falls within the period of time based on the third signal. Determining the severity of the injury suffered by the user may include determining the severity of the injury based on the determined severity of the impact experienced by the user and the determined distance over which the user falls.

In some implementations, generating the one or more notifications can include transmitting the first notification to a communication device remote from the mobile device. The first notification may include an indication that the user has fallen and an indication of the severity of the determined injury suffered by the user. The communication device may be an emergency response system.

In some implementations, the mobile device may be a wearable mobile device.

Particular implementations provide at least the following advantages. In some cases, implementations described herein may be used to determine whether a user has fallen and, in response, automatically take appropriate action. For example, a mobile device may monitor a user's movements (e.g., walking, running, sitting down, lying down, participating in sports or athletic activities, etc.) as they walk around in their daily lives. Based on the user's movements, the mobile device can determine whether the user has fallen, and if so, whether the user may need assistance (e.g., physical assistance when standing up and/or recovering from a fall, medical assistance for treating injuries in a fall, etc.). In response, the mobile device may automatically notify others (e.g., caregivers, doctors, medical responders, or bystanders) of the situation so that they may provide assistance to the user. Thus, help can be provided to the user more quickly and efficiently.

Additionally, implementations described herein may be used to more accurately determine whether a user has fallen and/or whether the user may need assistance. Thus, resources can be more efficiently utilized. For example, the mobile device may determine whether the user has fallen and/or whether the user may need assistance with fewer false positives. Thus, when the user does not need assistance, the mobile device is less likely to use computing resources and/or network resources to generate and transmit notifications to others. In addition, medical resources and logistical resources can be deployed to help users determine the resources needed with greater confidence, thereby reducing the likelihood of wastage. Thus, resources can be used more efficiently and in a manner that improves the effective response capability of the system.

Other implementations relate to systems, devices, and non-transitory computer readable media that include computer executable instructions for performing the techniques described herein.

The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims.

Drawings

Fig. 1 is an illustration of an example system for determining whether a user has fallen and/or may need assistance.

Fig. 2A is a diagram illustrating an example location of a mobile device on a user's body.

Fig. 2B is a diagram illustrating an example directional axis relative to a mobile device.

Fig. 3 is a diagram illustrating an example acceleration signal acquired by a mobile device.

Fig. 4 is an illustration of an example heatmap of motion data collected from a sample population of users who have not fallen while performing athletic activity, and data points indicating motion data collected from a sample population of users who have fallen.

Fig. 5 is an illustration of another example heatmap of motion data collected from a sample population of users who did not fall while performing athletic activity, and data points indicating motion data collected from a sample population of users who fallen.

Fig. 6 is an illustration of another example heatmap of athletic data collected from a sample population of users who did not fall while performing physical activity, and data points indicating athletic data collected from a sample population of users who fallen.

FIG. 7 is an illustration of an example technique for determining a duration in which a magnitude of an acceleration vector is below a threshold.

FIG. 8 is an illustration of an example decision tree for determining whether a user exhibits signs of trauma and/or athletic injury.

Fig. 9 is an illustration of an example decision tree for determining that a user has fallen and may need assistance, or that a user has not fallen or has fallen but does not need assistance.

Fig. 10 is an illustration of an example scatter plot of attitude angle data collected from a sample population of users who have fallen.

FIG. 11A shows a graph of an example acceleration signal acquired by a mobile device within a sliding sample window.

Fig. 11B shows a graph of an example rotation rate signal relative to an inertial frame acquired by a mobile device within a sliding sample window.

Fig. 11C shows a rotation axis signal indicating the instantaneous rotation axis of the mobile device within the sliding sample window.

Fig. 11D shows the total angular displacement signal corresponding to the total angular displacement of the mobile device within the sliding sample window.

Fig. 12 is an illustration of an example state machine for determining whether to use a mobile device to transmit a distress call.

Fig. 13 is a schematic representation of a fall detection technique based on user-specific sensitivity.

FIG. 14 is an illustration of an example state machine for selectively enabling and disabling a gyroscope.

Fig. 15 is a schematic representation of a fall detection technique based on multiple types of sensor measurements.

FIG. 16 is a schematic representation of an example use of determining information about user motion in conjunction with an accelerometer and a gyroscope.

Fig. 17 is a schematic representation of an example fall classifier.

Fig. 18 is a schematic representation of an example fall sensor fusion module.

Figure 19 is a schematic representation of an example distress call module.

Fig. 20 is a schematic representation of an example fall state machine.

Fig. 21 is an illustration of different types of falls that can be experienced by a user.

Fig. 22A is a schematic representation of an example fall detection module.

Fig. 22B is a flowchart of an example process for determining the likelihood that the user has fallen.

Fig. 23 shows an example line graph of the physical history activity level of the user and the expected severity of a fall experienced by the user.

Fig. 24 shows example acceleration data collected by accelerometers positioned on the user's body before, at and after a fall.

FIG. 25 is a flow chart of an example process for generating and transmitting notifications.

FIG. 26 is an illustration of an example state machine for determining whether to generate and transmit notifications.

Fig. 27 is a schematic representation showing a fall detection module for determining the severity of a user's injury due to a fall.

Fig. 28 is a flow chart of an example process for making a user-specific determination of whether a user has fallen.

Fig. 29 is a flow chart of an example process for determining whether a user has fallen and/or may need assistance.

FIG. 30 is a flow chart of an example process for determining whether a user has rolled over and/or may need assistance.

Fig. 31 is a flow chart of an example process for determining whether a user has fallen and/or may need assistance.

Fig. 32 is a flow chart of an example process for determining whether a user has fallen and/or may need assistance.

Fig. 33 is a flow chart of an example process for determining whether a user has fallen and/or may need assistance.

Fig. 34 is a flow chart of an example process for determining whether a user has fallen and/or may need assistance.

Fig. 35 is a flow chart of an example process for determining whether a user has fallen and/or may need assistance.

Fig. 36 is a flowchart of an example process for determining whether a user has fallen and/or may need assistance.

Fig. 37 is a block diagram of an example architecture for implementing the features and processes described with reference to fig. 1-36.

Detailed Description

SUMMARY

Fig. 1 illustrates an example system 100 for determining whether a user has fallen and/or may need assistance. The system 100 includes a mobile device 102, a server computer system 104, a communication device 106, and a network 108.

In an example use of the system 100, the user 110 positions the mobile device 102 on his body and walks around in his daily life. This may include, for example, walking, running, sitting, lying down, participating in an exercise or sporting activity, or any other physical activity. During this time, the mobile device 102 collects sensor data regarding the movement of the mobile device 102, the orientation of the mobile device 102, and/or other dynamic properties. Based on this information, the system 100 determines whether the user has fallen, and if so, whether the user may need assistance.

For example, user 110 may stumble and fall to the ground while walking. In addition, after a fall, the user 110 may not be able to re-stand himself and/or suffer injury from the fall. Thus, he may need assistance, such as physical assistance when standing up and/or recovering from a fall, medical assistance in the treatment of injuries in a fall, or other assistance. In response, the system 100 may automatically notify others of the situation. For example, mobile device 102 can generate and transmit notifications to one or more of communication devices 106 to notify one or more users 112 (e.g., caregivers, doctors, medical responders, emergency contacts, etc.) of the situation so that they can take action. As another example, mobile device 102 can generate and transmit notifications to one or more bystanders in the vicinity of the user (e.g., by broadcasting visual alerts and/or audible alerts) so that they can take action. As another example, mobile device 102 can generate and transmit notifications to server computer system 104 (e.g., to forward the notifications to others and/or to store information for future analysis). Thus, assistance may be provided to user 110 more quickly and efficiently.

In some cases, the system 100 can determine that the user 110 has experienced an external force, but has not fallen and does not need assistance. For example, user 110 may have been contacted during a sporting activity (e.g., hit by another person while playing a basketball), but not fallen due to the contact, and can resume without the assistance of other people. Thus, the system 100 may avoid generating notifications and transmitting notifications to others.

In some cases, the system 100 can determine that the user 110 has fallen, but the user does not need assistance. For example, the user 110 may fall as part of a sporting activity (e.g., slip while skiing), but be able to recover without the assistance of other people. Thus, the system 100 may avoid generating notifications and transmitting notifications to others.

In some cases, the system 100 can determine that the user 110 has fallen, determine the severity of the injury the user 110 suffered as a result of the fall, and in response perform (or refrain from performing) one or more actions. For example, the system 100 can determine that the user 110 has fallen, but has suffered no or little injury, so that he can recover without the assistance of other people. In response, the system 100 may refrain from generating the notification and transmitting the notification to others. As another example, the system 100 can determine that the user 110 has fallen and has suffered more serious injury, so that he may need assistance. In response, the system 100 may generate a notification and transmit the notification to others to assist the user.

In some cases, the system 100 may make these determinations based on motion data acquired before, during, and/or after the impact experienced by the user 110. For example, the mobile device 102 may collect motion data (e.g., acceleration signals acquired by a motion sensor of the mobile device 102), and the system 100 may use the motion data to identify a point in time when the user experienced an impact. Upon identifying the time of impact, the system 100 may analyze motion data acquired during, before, and/or after the impact to determine whether the user has fallen, and if so, the severity of the injury suffered by the user and/or whether the user may need assistance.

In some cases, the system 100 can also make these determinations in a manner that eliminates or otherwise reduces the incidence of false positives (e.g., incorrect determinations that the user has fallen and/or needs help when the user has not actually fallen and/or does not actually need help). In some implementations, these determinations may be made on a user-specific basis. For example, such determinations may be made for a particular user based on previously collected historical sensor data about the user, historical information about the user's previous activities, personal characteristics of the user (e.g., physical characteristics, age, demographics, etc.), and/or other information specific to the user.

Implementations described herein enable the system 100 to more accurately determine whether a user has fallen and/or whether the user may need assistance so that resources may be used more efficiently. For example, the system 100 can determine whether the user has fallen and/or whether the user may need assistance with fewer false positives. Thus, when a user does not need assistance, the system 100 uses computing resources and/or network resources to generate and transmit notifications to others less likely. In addition, medical resources and logistical resources can be deployed to help users determine the resources needed with greater confidence, thereby reducing the likelihood of wastage. Thus, resources may be more efficiently used and utilized in a manner that increases the effective response capability of one or more systems (e.g., computer systems, communication systems, and/or emergency response systems).

The mobile device 102 may be any portable electronic device for receiving, processing, and/or transmitting data, including but not limited to a cellular telephone, a smart phone, a tablet computer, a wearable computer (e.g., a smart watch), and the like. The mobile device 102 is communicatively connected to the server computer system 104 and/or the communication device 106 using a network 108.

The server computer system 104 is communicatively connected to the mobile device 102 and/or the communication device 106 using a network 108. The server computer system 104 is shown as a respective single component. In practice, however, it may be implemented on one or more computing devices (e.g., each computing device including at least one processor such as a microprocessor or microcontroller). The server computer system 104 may be, for example, a single computing device connected to the network 108. In some implementations, the server computer system 104 can include multiple computing devices connected to the network 108. In some implementations, the server computer system 104 need not be local to the rest of the system 100, and portions of the server computer system 104 may be located in one or more remote physical locations.

Communication device 106 may be any device for transmitting and/or receiving information transmitted over network 108. Examples of communication devices 106 include computers (such as desktop computers, notebook computers, server systems, etc.), mobile devices (such as cellular telephones, smart phones, tablet computers, personal data assistants, notebook computers with networking capabilities), telephones, faxes, and other devices capable of transmitting and receiving data from network 108. The communication device 106 may include devices that operate using one or more operating systems (e.g., microsoft Windows, apple OS X, linux, unix, android, apple iOS, etc.) and/or architectures (e.g., X86, powerPC, ARM, etc.). In some implementations, one or more of the communication devices 106 need not be located locally with respect to the rest of the system 100, and one or more of the communication devices 106 may be located in one or more remote physical locations.

Network 108 may be any communication network through which data may be transmitted and shared. For example, the network 108 may be a Local Area Network (LAN) or a Wide Area Network (WAN), such as the Internet. As another example, the network 108 may be a telephone or cellular communication network. The network 108 may be implemented using various network interfaces, such as a wireless network interface (such as Wi-Fi, bluetooth, or infrared) or a wired network interface (such as an ethernet or serial connection). The network 108 may also include a combination of more than one network and may be implemented using one or more network interfaces.

As described above, the user 110 may position the mobile device 102 on his body and walk around in his daily life. For example, as shown in fig. 2A, mobile device 102 may be a wearable electronic device or a wearable computer (e.g., a smartwatch) that is secured to wrist 202 of user 110. Mobile device 102 may be secured to user 110, for example, by a strap or strap 204 around wrist 202. In addition, the orientation of the mobile device 102 may vary depending on its location on the user's body and the user's positioning of its body. For example, orientation 206 of mobile device 102 is shown in fig. 2A. Orientation 206 may refer, for example, to a vector projected from a front edge (e.g., the y-axis shown in fig. 2B) of mobile device 102.

Although an example mobile device 102 and an example location of the mobile device 102 are shown, it should be understood that these are merely illustrative examples. In practice, mobile device 102 may be any portable electronic device for receiving, processing, and/or transmitting data, including, but not limited to, a cellular telephone, a smart phone, a tablet computer, a wearable computer (e.g., a smart watch), and the like. For example, mobile device 102 can be implemented in accordance with architecture 2500 shown and described with respect to fig. 25. In addition, in practice, the mobile device 102 may be positioned on other locations of the user's body (e.g., arms, shoulders, legs, hips, head, abdomen, hands, feet, or any other location).

As the user 110 walks around in his daily life (e.g., walks, runs, sits down, lies down, takes part in an athletic or physical activity, or any other physical activity) with the mobile device 102 on his body, the mobile device 102 gathers sensor data about the movement of the user 110. For example, using the motion sensor 2510 (e.g., one or more accelerometers) shown in fig. 25, the mobile device 102 can measure the acceleration experienced by the motion sensor 2510 and, accordingly, the acceleration experienced by the mobile device 102. In addition, using the motion sensor 2510 (e.g., one or more compasses or gyroscopes), the mobile device 102 can measure an orientation of the motion sensor 2510 and, accordingly, the mobile device 102. In some cases, the motion sensor 2510 may collect data continuously or periodically over a period of time or in response to a triggering event. In some cases, the motion sensor 2510 may collect motion data relative to one or more particular directions relative to the orientation of the mobile device 102. For example, the motion sensor 2510 may collect sensor data regarding acceleration of the mobile device 102 relative to an x-axis (e.g., a vector protruding from a side edge of the mobile device 102, as shown in fig. 2B), a y-axis (e.g., a vector protruding from a front edge of the mobile device 102, as shown in fig. 2B), and/or a z-axis (e.g., a vector protruding from a top surface or screen of the mobile device 102, as shown in fig. 2B), where the x-axis, y-axis, and z-axis refer to cartesian coordinate systems (e.g., a "body" reference frame) in a reference frame fixed to the mobile device 102.

For example, as shown in fig. 3, as the user 110 moves, the mobile device 102 may use the motion sensor 2510 to continuously or periodically collect sensor data regarding the acceleration experienced by the motion sensor 2510 relative to the y-axis over a period of time. The resulting sensor data may be presented in the form of a time-varying acceleration signal 300. In some cases, the acceleration system 300 may acquire acceleration samples at a sample frequency of 800Hz using a motion sensor 2510 having a sampling bandwidth of 200 Hz. In practice, other sampling frequencies and/or sampling bandwidths are possible.

In the above example, the acceleration signal 300 is indicative of the acceleration experienced by the mobile device 102 relative to the y-axis of the mobile device. In some cases, the acceleration signal 300 may also be indicative of acceleration experienced by the mobile device 102 relative to a plurality of different directions. For example, the acceleration signal 300 may include an x-component, a y-component, and a z-component, which refer to accelerations experienced by the mobile device 102 relative to the x-axis, the y-axis, and the z-axis of the mobile device 102, respectively. Each component may also be referred to as a channel of the acceleration signal (e.g., "x-channel", "y-channel", and "z-channel").

The mobile device 102 can analyze the acceleration signal 300 to determine if the user has fallen. For example, if the user has fallen, the mobile device 102 may experience a relatively strong impact (e.g., when the user's body impacts the ground). Such impacts may be identified based on the magnitude of the acceleration experienced by the mobile device 102 (e.g., the rate of change of the velocity of the mobile device), the magnitude of the jitter experienced by the mobile device (e.g., the rate of change of the acceleration of the mobile device), and the oscillatory behavior of the acceleration experienced by the mobile device 102. Each of these parameters may be determined using the acceleration signal 300.

For example, for each channel of the acceleration signal, the magnitude of the acceleration experienced by the mobile device 102 may be determined using the following relationship:

mag=maximum value (abs (a (n))),

where mag is the magnitude of the acceleration of the channel, a (n) is the nth sample of the acceleration signal 300 of the channel, and the maximum is the sliding window n of samples of the acceleration signal 300 _window Is calculated as the maximum value of the above. In some cases, n _window May correspond to the number of samples spanning an interval of 0.2 seconds or about 0.2 seconds. For example, if the sampling frequency of the acceleration signal 300 is 800Hz, n _window May be 160. In practice, n _window Other values of (2) are also possible.

Alternatively, for each channel of the acceleration signal, the magnitude of the acceleration experienced by the mobile device 102 may be determined using the following relationship:

mag=maximum (a (n)) -min (a (n)),

where mag is the magnitude of the acceleration of the channel, a(n) is the nth sample of the acceleration signal 300 of the channel, the maximum value is the sliding window n of the sample _window The maximum value calculated above, and min is the minimum value calculated over the sample window of the acceleration signal 300, n _window . As described above, in some cases, n _window The number of samples, which may correspond to a time interval spanning 0.2 seconds or about 0.2 seconds, but in practice n _window Other values of (2) are also possible.

If the acceleration signal 300 includes an acceleration measurement relative to a single direction (e.g., has a single channel, such as a y-channel), the above-described relationship may be used to determine the magnitude of the acceleration relative to that direction. The resulting value represents the magnitude of the acceleration signal 300. Alternatively, the window (e.g., n _window ) The total energy of all three channels within can be used as the total value of acceleration. For example, one concept of total energy can be calculated as:

mag=sqrt (maximum (|x|) ² +maximum (|y|) ² +maximum (|z|) ² )。

If the acceleration signal 300 includes acceleration measurements relative to multiple directions (e.g., has multiple channels, such as an x-channel, a y-channel, and a z-channel), the magnitude of the acceleration relative to each direction may be determined one by one using the above relationship, resulting in three individual magnitudes (corresponding to the three channels, respectively). The largest magnitude may be selected to represent the magnitude of the acceleration signal 300.

As another example, for each channel of acceleration, the following relationship may be used to determine the magnitude of jitter experienced by the mobile device 102:

where jerk is the magnitude of the channel jitter, a (n) is the nth sample of the acceleration signal 300 for the channel, and Δt is the sampling interval (e.g., the inverse of the sampling frequency) of the acceleration signal 300.

If the acceleration signal 300 includes an acceleration measurement relative to a single direction (e.g., has a single channel, such as a y-channel), the above-described relationship may be used to determine the magnitude of the jitter relative to that direction. The resulting value represents the magnitude of the jitter of the acceleration signal 300.

If the acceleration signal 300 includes acceleration measurements relative to multiple directions (e.g., has multiple channels, such as an x-channel, a y-channel, and a z-channel), the magnitude of the jitter relative to each direction may be determined one by one using the above relationship, resulting in three individual magnitudes (corresponding to the three channels, respectively). The largest magnitude may be selected to represent the magnitude of the jitter of the acceleration signal 300.

As another example, the oscillatory behavior of the acceleration experienced by the mobile device 102 may be determined by identifying a "third zero crossing" of the acceleration signal 300. The third zero-crossing refers to a point in time at which the acceleration signal 300 changes from negative to positive or vice versa a third time in a particular oscillation cycle (e.g., a third "pass" through zero). For example, within the window shown in fig. 3, a third zero crossing of the acceleration signal 300 occurs at point 302. The time between the maximum value (e.g., point 304) of the acceleration signal 300 and the third zero crossing 302 after the maximum value is referred to as the time to the third zero crossing 306. This value may be used as an estimate of the oscillation period of the acceleration signal 300 and may be used to estimate the oscillation frequency of the acceleration signal 300. For example, this may be useful because the impact response of a component of the motion sensor (e.g., a microelectromechanical system [ MEMS ] component) may be similar to the impact response of a second-order underdamped system. Thus, the period or frequency of the oscillation can be used as an approximation of the response.

The magnitude of the acceleration experienced by the mobile device 102, the magnitude of the jitter experienced by the mobile device 102, and the oscillatory behavior of the acceleration experienced by the mobile device 102 (e.g., the third zero crossing of the acceleration signal 300) may be used in combination to identify an impact that may be indicative of a user falling. In some cases, such a determination may be made using a statistical model or a probabilistic model (e.g., a bayesian model).

For example, fig. 4 shows a heat map 400 of athletic data collected from a sample population of users as they perform daily activities (e.g., "active daily living"). The motion data does not include any motion data caused by a fall of the user. The heat map 400 compares (i) the magnitude of acceleration experienced by the mobile device (x-axis) and (ii) the magnitude of jitter experienced by the mobile device for a sample population (y-axis). In addition, the distribution of motion data relative to each axis in the heat map 400 is shown by a respective cumulative data function 402a or 402 b. As shown in fig. 4, the motion data is mainly concentrated in the region corresponding to the acceleration of a relatively low magnitude in combination with the shake of a relatively low magnitude.

Fig. 4 also shows a plurality of

points

404a and 404b representing motion data collected from a sample population of fallen users. The dot 404a represents a "radial dominant fall" and the square dot 404b represents a "tangential dominant fall". These terms refer to the primary direction of movement of the mobile device 102 when the user falls relative to the oscillating movement of the mobile device 102 when the user walks (e.g., the swinging movement of the user's arms and wrists when the user walks) (e.g., the movement of the user's arms and wrists when the user falls). As shown in fig. 4, each of these

points

404a and 404b is located in an area corresponding to a relatively high magnitude of acceleration in combination with a relatively high magnitude of shake. Thus, an impact that may indicate a user fall may be identified at least in part by identifying motion data that has a sufficiently high magnitude of acceleration (e.g., greater than a particular threshold acceleration value) in combination with a sufficiently high magnitude of jitter (e.g., greater than a particular threshold jitter value). In practice, the threshold acceleration value and/or the threshold jitter value may vary (e.g., based on empirically collected sample data).

As another example, fig. 5 shows a heat map 500 of athletic data collected from a sample population of users as they perform daily activities (e.g., "active daily living"). As with the heat map 400 of fig. 4, the motion data also does not include any motion data caused by a user falling. The heat map 500 compares (i) the magnitude of acceleration experienced by the mobile device (x-axis), and (ii) the time of the third zero crossing of the acceleration signal acquired by the mobile device for the sample population (y-axis). In addition, the distribution of motion data relative to each axis in the heat map 500 is shown by a respective cumulative data function 502a or 502 b. As shown in fig. 5, the motion data is mainly concentrated in the region corresponding to the acceleration of relatively low magnitude in combination with the relatively high time to the third zero crossing of the acceleration signal.

Fig. 5 also shows a plurality of

points

504a and 504b representing motion data collected from a sample population of fallen users. In a similar manner to fig. 4, the dots 504a represent a "radial dominant fall" and the square dots 504b represent a "tangential dominant fall". As shown in fig. 5, each of these

points

504a and 504b is located in an area corresponding to a relatively high magnitude of acceleration in combination with a relatively low time to the third zero crossing of the acceleration signal. Thus, an impact that may indicate a user fall may be identified at least in part by identifying an acceleration having a sufficiently high magnitude (e.g., greater than a particular threshold acceleration value) in combination with sufficiently low motion data to a time of a third zero crossing of the acceleration signal (e.g., less than a particular threshold time value). In practice, the threshold acceleration value and/or the threshold time value may vary (e.g., based on empirically collected sample data).

As described herein, the mobile device 102 may collect motion data and use the motion data to identify a point in time at which the user experienced an impact. Upon identifying the time of impact, the mobile device 102 may analyze motion data acquired during, before, and/or after the impact to determine whether the user has fallen, and if so, whether the user may need assistance. In some cases, mobile device 102 may facilitate such analysis by continuously collecting and buffering motion data. The motion data may include acceleration signals (e.g., acceleration signal 300) describing acceleration experienced by the mobile device 102 and/or orientation signals (e.g., signals representing orientation measurements obtained using the motion sensor 2510, such as using a gyroscope or compass) describing an orientation of the mobile device 102. For example, the mobile device 102 may maintain portions of the acceleration signal and portions of the orientation signal on an ongoing basis, the portions corresponding to time. Is provided. Upon identifying the impact (e.g., using acceleration signals as described above), the mobile device 102 may retrieve portions of the buffered motion data corresponding to measurements taken during, before, and/or after the impact, and analyze those portions to determine the condition of the user.

For example, the mobile device 102 may analyze portions of the acceleration signal and portions of the orientation signal corresponding to measurements taken prior to the identified impact ("pre-impact" information). Based on the analysis, the mobile device 102 can determine whether the impact is likely to correspond to a user falling. In some cases, such a determination may be made using a statistical model or a probabilistic model.

For example, certain types of activities of the user may be more likely to trigger a fall (e.g., walking, running, going up and down stairs, or other such activities). Thus, the activity classifier may be used to identify whether the user is performing one of these types of activities prior to the impact. If so, the likelihood of the user falling down may be higher. Otherwise, the likelihood of the user falling down may be lower.

For example, the activity classifier may be implemented by collecting motion data from a sample population (e.g., acceleration signals and/or orientation signals describing the motion of each individual of the sample population). In addition, information about the physical activity being performed by the sample population may be collected (e.g., an indication of the physical activity being performed by each individual while measuring athletic data, such as walking, running, cycling, golf, boxing, skiing, basketball, or any other activity). This information may be obtained, for example, from a server computer system 104 (e.g., a server computer system for collecting and aggregating data from a plurality of different mobile devices). Based on this information, a particular trait or pattern may be determined for each different type of physical activity. Thus, when motion data is collected from an additional user, the motion data may be compared to previously collected motion data to classify the user's activity.

In addition, the activity classifier may be implemented using a variety of techniques. For example, in some cases, the activity classifier may be implemented using "machine learning" techniques, such as decision tree learning, association rule learning, artificial neural networks, deep learning, inductive logic programming, support vector machine clustering, bayesian networks, reinforcement learning, representation learning, similarity and metric learning, sparse dictionary learning, genetic algorithms, rule-based machine learning, learning classifier systems, and so forth.

As another example, the user's historical activity level may indicate whether the user is more likely to fall and needs assistance. For example, a sports user who is historically highly active (e.g., historically exhibiting frequent and violent movements) may be at a lower risk of falling and needing assistance. However, a frail user that is historically inactive (e.g., historically exhibiting infrequent and slight movements) may be at a higher risk of falling and needing assistance. Thus, the activity level classifier can be used to identify the user's historical activity level, and can adjust the likelihood that the user has fallen and needs assistance based on the classification.

For example, the activity level classifier may also be implemented by collecting motion data from a sample population (e.g., acceleration signals and/or orientation signals describing the motion of each individual of the sample population). Additionally, information about the activity level of the sample population (e.g., an indication of each individual's movement, physical health, and/or activity capabilities when measuring movement data) may be collected. This information may be obtained, for example, from a server computer system 104 (e.g., a server computer system for collecting and aggregating data from a plurality of different mobile devices). Based on this information, a particular trait or pattern may be determined for each different type of activity level. Thus, when motion data is collected from an additional user, the motion data may be compared to previously collected motion data to classify the activity level of the user.

In addition, the activity level classifier may also be implemented using a variety of techniques. For example, in some cases, the activity level classifier may be implemented using "machine learning" techniques, such as decision tree learning, association rule learning, artificial neural networks, deep learning, inductive logic programming, support vector machine clustering, bayesian networks, reinforcement learning, representation learning, similarity and metric learning, sparse dictionary learning, genetic algorithms, rule-based machine learning, learning classifier systems, and so forth.

In some cases, the statistical or probabilistic model may consider additional information about the user, such as the user's age, the user's walking speed, the number of steps the user takes per day, the number of calories the user consumes per day, the user's maximum effort during the day, the user's gender, the user's vascular health (e.g., quantifying the use of a device such as the user's maximum oxygen intake (VO) ₂ max) or other characteristics of the user. The likelihood of a user falling may be increased or decreased based on each of these characteristics. For example, older users may be at a higher risk of falling and needing assistance, while younger users may be at a lower risk of falling and needing assistance. In addition, users with lower maximum walking speeds may be at a higher risk of falling and needing assistance, while users with higher maximum walking speeds may be at a lower risk of falling and needing assistance. In addition, having lower vascular health (e.g., lower VO ₂ max) may be at a higher risk of falling and needing assistance, while having higher vascular health (e.g., higher VO ₂ max) may be at a lower risk of falling and needing assistance. VO (VO) ₂ max may be measured, for example, by measuring ventilation of the user, and the concentration of inhaled and exhaled oxygen and carbon dioxide as the user performs physical activities (e.g., walking on a grading surface, treadmill, manometer, etc.). In some implementations, at least some of these measurements may be obtained using air that includes one or more sensors in and/or communicatively coupled with the mobile device.

As another example, the gender of the user corresponds to a higher or lower risk of falling for assistance.

In addition, a number of different characteristics can be used in combination to determine the fall risk of the user. For example, the statistical model or probabilistic model may include a multi-dimensional risk assessment model (e.g., a multi-dimensional risk matrix), wherein each dimension corresponds to a different characteristic of the user and its contribution to the user's overall risk. Information about each user may be collected from a server computer system 104 (e.g., a server computer system for collecting and aggregating data from a plurality of different mobile devices).

In some cases, the mobile device 102 may consider other types of movement that the user makes prior to the identified impact. As another example, when a user falls, the user often attempts to prop himself up from the ground. Thus, the mobile device 102 may determine whether the user is performing a supporting motion with his arm prior to the identified impact. If so, the likelihood of the user falling down may be higher. Otherwise, the likelihood of the user falling down may be lower.

When performing the supporting movement, the user's hand and, accordingly, the mobile device 102 may accelerate towards the ground for a period of time. Thus, the mobile device 102 may observe positive acceleration in the inertial z-direction (e.g., a direction perpendicular to the ground or a direction of gravity). By measuring the acceleration in the inertial z-direction, the mobile device 102 can determine the duration of the fall. For example, the duration of a fall may be estimated as the period of time in which the acceleration signal in the inertial z-direction is above a certain threshold.

As another example, when a user falls, he often swings his arm. For example, if a user slips, trips, or rolls over while walking, the user may irregularly or swiftly move his arm before striking the ground in an attempt to balance himself. The mobile device 102 may then determine whether the user is swinging his arm prior to the identified impact. If so, the likelihood of the user falling down may be higher. Otherwise, the likelihood of the user falling down may be lower.

The wiggle motion may be detected in part by estimating a "pose angle" of the mobile device or an orientation relative to an inertial system (e.g., the earth system). This may be determined, for example, using both the acceleration signal (e.g., to identify the direction of gravity or the inertial z-direction) and the orientation signal (e.g., to determine the orientation of the device relative to the direction of gravity). Using this information, a change in the attitude angle of the mobile device 102 over time may also be determined (e.g., over a sample window such as n _window The largest difference in internal attitude angle). A relatively large change in the pose angle over a period of time (e.g., a large change in the orientation of the mobile device 102, and accordingly, the orientation of the user's wrist and arm) may be indicative of useThe user is more likely to be performing a swinging motion.

In some cases, the amount of "confusion" in the user's motion may be determined by acquiring an acceleration signal corresponding to the user's motion during a period of time and determining the path length of the acceleration signal. When the user moves in an irregular manner (e.g., performs a swinging motion), the acceleration signal will exhibit a higher degree of variation during this period of time. Therefore, the path length of the acceleration signal will be longer. In contrast, when the degree of user movement irregularity is low, the acceleration signal will exhibit a smaller degree of variation during this period of time. Therefore, the path length of the acceleration signal will be shorter.

In some cases, the path length of the acceleration signal may be calculated using the following formula:

(path length of acceleration signal) = Σ|a _n -a _n-1 |

Wherein a is _n Is the nth sample in the acceleration signal. The path length may be determined by summing samples taken within a sliding window (e.g., a 1.4 second window around a fall or bump). In some cases, other techniques may be used to determine clutter (e.g., by measuring entropy of acceleration signals).

This "pre-impact" information can be used in combination to determine if the user is likely to have fallen. For example, fig. 6 shows a heat map 600 of athletic data collected from a sample population of users as they perform daily activities (e.g., "active daily living"). The motion data does not include any motion data caused by a fall of the user. The heat map 600 compares (i) the change in attitude angle of the mobile device (x-axis), and (ii) the fall duration of the mobile device for the sample population (y-axis). In addition, the distribution of motion data relative to each axis in the heat map 600 is shown by a respective cumulative data function 602a or 602 b. As shown in fig. 6, the motion data is mainly concentrated in the region corresponding to the relatively low change in attitude angle in combination with the relatively low fall duration.

Fig. 6 also shows a plurality of points 604 representing movement data collected from a sample population of fallen users, wherein the movement data was collected immediately prior to the user falling. As shown in fig. 6, each of these points 604 is located in an area corresponding to a relatively high change in attitude angle in combination with a relatively high fall duration. Thus, a user fall can be identified at least in part by identifying motion data (collected immediately prior to an identified impact) that has a sufficiently large change in attitude angle (e.g., greater than a particular threshold angle) in combination with a sufficiently long fall duration (e.g., greater than a particular threshold amount of time). In practice, the threshold angle and/or the threshold amount of time may vary (e.g., based on empirically collected sample data).

The mobile device 102 may also analyze portions of the acceleration signal and portions of the orientation signal corresponding to measurements taken after the identified impact ("post-impact" information). Based on the analysis, the mobile device 102 can determine whether the user may need assistance. In some cases, such a determination may be made using a statistical model or a probabilistic model.

For example, if the user falls and is injured or in distress from the fall, the user may exhibit signs of trauma and/or athletic injury. Signs of trauma and/or athletic injury may be identified based on the acceleration signal and/or the orientation signal. For example, the mobile device 102 may analyze portions of the acceleration signal and portions of the orientation signal corresponding to measurements taken after the identified impact. Based on the analysis, the mobile device 102 may determine whether the impact may correspond to signs of trauma and/or athletic injury.

The analysis may be performed, for example, using an activity classifier. For example, the activity classifier may be implemented by collecting motion data from a sample population after the sample population falls (e.g., acceleration signals and/or orientation signals describing the motion of each individual of the sample population after the sample population falls). In addition, information about the condition of each individual after a fall may be collected (e.g., an indication that some individuals show signs of trauma after a fall, an indication that some individuals show motor impairment after a fall, etc.). This information may be obtained, for example, from a server computer system 104 (e.g., a server computer system for collecting and aggregating data from a plurality of different mobile devices). Based on this information, specific characteristics or patterns (e.g., signs of trauma, no signs of trauma, athletic injury, no athletic injury, etc.) may be determined for each different type of condition. Thus, when motion data is collected from an additional user, the motion data can be compared to previously collected motion data to classify the user's condition after a fall.

In a manner similar to that described above, the activity classifier may be implemented using various techniques (e.g., decision tree learning, association rule learning, artificial neural networks, deep learning, inductive logic programming, support vector machine clustering, bayesian networks, reinforcement learning, representation learning, similarity and metric learning, sparse dictionary learning, genetic algorithms, rule-based machine learning, learning classifier systems, and so forth).

The mobile device 102 can also determine the user's condition after a fall by identifying one or more motion-based metrics. For example, based on the acceleration signal and/or the orientation signal, the mobile device 102 can determine whether the user has walked several steps after the fall (e.g., by identifying trends or patterns in the signal that represent the user's walking, and/or using a sensor such as a pedometer). If so, the likelihood that the user will need assistance is low. Otherwise, the likelihood that the user needs assistance is high.

As another example, based on the acceleration signal and/or the orientation signal, the mobile device 102 can determine whether the user has been standing after a fall (e.g., by identifying trends or patterns in the signal that represent the user standing up from the ground). If so, the likelihood that the user will need assistance is low. Otherwise, the likelihood that the user needs assistance is high.

As another example, mobile device 102 can determine a standard deviation of the acceleration vector and a duration for which the magnitude is below a threshold. If the magnitude of the acceleration vector is below a threshold value for a period of time, this may indicate that the user is stationary (e.g., does not move his body). In addition, if the user is stationary after his fall, this may indicate that the user is syncoped or injured. Thus, the likelihood that the user will need assistance is high. However, if the standard deviation and magnitude of the acceleration vector exceeds the threshold for that period of time, this may indicate that the user moves his body after his fall (e.g., the user is not stationary). Thus, the likelihood that the user will need assistance is low.

For example, as shown in fig. 7, the mobile device 102 may acquire a "raw" acceleration signal 700 (e.g., in a similar manner as described above with respect to the acceleration signal 300). In the example shown in fig. 7, the acceleration signal 700 contains three channels, each referencing acceleration measured according to a different direction (e.g., x-channel, y-channel, and z-channel) relative to the mobile device 102. In addition, the acceleration signal 700 exhibits a sharp increase 702 indicative of an impact.

In addition, the mobile device 102 can calculate a Vector Magnitude (VM) of the acceleration signal 700 to obtain the vector magnitude signal 704. In practice, when the mobile device 102 is stationary (e.g., not moving), the vector magnitude signal 704 will be 1 (or about 1). As the mobile device 102 moves, the vector magnitude signal 704 will deviate from 1. Thus, the relationship |VM-1| can be used as a measure of the stall. Similarly, the standard deviation of the acceleration signal may also be used as a measure of the dwell.

In addition, the mobile device 102 may identify a time period 706 after the impact during which the magnitude of the normalized acceleration signal 704 is less than a particular threshold (indicated by a black rectangle). In some cases, the threshold may be 0.025g. However, in practice, this threshold may vary depending on the implementation.

In addition, the mobile device 102 can sum each of the time periods 706 to determine a cumulative duration 708 that the magnitude of the normalized acceleration signal 704 is below a threshold. If the cumulative duration 708 is relatively large, the likelihood that the user will need assistance is high. However, if the cumulative duration 708 is relatively small, the likelihood that the user will need assistance is low.

As another example, based on the acceleration signal and/or the orientation signal, the mobile device 102 may determine a pose angle of the mobile device 102. The attitude angle may be determined, for example, based on the acceleration signal (e.g., the filtered acceleration signal) and/or the motion orientation of the mobile device 102 (e.g., information derived from a fusion of acceleration data and gyroscope data). The filtered acceleration signal may be, for example, an acceleration signal having one or more channels (e.g., an x-channel, a y-channel, and/or a z-channel) in which high frequency content is removed (e.g., greater than a particular threshold frequency). In some cases, the removed high frequency content may correspond to machine-induced motion (e.g., buses, trains, etc.). Based on these information sources, mobile device 102 can determine a direction in which the wearer's forearm points (e.g., approximately the attitude angle of mobile device 102 in the x-direction).

If the wearer's forearm is pointed at the horizon for a longer period of time (e.g., greater than a threshold amount of time) and there is no movement below or above the horizon, the user may be more likely to be injured. For example, this may indicate that the user is lying on the floor and does not move his arm up or down. However, if the user's forearm is repeatedly moved above and below the horizontal line with a relatively large motion (e.g., 45 °), this may indicate that the user is less injured, especially if the user is determined to be ascending and descending stairs.

As another example, based on the acceleration signal and/or the orientation signal, the mobile device 102 may determine a number of times the acceleration signal (e.g., the filtered acceleration signal) crosses a given threshold T. This metric may be referred to as a threshold crossing number. As described above, the filtered acceleration signal may be, for example, an acceleration signal having one or more channels and having high frequency content removed. Symmetrical crossing requires that the acceleration signal both rise above T and fall below (-T) (or vice versa) while the asymmetric crossing statistic signal varies by more than + -T times, whether or not it crosses + -T afterwards.

These threshold crossings represent the movement of the person. For example, a user walking will typically generate a symmetric threshold crossing, while a user touching something or rotating the wrist will typically result in an asymmetric crossing, and so on. By counting the threshold number of crossings, mobile device 102 can determine whether an individual is likely to be injured. For example, the greater the threshold number of crossings, the less likely a user is injured.

In some cases, this "post-impact" information may be used in combination to determine if the user may need assistance. For example, the mobile device 102 can determine the condition of the user after their fall based on sample data collected from a sample population. For example, the mobile device may collect motion data (e.g., acceleration signals and/or orientation signals) from a sample population. In addition, information about the condition of each individual may be collected (e.g., an indication that some individuals exhibit signs of trauma when measuring athletic data, an indication that some individuals exhibit athletic injuries when measuring athletic data, etc.). This information may be obtained, for example, from a server computer system 104 (e.g., a server computer system for collecting and aggregating data from a plurality of different mobile devices).

Using this information, one or more correlations may be determined between the characteristics of the user's movements and the user's condition. For example, based on sample data collected from a sample population, a correlation may be determined between one or more particular characteristics of user movement corresponding to users exhibiting signs of trauma. Thus, if mobile device 102 determines that the user's motion exhibits similar characteristics, mobile device 102 may determine that the user also exhibits signs of trauma. As another example, based on sample data collected from a sample population, a correlation may be determined between one or more particular characteristics of user movement corresponding to users exhibiting athletic injuries. Thus, if mobile device 102 determines that the user's motion exhibits similar characteristics, mobile device 102 may determine that the user also exhibits motion impairment.

These correlations may be determined using various techniques. For example, in some cases, these correlations may be determined using "machine learning" techniques such as decision tree learning, association rule learning, artificial neural networks, deep learning, inductive logic programming, support vector machine clustering, bayesian networks, reinforcement learning, representation learning, similarity and metric learning, sparse dictionary learning, genetic algorithms, rule-based machine learning, learning classifier systems, and the like.

For example, fig. 8 illustrates a decision tree 800 generated using sample data collected from a sample population (e.g., using sample data "training"). In this example, the sample data includes "positive control" information collected from a sleeping user (e.g., simulating the behavior of an individual exhibiting signs of trauma or motor impairment). The sample data includes "negative control" information collected from an intact user performing daily physical activity.

As shown in fig. 8, certain combinations of characteristics indicate that the user exhibits signs of trauma and/or athletic injury. For example, if (i) the magnitude of the acceleration signal is very low for a duration (parameter "duration_low_vm") of between 48.1 seconds and 51.2 seconds, and (ii) the user takes less than 0.5 steps (parameter "steps"), it may be determined that the user is exhibiting signs of trauma and/or athletic injury.

As another example, if (i) the magnitude of the acceleration signal is low for a duration (parameter "duration_low_vm") between 51.2 seconds and 58.9 seconds, (ii) the user takes less than 0.5 steps (parameter "steps"), and (iii) the user stands for less than 34.7 seconds (parameter "stand"), it may be determined that the user is exhibiting signs of trauma and/or athletic injury. In the example shown in fig. 8, the duration of time that the user stands during the post-fall period (parameter "stand") is indicated by an integer between 0 and 600, which refers to the duration (in tenths of a second) within the total sample window of 60 seconds. For example, if the "stand" value is 347, this indicates that the user has been standing 34.7 seconds after falling within a sample window of 60 seconds. In this example, this particular branch indicates that "sleep" (positive control) may be present if the user is standing for a time between 0 and 34.7 seconds (e.g., the user is standing for less than or equal to about half of the period of time after the fall). In practice, this reflects the expectation that the user will not stand after a fall.

As another example, if (i) the magnitude of the acceleration signal is low for a duration ("duration_low_vm") of between 51.2 seconds and 58.9 seconds, and (ii) the number of steps taken by the user is greater than 0.5 steps ("steps"), it may be determined that the user does not exhibit signs of trauma and/or athletic injury. Although example decision branches and parameter values are shown in fig. 8, these are merely illustrative examples. In practice, the decision branches of the decision tree and/or the parameter values may vary depending on the implementation.

In the above example, the mobile device 102 determines whether the user may need assistance based on the acceleration signal and the orientation signal. However, the mobile device 102 is not limited to using only these types of information. For example, in some cases, the mobile device 102 may consider additional information, such as a signal describing the heart rate of the user. For example, the mobile device 102 may determine that the user is more likely to need assistance when the user's heart rate deviates from a particular range (e.g., a "normal" or "healthy" range).

In some cases, the "pre-impact" information, the impact information, and the "post-impact" information may be used in combination to determine whether the user has fallen and may need assistance. In some cases, this determination may be made using a decision tree.

For example, fig. 9 shows a decision tree 900 for determining that (i) the user has fallen and may need help (indicated as "falling" in decision tree 900) or (ii) the user has not fallen, or has fallen but does not need help (indicated as "not falling" in decision tree 900). As shown in fig. 9, certain combinations of features indicate each result.

For example, if the user experiences a high impact (e.g., greater than a first threshold) and exhibits signs of trauma and/or athletic injury after the impact, it may be determined that the user has fallen and may need assistance. As another example, if the user experiences a moderate impact (e.g., a second threshold that is less than the first threshold, but greater than the first threshold), certain activities are being performed prior to the impact (e.g., walking, climbing stairs, etc.), swinging or supporting movements are being performed prior to the impact, and signs of trauma and/or athletic injuries are exhibited after the impact, it may be determined that the user has fallen and may need assistance. Otherwise, it may be determined that the user has not fallen, or has fallen but does not need assistance. Although example decision branches are shown in fig. 9, these are merely illustrative examples. In practice, the decision branches of the decision tree may vary depending on the implementation.

In some cases, the fall detector may be a bayesian classifier. For example, the posterior probability of a fall can be calculated given a set of features:

p(fall|f1，f2，f3，...)＝p(f1|fall)p(f2|fall)p(f3|fall)...p(fall)/p(f1，f2，f3，...)，

where f1, f2, f3, · is a feature calculated from the acceleration signal and the orientation signal (e.g., an impact magnitude, a shake, a time to third zero crossing, a pre-impact activity, presence of support or swing, and/or a post-impact feature determined from the sample population), and p (fall) is an a priori probability of falling, which may be determined based on age, gender, and/or other biometric information from the sample population. Similarly, the posterior probability of an impact that is not a fall can be calculated as:

(ADL|f1，f2，f3，...)＝p(f1|ADL)p(f2|ADL)p(f3|ADL)...p(ADL)/p(f1，f2，f3，...)，

where ADL represents activities of daily living that do not include any instance of a user falling. When the ratio p (fall|f1, f2, f3,)/p (adl|f1, f2, f3,) is greater than the threshold, the mobile device 102 can determine that a fall has occurred. In practice, the threshold may vary depending on the implementation.

Other types of information may also be used to determine if the user has fallen. For example, as discussed above, unstable motion over a period of time may indicate a fall to a greater extent (e.g., a user may swing their arm when falling). However, periodic movement over a period of time may indicate to a greater extent intentional movement of the user (e.g., periodic movement of an arm during handshaking, chopping food, etc.). Thus, the periodicity of the user movement can be used to determine whether the user has fallen.

In some cases, the periodicity of the user motion may be determined based on a first periodicity metric corresponding to a Fast Fourier Transform (FFT) of coherent energy of the acceleration signal within a time window (e.g., 10 seconds) after the detected impact. The coherent energy is the sum of peaks in the spectrum having a mass greater than a certain threshold. A larger first periodicity measure may indicate a larger movement periodicity, which may correspond to a lower likelihood that the user has fallen. A lower first periodicity metric may indicate a smaller movement periodicity, which may correspond to a higher likelihood that the user has fallen.

In some cases, the periodicity of the user motion may be determined based on a second periodicity metric corresponding to a quarter bit spacing (IQR) of peak-to-peak intervals of the acceleration signal within a time window (e.g., 10 seconds) after the detected impact. This may be calculated by identifying all peaks around the detected impact that are greater than a particular threshold, and then calculating the IQR of the spacing between adjacent peaks. If the repeated peak intervals are equal (or substantially equal), the IQR will be smaller, which indicates a periodic movement. Thus, a lower second periodicity measure may indicate a greater movement periodicity, which may correspond to a lower likelihood that the user has fallen. In contrast, a larger second periodicity metric may indicate a smaller movement periodicity, which may correspond to a higher likelihood that the user has fallen.

In some cases, the mobile device 102 can be used to distinguish between different types of falls. For example, the mobile device 102 can distinguish between tripping (e.g., a user tripping forward, such as a fall when their feet are captured by an obstacle), sliding (e.g., a user falling backward, such as a fall out of balance on a slippery surface), tumbling (e.g., a user rotating or tumbling about an axis of rotation, such as a fall when the user rolls down a mountain or stairs), and/or other types of falls. For example, this may be useful because different types of falls may have different effects on the user (e.g., some may be more harmful to the user, while other types may be less harmful to the user). Thus, the mobile device 102 can take more specific actions in response to the particular nature of the user's fall.

As described herein, the mobile device may determine whether the user has fallen based at least in part on a "pose angle" or orientation of the mobile device relative to an inertial frame (e.g., the earth frame). In some cases, this may be determined using information acquired by an accelerometer (e.g., determined by using an acceleration signal generated by the accelerometer to identify a direction of gravity or an inertial z-direction) and/or information from an orientation sensor (such as a gyroscope) (e.g., by using an orientation signal generated by the gyroscope to determine an orientation of the device relative to the direction of gravity).

In some cases, the mobile device may distinguish between stumbling and sliding based on changes in the attitude angle of the mobile device before and/or after the impact, and determining whether the measured attitude angle change indicates stumbling or falling. For example, when a user trips, he often moves his arm downward to support against the impact. Thus, if the mobile device is attached to the user's wrist, it may experience a negative change in attitude angle prior to impact. In addition, the attitude angle may be directed toward the ground when falling. As another example, when a user slips, he often throws his arm upward in an attempt to restore his balance. Thus, if the mobile device is attached to the user's wrist, it may experience a positive change in attitude angle prior to impact.

These characteristics can be used to distinguish tripping and slipping. For example, fig. 10 shows a scatter plot 1000 of pose angle data collected from a sample population of users experiencing different types of falls using a mobile device attached to each user's wrist. Each point 1002 indicates a particular type of fall (e.g., stumbling, sliding, or other type of fall), a corresponding change in attitude angle (e.g., in degrees) of the mobile device before the impact (shown in the x-axis), and a corresponding change in attitude angle of the mobile device after the impact (shown in the y-axis). The sign of the change in attitude angle may indicate the direction of motion. For example, a positive attitude angle change may indicate that the final attitude angle of the mobile device is higher than the initial attitude angle during the considered time window. Likewise, a negative value of the change in the attitude angle may indicate that the final attitude angle of the mobile device is lower than the initial attitude angle. As shown in fig. 10, when the user experiences a trip, the mobile device moves in a downward direction before striking because the user is preparing to support and then moves upward after striking the surface. Therefore, tripping often has a negative change in attitude angle before impact and a positive change in attitude angle after impact. In contrast, when the user experiences slipping, the mobile device moves in an upward direction because the user swings his arm before the impact and then moves downward after the impact. Thus, a slip often has a positive change in attitude angle before impact and a negative change in attitude angle after impact. The mobile device may then distinguish between stumbling and slipping based on changes in the attitude angle of the mobile device before and/or after the impact, and determining whether the measured attitude angle change indicates stumbling or a fall. In addition, if the magnitude of the change in attitude angle in either direction is insufficient to convincingly indicate a tripping or slip-like fall behavior, the fall can be classified as the inclusion category of "other" falls. This may be the case, for example, in the case of falls where the user syncs due to dehydration, in which case there may be no apparent swing or supporting movement prior to impact when the user is unconscious.

In some cases, the system can determine whether the user has fallen (e.g., fallen in a manner that the user has rotated or scrolled around a rotational axis). For example, this may be beneficial because a roll-over may be particularly harmful to the user and may correspond to a higher likelihood that the user needs assistance. Example tumblers include a user falling and rolling down stairs or hills, a user falling downhill (e.g., partially or fully tumbling a bucket), a user "tumbling" or another fall involving some degree of rolling or rotation.

As described above, the user 110 may position the mobile device 102 on his body and walk around in his daily life (e.g., walk, run, sit down, lie down, engage in sports or athletic activity, or any other physical activity). During this time, mobile device 102 collects sensor data regarding the movement of user 110. For example, using the motion sensor 2510 (e.g., one or more accelerometers) shown in fig. 25, the mobile device 102 can measure the acceleration experienced by the motion sensor 2510 and, accordingly, the acceleration experienced by the mobile device 102. In addition, using motion sensor 2510 (e.g., one or more compasses or gyroscopes), mobile device 102 can measure an orientation of mobile device 102. In some cases, the motion sensor 2510 may collect data continuously or periodically over a period of time or in response to a triggering event. Additionally, the mobile device 102 can determine acceleration and/or orientation relative to a reference frame (e.g., body frame) fixed to the mobile device 102 and/or relative to an inertial frame (e.g., earth frame).

The mobile device 102 can continuously measure acceleration and orientation of the mobile device within the sliding sample window (e.g., to generate a continuous sample buffer). The acceleration signal may be used to identify a direction of gravity (or inertial z-direction), and the orientation signal may be used to determine an orientation of the mobile device 102 relative to the direction of gravity. Using this information, mobile device 102 can determine a pose angle of mobile device 102 (approximately the orientation of user 110). In addition, using this information, the rate of rotation of the mobile device 102 may be determined relative to the body and inertial frame (approximately the rate of rotation of the user 110 relative to the body and inertial frame).

For example, graph 1100 in fig. 11A shows an acceleration signal 1102 acquired over a sliding sample window spanning approximately 2.5 seconds to 7.5 seconds, and graph 1104 in fig. 11B shows a corresponding rotation rate signal 1106 relative to the inertial frame over the same sliding sample window. In this example, the user has fallen a flight of stairs at about the 3 second mark, contacted the stairs and/or railing between about the 3 second mark and the 6.5 second mark, and began to roll down the stairs at the 6.5 second mark. The graph 1100 includes three different acceleration signals 1102, each corresponding to a different direction (e.g., an x-direction, a y-direction, and a z-direction in a Cartesian coordinate system in a reference or body system fixed to the mobile device 102). Similarly, graph 1104 includes three different rotation rate signals 1106, each corresponding to a different direction (e.g., x-direction, y-direction, and z-direction in a Cartesian coordinate system fixed to an inertial frame). Although a sliding sample window having an example length is shown, in practice the sliding sample window may have any length (e.g., 1 second, 2 seconds, 3 seconds, 4 seconds, or any other length of time). In addition, different reference frames may be used in addition to the Cartesian coordinate system. For example, in some cases, a quaternary coordinate system may be used.

In addition, using this information, the mobile device 102 can determine one or more instantaneous axes of rotation of the mobile device 102 within the sliding sample window, an average axis of rotation of the mobile device within the sliding sample window, and a degree of uncertainty (e.g., a variance value, a standard deviation value, or other measure of uncertainty) associated with the average axis of rotation. For example, graph 1108 in fig. 11C shows a rotation axis signal 1110 that indicates the instantaneous rotation axis of mobile device 102 relative to the inertial frame at any given point in time in the sliding sample window. An angle of 0 indicates that the mobile device 102 is rotating along an axis parallel to the direction of gravity at a particular point in time, while an angle of 90 indicates that the mobile device is rotating along an axis perpendicular to the direction of gravity at a particular point in time. The signal may be used to determine an average rotation axis of the mobile device within the sliding sample window and a degree of uncertainty associated with the average rotation axis (e.g., by averaging values of the signal and determining a variance, standard deviation, or other measure of uncertainty based on the value of the signal).

In addition, the mobile device 102 can determine whether the mobile device 102 is rotating within the sliding sample window relative to a uniform axis of rotation. For example, if the change or deviation between one or more instantaneous axes of rotation of the mobile device 102 and the average axis of rotation during sliding of the sample window is low, the mobile device 102 may determine that the mobile device 102 is rotating more consistently about a particular axis of rotation. However, if the change or deviation between one or more instantaneous axes of rotation of the mobile device 102 and the average axis of rotation during sliding of the sample window is high, the mobile device 102 may determine that the mobile device 102 is rotating less consistently about a particular axis of rotation. In some cases, if the change or deviation between one or more instantaneous axes of rotation of the mobile device 102 and the average axis of rotation during the sliding sample window is below a particular threshold, the mobile device 102 may determine that it is rotating relative to a consistent axis of rotation within the sliding sample window.

If the mobile device 102 determines that the mobile device 102 is rotating within the sliding sample window about a uniform axis of rotation, the mobile device 102 may determine an angular displacement of the mobile device 102 relative to the sliding sample window. This may be performed, for example, by angular integration of the rotational speed signal with respect to the inertial frame within a sliding sample window. For example, graph 1112 in fig. 11D shows a total angular displacement signal 1014 corresponding to a total angular displacement of mobile device 102 relative to the inertial frame during a sliding time period. The total angular displacement signal 1014 may be obtained, for example, by integrating one or more of the rotation rate signals 1006 shown in fig. 11B based on the sliding.

The mobile device 102 can determine whether the user has fallen based on the total angular displacement of the mobile device 102 and the instantaneous axis of rotation. For example, as shown in fig. 11C, the instantaneous axis of rotation of the mobile device 102 is relatively stable starting from about the 6.5 second mark (corresponding to the time the user begins to roll down stairs). In addition, during this time, the axis of rotation is always approximately 90 °, which indicates that the user is scrolling along an axis of rotation that is approximately perpendicular to the direction of gravity. In addition, as shown in fig. 11D, the mobile device 102 experiences a relatively large angular displacement over approximately the same period of time. A combination of these characteristics may indicate a roll-over.

In some cases, the mobile device 102 may determine that the user has flipped if one or more instantaneous axes of rotation are greater than a threshold angular value and/or if the angular displacement is greater than a threshold displacement value. In some cases, mobile device 102 may determine that the user has fallen if one or more instantaneous axes of rotation are relatively large (e.g., greater than a threshold angular value) and/or remain consistent for a threshold period of time (e.g., approximately 90 ° in 1 second, 2 seconds, 3 seconds, or some period of time).

In some cases, mobile device 102 can also identify different types of flip. For example, if one or more instantaneous axes of rotation are approximately 90 ° with respect to the direction of gravity, this may represent a user planting a spring onion upside down or a roll-over (e.g., a flip bucket or tumble). As another example, if one or more instantaneous axes of rotation are between 0 ° and 90 ° with respect to the direction of gravity, this may represent a user twisting a fall while falling.

In a similar manner as described above, upon identifying that the user has rolled over and may need assistance, mobile device 102 may automatically take appropriate action in response. For example, mobile device 102 can automatically notify the user or other person of this (e.g., emergency response system, emergency contact, or others). Similarly, in some cases, mobile device 102 may first notify the user before transmitting a message to others (e.g., before transmitting a notification to an emergency response system, emergency contact, or others) upon determining that the user has rolled over and may need assistance.

Implementations described herein are not limited to detecting a user's rollover. In some cases, one or more implementations may be used to determine an intentional roll-over or rotation of a user. For example, the device may use the rotation data to determine whether the user has turned a bucket or rolled (e.g., as part of an athletic activity such as gymnastics). As another example, the device may use the spin data to determine whether the user has performed a tumbling turn during swimming (e.g., counting the number of swimming strokes performed by the user in a swimming pool). Indeed, other applications are possible, depending on the implementation.

Upon identifying that the user has fallen and may need assistance, the mobile device 102 may automatically take appropriate action in response. For example, the mobile device 102 can determine that the user has fallen and may need assistance, and in response, automatically notify the user or other person of the situation. For example, the mobile device 102 can display a notification to the user to inform the user that he has fallen and that assistance may be needed. In addition, mobile device 102 can transmit a notification to a remote device (e.g., one or more of server computer system 104 and/or communication device 106) to notify others of the user's condition. This may include, for example, notifying an emergency response system, a computer system associated with a medical professional, a computer system associated with a caregiver of the user, a bystander, etc. The notification may include, for example, audible information (e.g., sound), textual information, graphical information (e.g., images, colors, patterns, etc.), and/or tactile information (e.g., vibration). In some cases, the notification may be transmitted in the form of an email, an instant chat message, a text message (e.g., a short message service [ SMS ] message), a telephone message, a facsimile message, a radio message, an audio message, a video message, a haptic message (e.g., one or more jolts or vibrations), or another message for conveying the information.

In some cases, the mobile device 102 may transmit the message to another system using predetermined contact information. For example, a user of mobile device 102 may provide contact information about an emergency contact, such as a phone number, an email address, a user name in an instant chat service, or some other contact information. Mobile device 102 can generate a message having a compatible data format (e.g., an audio telephone message, a video telephone message, a text message, an email message, a chat message, or some other message) and transmit the message to the emergency contact using the provided contact information (e.g., using one or more of server computer system 104 and/or communication device 106).

In some cases, upon determining that the user has fallen and may need assistance, mobile device 102 may first notify the user before transmitting the message to the other person (e.g., before transmitting a notification to an emergency response system, emergency contact, or other person). In response, the user may instruct the mobile device 102 not to notify others (e.g., if the user does not need assistance). Alternatively, the user may explicitly instruct the mobile device 102 to notify others (e.g., if the user needs assistance). In some cases, if the user does not respond to the notification, the mobile device 102 may automatically notify others (e.g., in the event that the user is disabled and unable to respond). In some cases, mobile device 102 may notify the user a number of different times and automatically notify others to obtain assistance if no response is received from the user after a period of time (e.g., 25 seconds or some other period of time).

In some cases, mobile device 102 can determine whether to generate a notification using a state machine and transmit the notification to others. The state machine may specify that the mobile device send a fall alert when a short dwell period (behavior that typically occurs after a fall) is observed. The mobile device then detects disability by checking for a "long" period of the user (e.g., a period of time the user is not moving or standing up). If a disability is detected, the mobile device may automatically transmit a distress call to a third party and/or instruct another device to transmit a distress call. The user may also cancel the distress call (e.g., if the user believes he does not need assistance).

An example state machine 1200 is shown in fig. 12. In this example, the mobile device begins with a "normal" state 1202 (e.g., a low alert state of the mobile device), a "possible fall" state 1204 (e.g., an elevated alert state of the mobile device when a user is detected to be likely to fall), a "waiting"

state

1206a and 1206b (e.g., a state in which the mobile device waits for additional information such as a user's possible input or user movement), a "alert"

state

1208a and 1208b (e.g., a state in which the mobile device alerts the user of the mobile device to be likely to fall and possibly sends a distress call to a third party), a "cancel" state 1210 (e.g., a state in which an upcoming distress call is cancelled), and a "SOS" state 1212 (e.g., a state in which a distress call is performed). The mobile device can transition between each state based on certain indications of a detected fall (e.g., as described herein), a detected dwell period (e.g., no movement by the user), and/or user input.

For example, the mobile device begins in a "normal" state 1202. Upon detection of a fall flag (e.g., a combination of sensor measurements and other information indicative of a fall), the mobile device transitions to a "possible fall" state 1204. Pause period T after fall detection _Q When the mobile device transitions to an "alert" state 1208b, alerting the user to detect a fall and informing the user that a distress call may be sent to a third party (e.g., an emergency responder). The mobile device transitions to a "wait" state 1206b and waits for possible movement by the user. If no user movement is detected during the elapse of a "timeout" period (e.g., 30 seconds) after the transmission of the fall alert, and a continuous dwell period (e.g., T) is detected during that time _LL =10 seconds), the mobile device transitions to the "SOS" state 1212 and transmits a distress call. The mobile device then returns to the "normal" state 1202. This may be useful, for example, if the user has fallen and is incapacitated for a long period of time. The mobile device may automatically seek assistance for the user even without user input.

As another example, the mobile device begins in a "normal" state 1202. Upon detection of a fall flag, the mobile device transitions to a "possible fall" state 1204. Pause period T after fall detection _Q When that is the case, the mobile device transitions to an "alert" state 1208b and alerts the user to detect a fall and informs the user that a distress call may be sent to a third party. The mobile device transitions to a "wait" state 1206b and waits for possible movement by the user. If user movement is detected before a "timeout" period (e.g., 30 seconds) has elapsed after the transmission of the fall alert, the mobile device transitions to a "cancel" state 1210 and cancels the distress call. The mobile device then returns to the "normal" state 1202. This may be useful, for example, if the user has fallen and has indicated with certainty that he does not need assistance. In this case, the mobile device does not automatically seek assistance for the user.

As another example, the mobile device begins in a "normal" state 1202. Upon detection of a fall flag, the mobile device transitions to a "possible fall" state 1204. Upon detecting some type of movement (e.g., walking movement, standing movement, highly dynamic movement, or any other movement that indicates user recovery) after a fall, the mobile device transitions to a "waiting" state 1206a. Detecting a dwell period T after the user stops moving _Q When that is the case, the mobile device transitions to an "alert" state 1208a and alerts the user to the detected fall. The mobile device transitions to the "cancel" state 1210 and does not transmit a distress call. The mobile device then returns to the "normal" state 1202. This may be useful, for example, if the user has fallen but shows signs of recovery. The mobile device can alert the user about a fall, but in this case will not automatically seek assistance for the user.

As another example, the mobile device begins in a "normal" state 1202. Upon detection of a fall flag, the mobile device transitions to a "possible fall" state 1204. Upon detecting some type of movement (e.g., walking movement, standing movement, highly dynamic movement, or any other movement that indicates user recovery) after a fall, the mobile device transitions to a "waiting" state 1206a. The pause period T is not detected after 25 seconds have elapsed _Q At this point, the mobile device returns to the "normal" state 1202. The mobile device then returns to the "normal" state 1202. For example, if the user has alreadyIt may be useful to fall but show signs of recovery and continue to move long after the fall. In this case, the mobile device may avoid alerting the user or automatically seek assistance for the user.

Although time values are shown in fig. 12, these are merely illustrative examples. In practice, one or more of the time values may vary depending on the particular implementation. In some cases, one or more of the time values may be an adjustable parameter (e.g., a parameter empirically selected to distinguish between different types or events or conditions).

In some cases, the response sensitivity of the mobile device 102 may vary depending on the characteristics of the user. For example, the mobile device 102 can determine a probability that the user has fallen and may need assistance. If the probability is greater than a threshold level, the mobile device can automatically notify the user and/or others of the user's fall. The threshold level may vary based on each particular user. For example, if the user is at a higher risk of falling, the threshold level may be lower (e.g., so that the mobile device 102 is more likely to notify the user and/or others of the fall). However, if the user is at a lower risk of falling, the threshold level may be higher (e.g., making the mobile device 102 less likely to notify the user and/or others of the fall). The threshold level for each user may be varied based on one or more behaviors and/or demographics of the user, such as the user's age, activity level, walking speed (e.g., maximum walking speed observed over a period of time), or other factors.

For example, fig. 13 shows a schematic representation 1300 of a fall detection technique based on user-specific sensitivity. The mobile device receives the motion data 1302 and calculates behavioral characteristics of the user's motion (e.g., swing motion, support motion, chaotic motion, periodic motion, or other characteristics as described herein) (1304). In addition, the mobile device determines a sensitivity threshold based on the user's behavior and/or demographics (1306). The mobile device determines whether the user has fallen and/or is transmitting a distress call based on the determined characteristics and thresholds (e.g., using the fall detector 1308 to perform one or more of the techniques described herein).

As described herein, the mobile device may determine whether the user has fallen based at least in part on a "pose angle" or orientation of the mobile device relative to the inertial frame. In some cases, this may be determined using information acquired by an accelerometer and/or information from an orientation sensor (such as a gyroscope).

In practice, the dynamic range of the accelerometer may vary depending on the implementation. For example, an accelerometer may have a dynamic range of 16 g. As another example, an accelerometer may have a dynamic range of 32g (e.g., to detect a greater range of accelerations).

In some cases, the accelerometer and gyroscope may each obtain measurements according to the same sampling rate (e.g., 200Hz, 400Hz, 800Hz, or some other frequency). In some cases, the accelerometer and gyroscope may each obtain measurements from different sampling rates. For example, an accelerometer may obtain measurements from a higher sampling rate (e.g., 800 Hz), while a gyroscope may obtain measurements from a lower sampling rate (e.g., 200 Hz). This may be useful, for example, in selectively reducing the power consumption of one sensor (e.g., a sensor that consumes more power during operation) relative to other sensors to improve the power efficiency of the mobile device. In some cases, the sampling rate of the accelerometer and/or gyroscope may be dynamically adjusted during operation. For example, the sampling rate of the accelerometer and/or gyroscope may be selectively increased during certain periods of time and/or in response to certain conditions (e.g., greater user motion) and decreased during certain other periods of time and/or in response to certain other conditions (e.g., less user motion).

In some cases, one of the accelerometer or gyroscope may be used to obtain a measurement while the other sensor is disabled (e.g., such that it does not collect a measurement). The disabled sensor may be selectively activated based on measurements taken from the active sensor. This may be useful, for example, in reducing power consumption of the mobile device (e.g., by operating only one of the accelerometer or gyroscope during certain periods of time, and selectively operating both the accelerometer or gyroscope in response to certain conditions).

For example, the mobile device may disable the gyroscope and use the accelerometer to obtain acceleration measurements over a period of time. If the measured acceleration is greater than a particular threshold level (e.g., the root mean square RMS of the energy of the accelerometer's acceleration signal is above a threshold energy level), the mobile device may activate the gyroscope and collect orientation information. Thus, the gyroscope is selectively activated in response to the accelerometer detecting "significant" motion.

In some cases, if significant motion is no longer detected, the gyroscope may be disabled. For example, if the RMS energy of the accelerometer's acceleration signal is less than a threshold energy level for a particular period of time (e.g., a predefined time interval), the mobile device may disable the gyroscope and continue to operate the accelerometer. In some cases, the gyroscope may be disabled for a particular period of time after the gyroscope is activated (e.g., after a predefined time interval has elapsed since the gyroscope was turned on). In some cases, the gyroscope may be disabled if the RMS energy of the accelerometer's acceleration signal is less than a threshold energy level for a first time interval, or if a second time interval has elapsed since the accelerometer was activated, whichever occurs first. In practice, the time interval may vary depending on the implementation.

In some cases, the gyroscope may be selectively enabled and disabled according to a state machine. Fig. 14 shows that example state machines include a "gyro off" state 1402 (corresponding to a disabled state of a gyroscope), a "gyro on" state 1404 (corresponding to an enabled state of a gyroscope), and a "wait" state 1406 (corresponding to a state in which the mobile device waits for further information before adjusting operation of the gyroscope).

The mobile device begins with a "gyro off" state 1402 in which the gyro is disabled. The mobile device transitions to a "gyro on" state 1404 and enables the gyro upon detection of a non-dwell period based on the acceleration signal acquired from the accelerometer. Upon detecting a dwell and low rotation based on the acceleration signal and orientation signal from the gyroscope, the mobile device transitions to a "wait" state 1406. If the dwell and low rotation continue, the mobile device periodically increments a counter over time. If the counter exceeds the threshold, the mobile device returns to the "gyro off" state 1402 and disables the gyro and resets the counter. However, if a non-stop and/or a high enough degree of rotation is detected, the mobile device in turn returns to the "gyro on" state 1404 and keeps the gyro in an enabled state and resets the counter. In this way, the mobile device selectively enables the gyroscope in response to detecting movement and disables it after a pause and low rotation of the time period.

In some cases, the "pause" condition shown in fig. 14 may be a boolean value that is true when the following formula is satisfied:

thr1≤k1*VM+k2*dVM＜thr2

where VM is the magnitude of the acceleration signal, dVM is the rate of change of the magnitude of the acceleration signal, and thr1 and thr2 are adjustable thresholds.

In some cases, the "pause and low rotation" condition shown in fig. 14 may be a boolean value that is true when the following formula is satisfied:

(thr1+δ.ltoreq.k1.VM+k2. dVM < thr2- δ) and (rot < thr3)

Where VM is the magnitude of the acceleration signal, dVM is the rate of change of the magnitude of the acceleration signal, rot is the rate of rotation (e.g., determined based on a gyroscope), δ is the adjustable offset value, and thr1, thr2, and thr3 are the adjustable thresholds.

In some cases, a fall may be detected based on a combination or "fusion" of a number of different types of sensor measurements. For example, a fall may be detected based on acceleration measurements (e.g., acquired by an accelerometer), orientation measurements (e.g., acquired by a gyroscope), air pressure measurements (e.g., acquired by a pressure sensor or barometer), altitude measurements (e.g., acquired by an altimeter, pressure sensor, accelerometer, or other sensor), heart rate measurements (e.g., acquired by a heart rate sensor), and/or other types of measurements.

For example, fig. 15 shows a schematic representation 1500 of a fall detection technique based on multiple types of sensor measurements. The accelerometer is used to detect a severe impact (step 1502). For example, the acceleration a measured by the accelerometer within the sliding window may be based on _mag Is of the magnitude and jitter j _mag Is a magnitude of the detection of a severe impact. The magnitude of acceleration within the sliding window may be calculated using the following formula:

a _mag =sqrt (maximum (|x|) ² +maximum (|y|) ² +maximum (|z|) ² )

Where x, y, and z are the x, y, and z components of the acceleration signal, respectively, and the maximum is acquired within a window of 0.2 seconds.

The magnitude of jitter within a sliding window may be calculated using the following formula:

j _mag ＝max(sqrt(dx ² +dy ² +dz ² ))

where dx, dy, and dz are the derivatives of the x, y, and z components of the acceleration signal, respectively, and the maximum is taken within a window of 0.2 seconds.

If a is _mag Greater than threshold thr1 and j _mag Above the threshold thr2, the mobile device acquires gyroscope measurements (step 1504), altitude or altitude measurements (step 1506) and heart rate information (step 1508), and determines whether a fall has occurred based on the measurements (step 1510).

For example, accelerometer and gyroscope measurements may be used to determine the impact direction and attitude angle of a mobile device before, during, and/or after impact (e.g., as described herein). In some cases, accelerometer measurements may be used to estimate altitude or altitude of the mobile device (e.g., when the mobile device is stationary). In some cases, accelerometer and gyroscope measurements may be combined to estimate altitude or altitude of the mobile device (e.g., when the mobile device is in motion).

As another example, pressure sensors may be used to detect multi-level falls (e.g., a user falling off a ladder). As another example, a heart rate sensor may be used to detect a change in heart rate, such as an increase in heart rate (e.g., due to non-war i.e., escape response) or a heart rate decay curve (e.g., the heart rate decay of a person after a fall may have a different characteristic, such as a smaller time constant, than the heart rate decay after an exercise is completed). As another example, an accelerometer may be used to detect altitude or altitude (e.g., when the device is stationary). As another example, an accelerometer may be used to detect altitude or altitude (e.g., when the device is stationary).

FIG. 16 illustrates an example use of combining an accelerometer and a gyroscope to determine information about user motion (e.g., as part of step 1504 shown in FIG. 15). Accelerometer and gyroscope generate accelerometer and gyroscope measurements (step 1602). Based on this information, the gyroscope controller may selectively turn off the gyroscope (e.g., as described with respect to fig. 14) during a period of time (step 1604). Accelerometer and gyroscope measurements are used in combination (e.g., "fused") to obtain information about the device, such as the altitude or elevation of the device (step 1606). In addition, this information may be used to determine other information about the device, such as the pose angle and direction in which the mobile device experienced an impact (step 1608). The mobile device determines whether a fall has occurred based on the measurements (step 1610).

Fig. 17 shows an example fall classifier 1700. The fall classifier 1700 can be used to determine whether the user has fallen, and if so, the type or nature of the fall. The fall classifier 1700 can be implemented, for example, using one or more of the components of the system 100 shown in fig. 1 (e.g., the mobile device 102).

The fall classifier 1700 receives input indicating the measured movement of the user and outputs information indicating whether the user has fallen, and if so, the type or nature of the fall. For example, as shown in diagram 1700, the fall classifier receives acceleration data 1702 indicative of acceleration experienced by a mobile device worn by a user (e.g., measured using an accelerometer), and gyroscope data 1704 indicative of an orientation of the mobile device (e.g., measured using a gyroscope).

The acceleration data 1702 and the gyroscope data 1704 are combined or "fused" together by the sensor fusion module 1706 (e.g., using one or more techniques described herein) and considered together by the fall classifier 1700. In some cases, the acceleration data 1702 and the gyroscope data 1704 may be combined with respect to one or more (e.g., six) spatial axes.

The acceleration data 1702 and the gyroscope data 1704 may be used in combination to determine information about the user's motion characteristics. For example, the data may be used to determine the height 1708 of the mobile device.

Additionally, the acceleration data 1702 and the gyroscope data 1704 may be input into a feature extraction module 1710 that identifies one or more features or characteristics of the acceleration data 1702 and the gyroscope data 1704. Feature extraction module 1710 may perform one or more of the techniques described herein. For example, feature extractor 1710 may determine a wrist angle 1710 of a user (e.g., by determining a pose angle of a mobile device when worn on a wrist of the user).

Additionally, behavior modeling module 1712 may be used to determine the behavior of the user. The behavior of the user may be modeling one or more of the techniques described herein. For example, based on a change in the attitude angle of the mobile device 102, the behavior modeling module 1712 may determine behavior information 1714, such as whether the user is performing a supporting motion (e.g., extending his arm to stop a forward stroke), a balancing motion (e.g., extending his arm to restore balance), a swinging motion (e.g., swinging his arm during and after an impact), or another motion. In some cases, the support motion may be detected based on the following features: such as the wrist traversing the length of the negative arc prior to impact and pointing toward the ground at the moment of impact. In some cases, the balancing motion may be detected based on features such as the wrist traversing the length of the positive arc when the user attempts to restore balance. In some cases, the oscillatory motion may be detected based on features such as one or more rapid reversals of the wrist in motion (as part of a grip reflex, or due to repeated secondary impacts with the ground). The behavior information 1714 can be input into the classification module 1716 to assist in detection and classification of falls.

The fall classifier 1700 can also analyze various aspects of the acceleration data 1702 independent of the gyroscope data 1704. For example, the acceleration data 1702 may be input into a feature extraction module 1718 that identifies one or more features or characteristics of the acceleration data 1702. The feature extraction module 1718 may perform one or more of the techniques described herein. For example, the feature extractor 1718 may determine impact information 1720, such as the magnitude of an impact experienced by the user, the motion made by the user prior to the impact, and the motion made by the user after the impact. As another example, the feature extractor 1718 may determine a degree of confusion in user motion over a period of time.

Impact information 1720 may be input into an impact detector 1722 that determines whether the user has actually experienced an impact, and if so, the type or nature of the impact. The impact detector 1722 may perform one or more of the techniques described herein. For example, the impact detector 1722 may output an indication 1724 as to whether the user has experienced an impact.

Information from the impact detector 1722 and the behavior modeling module 1712 can be used to determine whether the user has fallen, and if so, the type or nature of the fall. For example, based on input from the impact detector 1722 and the behavior modeling module 1712, the classification module 1716 may determine that the user has slipped, stumbled, rolled, or experienced some other type of fall. As another example, based on inputs from the impact detector 1722 and the behavior modeling module 1712, the classification module 1716 may determine that the user has experienced an impact but has not fallen. As another example, based on inputs from the impact detector 1722 and the behavior modeling module 1712, the classification module 1716 may determine that the user has fallen but has recovered. The classification module 1716 outputs fall information 1726 that indicates whether the user has fallen, and if so, the type or nature of the fall.

As described above, multiple types of sensor measurements may be used in combination to determine a user's motion characteristics. For example, fig. 18 shows a fall sensor fusion module 1800 for determining whether a user has fallen, and if so, the type or nature of the fall. The sensor fusion module 1800 may be implemented, for example, using one or more of the components of the system 100 shown in fig. 1 (e.g., the mobile device 102).

The fall sensor fusion module 1800 receives input from a number of different sensors. For example, the fall sensor fusion module 1800 receives acceleration data 1802a that indicates the acceleration experienced by the mobile device worn by the user (e.g., measured using an accelerometer), and gyroscope data 1802b that indicates the orientation of the mobile device (e.g., measured using a gyroscope). As another example, the sensor fusion module 1800 receives location data 1802c (e.g., measured using a global navigation satellite system receiver (such as a global positioning system receiver) and/or a wireless transceiver (such as a Wi-Fi radio)) indicative of a location of the mobile device. As another example, the sensor fusion module 1800 receives altitude data 1802d (e.g., measured using altimeters, barometers, or other altitude sensors) indicative of the altitude or elevation of the device. As another example, the sensor fusion module 1800 receives heart rate data 1802d that is indicative of a heart rate of a user wearing the mobile device (e.g., measured using a heart rate sensor).

Acceleration data 1802a and gyroscope data 1802b may be used to determine whether a user has fallen, as described herein. For example, acceleration data 1802a and gyroscope data 1802b may be input into the fall classifier 1804. In general, the fall classifier 1804 can operate in a similar manner as described with respect to fig. 17. For example, the fall classifier 1804 can determine one or more features based on the acceleration data 1802a, and based on these features, determine whether the user has experienced an impact. In addition, the fall classifier 1804 can determine one or more features based on both the acceleration data 1802a and the gyroscope data 1802b, and model the user's behavior based on the features. Additionally, the fall classifier 1804 can determine whether the user has fallen based on the detected impacts and/or modeled behavior.

In addition, the fall classifier 1804 can determine whether the user has fallen based at least in part on the location data 1802 c. For example, the location data 1802c may be input into the threshold module 1806. The threshold module 1806 determines information about the location of the mobile device 1806. For example, the threshold module 1806 may determine whether the mobile device is at the user's home, at the user's work site, in a public area (e.g., store, gym, swimming pool, etc.), or in some other location. As another example, the threshold module 1806 may determine whether the mobile device is being worn by the user while the user is driving, riding a bicycle, skating, skateboarding, or traveling using some other mode of transportation. This information can be input into the fall classifier 1804 to improve detection of falls. For example, a user may be more likely to fall while at home than while riding in a car. Thus, the fall classifier 1804 can increase the sensitivity of determining that the user is at home to detect falls, as compared to when the user is driving a car. As another example, a user may be more likely to fall while out when snowing or raining than when not. Thus, the fall classifier 1804 may increase the sensitivity of detecting falls when it is determined that the user is outdoors and that the location is raining or snowing (e.g., based on information obtained from a weather service) than when the user is not raining or snowing at the location.

The fall classifier 1804 outputs fall data 1808 that indicates whether the user has experienced a fall, and if so, the type or nature of the fall. The fall data 1808 can be input into a fusion module 1810 that rejects false positives of the fall classifier 1804. For example, the fusion module 1810 can receive fall data 1808 that indicates that a fall has occurred. However, based on the additional information received by the fusion module 1810, the fusion module 1810 can overwrite the fall data 1808 and determine that no fall has occurred. The fusion module 1810 outputs confirmation data 1812 that confirms whether the user has experienced a fall, and if so, the type or nature of the fall.

In some cases, the fusion module 1810 can determine whether the fall data 1808 is false positive based on the height data 1802 d. For example, the height data 1802d may be input into the filter module 1814. The filter module 1814 may be used to isolate a particular component (e.g., a particular frequency or frequency range) of the height data 1802 d. The filtered altitude data 1802d is input into a feature extraction module 1816 that determines feature data 1818 that is indicative of one or more features of the altitude of the mobile device. For example, the feature extraction module 1816 may determine a change in the altitude or elevation of the mobile device over a period of time. The feature data 1818 is input into the fusion module 1810 and may be used to identify potential false positives. For example, if the mobile device experiences a significant change in altitude (e.g., a few feet, or a few levels or floors), the fusion module 1810 may determine that false positives are unlikely. As another example, if the mobile device does not experience any altitude changes, the fusion module 1810 may determine that false positives are more likely.

In some cases, the fusion module 1810 can determine whether the fall data 1808 is a false positive based on the heart rate data 1802 e. For example, heart rate data 1802e may be input into a filter module 1820. The filter module 1820 may be used to isolate a particular component (e.g., a particular frequency or frequency range) of the heart rate data 1802 e. The filtered heart rate data 1802e is input into a feature extraction module 1822 that determines feature data 1824 that is indicative of one or more features of the heart rate of a user wearing the mobile device. As another example, the feature extraction module 1822 may determine an increase or increase in the user's heart rate after a fall. As another example, the feature extraction module 1822 may determine a subsequent decay or restoration of the heart rate (e.g., when the heart rate returns to normal). As another example, the feature extraction module 1822 may determine an decay time constraint (e.g., a time constant indicating a decay rate after rising) associated with a decay or recovery of the heart rate. The feature data 1824 is input into the fusion module 1810 and may be used to identify potential false positives. For example, if the user's heart rate increases (e.g., due to non-war i.e., escaping responses), fusion module 1810 may determine that false positives are unlikely. As another example, the heart rate of a user generally decays faster after a fall than after an exercise period. Thus, the mobile device may compare the decay time constant of the user with the decay time constant sampled after the user exercises. If the user's decay time constant is less than the exercise-related decay time constant, fusion module 1810 may determine that false positives are unlikely.

As described above, the mobile device can automatically transmit a distress call to a third party (e.g., an emergency responder) in response to detecting a fall and determining that the user may need assistance. In a similar manner as described above (e.g., with respect to fig. 12), the mobile device may first alert the user that a distress call may be transmitted (e.g., after a fall), and allow the user to confirm whether to proceed with the call. The user may manually initiate a distress call in response to the alert (e.g., a command may be entered into the mobile device). However, if the user does not respond to the alert, the mobile device may determine whether the call should continue based on user behavior after the call. For example, if the user has not moved after a fall (e.g., indicating that the user is injured or unresponsive), the mobile device may continue the call. However, if the user exhibits activity (e.g., indicating that the user has resumed), the mobile device may cancel the call. Such progressive upgrades may be beneficial, for example, to reduce the number of false positives on falls, and to reduce the likelihood of unnecessary calls to third parties.

In some cases, the mobile device may determine whether to transmit a distress call based on measurements taken by a plurality of different sensors. For example, fig. 19 illustrates a distress call module 1900 for determining whether to transmit a distress call to a third party. For example, distress call module 1900 may be implemented using one or more of the components of system 100 (e.g., mobile device 102) shown in fig. 1, for example.

The distress call module 1900 includes a fusion module 1902 for determining whether the user has fallen, and if so, for determining the type or nature of the fall. The fusion module 1902 can operate in a similar manner as described with respect to fig. 18. For example, the fusion module 1902 may receive several types of sensor data 1804, such as acceleration data, gyroscope data, position data, altitude data, and/or heart rate data. Based on this information, the fusion module 1902 can determine whether the user wearing the mobile device has fallen and identify potential false positives. The fusion module 1902 outputs acknowledgement data 1804 that confirms whether the user has experienced a fall, and if so, the type or nature of the fall.

In addition, the distress call module 1900 determines information about the movement of the user wearing the mobile device. For example, acceleration data and gyroscope data may be input into the feature extraction module 1806. The feature extraction module 1806 determines one or more features regarding the user's movement, such as whether the user has moved some time after a fall, whether the user can walk several steps after a fall, whether the user has stood after a fall, or other features. The extraction module 1806 outputs feature data 1808, which indicates each of the extracted features.

The validation data 1804 and the feature data 1806 can be input into the fall state machine 1810. The fall state machine 1810 determines whether to transmit a distress call to a third party based on the input. An example fall state machine 1810 is shown in fig. 20.

In this example, the mobile device begins with a "normal" state 2002 (e.g., a low alert state), a "confirm fall" state 2004 (e.g., an elevated alert state when the fusion module 1802 detects a fall), a "alert" state 2006 (e.g., a state in which the mobile device alerts the user of the mobile device to a fall and possibly sends a distress call to a third party), a "wait" state 2008 (e.g., a state in which the mobile device waits for additional information, such as potential inputs by the user), a "cancel" state 2010 (e.g., a state in which an upcoming distress call is cancelled), and a "SOS" state 2012 (e.g., a state in which a distress call is made). The mobile device can transition between each state based on certain indications of a detected fall (e.g., as described herein), a detected dwell period (e.g., no movement by the user), and/or user input.

For example, the mobile device begins in a "normal" state 2002. Upon detection of a confirmed fall (e.g., by fusion module 1802), the mobile device transitions to a "confirm fall" state 2004. Pause period T after fall detection _Q When that occurs, the mobile device transitions to an "alert" state 2006 and alerts the user to detect a fall and informs the user that a distress call may be sent to a third party (e.g., an emergency responder). The mobile device transitions to a "wait" state 2008 and waits for input by the user in response to the alert. If a long lying period T after transmission of a fall warning _LL If no user movement is detected during (e.g., 30 seconds), the mobile device transitions to "SOS"Status 2012 and transmit distress call. The mobile device then returns to the "normal" state 2002. This may be useful, for example, if the user has fallen and is incapacitated for a long period of time. The mobile device may automatically seek assistance for the user even without user input.

As another example, the mobile device begins in a "normal" state 2002. Upon detection of a confirmation fall, the mobile device transitions to a "confirmation fall" state 2004. In a pause period T after a fall _Q Upon detection of certain types of movement (e.g., walking movement, standing movement, highly dynamic movement, or any other movement that indicates user recovery), the mobile device returns to the "normal" state 2002. This may be useful, for example, if the user has fallen, but shows signs of movement and recovery after the fall. In this case, the mobile device may avoid alerting the user or automatically seek assistance for the user.

For example, the mobile device begins in a "normal" state 2002. Upon detection of a confirmed fall (e.g., by fusion module 1802), the mobile device transitions to a "confirm fall" state 2004. Pause period T after fall detection _Q When that occurs, the mobile device transitions to an "alert" state 2006 and alerts the user to detect a fall and informs the user that a distress call may be sent to a third party (e.g., an emergency responder). The mobile device transitions to a "wait" state 2008 and waits for input by the user in response to the alert. For a long lying period T after transmission of a fall alert _LL Upon detection of certain types of movement (e.g., walking movement, standing movement, high dynamic movement, or any other movement that indicates user recovery), the mobile device transitions to the "cancel" state 2010 and does not transmit a distress call. The mobile device then returns to the "normal" state 2002. This may be useful, for example, if the user has fallen but shows signs of recovery. The mobile device can alert the user about a fall, but in this case will not automatically seek assistance for the user.

Although time values are described with respect to fig. 20, these are merely illustrative examples. In practice, one or more of the time values may vary depending on the particular implementation. In some cases, one or more of the time values may be an adjustable parameter (e.g., a parameter empirically selected to distinguish between different types or events or conditions).

As an illustrative example, fig. 21 shows a diagram 2100 that classifies different types of falls that a user may experience. In this example, the set of all types of falls that the user may experience is represented by boundary 2102. In addition, the subset of falls that lead to user injury is represented by boundary 2104 within boundary 2102. Further, the subset of falls that lead to injury to the user and render the user unresponsive is represented by boundary 2106 within boundary 2104.

In some implementations, the system 100 can distinguish between each of these types of falls and perform a different action in response. For example, if the user has suffered a fall that has caused his injury and is unresponsive, the system 100 can generate a notification and transmit the notification to other people to assist the user even though the user does not explicitly instruct the system 100 to do so after the fall. As another example, if the user has suffered a fall that has caused his or her injury but has not caused him or her to fail, the system 100 can generate a notification and transmit the notification to others to assist the user (e.g., after receiving confirmation from the user after the fall). As another example, if the user has suffered a fall that leaves him neither injured nor responding, the system 100 can avoid generating a notification and transmitting the notification to others to assist the user.

Although fig. 21 shows example categories or types of falls, these are merely illustrative examples. In practice, falls may be classified according to other categories or types, instead of or in addition to those shown in fig. 21.

As described herein, in some implementations, the system 100 can determine whether the user has fallen based on motion data acquired before, during, and/or after the impact experienced by the user 110. In some implementations, the system 100 can make this determination by detecting a motion characteristic of the user and determining whether the motion characteristic indicates that the user has experienced an impact, an expected fall (e.g., performing a supporting motion), and/or a reaction to a fall. Based on this information, the system 100 can determine the probability that the user has experienced a fall.

For example, fig. 22A illustrates a fall detection module 2200, and fig. 22B illustrates a process 2250 that may be at least partially performed by the fall detection module 2200. The fall detection module 2200 can be implemented as, for example, one or more of the components of the system 100 shown in fig. 1 (e.g., the mobile device 102).

As shown in fig. 22A, the fall detection module 2200 includes an arm motion tracker module 2202, an arm motion classifier module 2204, an impact detector module 2206, an impact classifier module 2208, and a fall event classifier module 2210.

During operation of the fall detection module 2200, the arm motion tracker module 2202 receives sensor measurements from one or more sensors worn by the user on their arm. For example, the arm motion tracker module 2202 may receive acceleration data acquired from one or more accelerometers worn by a user on their arm, and orientation data acquired from one or more gyroscopes worn by the user on their arm. Based on these sensor measurements, the arm motion tracker module 2202 determines the motion of the user's arm over a period of time. For example, the arm motion tracker module 2202 may determine that the user has rotated along one or more axes and/or has translated along one or more axes.

The arm motion tracker module 2204 provides information about the motion of the user's arm to the arm motion classifier module 2204. Based on this information, the arm motion classifier module 2204 determines the type of motion performed by the user's arm. For example, the arm motion classifier module 2204 may determine that the user has performed a supporting motion (e.g., extending his arm to stop a forward stroke), a balancing motion (e.g., extending his arm to restore balance), a swinging motion (e.g., swinging his arm during and after an impact), or another motion using his arm. For example, with reference to fig. 6, an example technique for classifying movement of a user's arm is described. The arm motion tracking module 2204 provides information about the type of motion performed by the user's arm to the fall event classifier module 2210 for further processing.

During operation of the fall detection module 2200, the impact detector module 2206 also receives sensor measurements from one or more sensors worn by the user on their arm. For example, the impact detector module 2206 may receive acceleration data acquired from one or more accelerometers worn by a user on their arm. Based on these sensor measurements, the impact detector module 2206 determines one or more characteristics or properties that may be indicative of an impact experienced by the user over a period of time. For example, the impact detector module 2206 may determine a magnitude of acceleration measured by one or more features relative to one or more dimensions, jitter measured by one or more features relative to one or more dimensions, a time associated with each of the measurements, and/or any other attribute that may be indicative of an impact.

The impact detector module 2206 provides information about the determined feature or characteristic to the impact classifier module 2208. Based on these sensor measurements, the impact classifier module 2208 determines a likelihood that the user has experienced an impact. The impact classifier module 2208 may also use additional sensor information to make this determination. For example, the fall detection module may make this determination based on acceleration data obtained from one or more accelerometers worn by the user on their arm and orientation data obtained from one or more gyroscopes worn by the user on their arm. The impact classifier module 2208 provides information about the likelihood that the user has experienced an impact identified by the fall event classifier module 2210 for further processing.

The fall event classifier module 2210 determines the likelihood of the user experiencing a fall based on information received from the arm motion classifier module 2204 and the impact classifier module 2208. For example, if the user has experienced an impact, the fall event classifier module 2210 can determine that the user is more likely to have experienced a fall. As another example, if the user's arm moves in a particular manner (e.g., the user stretches his arm forward, swings his arm, etc.) before the impact, the fall event classifier module 2210 can determine that the user is more likely to have experienced a fall.

In some implementations, the fall detection module 2200 can use the process 2250 shown in fig. 22B to determine the likelihood that the user has fallen. According to process 2250, the fall detection module 2200 tracks the motion of the user's arm (e.g., using sensor data from one or more sensors worn by the user on their arm, and the arm motion tracker module 2202) (step 2252). Based on the sensor measurements, the fall detection module 2200 determines the type of motion performed by the user's arm (e.g., using the arm motion classifier module 2204) (step 2254). For example, the arm motion classifier module 2204 may determine that the user has performed a supporting motion, a balancing motion, a swinging motion, or another motion using his arm.

While tracking the movement of the user's arm and determining the type of movement performed by the user's arm, the fall detection module 2200 collects sensor data (e.g., using the impact detector module 2206) that may indicate that the user is experiencing an impact (step 2256). Based on the sensor data, the fall detection module 2200 determines a likelihood that the user has experienced an impact (e.g., using the impact classifier module 2208) (step 2258). If the fall detection module 2200 determines that the user is unlikely to have experienced an impact, the fall detection module 2200 continues to collect sensor data and awaits the occurrence of an impact.

If the fall detection module 2200 determines that the user may have experienced an impact, the fall detection module 2200 determines the likelihood that the user has experienced a fall (e.g., using the fall event classifier module 2210) (step 2260). If the fall detection module 2200 determines that the user is unlikely to have experienced a fall, the fall detection module 2200 continues to collect sensor data and await the occurrence of an impact and track the movement of the user's arm.

If the fall detection module 2200 determines that the user is unlikely to have experienced a fall, the fall detection module 2200 can provide this information to the system 100. Based on this information, the system 100 may determine whether to perform (or refrain from performing) one or more actions in response to the determination. For example, such as using one or more of the techniques described herein, the system 100 can generate a notification and transmit the notification to others to assist the user (or avoid doing so).

As described herein, in some implementations, the system 100 can determine whether the user has fallen and needs assistance based on the user's activity level. For example, a sports user that is highly active (e.g., historically exhibiting frequent and violent movements) may be at a lower risk of falling and needing assistance. However, an inactive weak user (e.g. historically exhibiting infrequent and slight movements) may be at a higher risk of falling and needing assistance.

For example, fig. 23 shows an example line graph of the physical activity level of the user (horizontal axis) and the expected severity of a fall experienced by the user (vertical axis). A user with a relatively high physical activity level (e.g., corresponding to the right portion of the line graph) may be expected to experience a less severe fall (e.g., corresponding to the lower portion of the line graph). Thus, for these users, the system 100 may be configured to make it less likely to generate notifications and transmit the notifications to other users (e.g., to reduce false positives).

In contrast, a user with a relatively lower physical activity level (e.g., corresponding to the left portion of the line graph) may be expected to experience a more severe fall (e.g., corresponding to the upper portion of the line graph). Thus, for these users, the system 100 may be configured to make it more likely to generate notifications and transmit the notifications to other users (e.g., to reduce false positives).

In some implementations, the activity level of the user may be determined based on factors such as the age of the user and/or historical information about the user's physical activity over time (e.g., the average number of steps the user takes per day).

As described herein, in some implementations, the system 100 can determine that the user may have fallen and selectively perform certain actions (e.g., generating a notification and transmitting the notification to others) based on the user's movement characteristics before and after the fall. For example, in some implementations, the system 100 can compare the user activity before the fall with the user activity after the fall. If after a fall, the user cannot resume his pre-fall activity, the system 100 can determine that the user may have suffered injury and in response perform one or more actions (e.g., generate a notification and transmit the notification to others to assist the user).

For example, fig. 24 shows that the user's body is not occupied by a fall (time interval t ₀ To t ₁ ) At the fall time (time t ₁ ) After a fall (t ₁ Later time interval) collected example acceleration data 2400. In this example, the user performs a physical activity before falling (corresponding to at time interval t ₀ To t ₁ Changes in acceleration data collected during) and at that time t ₁ Undergo a fall (corresponding to time t ₁ Larger peaks in the collected accelerometer data). After a fall, the user does not move during the "pause" period (corresponding to after a fall until a certain time t ₂ Acceleration data collected, which has little to no change). For example, the user may have stun from a fall and/or be recovering from a fall. After the dwell period, the user begins to move again (corresponding to at this time t ₂ Changes in acceleration data collected thereafter).

The system 100 can determine whether to perform certain actions (e.g., generate a notification and transmit the notification to others) based on the user's movement characteristics before and after the fall. For example, fig. 25 illustrates an example process 2500 for generating and transmitting notifications. Process 2500 may be implemented, for example, in one or more of the components of system 100 shown in fig. 1 (e.g., mobile device 102).

According to process 2500, a device (e.g., mobile device 102 and/or one or more other components of system 100) collects sensor data from one or more sensors positioned on a user's body and uses the sensor data to determine whether the user has fallen (step 2502). Example techniques for determining whether a user has fallen are described herein.

Upon determining that the user has fallen, the device waits for a "pause" period after the fall (step 2504). During this time, the device does not generate any notification nor transmit the notification to others. However, the device continues to collect sensor data to track the user's movements. In practice, the length of the dwell period may vary depending on the implementation. In some implementations, the dwell period may be empirically selected.

After the dwell period has elapsed, the device determines the type of activity being performed by the user (step 2506). For example, the device may determine that the user is performing some type of activity, such as walking, running, swimming, doing sports, and so forth. For example, the determination may be made using an activity classifier based on sensor data (e.g., as described herein).

In addition, the device compares the user's "post-fall" activity with the user's "pre-fall" activity (step 2508). The pre-fall activity of the user may be determined in a similar manner to the post-fall activity of the user. For example, the type of activity that the user is performing before a fall may be determined by the activity classifier based on sensor data acquired before the fall.

If the user's pre-fall activity is different from (or substantially dissimilar to) the pre-fall activity (e.g., free fall activity < > post-fall activity, or free fall activity +.post-fall activity), the device may perform one or more actions in response. For example, the device may generate a notification and transmit the notification to others to assist the user (step 2508). For example, if the user runs before a fall, but no longer runs during "post-fall", the device may determine that the user is likely to be injured, and may seek assistance for the user. For example, if the user swims before a fall, but no longer swims during "post-fall", the device may determine that the user is likely to be injured, and may seek assistance for the user.

If the user's pre-fall activity is the same as (or substantially similar to) the pre-fall activity (e.g., free fall activity = post-fall activity), the device may avoid generating a notification and transmitting the notification to others, but may continue to collect sensor data and monitor the user for falls. For example, if the user runs before a fall, and also during a "post-fall," the device may determine that the user is less likely to be injured, and may avoid seeking help for the user.

As described herein, the system 100 can determine whether to generate a notification using a state machine and transmit the notification to others. FIG. 26 illustrates an example state machine. In this example, the device (e.g., mobile device 102) starts in a "no fall" state 2602 (e.g., corresponding to the user not detecting a fall). If the device determines that the user has fallen (e.g. at time t ₀ ) The device transitions to a "fall" state 2604 and resets the timer. After reset, the timer begins to increase over time.

After transitioning to the "fall" state, the device waits for a first period of time (e.g., until time t ₀ Time t thereafter ₁ ). This may be advantageous, for example, because it enables the user's movement to "stabilize" after a fall before the device analyses the nature of the fall and decides whether to take any action. In practice, the length of this first period of time may vary depending on the implementation. For example, the first time period may be one second, two seconds, three seconds, or any other time period. In some implementations, the length of the second time period may be selected empirically.

After waiting for the first period of time, the device determines whether the user has been during the second period of time (e.g., at time t ₁ And time t ₂ Between, wherein time t ₂ At time t ₁ After that) move (like)State 2606). If the device determines that the user has been during the second period of time (e.g., at time t ₁ And time t ₂ Between) moves, the device returns to the "not fallen" state 2602 and avoids generating a notification about the detected fall and transmitting the notification to other people. In practice, the length of this second period of time may vary depending on the implementation. For example, the second time period may be one second, two seconds, three seconds, or any other time period. In some implementations, the length of the second time period may be selected empirically.

The device may determine whether the user has moved based on measurements taken by one or more sensors worn by the user. For example, the device may receive acceleration measurements taken by one or more acceleration sensors worn by the user. If the acceleration measurement changes to a sufficient extent during a second period of time following the detected fall (e.g., the acceleration changes beyond a certain threshold level), the device may determine that the user has moved after the fall. As another example, the device may receive orientation measurements taken by one or more orientation sensors (e.g., gyroscopes) worn by a user. If the orientation measurement changes to a sufficient extent during a second period of time following the detected fall (e.g., the orientation changes beyond a certain threshold level), the device may determine that the user has moved after the fall. In some implementations, the threshold and/or the length of the second time period may be selected empirically (e.g., based on experimental studies that identify differences between situations where the user needs assistance and situations where the user does not need assistance).

As another example, in some implementations, the device can determine whether the user has moved by determining whether the user is walking or has been standing for a second period of time after the detected fall. For example, the device may obtain acceleration measurements (e.g., by one or more acceleration sensors), orientation measurements (e.g., by one or more orientation sensors such as gyroscopes), altitude measurements (e.g., by one or more altitude sensors such as barometers), and/or other measurements obtained by one or more sensors worn by the user. If the device determines that the measurement value indicates that the user walks and/or stands during a second period of time after the detected fall, the device may determine that the user has moved. In some implementations, the device may make this determination by determining whether the measurements match or are sufficiently similar to a predetermined pattern of measurements known to instruct the user to walk or stand. In some implementations, the time intervals and/or predetermined patterns of measurements may be selected empirically (e.g., based on experimental studies that identify distinguishing characteristics of measurements of a user walking and/or standing relative to measurements of a user performing other types of activities).

If the device determines that the user has not moved during the second period of time, the device transitions to an "alert" state 2608 and is ready to seek assistance for the user. For example, at the end of the second time period (e.g., at time t ₂ ) The device may present a notification (e.g., visual, audible, and/or tactile notification) to the user indicating that the request for assistance is to be transmitted. If the user does not instruct the device at the end of the third time period (e.g., at time t ₂ And time t ₃ Between, wherein time t ₃ At time t ₂ Thereafter) the transmission is canceled, the device may transition to an "auto-dial" state and automatically transmit a notification to the emergency responder (state 2610). For example, the device may automatically dial an emergency response system, such as "911", or initiate some other communication session. This may be advantageous, for example, because it provides the user with an opportunity to cancel the transmission manually if no assistance is needed. However, if the user does not respond (or cannot respond), the device will continue to transmit. In practice, the length of the third time period may vary depending on the implementation. For example, the length of the third time period may be one second, two seconds, three seconds, or any other time period. In some implementations, the length of the third time period may be empirically selected.

In addition, the device may provide information to assist emergency responders in locating and treating the user. For example, the device may transmit the location of the user (e.g., as determined by determining the location of a subsystem of the device), the nature of the emergency, or other information.

The user may also instruct the device to cancel the automatic notification and/or terminate the communication system when the device is in the automatic dialing state 2610. For example, at time t ₃ Previously, the user may instruct the device to cancel the automatic notification. In response, the device transitions back to the "not fallen" state 2602 and refrains from generating a notification and transmitting the notification to other people. As another example, at time t ₃ Thereafter, the user may instruct the device to cancel the automatic notification. In response, the device transitions back to the "not fallen" state 2602 and terminates the transmission.

In some implementations, the system 100 can determine the severity of the injury suffered by the user 110 due to a fall based on multiple types of sensor data. For example, the system 100 may make this determination based on accelerometer data indicative of the user's movement, gyroscope data indicative of changes in the orientation of the user's body, and altitude data indicative of changes in the user's altitude.

For example, fig. 27 shows a fall detection module 2700 for determining the severity of a user injury due to a fall. The fall detection module 2700 can be implemented as one or more of the components of the system 100 shown in fig. 1 (e.g., the mobile device 102), for example.

As shown in fig. 27, the fall detection module 2700 includes a fall height classifier module 2702, an impact detector module 2704, and an impact classifier module 2706, and an injury severity classifier module 2708.

During operation of the fall detection module 2700, the fall height classifier module 2702 receives sensor measurements from one or more sensors worn on its body by a user. For example, the fall height classifier module 2702 can receive height data obtained from one or more altimeters worn on the body of the user. Based on these sensor measurements, the fall height classifier module 2702 determines the height of the user over a period of time.

Further, the fall height classifier module 2702 can classify changes in the height of the user over time. For example, based on sensor data, the fall height classifier module 2702 can determine whether the height of the user changes in a manner that indicates whether the user has fallen a relatively short distance (e.g., a "single-layer" fall, such as a fall from a flight of stairs) or a relatively long distance (e.g., a "multi-layer" fall, such as a fall from a ladder or balcony). In some implementations, the fall height classifier module 2702 can make this determination by determining a change in the height of the user over a particular period of time, and determining whether the likelihood of a fall at that height falls within a different range (e.g., a relatively short range of distances for a single-stage fall or a relatively long range of distances for a multi-stage fall). The range can be determined empirically (e.g., based on observations that the subject falls under various conditions).

Although two example classifications are described above, these are merely illustrative examples. In practice, the fall height classifier module 2702 may classify the user's height variations according to any number of different categories or types, as an alternative or in addition to those described above.

During operation of the fall detection module 2700, the impact detector module 2704 also receives sensor measurements from one or more sensors worn by the user on their arm. For example, the impact detector module 2704 may receive acceleration data acquired from one or more accelerometers worn by a user on their arm. Based on these sensor measurements, the impact detector module 2704 determines one or more characteristics or properties that may be indicative of an impact experienced by the user over a period of time. For example, the impact detector module 2704 may determine a magnitude of acceleration measured by one or more features relative to one or more dimensions, jitter measured by one or more features relative to one or more dimensions, a time associated with each of the measurements, and/or any other attribute that may be indicative of an impact.

The impact detector module 2704 provides information regarding the determined feature or characteristic to the impact classifier module 2706. Based on these sensor measurements, the impact classifier module 2706 determines the severity of the impact experienced by the user. The impact classifier module 2706 may also use additional sensor information to make this determination. For example, the fall detection module may make this determination based on acceleration data obtained from one or more accelerometers worn by the user on their arm and orientation data obtained from one or more gyroscopes worn by the user on their arm. For example, if the magnitude of the change in sensor data is high (e.g., the change in acceleration, shake, and/or orientation is high), the impact classifier module 2706 may classify the impact as more severe. As another example, if the magnitude of the change in sensor data is low (e.g., the change in acceleration, shake, and/or orientation is low), the impact classifier module 2706 may classify the impact as less severe. The impact classifier module 2706 provides information to the injury severity classifier module 2708 regarding the severity of the impact experienced by the user for further processing.

The injury severity classifier module 2708 determines the severity of the injury suffered by the user based on information received from the fall height classifier module 2702 and the impact classifier module 2706. For example, if the user has experienced a severe impact and/or has fallen a longer distance, the injury severity classifier module 2708 may determine that the user is more likely to have experienced severe injury. As another example, if the user experiences a less severe impact and/or has fallen a shorter distance, the injury severity classifier module 2708 may determine that the user is more likely to have experienced less severe injury.

As another example, if the user has experienced multiple impacts over time, the injury severity classifier module 2708 may determine that the user is more likely to have experienced serious injury. As another example, if the user has experienced fewer impacts over time, the injury severity classifier module 2708 may determine that the user is more likely to have experienced less severe injury.

As another example, if the user has experienced multiple impacts over time, the injury severity classifier module 2708 may determine that the user is more likely to have experienced severe injury, wherein at least some of the impacts are different from one another. This may be advantageous, for example, because a flip (where a user is more likely to experience multiple changes in impact in sequence) may be more likely to cause serious injury.

For example, severity classifier module 2708 may determine, for each impact experienced by a user, a set of characteristics associated with the impact. The characteristics may include, for example, the magnitude of the impact, the distance the user falls, one or more axes of rotation of the user, the acceleration of the user, the shake of the user, the orientation of the user, any changes thereto, and/or any other characteristics of the impact. If the users experience impacts that are different relative to each other, the injury severity classifier module 2708 may determine that the user is more likely to have experienced serious injury. Further, if the users experience relatively similar impacts with respect to each other, the injury severity classifier module 2708 may determine that the user is unlikely to have experienced serious injury. The dissimilarity between the characteristics may be determined, for example, based on an arithmetic difference between the characteristics of the impacts and/or by determining a percentage change between the characteristics of the impacts. In some implementations, the impacts may be similar to each other if the difference between their respective characteristics is less than a particular threshold and/or the percentage of change between their respective characteristics is less than a threshold percentage. In some implementations, impacts may be different from each other if the difference between their respective characteristics is greater than a particular threshold and/or the percentage of change between their respective characteristics is greater than a threshold percentage.

In some implementations, the injury severity classifier module 2708 may determine the severity of the injury suffered by the user based on one or more of these factors in combination.

As described herein, the system 100 can determine whether the user has fallen in a manner that eliminates or otherwise reduces the incidence of false positives (e.g., when the user has not actually fallen and/or does not actually need help, it is determined that the user has fallen and/or needs help incorrectly). In some implementations, these determinations may be made on a user-specific basis. For example, such determinations may be made for a particular user based on previously collected historical sensor data about the user, historical information about the user's previous activities, personal characteristics of the user (e.g., physical characteristics, age, demographics, etc.), and/or other information specific to the user.

In some implementations, the system 100 can generate and maintain a database of information specific to each user. The database includes one or more data records, each of which includes information about the impact or "event" that the user has previously experienced. For example, the data record may include an indication (e.g., a unique identifier) of an impact previously experienced by the user. Further, the data records may include sensor data (e.g., accelerometer data, gyroscope data, altitude data, or any other sensor data) collected by sensors worn on the user's body before, during, and/or after the impact. Further, the data record may include an indication of the type of activity (e.g., walking, running, swimming, doing sports, cycling, etc.) that the user is performing at the time of the event as determined by the activity classifier module. Further, the data record may include metadata about the impact, such as the time the impact occurred, the date the impact occurred, the day of the week the impact occurred, and/or the location of the impact occurred. Further, the data record may include an indication of whether the impact has been previously categorized as a fall. The classification may be determined automatically (e.g., based on one or more of the techniques described herein) or manually by the user (e.g., based on input or feedback provided by the user after the impact).

The database may be used to determine user-specific characteristics that may indicate a fall. For example, the system 100 can cluster data records for one or more information dimensions (e.g., time, day of week, sensor data, activity type, whether the user has fallen, etc.) into two or more clusters. At least one of the clusters may correspond to an instance of the user previously experienced an impact and fall. Further, at least one of the clusters may correspond to an instance in which the user previously experienced an impact without falling. In some implementations, clustering techniques such as K-means clustering may be used to cluster data records.

The clustered data records can be used as a "template" to interpret the newly collected sensor data to determine if the user has fallen. For example, the system 100 may collect sensor data regarding the user's daily life. When a user experiences an impact, the system 100 may collect information about the impact, such as sensor data collected from the user before, during, and/or after the impact, the time of day of the impact, the week of the impact, the geographic location of the impact, the type of activity being performed by the user, and so forth. Further, the system 100 may compare the collected information with information contained in the data records of the clusters. If the collected information is similar to the information of one of the clusters, the system 100 can determine whether the user has fallen based on whether the cluster corresponds to an instance of the user falling or an instance of the user not falling.

For example, if a user regularly engages in regularly scheduled contact sports (e.g., football), he may experience impacts according to repeated schedules (e.g., at the same time of day, at the same time of week, etc.) and/or at the same geographic location (e.g., in certain soccer fields). Further, the type of activity of the user may be similar during each of those collisions (e.g., playing football), and the motion characteristics of the user may be similar during each of those collisions (e.g., corresponding to the user being robbed by others). Furthermore, after those impacts, he may provide feedback to the system 100 indicating that each of those impacts is associated with motion and does not fall into the situation where he needs medical assistance. The system 100 may generate data records of these impacts or "events" and then cluster them together into one or more clusters (e.g., based on their similarity in time, location, activity type, and sensor data). Further, the system 100 can indicate that the one or more clusters are not associated with a fall.

In the future, the user may play the football again according to his repeated arrangement and typical position. During this time, the system 100 may collect sensor data to detect whether the user has experienced an impact (e.g., one or more of the techniques described herein). Upon detecting an impact, the system 100 may determine whether characteristics of the impact (e.g., time, location, activity type, and/or sensor data regarding the impact) are similar to characteristics of a data record of the cluster. In this example, when the user plays in a similar manner, at a similar time, and at a similar location. Thus, the system 100 may determine that the newly detected impact is similar to an impact in one or more clusters of data records not associated with the phase fall. Thus, the system 100 may avoid generating notifications and transmitting the notifications to others to assist the user.

After playing football, the user may return to his home and fall from a stretch of stairs. During this time, the system 100 may collect sensor data to detect whether the user has experienced an impact. Upon detecting an impact, the system 100 may determine whether characteristics of the impact are similar to those of the cluster data record. In this example, since the user is in a different location than in the "non-fall" cluster of data records, at some time of the day has experienced an impact that is different from their usual football arrangement, is performing a different type of activity, and has experienced an impact of a different movement characteristic than that associated with playing a football. Thus, the system 100 can determine that the newly detected impact is sufficiently different from impacts in one or more clusters of data records that are not associated with a fall. In response, the system 100 may generate a notification and transmit the notification to others to assist the user.

In this way, the system 100 can adjust its fall detection behavior based on the specific behavior and characteristics of the user over time. Further, the system 100 may continuously "learn" and improve its detection over time. For example, as the system 100 collects more information about the user over the course of several impacts, the system 100 may update its database with new data records and re-cluster the data records to find new associations and trends about the collected information. Based on the re-clustered data records, the system 100 can determine whether the new impact experienced by the user is likely to be a fall.

In some implementations, the system 100 may discard, ignore, and/or give some data records a lower weight based on their age. For example, if the data record is greater than a particular threshold age, the system 100 may discard or ignore the data record such that it is not included in the cluster (e.g., such that the impact associated with the impact is no longer considered). As another example, the system 100 can give the data record a lower weight in the clustering process (e.g., such that the impact associated with the impact has less impact in determining whether the user has fallen). This may be advantageous, for example, because it enables the system 100 to accommodate changes in user behavior over time.

As described above, in some implementations, the user can manually provide input or feedback as to whether the impact is a fall (e.g., he needs medical assistance) or a non-fall (e.g., he does not need medical assistance). For example, after the system 100 detects an impact, the system 100 may prompt the user to classify the impact as falling or non-falling. The system 100 may include an input to a user in the data records for the impact and may use the input to cluster the data records in the database.

In some implementations, the system 100 can apply user input relative to one impact to other impacts having similar characteristics. This may be advantageous, for example, in reducing the number of times the system 100 requires manual input by a user. For example, if the user experiences multiple similar impacts, the system 100 may cluster the data records associated with those impacts into a common cluster. If the user indicates that one of those collisions does not correspond to a fall, the system 100 can update each of the data records in the cluster to indicate that it is not associated with a fall. In this way, the number of false positives can be reduced or eliminated even if the user has previously indicated erroneously that other impacts within the same cluster have corresponded to falls.

In some implementations, the system 100 can cluster data records from multiple types of clusters according to the likelihood that each cluster corresponds to a fall. For example, the system 100 may be based on at least three clusters: "Fall" clusters, "non-fall" clusters, and "near-fall" clusters are clustered for data logging. A "fall" cluster is more likely to be associated with a fall (e.g. the information contained therein shows a stronger correlation or trend than its "non-fall" cluster). "near-fall" clusters are less associated with falls (e.g., the information contained therein shows a less strong correlation or trend than its "non-fall" clusters). In some implementations, the likelihood that the cluster corresponds to a fall can be expressed as a posterior ratio or probability.

When determining whether a newly detected impact corresponds to a fall, the system 100 may ignore "near-fall" clusters (e.g., only moderately likely to correspond to a cluster of falls), and consider only "fall" clusters (e.g., more likely to correspond to a cluster of falls) and "non-fall" clusters (e.g., less likely to correspond to a cluster of falls). This may be useful, for example, because it enables the system 100 to classify a newly detected impact based on more representative or indicative or fall and non-fall "template" information (rather than information that is only weakly related to one or the other, and may result in ambiguous results). Thus, the system 100 may classify more accurately and/or in a predictable manner.

In some implementations, the information database specific to a particular user may be maintained uniquely on the user's personal device (e.g., his smart watch and/or smart phone, rather than a remote server). Furthermore, the clustering and categorization process may be performed exclusively on the user's personal device. This may be used, for example, to maintain the privacy of the user. However, in some implementations, at least some of the information data may be maintained on one or more remote servers in addition to or in lieu of the user's personal device. Similarly, in some implementations, at least a portion of the clustering and categorization process may be performed on one or more remote servers in addition to or in lieu of the user's personal device.

Fig. 28 shows an example process 2800 for making a user-specific determination of whether a user has fallen. Process 2800 may be implemented, for example, in one or more of the components of system 100 shown in fig. 1 (e.g., mobile device 102).

According to process 2800, a device (e.g., mobile device 102) makes an initial determination of whether a user has fallen based on sensor data acquired from sensors worn by the user on his body (step 2802). For example, the device may obtain sensor data about the impact (e.g., acceleration data and/or shake data), a pose angle or orientation of the device worn by the user and/or user, and/or movement of the device and/or user (e.g., whether the user is stationary or active). Example techniques for determining whether a user has fallen based on sensor data are described herein.

In some implementations, the device can determine whether the user has fallen by calculating a probability metric (e.g., a posterior ratio). If the metric is above the first threshold, the device may determine that the user has fallen. If the metric is below the first threshold, the device may determine that the user has not fallen. In practice, the first threshold may be selected empirically.

If the device determines that the user has fallen, the device determines whether the detected fall is similar to an event previously recorded in the database (step 2804).

For example, if the newly detected fall has similar characteristics (e.g., similar time, day, activity type, sensor data, etc.) to those stored in one or more data records (or clusters of data records) in a user-specific database, the device may determine that the newly detected fall is similar to one or more of the previously detected falls by the user. In response, the device may update the database to include information about the newly detected fall (step 2806). For example, the device can generate a data record of a newly detected fall and include information about the fall. Example information includes sensor data collected by sensors worn by the user on his body before, during, and/or after the impact, the sensor data indicating the type of activity the user is performing at the time of the event, and metadata about the impact (e.g., the time of the fall, the date of the fall, the day of the week the fall occurred, and/or the location of the fall occurred). Further, the new data record may be clustered with its similar one or more data records (e.g., added to an existing cluster).

If the newly detected fall does not have similar characteristics to those in any of the data records stored in the database, the device may determine that the newly detected fall is not similar to one or more of the falls previously detected by the user. In response, the device may generate an alert to the user (e.g., a pop-up alert on the display device) asking for further input by the user (step 2808). In response, the user can provide additional information about the fall. For example, the user may provide feedback as to whether he has actually fallen, the type of activity he is performing, or any other information about the event.

Similarly, the device can update the database to include information regarding the newly detected fall (step 2806). For example, the device can generate a data record of a newly detected fall and include information about the fall. Example information includes sensor data collected by sensors worn by the user on his body before, during, and/or after the impact, the sensor data indicating the type of activity the user is performing at the time of the event, and metadata about the impact (e.g., the time of the fall, the date of the fall, the day of the week the fall occurred, and/or the location of the fall occurred). Furthermore, the new data record may include information provided by the user (e.g. whether he has actually fallen, the type of activity he is performing, or any other information about the event).

Referring again to step 2802, if the device in turn determines that the user has not fallen, the device determines whether the probability metric associated with the newly detected event is approaching a threshold level (step 2804). For example, the device may determine whether a probability metric associated with the newly detected event is above a second threshold level that is lower than the first threshold level. If the probability metric is below a second threshold (e.g., it is not "near" the first threshold level used to determine whether the user has fallen), the device may refrain from updating the database and continue to gather sensor information about the user.

If the probability metric is above a second threshold level (e.g., it is "close" to the first threshold level used to determine whether the user has fallen), the device may update the database to include information about the new event (step 2806). For example, the device may generate a data record of the newly detected event and include information about the event. Example information includes sensor data collected by sensors worn by a user on his body before, during, and/or after an event, the sensor data indicating the type of activity the user is performing at the time of the event occurrence, and metadata about the event (e.g., the time of the event occurrence, the date of the event occurrence, the day of the week of the event occurrence, and/or the location of the event occurrence).

During this process, the device may also update its database to "learn" and improve its detection over time based on the newly collected information (step 2812). For example, the device may re-cluster the data records to find new associations and trends with respect to the collected information. In some implementations, the device may continuously re-cluster data records over time. In some implementations, the device may periodically re-cluster the data records over time (e.g., every M days). In some implementations, the device may periodically re-cluster the data records based on the number of events it has detected (e.g., every N detected events or impacts). Further, in some implementations, the device may discard data records based on its age (e.g., discard data records for more than T days). In practice, each of M, N and T may vary depending on the particular implementation.

As described herein, the resulting database can be used to determine whether the user has fallen (e.g., in a manner that eliminates or otherwise reduces the incidence of false positives).

In some implementations, the device may also identify false positives by measuring impacts experienced by the user over a period of time and identifying one or more patterns that may indicate false positives.

For example, the device determines that the user has experienced multiple impacts within a time period according to a periodic or near-periodic sequence based on accelerometer measurements. For example, the device may determine that the user has experienced multiple impacts that repeat according to a particular frequency or according to a particular frequency within a particular tolerance range (e.g., ±5%, ±10%, ±15% or some other tolerance range). In response, the device may determine that the user is unlikely to have experienced a fall. This can be used, for example, to distinguish falls from repeated physical activity (e.g., golf, tennis, squash, badminton, baseball or software shots, boxing, etc.). In some implementations, the time period and/or tolerance range can be selected empirically (e.g., based on experimental studies that identify differences between falls and other types of activities).

As another example, the device determines that the user has experienced a number of similar impacts within a time period based on accelerometer measurements. For example, the device may determine that the user experienced multiple impacts of equal intensity, magnitude, and/or power to each other or within a particular tolerance range of each other (e.g., ±5%, ±10%, ±15%, or some other tolerance range). In response, the device may determine that the user is unlikely to have experienced a fall. This can be used, for example, to distinguish falls from repeated physical activity (e.g., golf, tennis, squash, badminton, baseball or software shots, boxing, etc.). In some implementations, the time period and/or tolerance range can be selected empirically (e.g., based on experimental studies that identify differences between falls and other types of activities).

In some implementations, the device may also identify false positives by determining smoothness of movement of the user over a period of time (e.g., guiding the user through and/or after the user experiences the impact). For example, if the user's movement is relatively smooth over the period of time, the device may determine that the user is unlikely to have experienced a fall. Conversely, if the user's movement is relatively less smooth over the period of time, the device may determine that the user is more likely to have experienced a fall. This can be used, for example, to distinguish falls (e.g., characterized by relatively unstable movements in some implementations) from physical activity characterized by relatively smooth movements. In some implementations, the time period can be selected empirically (e.g., based on experimental studies that identify differences between falls and other types of activity).

Smoothness of user movement may be determined, for example, by taking accelerometer measurements as the user moves, and determining the intensity of changes in the accelerometer measurements (e.g., "jitter") and/or the frequency of these changes. For example, a relatively smoother movement may correspond to less drastic and/or less frequent changes in the acceleration measurements over a particular period of time, and a relatively less smooth movement may correspond to more drastic and/or more frequent changes in the acceleration measurements over a particular period of time.

In some implementations, the device may also identify false positives by determining acceleration of the user with respect to a plurality of different directions or axes over a period of time. Higher accelerations relative to one or more axes associated with one or more other axes may be less likely to indicate a fall. For example, the device may obtain accelerometer measurements as the user moves and determine acceleration relative to a plurality of different components from an internal frame of reference. For example, the first component may correspond to acceleration in a first direction parallel to the direction of gravity ("vertical" direction) and the second component may correspond to acceleration in a second direction parallel to the direction of gravity ("horizontal" direction). If the acceleration in the horizontal direction is sufficiently high (e.g. above a first threshold level) and the acceleration in the vertical direction is sufficiently low (e.g. below a second threshold level), the device can determine that the user is unlikely to have experienced a fall. This can be used, for example, to distinguish falls (e.g., in some implementations, characterized by higher acceleration in the vertical direction than in the horizontal direction) from physical activity in which the user experiences acceleration primarily in the horizontal direction, rather than in the vertical direction. In some implementations, the time period, the first threshold, and/or the second threshold can be selected empirically (e.g., based on experimental studies that identify differences between falls and other types of activities).

Although the foregoing examples are described with respect to one mobile device at a time (e.g., mobile device 102), in some implementations, the techniques described herein may be performed in conjunction with multiple devices. For example, multiple devices (e.g., a smart watch, a smart phone, and/or a remote server) may be communicatively coupled to each other (e.g., via one or more wireless communication links) and may exchange sensor data and/or other information collected by each device. In some implementations, one device may collect sensor data and provide the sensor data to another device for processing in accordance with the techniques described herein. For example, the smart watch may collect sensor data (e.g., because the smart watch may be more likely to be consistently positioned on the user's body than the smart phone) and provide the sensor data to the smart phone and/or a remote server for processing (e.g., because the smart phone and/or a removal server may have greater access to computing resources than the smart watch). In some implementations, multiple devices may collect sensor data simultaneously and provide the sensor data to other devices for processing according to the techniques. For example, the smart watch and smart phone may collect sensor data simultaneously and provide the sensor data to each other and/or a remote server for processing.

Exemplary procedure

An example process 2900 for determining whether a user has fallen and/or may need assistance using a mobile device is shown in fig. 29. Process 2900 may be performed, for example, by one or more of the components of system 100 shown in fig. 1 (e.g., mobile device 102). In some cases, some or all of process 2900 may be performed by a coprocessor of the mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 2900, a mobile device (e.g., mobile device 102 and/or one or more other components of system 100) obtains motion data indicative of motion measured by a motion sensor over a period of time (step 2902). The sensor is worn by the user. For example, as described with respect to fig. 1 and 2A, a user may attach a mobile device, such as a smartwatch, to his arm or wrist and walk around in daily life. This may include, for example, walking, running, sitting, lying down, participating in an exercise or sporting activity, or any other physical activity. During this time, the mobile device uses a motion sensor (e.g., accelerometer) in the mobile device to measure the acceleration experienced by the sensor over a period of time. The sensor data may be presented in the form of a time-varying acceleration signal (e.g., as shown in fig. 3).

The mobile device determines, based on the motion data, an impact experienced by the user (step 2904), the impact occurring during a first interval of the period of time. Example techniques for determining an impact are described above (e.g., with respect to fig. 3-5).

The mobile device determines one or more first motion characteristics of the user during a second interval of the time period based on the motion data (step 2906), the second interval occurring before the first interval. The second interval may be, for example, a "pre-impact" period of time. Determining the first motion characteristic may include, for example, determining that the user is walking during the second interval, determining that the user is ascending or descending stairs during the second interval, and/or determining that the user is moving the body part according to a swing motion or a support motion during the second interval. Example techniques for determining motion characteristics during a "pre-impact" period are described above (e.g., with respect to fig. 6).

The mobile device determines one or more second motion characteristics of the user during a third interval of the time period based on the motion data (step 2908), the third interval occurring after the first interval. The third interval may be, for example, a "post-impact" period of time. Determining the second motion characteristic may include, for example, determining that the user is walking during a third interval, determining that the user is standing during the third interval, and/or determining that the orientation of the user's body part has changed N or more times during the third interval. Example techniques for determining motion characteristics during a "pre-impact" period are described above (e.g., with respect to fig. 7 and 8).

The mobile device determines that the user has fallen based on the impact, the one or more first motion characteristics of the user, and the one or more second motion characteristics of the user (step 2910). The mobile device can also determine whether the user may need assistance (e.g., the result of a fall). Example techniques for determining whether a user has fallen and may need assistance are described above (e.g., with respect to fig. 9).

For example, the mobile device may determine that the impact is greater than a first threshold based on the motion data and determine that the user's motion is impaired during a third interval based on the motion data. Based on these determinations, the mobile device can determine that the user has fallen and may need assistance.

As another example, the mobile device may determine that the impact is less than a first threshold and greater than a second threshold based on the motion data. Additionally, the mobile device may determine, based on the motion data, whether the user walks during a second interval, steps up during the second interval, or steps down during the second interval. Additionally, the mobile device may determine, based on the motion data, that the user is moving the body part according to a swinging motion or a supporting motion during the second interval. Additionally, the mobile device may determine that the user's motion was impaired during the third interval based on the motion data. Based on these determinations, the mobile device can determine that the user has fallen and may need assistance.

In some cases, the mobile device may determine that the user has fallen based on a statistical model (e.g., a bayesian statistical model). For example, a statistical model may be generated based on the impact of the one or more samples, the first motion characteristic of the one or more samples, and the second motion characteristic of the one or more samples. The impact of the one or more samples, the first motion characteristic of the one or more samples, and the second motion characteristic of the one or more samples may be determined based on sample motion data collected from the sample population. The sample motion data may be indicative of motion measured by one or more additional motion sensors over one or more additional time periods, wherein each additional motion sensor is worn by a respective user of the sample population. Example techniques for generating and using statistical models are described above. In some implementations, the one or more sampled first motion characteristics may include an indication of a type of activity performed by the particular additional user with respect to the sample motion data, an indication of a level of activity of the particular additional user with respect to the sample motion data, and/or an indication of a speed of travel of the particular additional user with respect to the sample motion data.

In response to determining that the user has fallen, the mobile device generates a notification indicating that the user has fallen (step 2912). For example, the mobile device may present an indication on a display device and/or an audio device of the mobile device that the user has fallen. As another example, the mobile device may transmit data to a communication device remote from the mobile device indicating that the user has fallen. This may include, for example, an email, instant chat message, text message, telephone message, facsimile message, radio message, audio message, video message, haptic message, or another message for conveying information. Example techniques for generating notifications are described above.

Fig. 30 illustrates another example process 3000 for determining whether a user has fallen and/or assistance that may be needed using a mobile device. The process 3000 may be performed, for example, using one or more of the components of the system 100 shown in fig. 1 (e.g., the mobile device 102). In some cases, some or all of process 3000 may be performed by a coprocessor of a mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 3000, a mobile device (e.g., mobile device 102 and/or one or more other components of system 100) obtains a first signal indicative of acceleration measured by an accelerometer over a period of time (step 3002), and a second signal indicative of orientation measured by an orientation sensor over the period of time (step 3004). The accelerometer and the orientation sensor are physically coupled to the user. For example, as described with respect to fig. 1 and 2A, a user may attach a mobile device, such as a smartwatch, to his arm or wrist and walk around in daily life. This may include, for example, walking, running, sitting, lying down, participating in an exercise or sporting activity, or any other physical activity. During this time, the mobile device uses sensors in the mobile device (e.g., accelerometers and orientation sensors such as gyroscopes) to measure the acceleration experienced by the sensor over a period of time and the orientation of the sensor over that period of time. The sensor data may be presented in the form of a time-varying signal (e.g., as shown in fig. 11A).

The mobile device determines rotation data regarding the amount of rotation the user experienced during the time period (step 3006). The rotation data may include a third signal corresponding to a rotation rate of rotation of the user during a time period, an indication of one or more axes of rotation (e.g., one or more instantaneous axes of rotation) of the user during the time period in a reference frame, and/or an indication of an average axis of rotation of the user during the time period. For example, example rotation data is shown and described in fig. 11A-11D.

The mobile device determines whether the user has flipped based on the rotation data (step 3008). In some cases, this may be performed by determining a change between one or more axes of rotation of the user during the time period and an average axis of rotation of the user during the time period. Additionally, the mobile device may determine that the change is less than a first threshold. In response to determining that the change is less than the first threshold, the mobile device may determine a fourth signal corresponding to an angular displacement of the user during the time period based on the third signal (e.g., by integrating the third signal relative to the time period).

Additionally, the mobile device may determine that the angular displacement of the user during the time period is greater than a second threshold and determine that at least one of the one or more axes of rotation of the user during the time period is greater than a third threshold. In response to determining that the angular displacement of the user during the time period is greater than the second threshold and that at least one of the one or more axes of rotation of the user during the time period is greater than the third threshold, the mobile device can determine that the user has fallen. Otherwise, the mobile device may determine that the user has not flipped.

In response to determining that the user has fallen, the mobile device generates a notification indicating that the user has fallen (step 3010). Generating the notification may include presenting an indication on at least one of a display device or an audio device of the mobile device that the user has flipped over and/or transmitting data to a communication device remote from the mobile device. This may include, for example, an email, instant chat message, text message, telephone message, facsimile message, radio message, audio message, video message, haptic message, or another message for conveying information. The data may include an indication that the user has fallen. Example techniques for generating notifications are described above.

Fig. 31 illustrates another example process 3100 for determining whether a user has fallen and/or assistance that may be needed using a mobile device. Process 3100 may be performed, for example, using one or more of the components of system 100 shown in fig. 1 (e.g., mobile device 102). In some cases, some or all of process 3100 may be performed by a coprocessor of the mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 3100, a mobile device (e.g., mobile device 102 and/or one or more other components of system 100) obtains motion data indicative of motion measured by one or more motion sensors over a first period of time (step 3102). One or more motion sensors are worn by a user. The one or more motion sensors may include an accelerometer and/or a gyroscope. The mobile device may be a wearable mobile device. Example techniques for acquiring motion data are described above.

The mobile device determines that the user has fallen based on the motion data (step 3104). In some implementations, the mobile device can determine that the user has fallen by determining that the user has experienced an impact based on the motion data). In some implementations, the mobile device can determine that the user has fallen by determining the user's behavior during the first period of time. Example techniques for determining whether a user has fallen are described above.

In response to determining that the user has fallen, the mobile device generates one or more notifications indicating that the user has fallen (step 3106). In some implementations, generating the one or more notifications can include presenting a first notification to the user indicating that the user has fallen. The first notification may include at least one of a visual message, an audio message, or a haptic message. Example techniques for generating notifications are described above.

In some implementations, the mobile device can receive input from the user in response to the first notification (e.g., input indicating a user's request for assistance). In response to receiving the input, the mobile device may transmit a second notification to a communication device remote from the mobile device indicating a request for assistance. The communication device may be an emergency response system. Additionally, the second notification may indicate a location of the mobile device.

In some implementations, the mobile device can determine that the user has not moved (e.g., indicating that the user is injured or disabled) during a second period of time after the user falls. In response to determining that the user has not moved during the second period of time, the mobile device may transmit a second notification to a communication device remote from the mobile device indicating a request for assistance.

In some implementations, the mobile device can determine that the user has moved (e.g., walked, standing, or possibly some other type of movement) during a second period of time after the user has fallen. In response to determining that the user has moved during the second period of time, the mobile device may refrain from transmitting a second notification to a communication device remote from the mobile device indicating a request for assistance.

In some implementations, one or more notifications can be generated according to a state machine. An example state machine is shown in fig. 12 and 20.

Fig. 32 illustrates another example process 3200 for determining whether a user has fallen and/or assistance that may be required using a mobile device. Process 3200 may be performed, for example, using one or more of the components of system 100 shown in fig. 1 (e.g., mobile device 102). In some cases, some or all of process 3200 may be performed by a coprocessor of a mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 3200, a mobile device (e.g., mobile device 102 and/or one or more other components of system 100) obtains sample data generated by a plurality of sensors over a period of time (step 3202). The plurality of sensors is worn by a user. The sample data includes motion data indicative of user motion acquired from one or more motion sensors of the plurality of sensors. The sample data also includes at least one of the following: position data indicative of a position of the mobile device acquired from one or more of the plurality of sensors, altitude data indicative of an altitude of the mobile device acquired from one or more of the plurality of sensors, or heart rate data indicative of a heart rate of the user acquired from one or more of the plurality of sensors. The mobile device may be a wearable mobile device. Example techniques for acquiring sample data are described above.

In some implementations, the one or more motion sensors may include an accelerometer and/or a gyroscope. In some implementations, the accelerometer and gyroscope may be independently operable to collect motion data. For example, an accelerometer may be used to obtain acceleration data during a first time interval during the time period. The gyroscope may be disabled during the first time interval. Additionally, based on acceleration data acquired during the first time interval, the mobile device may determine that the movement of the user exceeds a threshold level during the first time interval. In response to determining that the movement of the user exceeds the threshold level during a first time interval, the mobile device may acquire acceleration data using the accelerometer and gyroscope data using the gyroscope during a second time interval subsequent to the first time interval. In some cases, the accelerometer and gyroscope may be operated according to a state machine. An example state machine is shown in fig. 14.

In some implementations, the one or more height sensors may include at least one of an altimeter or a barometer. The height sensor may be used, for example, to measure a specific change in height indicative of a fall (e.g., indicative of a fall in height from a ladder or structure).

In some implementations, the one or more position sensors can include at least one of a wireless transceiver (e.g., wi-Fi radio or cellular radio) or a global navigation satellite system receiver (e.g., GPS receiver).

The mobile device determines that the user has fallen based on the sample data (step 3204). Example techniques for determining whether a user has fallen are described above.

In some implementations, the mobile device can determine that the user has fallen by determining a change in orientation (e.g., a gesture angle) of the mobile device during the period of time based on the motion data, and determining that the user has fallen based on the change in orientation.

In some implementations, the mobile device can determine that the user has fallen by determining, based on the motion data, an impact experienced by the user during the period of time, and determining, based on the impact, that the user has fallen.

In some implementations, the mobile device can determine that the user has fallen by determining a change in the height of the mobile device during the period of time based on the height data, and determining that the user has fallen based on the change in height.

In some implementations, the mobile device can determine that the user has fallen by determining a change in the user's heart rate during the period based on the heart rate data, and determining that the user has fallen based on the change in heart rate. Determining a change in the heart rate of the user during the time period may include determining a decay rate (e.g., a time constant associated with the decay rate) of the heart rate of the user during the time period.

In some implementations, the mobile device can determine that the user has fallen by determining an environmental condition at the location of the mobile device based on the location data and determining that the user has fallen based on the environmental condition. The environmental condition may be weather (e.g., rain, snow, etc.) at the location.

In some implementations, the mobile device can determine that the user has fallen based on the motion data, the location data, the altitude data, and the heart rate data (e.g., in combination).

In response to determining that the user has fallen, the mobile device generates one or more notifications indicating that the user has fallen (step 3206). Generating the one or more notifications may include transmitting the notification to a communication device remote from the mobile device. The communication device may be an emergency response system. Example techniques for generating notifications are described above.

Fig. 33 illustrates another example process 3300 of using a mobile device to determine if a user has fallen and/or assistance that may be needed. Process 3300 may be performed, for example, using one or more of the components of system 100 shown in fig. 1 (e.g., mobile device 102). In some cases, some or all of process 3300 may be performed by a coprocessor of the mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 3300, a mobile device (e.g., mobile device 102 and/or one or more other components of system 100) receives motion data acquired by one or more sensors over a first period of time (step 3302). The one or more sensors are worn by a user. In some implementations, the one or more sensors may include one or more accelerometers, gyroscopes, and/or altimeters or barometers.

The mobile device may determine an impact experienced by the user during the period of time based on the motion data (step 3304)

The mobile device determines one or more of the characteristics of the user (step 3306).

The mobile device determines a likelihood that the user needs assistance after the impact based on the motion data and one or more characteristics of the user (step 3308).

In some implementations, the one or more characteristics of the user can include an age of the user. Furthermore, the likelihood may increase as the age of the user increases.

In some implementations, the one or more characteristics of the user can include a gender of the user. Furthermore, the likelihood may depend on the gender of the user.

In some implementations, the one or more characteristics of the user can include a historical physical activity level of the user. Furthermore, the likelihood may increase as the user's historical physical activity level decreases. In some implementations, the historical physical activity level may indicate a frequency of movement of the user prior to the impact. In some implementations, the historical physical activity level may indicate the strength of movement of the user prior to the impact.

In some implementations, the one or more characteristics of the user can include vascular health of the user. Furthermore, the likelihood may increase as the vascular health of the user decreases. The vascular health of the user may be determined based on the user's maximum oxygen uptake (VO 2 max).

The mobile device generates one or more notifications based on the likelihood (step 3310).

In some implementations, generating the one or more notifications can include transmitting the first notification to a communication device remote from the mobile device. The first notification may include an indication that the user has fallen. The communication device may be an emergency response system.

In some implementations, the mobile device is a wearable mobile device (e.g., a smart watch). In some implementations, at least some of the one or more sensors may be disposed on or in the mobile device. In some implementations, at least some of the one or more sensors may be remote from the mobile device.

Additional details regarding process 3300 are described above (e.g., with reference to fig. 23).

Fig. 34 illustrates another example process 3400 of using a mobile device to determine if a user has fallen and/or assistance that may be needed. The process 3400 may be performed, for example, using one or more of the components of the system 100 shown in fig. 1 (e.g., the mobile device 102). In some cases, some or all of the process 3400 may be performed by a coprocessor of the mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 3400, a mobile device (e.g., mobile device 102 and/or one or more other components of system 100) obtains a database comprising a number of data records (step 3402). Each data record includes an indication of a respective impact previously experienced by a user of the mobile device, and sensor data generated by one or more first sensors worn by the user during the impact. In some implementations, the one or more first sensors may include one or more accelerometers, gyroscopes, and/or altimeters or barometers.

The mobile device obtains additional sensor data generated by one or more second sensors worn by the user over a period of time (step 3404).

In some implementations, the one or more first sensors and the one or more second sensors may include at least one of an accelerometer or an orientation sensor. Further, for each data record, the sensor data may include one or more first signals indicative of acceleration measured by the accelerometer during an impact associated with the data record, and one or more second signals indicative of orientation measured by the orientation sensor during the impact associated with the data record.

In some implementations, the additional sensor data may include one or more additional first signals indicative of acceleration measured by the accelerometer during the time period, and one or more additional second signals indicative of orientation measured by the orientation sensor during the time period.

In some implementations, each data record can include metadata about an impact associated with the data record. The metadata may include at least one of an indication of a respective time of impact associated with the data record or an indication of a respective day of the week of impact associated with the data record.

The mobile device determines whether the user has fallen during the period of time based on the database and the additional sensor data (step 3406).

In some implementations, determining whether the user has fallen may include determining that the user has experienced an impact based on additional sensor data. Further, in response to determining that the user has experienced an impact, a likelihood that the impact corresponds to a fall of the user may be determined based on the additional sensor data.

In some implementations, determining whether the user has fallen may include the determined likelihood determining that the user has fallen. Further, in response to determining that the user has fallen, a similarity measure may be determined. The similarity metric may indicate a similarity between the additional sensor data and the sensor data of one or more clusters in the plurality of data records.

In some implementations, determining whether the user has fallen may include determining that the user has experienced multiple impacts during the time period based on the additional sensor data, and determining that the multiple impacts are similar to one another based on the additional sensor data. In response to these determinations, it may be determined that the user is unlikely to have fallen during the period of time.

In some implementations, determining whether the user has fallen may include determining, based on the additional sensor data, that the user has experienced a periodic sequence of multiple impacts during the period of time, and in response, determining that the user is unlikely to have fallen during the period of time.

In some implementations, determining whether the user has fallen may include determining a smoothness of movement of the user during the time period based on the additional sensor data, and determining whether the user has fallen during the time period based on the smoothness of movement of the user.

In some implementations, determining whether the user has fallen may include determining an acceleration of the user during the period of time relative to the first direction and a second direction orthogonal to the first direction based on the additional sensor data, and determining that the acceleration in the first direction is greater than a first threshold and the acceleration in the second direction is less than a second threshold. In response to these determinations, it may be determined that the user is unlikely to have fallen during the period of time. In some implementations, the first direction may be orthogonal to the direction of gravity and the second direction may be parallel to the direction of gravity.

The mobile device generates one or more notifications based on determining whether the user has fallen during the period of time (step 3408).

In some implementations, generating the one or more notifications can include determining that the similarity measure is below a threshold level, and in response, generating a first notification to the user to confirm whether the user has fallen.

In some implementations, generating the one or more notifications can include receiving input from the user indicating that the user has fallen, and in response, transmitting a second notification to a communication device remote from the mobile device. The second notification may include an indication that the user has fallen. In some implementations, the communication device may be an emergency response system.

In some implementations, the process 3400 may also include generating additional data records based on the additional sensor data and including the additional data records in the database.

In some implementations, the process 3400 may also include generating, by the mobile device, one or more clusters of data records based on similarities between the data records. One or more clusters may be generated using K-means clustering.

Additional details regarding process 3400 are described above (e.g., with reference to fig. 28).

Fig. 35 illustrates another example process 3500 for determining whether a user has fallen and/or assistance that may be required using a mobile device. Process 3500 may be performed, for example, using one or more of the components of system 100 shown in fig. 1 (e.g., mobile device 102). In some cases, some or all of process 3500 may be performed by a coprocessor of the mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 3500, the mobile device (e.g., mobile device 102 and/or one or more other components of the components of system 100) receives motion data acquired by the one or more sensors (step 3502). The one or more sensors are worn by a user.

The mobile device determines that the user has fallen at the first time based on the motion data (step 3504).

The mobile device determines whether the user has moved between a second time and a third time after the first time based on the motion data (step 3506).

In some implementations, the one or more sensors include one or more accelerometers. Further, the motion data may include one or more acceleration signals acquired by one or more accelerometers. Further, determining whether the user has moved between the first time and the second time may include determining a change in one or more acceleration signals between the first time and the second time.

In some implementations, the one or more sensors include one or more orientation sensors. Further, the motion data may include one or more orientation signals acquired by one or more orientation sensors. Further, determining whether the user has moved between the first time and the second time may include determining a change in one or more orientation signals between the first time and the second time.

Upon determining that the user has not moved between the second time and the third time, the mobile device initiates a communication request emergency response service at a fourth time subsequent to the third time (step 3508). The communication includes an indication that the user has fallen and the user's location.

In some implementations, the mobile device may refrain from initiating the communication request emergency response service upon determining that the user has moved between the second time and the third time.

In some implementations, after initiating the communication request emergency response service, the mobile device can receive a command to terminate the communication from the user. In response, the mobile device may terminate the communication.

In some implementations, the mobile device may be a wearable mobile device. In some implementations, at least some of the one or more sensors may be disposed on or in the mobile device. In some implementations, at least some of the one or more sensors may be remote from the mobile device.

Additional details regarding process 3500 are described above (e.g., with reference to fig. 26).

Fig. 36 illustrates another example process 3600 for determining whether a user has fallen and/or assistance that may be needed using a mobile device. The process 3600 may be performed, for example, using one or more of the components of the system 100 shown in fig. 1 (e.g., the mobile device 102). In some cases, some or all of process 3600 may be performed by a coprocessor of a mobile device. The co-processor may be configured to receive motion data acquired from the one or more sensors, process the motion data, and provide the processed motion data to one or more processors of the mobile device.

In process 3600, a mobile device (e.g., mobile device 102 and/or one or more other components of system 100) obtains sample data generated by one or more sensors over a period of time (step 3602). The one or more sensors are worn by a user.

The mobile device determines that the user has fallen based on the sample data (step 3604).

The mobile device determines the severity of the injury suffered by the user based on the sample data (step 3606).

The mobile device generates one or more notifications based on the determination that the user has fallen and the determined severity of the injury (step 3608). In some implementations, generating the one or more notifications can include transmitting the first notification to a communication device remote from the mobile device. The first notification may include an indication that the user has fallen and an indication of the severity of the determined injury suffered by the user. The communication device may be an emergency response system.

In some implementations, the one or more sensors may include at least one of an accelerometer, an orientation sensor, or an altimeter. Further, the sample data may include movement data indicating movement of the user over a period of time. Further, determining that the user has fallen may include determining, based on the motion data, a first impact experienced by the user during the period of time, and determining, based on the motion data, a change in orientation of a portion of the user's body during the period of time. The portion of the user's body may be (or may include) the user's wrist.

In some implementations, the motion data may include a first signal indicative of acceleration measured by the accelerometer over a period of time, and a second signal indicative of orientation measured by the orientation sensor over the period of time. Further, determining the severity of the injury suffered by the user may include determining the severity of the first impact experienced by the user based on the first signal and the second signal.

In some implementations, the severity of the injury suffered by the user may be determined based on a similarity between the impacts experienced by the user. For example, a first set of characteristics associated with a first impact may be determined based on the first signal and the second signal. Further, a second set of characteristics associated with a second impact experienced by the user during the period of time may be determined based on the first signal and the second signal. Further, a similarity between the first set of characteristics and the second set of characteristics may be determined. In some implementations, a greater degree of similarity between the characteristics of the first impact and the second impact may correspond to a lesser severity of injury. In some implementations, a lesser degree of similarity between the characteristics of the first impact and the second impact may correspond to a greater severity of injury.

In some implementations, the motion data can include a third signal indicative of an altitude measured by the altimeter over a period of time. Determining the severity of the injury suffered by the user may further comprise determining a distance over which the user falls within the period of time based on the third signal. The severity of the injury may be determined based on the determined severity of the impact experienced by the user and the determined distance the user falls.

Additional details regarding process 3500 are described above (e.g., with reference to fig. 27).

Example Mobile device

Fig. 37 is a block diagram of an example device architecture 2500 for implementing the features and processes described with reference to fig. 1-36. For example, architecture 3700 may be used to implement one or more of mobile device 102, server computer system 104, and/or communication device 106. Architecture 3700 may be implemented in any device for generating the features described with reference to fig. 1-36, including, but not limited to, a desktop computer, a server computer, a portable computer, a smart phone, a tablet computer, a game console, a wearable computer, a set-top box, a media player, a smart television, and the like.

Architecture 3700 may include memory interface 3702, one or more data processors 3704, one or more data coprocessors 3774, and peripheral interface 3706. Memory interface 3702, one or more processors 3704, one or more coprocessors 3774, and/or peripheral interfaces 3706 may be separate components or may be integrated into one or more integrated circuits. One or more communication buses or signal lines may couple the various components.

The one or more processors 3704 and/or the one or more coprocessors 3774 may cooperate to perform the operations described herein. For example, the one or more processors 3704 may include one or more Central Processing Units (CPUs) configured to act as host computer processors for the architecture 3700. For example, the one or more processors 3704 may be configured to perform the generalized data processing tasks of the architecture 3700. In addition, at least some of the data processing tasks may be offloaded to one or more coprocessors 3774. For example, specialized data processing tasks (such as processing motion data, processing image data, encrypting data, and/or performing some type of arithmetic operation) may be offloaded to one or more specialized coprocessors 3774 for processing these tasks. In some cases, the one or more processors 3704 may be relatively more powerful than the one or more coprocessors 3774 and/or may consume more power than the one or more coprocessors 3774. This may be useful, for example, because it enables the one or more processors 3704 to quickly process generalized tasks while also offloading certain other tasks to the one or more coprocessors 3774 that may perform those tasks more efficiently and/or effectively. In some cases, one or more coprocessors may include one or more sensors or other components (e.g., as described herein) and may be configured to process data acquired using these sensors or components and provide the processed data to one or more processors 3704 for further analysis.

Sensors, devices, and subsystems can be coupled to the peripheral interface 3706 to facilitate multiple functions. For example, motion sensor 3710, light sensor 3712, and proximity sensor 3714 may be coupled to peripheral interface 3706 to facilitate orientation, lighting, and proximity functions of architecture 3700. For example, in some implementations, the light sensor 3712 can be utilized to help adjust the brightness of the touch surface 3746. In some implementations, the motion sensor 3710 can be used to detect movement and orientation of the device. For example, the motion sensor 3710 may include one or more accelerometers (e.g., to measure acceleration experienced by the motion sensor 3710 and/or the architecture 3700 over a period of time) and/or one or more compasses or gyroscopes (e.g., to measure orientation of the motion sensor 3710 and/or the mobile device). In some cases, the measurement information acquired by the motion sensor 3710 may be in the form of one or more time-varying signals (e.g., a time-varying plot of acceleration and/or orientation over a period of time). In addition, display objects or media may be presented according to the detected orientation (e.g., according to a "portrait" orientation or a "landscape" orientation). In some cases, the motion sensor 3710 may be integrated directly into the coprocessor 3774 configured to process measurements taken by the motion sensor 3710. For example, coprocessor 3774 may include one or more accelerometers, compasses, and/or gyroscopes, and may be configured to obtain sensor data from each of these sensors, process the sensor data, and transmit the processed data to one or more processors 3704 for further analysis.

Other sensors may also be connected to the peripheral interface 3706, such as temperature sensors, biometric sensors, or other sensing devices to facilitate related functions. For example, as shown in fig. 37, architecture 3700 may include a heart rate sensor 3732 that measures the heart beat of the user. Similarly, these other sensors may also be integrated directly into one or more coprocessors 3774 configured to process measurements taken from those sensors.

The location processor 3715 (e.g., a GNSS receiver chip) may be connected to the peripheral interface 3706 to provide a geographic reference. An electronic magnetometer 3716 (e.g., an integrated circuit chip) can also be connected to the peripheral interface 3706 to provide data that can be used to determine the direction of magnetic north. Thus, the electronic magnetometer 3716 can be used as an electronic compass.

Camera subsystem 3720 and optical sensor 3722 (e.g., a charge coupled device [ CCD ] or a complementary metal oxide semiconductor [ CMOS ] optical sensor) may be utilized to facilitate camera functions, such as taking photographs and video clips.

Communication functions may be facilitated through one or more communication subsystems 3724. Communication subsystem 3724 may include one or more wireless and/or wired communication subsystems. For example, the wireless communication subsystem may include radio frequency receivers and transmitters and/or optical (e.g., infrared) receivers and transmitters. As another example, a wired communication system may include a port device (e.g., a Universal Serial Bus (USB) port) or some other wired port connection that may be used to establish a wired connection to other computing devices, such as other communication devices, network access devices, personal computers, printers, display screens, or other processing devices capable of receiving or transmitting data.

The specific design and implementation of communication subsystem 3724 may depend on the communication network(s) or medium(s) through which architecture 3700 is intended to operate. For example, architecture 3700 may include a network designed to communicate over a global system for mobile communications (GSM) network, a GPRS network, an Enhanced Data GSM Environment (EDGE) network, an 802.X communication network (e.g., wi-Fi, wi-Max), a Code Division Multiple Access (CDMA) network, NFC, and bluetooth ^TM A network-operated wireless communication subsystem. The wireless communication subsystem may also include a host protocol such that architecture 3700 may be configured as a base station for other wireless devices. As another example, the communication subsystem may use one or more protocols, such as TCP/IP protocol, HTTP protocol, UDP protocol, and any other known protocol to allow architecture 3700 to synchronize with a host device.

An audio subsystem 3726 may be coupled to speaker 3728 and one or more microphones 3730 to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording, and telephony functions.

I/O subsystem 3740 may include touch controller 3742 and/or one or more other input controllers 3744. Touch controller 3742 can be coupled to touch surface 3746. The touch surface 3746 and touch controller 3742 can detect contact and movement or vice versa, for example, using any of a variety of touch sensitive technologies including, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch surface 3746. In one implementation, touch surface 3746 may display virtual buttons or soft buttons and a virtual keyboard, which the user may use as input/output devices.

Other input controllers 3744 may be coupled to other input/control devices 3748, such as one or more buttons, rocker switches, thumb-wheel, infrared ports, USB ports, and/or pointer devices (such as a stylus). The one or more buttons (not shown) may include an up/down button for volume control of speaker 3728 and/or microphone 3730.

In some implementations, architecture 3700 can present recorded audio files and/or video files, such as MP3, AAC, and MPEG video files. In some implementations, architecture 3700 may include the functionality of an MP3 player and may include pin connectors for connecting to other devices. Other input/output devices and control devices may be used.

Memory interface 3702 may be coupled to memory 3750. Memory 3750 may include high-speed random access memory or nonvolatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, or flash memory (e.g., NAND, NOR). Memory 3750 may store operating system 3752, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system (such as VxWorks). Operating system 3752 can include instructions for handling basic system services and for performing hardware-related tasks. In some implementations, the operating system 3752 can include a kernel (e.g., a UNIX kernel).

Memory 3750 may also store communication instructions 3754 to facilitate communication, including peer-to-peer communication, with one or more additional devices, one or more computers, or servers. The communication instructions 3754 may also be used to select an operating mode or communication medium for use by the device based on the geographic location of the device (as acquired by the GPS/navigation instructions 3768). Memory 3750 can include graphical user interface instructions 3756 that facilitate graphical user interface processing, including touch models for interpreting touch inputs and gestures; sensor processing instructions 3758 that facilitate sensor-related processes and functions; phone instructions 3760 that facilitate phone-related processes and functions; electronic message processing instructions 3762 that facilitate processes and functions related to electronic message processing; web browsing instructions 3764 that facilitate web browsing-related processes and functions; media processing instructions 3766 to facilitate media processing-related procedures and functions; GPS/navigation instructions 3769 to facilitate GPS and navigation-related processes; camera instructions 3770 that facilitate camera-related processes and functions; and other instructions 3772 for performing some or all of the processes described herein.

Each of the above-identified instructions and applications may correspond to a set of instructions for performing one or more functions described herein. These instructions need not be implemented as separate software programs, processes or modules. Memory 3750 may include additional instructions or fewer instructions. Furthermore, the various functions of the device may be performed in hardware and/or software, including in one or more signal processing and/or Application Specific Integrated Circuits (ASICs).

The features may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The features may be implemented in a computer program product tangibly embodied in an information carrier, e.g., in a machine-readable storage device, for execution by a programmable processor; and method steps may be performed by a programmable processor executing a program of instructions to perform functions of the implementations by operating on input data and generating output.

The described features may be advantageously implemented in one or more computer programs that are executable on a programmable system including at least one input device, at least one output device, and at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language, including compiled and interpreted languages, such as Objective-C, java, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of the multiple processors or cores of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer may communicate with a mass storage device to store data files. These mass storage devices may include magnetic disks, such as internal hard disks and removable disks; magneto-optical disk; an optical disc. Storage devices suitable for tangibly embodying computer program instructions and data include: all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disk; CD-ROM and DVD-ROM discs. The processor and the memory may be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

To provide for interaction with a user, these features can be implemented on a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the author and a keyboard and a pointing device, such as a mouse or a trackball, by which the author can provide input to the computer.

These features may be implemented in a computer system that includes a back-end component such as a data server or that includes a middleware component such as an application server or an Internet server, or that includes a front-end component such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system can be connected by any form of digital data communication, such as a communication network, or medium of such digital data communication. Examples of communication networks include LANs, WANs, and computers and networks forming the internet.

The computer system may include clients and servers. The client and server are typically remote from each other and typically interact through a network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

One or more features or steps of the disclosed embodiments can be implemented using an Application Programming Interface (API). An API may define one or more parameters that are passed between invoking an application and other software code (e.g., an operating system, an inventory program, a function) that provides a service, provides data, or performs an operation or calculation.

An API may be implemented as one or more calls in program code that send or receive one or more parameters through a parameter list or other structure based on calling conventions defined in an API specification document. The parameter may be a constant, a key, a data structure, a target class, a variable, a data type, a pointer, an array, a list, or another call. The API calls and parameters may be implemented in any programming language. The programming language may define the vocabulary and calling conventions that the programmer will use to access the functions that support the API.

In some implementations, the API call may report to the application the capabilities of the device to run the application, such as input capabilities, output capabilities, processing capabilities, power capabilities, communication capabilities, and the like.

As described above, some aspects of the subject matter of this specification include the collection and use of data from various sources to improve services that mobile devices can provide to users. The present disclosure contemplates that in some cases, the collected data may identify a particular location or address based on device usage. Such personal information data may include location-based data, addresses, subscriber account identifiers, or other identifying information.

The present disclosure also contemplates that the entity responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will adhere to established privacy policies and/or privacy practices. In particular, such entities should exercise and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. For example, personal information from a user should be collected for legal and legitimate uses of an entity and not shared or sold outside of those legal uses. In addition, such collection should be done only after the user's informed consent. In addition, such entities should take any required steps to secure and protect access to such personal information data and to ensure that other people who are able to access the personal information data adhere to their privacy policies and procedures. In addition, such entities may subject themselves to third party evaluations to prove compliance with widely accepted privacy policies and practices.

In the case of advertisement delivery services, the present disclosure also contemplates embodiments in which the user selectively blocks use or access to personal information data. That is, the present disclosure contemplates that hardware elements and/or software elements may be provided to prevent or block access to such personal information data. For example, with respect to advertisement delivery services, the techniques of the present invention may be configured to allow a user to choose to "opt-in" or "opt-out" to participate in the collection of personal information data during a registration service.

Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that the various embodiments may be implemented without accessing such personal information data. That is, various embodiments of the present technology do not fail to function properly due to the lack of all or a portion of such personal information data. For example, the content may be selected and delivered to the user by inferring preferences based on non-personal information data or absolute minimum amount of personal information such as content requested by a device associated with the user, other non-personal information available to the content delivery service, or publicly available information.

A number of implementations have been described. However, it should be understood that various modifications may be made. Elements of one or more implementations may be combined, deleted, modified, or supplemented to form additional implementations. As another example, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Further, other steps may be provided or steps may be eliminated from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A method for detecting falls, comprising:

obtaining, by a mobile device, sample data generated by a plurality of sensors over a period of time, wherein the plurality of sensors are worn by a user, and wherein the plurality of sensors include an accelerometer and an orientation sensor;

determining, by the mobile device, that the user has fallen based on the sample data;

determining, by the mobile device, a severity of injury suffered by the user based on the sample data; and

generating by the mobile device one or more notifications based on the determination that the user has fallen and the determined severity of the injury,

wherein the sample data includes motion data indicative of motion of the user over the period of time, comprising:

a first signal indicative of acceleration measured by the accelerometer over the period of time, an

A second signal indicative of an orientation measured by the orientation sensor over the period of time,

wherein determining the severity of the injury suffered by the user comprises:

a first set of characteristics associated with a first impact experienced by the user during the time period is determined based on the first signal and the second signal,

Determining a second set of characteristics associated with a second impact experienced by the user during the time period based on the first signal and the second signal,

determining a similarity between the first set of characteristics and the second set of characteristics, an

A severity of the injury suffered by the user is determined based on the determined similarity.

2. The method of claim 1, wherein the plurality of sensors further comprises an altimeter.

3. The method of claim 2, wherein determining that the user has fallen comprises:

determining the first impact experienced by the user during the period of time based on the motion data, and

an orientation change of a portion of the user's body during the period of time is determined based on the motion data.

4. A method according to claim 3, wherein the portion of the user's body comprises the user's wrist.

5. The method of claim 2, wherein determining the severity of the injury suffered by the user comprises:

a severity of the first impact experienced by the user is determined based on the first signal and the second signal.

6. The method of claim 5, wherein determining the severity of the injury suffered by the user comprises:

Determining that the user has experienced a plurality of impacts including the first impact and the second impact during the period of time based on the first signal and the second signal.

7. The method of claim 5, wherein the motion data further comprises a third signal indicative of an altitude measured by the altimeter over the period of time.

8. The method of claim 7, wherein determining the severity of the injury suffered by the user further comprises:

a distance from which the user falls within the period of time is determined based on the third signal.

9. The method of claim 8, wherein determining the severity of the injury suffered by the user further comprises:

the severity of the injury is determined based on the determined severity of the first impact experienced by the user and the determined distance over which the user falls.

10. The method of claim 1, wherein generating the one or more notifications comprises:

transmitting a first notification to a communication device remote from the mobile device, the first notification comprising an indication that the user has fallen and an indication of the determined severity of the injury suffered by the user.

11. The method of claim 10, wherein the communication device is an emergency response system.

12. The method of claim 1, wherein the mobile device is a wearable mobile device.

13. The method of claim 1, wherein at least some of the plurality of sensors are disposed on or in the mobile device.

14. The method of claim 1, wherein at least some of the plurality of sensors are remote from the mobile device.

15. A system for detecting falls, comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

wherein determining the severity of the injury suffered by the user comprises:

16. The system of claim 15, wherein the mobile device is a wearable mobile device.

17. The system of claim 15, wherein at least some of the plurality of sensors are disposed on or in the mobile device.

18. The system of claim 15, wherein at least some of the plurality of sensors are remote from the mobile device.