KR20230061916A - Apparatus and Method for Fall Recognition based on Detecting Point of No Return - Google Patents

Abstract

In accordance with an embodiment of the present invention, disclosed are an apparatus and a method for fall recognition based on point-of-no-return detection. In accordance with an embodiment of the present invention, the apparatus for fall recognition based on point-of-no-return detection includes a memory in which at least one program and a pre-trained fall step classification model are recorded, and a processor executing the program. The fall step classification model is pretrained to subdivide and predict a critical step from a time point, in which a fall begins, to a time point, in which a human body comes in contact with the ground, into an imbalance section and a falling down section depending on whether a posture is returnable, wherein the program includes the following steps of: acquiring fall data measuring a posture of a user; predicting a fall progression step corresponding to the fall data using a fall step classification model; and determining a situation, in which the predicted fall progression step advances to the falling down section and reaches a point of no return (PONR), as a fall.

Description

Apparatus and Method for Fall Recognition based on Detecting Point of No Return}

The disclosed embodiment relates to a fall prediction technology, and more particularly, measures to protect the body by predicting the possibility of a fall before colliding with the ground when a fall occurs using body movement data measured from a wearable sensor attached to the user's body. It is about a fall inference method using artificial intelligence that enables to take

Studies on falls have been studied for a long time from various perspectives. In addition, as the proportion of the elderly population increases exponentially, falls in the elderly are emerging as a social problem. In Korea, as can be seen from the report on the results of a recent survey of the elderly by the Ministry of Health and Welfare, the proportion of the elderly who experienced a fall for one year (elderly fall rate) was 15.9% in 2017.

Looking at the existing technology for determining falls, the fall detection process determines whether the user's lying state is due to a fall in the section before and after the fall, and detects the fall through a two-step algorithm that uses the amount of impact at the time of the fall. Determine This technology is applied to a fall detection device that is worn on the body and is linked to an integrated fall accident management system.

Another prior art uses a 3-axis acceleration sensor to detect movement and motion status, recognizing it as a fall or physical crisis situation when the motion disappears after rapid motion lasts for several seconds, and sends an emergency text message to a previously reserved contact. It is a transmission technology.

In the prior art using a camera, it is determined whether or not the user has fallen through the pressure generated while the user falls, the user's voice related to the fall, and the user's state following the fall (photographed by a camera), and in case of a fall, the user who has fallen through a terminal. It is a technique that allows the guardian to take appropriate action according to the state of the child. This method can be used only in places where cameras can be installed nearby, so it can be used only in specific places such as hospitals.

In the prior art, there is also a technology for performing a fall by including an acceleration sensor and a gyro sensor in a smart band wearable on a user's wrist, and determining whether or not to fall using values detected by the acceleration sensor and the gyro sensor.

The prior art relates to a fall detection method for detecting a user's fall before colliding with the floor, and a threshold value before the user's body collides with the floor using an acceleration or gyro measurement value measured by an inertial sensor worn by a tester. is measured and used as a threshold value to determine whether the user has fallen, and the threshold value is continuously compared with the inertial sensor value measured by the inertial sensor worn on the user's body, and if it is greater than the threshold value, a separate control unit is used. Thus, a fall detection method including a determination step of determining a fall before the user collides with the floor is described.

In the prior art as described above, a method of determining a fall is mainly used when the sensors used in the fall exceed a predetermined reference value.

An object of the described embodiment is to determine whether or not a person has fallen while monitoring a process in which a fall is progressing in real time.

An object of the disclosed embodiments is to improve the accuracy of predicting a fall, which can vary depending on whether a user can recover his balance by himself after a fall inducing factor starts in the course of a fall.

An apparatus for detecting a fall based on point of no recovery according to an embodiment includes a memory in which at least one program and a pre-learned fall stage classification model are recorded, and a processor executing the program, wherein the fall stage classification model is configured at a fall start point. Pre-trained to predict the critical step until the body touches the ground by subdividing it into an imbalance section and a falling section depending on whether posture recovery is possible, but the program uses fall data that measures the user's posture Obtaining a fall stage classification model, predicting a fall progression stage corresponding to the acquired fall data using a fall stage classification model, and the predicted fall progression stage progressing to a falling down section to a point of no return (Point of No Return). , PONR) may be determined as a fall.

According to the described embodiment, it is possible to determine whether or not a fall occurs while monitoring a process in which a fall progresses in real time.

According to the described embodiments, it is possible to improve the accuracy of predicting a fall, which may vary depending on whether the user can recover his/her balance after the fall inducing factor starts in the course of the fall.

1 is a schematic block diagram of an apparatus for detecting a fall based on point of no recovery detection according to an embodiment.
2 is a graph for explaining a fall progression stage predicted by a fall stage classification model according to an embodiment.
3 and 4 are exemplary diagrams of a fall stage classification model according to an embodiment.
5 and 6 are exemplary views of labeling fall data according to an embodiment.
7 is a flowchart illustrating a method for recognizing a fall based on detection of a point of no recovery according to an exemplary embodiment.
8 is a diagram showing the configuration of a computer system according to an embodiment.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

Although "first" or "second" is used to describe various elements, these elements are not limited by the above terms. Such terms may only be used to distinguish one component from another. Therefore, the first component mentioned below may also be the second component within the technical spirit of the present invention.

Terms used in this specification are for describing embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" or "comprising" implies that a stated component or step does not preclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used herein may be interpreted as meanings commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

Hereinafter, an apparatus and method for detecting a fall based on detection of a point of no recovery according to an embodiment will be described in detail with reference to FIGS. 1 to 8 .

1 is a schematic block diagram of an apparatus for detecting a fall based on point of no recovery detection according to an embodiment.

Referring to FIG. 1 , an apparatus for detecting a fall based on point of no recovery detection according to an embodiment 100 may include a fall data acquisition unit 110 , a fall determination unit 130 , and a fall stage classification model 140 . Additionally, a pre-processing unit 120 and a post-fall action performing unit 150 may be further included.

The fall data acquisition unit 110 acquires fall data obtained by measuring a user's posture.

In this case, the fall data may include data measured by at least one sensor worn by the user or image data photographing the user.

In addition, the fall recognition apparatus 100 based on the point of no recovery detection of the fall data acquisition unit 110 may acquire fall data of a user collected by a separate, physically separated server device through wired/wireless communication.

The preprocessor 120 may convert the fall data obtained from the fall data acquisition unit 110 into data that can be input to a fall stage classification model. Alternatively, feature data may be extracted from fall data.

The fall stage classification model 140 is designed based on a deep learning neural network algorithm, and the critical stage from the start of the fall to the point when the body touches the ground is divided into an imbalance section and a fall ( It is pre-learned to predict by subdividing into sections (falling down).

2 is a graph for explaining a fall progression stage predicted by a fall stage classification model according to an embodiment.

Referring to FIG. 2, there are two graphs showing the progress of a fall. In each of the two graphs, the x-axis represents the flow of time, and the y-axis represents the position of the center of mass (COM) in the direction of gravity .

The graph shown at the top of FIG. 2 is a conventional fall process, which is divided into four stages: Pre Fall Phase, Critical Phase, Post Fall Phase, and Recovery Phase. Separated.

At this time, the time point T ₀ is the time point when the balance starts to be disturbed by the factor causing the fall, the time point T ₁ is the point where the chest or hip of the body touches the ground, and the fall is completed after the point T ₁ .

As shown in FIG. 2 , in the conventional fall process, the time interval T ₀ to T ₁ is simply classified as a critical phase. Accordingly, in the related art, it is possible to reliably determine whether or not a fall has occurred only at time T ₁ when the critical phase ends. This is the point at which the fall has already been completed, and the prediction of the fall is delayed, which may lead to a serious accident due to delay in follow-up measures.

Therefore, in the embodiment, it is intended that the fall determination be made before the time point T ₁ when the chest, hip, etc. of the body touches the ground, so that follow-up measures are taken.

To this end, in the embodiment, as shown in the graph shown at the bottom of FIG. 2 and Table 1 below, the imbalance section and the falling down section depend on whether or not posture recovery is possible in the critical phase. subdivided into , and detects the Point of No Return (PONR).

4 stages of a fall Critical step segmentation Pre Fall Phase standing (S) ADL stage walking (W) Critical Phase Imbalance(I) fall progress Falling down (F) Post Fall Phase -Post fall (P) Stages after a fall Recovery Phase -Recovery(R)

Referring to FIG. 2 and <Table 1>, the pre-fall phase refers to daily activities. For example, it may include “standing” and “walking” activities. As described above, according to the embodiment, points T ₀ to T ₁ corresponding to critical phases are subdivided into imbalance sections and falling down sections starting from point Tα.

That is, the imbalance period is T _{0 to} Tα, and is a period in which the user's balance has started to be disturbed due to factors that cause falls, but the balance can be restored by itself according to the user's exercise ability so as not to lead to falls. Therefore, it is difficult to accurately determine a fall in an imbalance section even if the balance is disturbed by a factor causing a fall.

The falling down section is a point between Tα and T ₁ , and may be a section in which balance is disturbed to some extent and it is difficult to recover the posture with the user's exercise ability. Therefore, a point at which a transition from an imbalance section to a falling down section is detected as a Point of No Return (PONR). After the point of no recovery (PONR) detected in this way, whether or not a fall will occur can be predicted with certainty. Accordingly, in the embodiment, the fall determination may be determined after detecting the point of no recovery (PONR) as described above.

That is, the fall determination unit 130 predicts an input fall progress stage corresponding to the acquired fall data using the fall stage classification model 140, and the predicted fall progress stage proceeds to a falling down section. If the fall reaches the point of no return (PONR), it can be judged as a fall.

Meanwhile, the fall stage classification model 140 discretely infers the prediction result of the fall progress stage corresponding to the data received from the sensor worn by the user from the start of the fall at predetermined time intervals, for example, 10 ms intervals. can do. Therefore, the fall determining unit 130 determines the PONR after the predicted fall progression step is switched from the Imbalance section to the Falling down section, and then the prediction result of the Falling down section is continuously displayed. When a predetermined number, for example, 3 or 4 repetitions, it may be determined that the PONR has been reached. Therefore, after the PONR, the user's state may be determined as a fall. That is, it may already be determined as a fall before the point at which the user's body touches the ground.

If the fall is determined by the fall determination unit 130, the post-fall action performing unit 150 may take post-fall measures according to the user's fall. For example, a safety device to protect the user may be operated, or an alarm or contact may be made to a guardian or an emergency facility. Depending on the embodiment, since the fall determination is quickly made before the user's body touches the ground, it is possible to prevent a post-fall measure or a serious accident caused by rapid falls.

Next, learning of the fall stage classification model 140 as described above will be described with reference to FIGS. 3 to 6 .

3 and 4 are exemplary diagrams of a fall stage classification model according to an embodiment.

Referring to FIGS. 3 and 4 , a fall stage classification model 140 according to an embodiment may be designed based on various deep learning neural network algorithms.

For example, as shown in FIG. 3 , the fall stage classification model 140 applies a long short-term memory (LSTM) algorithm to learn sensor data as much as a window size, and through the prediction result, the fall stage classification model 140 It can be taught to judge progress.

Alternatively, as shown in FIG. 4 , the fall stage classification model 140 calculates the window size of IMU sensor data and the number of inputs of sensors based on a Convolutional Neural Network (CNN) into image data ( By mapping to (height, width) of image data and learning, it is possible to classify fall data input as a momentary still scene among fall progression steps. In addition, the CNN algorithm has an advantage that it can be applied to a more lightweight embedded system because the execution time is shorter than that of the LSTM algorithm.

At this time, depending on the embodiment, the window size is the 'Imbalance (I) period' and 'Imbalance (I) period' in which real-time falls occur, rather than the duration of the 'standing (S) period' or 'walking (W) period' in <Table 1>. It may be determined in consideration of the duration of the 'Falling down (F) section'. For example, when the average duration of the 'Imbalance (I) section' of fall data is 420 [ms] and the average duration of the 'Falling down (F) section' is 250 [ms], 10 [ms] Since fall data is input every time, when the window size is 25, the time size of the window can be set to 250 [ms], which is the average duration of the 'Falling down (F) section'.

Meanwhile, in order to collect learning data for learning the fall stage classification model 140 designed as described above, a data capture device, real-time data processing S/W, acquired data preview, and a visual tool for editing may be configured.

Commercial data capture devices usually acquire motion data of the whole body with 17 IMU sensors, and a single sensor is capable of data at a fast cycle of 100 [Hz], but when using the whole, the capture cycle becomes longer depending on the transmission capacity. In addition, there are cases in which real-time data cannot be directly used because only data stored in dedicated S/W can be used.

Therefore, the data capture device in the embodiment is easy to wear by configuring five IMU sensors in a daisy chain using a USB hub. Real-time data processing S/W is a server-client structured S/W that manages data coming from 5 sensors in packet form and transmits the collected data using wired/wireless network communication. The visual tool for previewing and editing acquired data checks the movement of the data acquired from the data capture device into a skeleton-type image, prevents incorrect data from being collected, and enables label editing in preparation for manual labeling. in order to be able to

Fall data acquisition first outputs quaternion, euler, accelerometer, gyroscope, and magnetic data at 100 [Hz] from the IMU sensor of the data capture device, and the real-time data processing S/W that receives it manages the received data while wired/wireless It is transmitted to Visual Tool through the network. Visual tool monitors real-time data through Preview, saves in CSV format, and edits labels as needed.

The fall learning data acquired as described above may be labeled based on the center of mass (COM) of the human body. This COM is an indicator for evaluating the gait and postural stability of the human body, and provides very intuitive and important information. The gait and posture of the human body are more stable as the COM is located closer to the BOS. This concept has been actively used in algorithms such as ZMP (Zero Moment Point) control method for posture control of foot-type robots, especially biped walking robots, as well as human body posture analysis. Therefore, in the embodiment, a labeling technique for distinguishing postural stability of the human body through COM is applied.

First, COM can be calculated by the following <Equation 1>.

<Equation 1>

와

는

번째 인체 부분에 가해지는 모멘트와 힘을 뜻하고,

는

번째 인체 부분의 무게,

는

번째 인체 부분의 무게 중심의 위치를 뜻한다. here,

and

Is

It refers to the moment and force applied to the second human body part,

Is

the weight of the second human body part,

Is

It refers to the position of the center of gravity of the second human body part.

방향의 무게중심 계산식을 동일하게 적용하여

와

방향의 무게중심 또한 계산할 수 있다.The human body part means a part of the human body classified for each joint of the human body, for example, the upper arm and lower arm of the arm, the thigh and calf of the leg, etc., calculated by <Equation 1>

By applying the same formula for calculating the center of gravity of the direction

and

The center of gravity of the direction can also be calculated.

Theoretically, the center of gravity of the human body applies at least 17 IMU sensors to consider all human body parts. By the way, in the embodiment, since the analysis of the posture change by the movement of the lower body is focused, the center of gravity is inferred using only the five IMU sensors attached to the lower body and the waist. That is, for example, fall data may be labeled using norm information of COMs inferred from five IMU sensors.

5 and 6 are exemplary views of labeling fall data according to an embodiment.

5 and 6, the black dotted line is the standing section of ADL (Activities of Daily Living) and is TAG1, the yellow dotted line is the walking section of ADL, TAG2, the blue dotted line is the imbalance section and is TAG3, The red dotted line is the falling down section and is labeled as TAG4. index 1 is in units of 10 [ms]. COM(x,y) represents the label value of the fall data as the coordinates in which the COM moves on the plane.

Figure 6 is a case of a backward fall. In fact, walking was not performed, but looking at COM(x, y), it can be seen that the inside of the circle constituting the BOS boundary is also divided into two sections.

In Figure 5, the black dots of the standing section of COM(x, y) are concentrated at one point and the yellow dots of the walking section cover it, so they are not clearly distinguished. Looking at the red dotted line section (falling section), the end point of the red dotted line means the point where the body touches the ground.

Therefore, the fall stage classification model 100 designed as shown in FIGS. 3 and 4 can receive the acquired fall data for learning as described above and learn to minimize an error with the labeled correct value.

7 is a flowchart illustrating a method for recognizing a fall based on detection of a point of no recovery according to an exemplary embodiment.

Referring to FIG. 7 , the fall recognition device 100 (hereinafter, referred to as 'device') based on detection of a point of no recovery according to an embodiment obtains real-time fall data obtained by measuring a user's posture (S210).

In this case, the fall data may include data measured by at least one sensor worn by the user or image data photographing the user.

In addition, fall data of a user collected by a separate server device physically separated from the fall recognition apparatus 100 based on point of no recovery detection may be obtained through wired/wireless communication.

The apparatus 100 may pre-process the acquired fall data to be converted into data that can be input to the fall stage classification model (S220). Alternatively, at this time, feature data may be extracted from fall data.

The apparatus 100 classifies an input fall progress stage corresponding to the acquired fall data using the fall stage classification model 140 described above (S230).

The apparatus 100 determines whether the fall progression phase has entered a critical phase, that is, whether a fall inducing factor has occurred (S240).

As a result of the determination in S240, if the critical step has not been entered, the device 100 proceeds to S210.

On the other hand, as a result of the determination in S240, if it has entered the critical stage, the device 100 monitors whether the predicted fall progress stage progresses to the falling section and reaches the Point of No Return (PONR). (S250). At this time, according to an embodiment, the fall stage classification model 140 discretely infers the prediction result of the fall progress stage corresponding to the data received from the sensor worn by the user at predetermined time intervals, for example, 10 ms intervals. can do.

Therefore, in S250, the device 100 determines the PONR, after the predicted fall progress step is switched from the Imbalance section to the Falling down section, the prediction result of the Falling down section is continuously determined. When the number, for example, 3 to 4 repetitions, it can be determined that the PONR has been reached.

Accordingly, the device 100 may determine the user's state as a fall after the PONR (S260). That is, it may already be determined as a fall before the point at which the user's body touches the ground.

After that, if it is determined that the fall occurred in S260, the apparatus 100 may take follow-up measures according to the occurrence of the user's fall (S270). For example, a safety device to protect the user may be operated, or an alarm or contact may be made to a guardian or an emergency facility. Depending on the embodiment, since the fall determination is quickly made before the user's body touches the ground, it is possible to prevent a post-fall measure or a serious accident caused by rapid falls.

8 is a diagram showing the configuration of a computer system according to an embodiment.

The fall detection device 100 based on point of no recovery detection according to the embodiment may be implemented in a computer system 1000 such as a computer-readable recording medium.

Computer system 1000 may include one or more processors 1010, memory 1030, user interface input devices 1040, user interface output devices 1050, and storage 1060 that communicate with each other over a bus 1020. can In addition, computer system 1000 may further include a network interface 1070 coupled to network 1080 . The processor 1010 may be a central processing unit or a semiconductor device that executes programs or processing instructions stored in the memory 1030 or the storage 1060 . The memory 1030 and the storage 1060 may be storage media including at least one of volatile media, nonvolatile media, removable media, non-removable media, communication media, and information delivery media. For example, memory 1030 may include ROM 1031 or RAM 1032 .

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art can implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand that there is Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting.

100: Fall recognition device based on point of no recovery detection
110: fall data acquisition unit 120: pre-processing unit
130: fall determination unit 140: fall stage classification model
150: post-fall action department

Claims (1)

a memory in which at least one program and a pre-learned fall stage classification model are recorded; and
A processor that executes a program;
The fall stage classification model,
It is pre-learned to predict the critical step from the start of the fall to the point when the body touches the ground by subdividing it into an imbalance section and a falling down section depending on whether posture recovery is possible,
program,
acquiring fall data obtained by measuring a user's posture;
predicting a fall progress stage corresponding to the acquired fall data using a fall stage classification model; and
A fall recognition device based on point of no recovery detection that determines a fall when the predicted fall progression progresses to the falling down section and reaches the point of no return (PONR).

Images