CN114694252B - Old people falling risk prediction method - Google Patents

Abstract

The invention discloses a method for predicting the falling risk of the elderly, which comprises the following steps: s1, data acquisition and pretreatment; s2, performing skeleton modeling; s3, video time segmentation based on action characteristics: dividing the video into 7 segments, labeled S ₀ ～S ₆ The method comprises the steps of carrying out a first treatment on the surface of the The total 6 time division points among the 7 fragments are respectively marked as T in sequence ₁ ～T ₆ The method comprises the steps of carrying out a first treatment on the surface of the S4, judging time risk; s5, extracting gesture features of each action stage according to frames from the video of each stage obtained by segmentation in the step S3; s6, judging the attitude risk according to the attitude characteristics obtained in the step S5; and S7, carrying out final comprehensive risk judgment according to the time risk and the gesture risk obtained in the steps S4 and S6. According to the invention, the time characteristics and the gesture characteristics of the movement process are combined, the obtained risk assessment result can reflect the real balance and movement capability, and the accuracy of the falling risk prediction result is improved.

Description

Old people falling risk prediction method

Technical Field

The invention relates to a fall risk prediction method for old people.

Background

The physical quality, skeletal muscle function and other aspects of the old people can be naturally weakened, the falling risk is increased, and the falling easily causes injuries such as fracture and the like. In order to discover the falling risk early and take effective protective measures, the falling risk assessment of the old has important practical significance. TUG Test, a time-to-rise walking Test, is a well-established method of predicting fall risk for elderly people today, is of great interest and has proven effectiveness.

In recent years, researchers have applied computer technology to TUG testing, i.e., to automate the TUG test. The computer automation of the TUG test can not only keep the utility of the TUG test, but also reduce the direct intervention of professional medical staff, thereby improving the popularity of the test, and effectively reducing the possible human error problem in operation.

The patent application with the application number of 201911412802.2 discloses a falling risk assessment method and system for old people, which aims at extracting gait characteristics such as single-leg suspension displacement, knee joint angle, ankle joint angle and the like in the walking process of the people and judging the falling risk of the old people.

The existing TUG automation method is limited to the timing characteristics of the TUG, ignores rich motion characteristics and gesture characteristics in the TUG process, performs characteristics and algorithms used for time segmentation, and is poor in robustness in practical application. The complete TUG action flow includes activities such as standing, walking, turning around, sitting down, etc., which can reflect the physical strength, balance level, etc. characteristics of the elderly from different angles. The 201911412802.2 application mainly focuses on extracting features of the walking stage to determine the risk of falling, which affects the accuracy of the risk determination of falling.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide the old people falling risk prediction method which integrates the time characteristics and the gesture characteristics of the movement process, and the obtained risk assessment result can reflect the real balance and movement capability and can improve the accuracy of the falling risk prediction result.

The aim of the invention is realized by the following technical scheme: a method for predicting fall risk of an elderly person, comprising the steps of:

s1, data acquisition and preprocessing, which comprises the following substeps:

s11, arranging a data acquisition environment according to a TUG standard: a chair with the height of 60cm is opposite to a flat pavement with the length of 3 m; the pixel of the image pickup device is required to be at least 720p and is fixedly arranged at the height of 1.5 m;

s12, starting shooting when the subject sits at the end, then rising up and advancing, turning back after reaching the end point of the walk, returning to the starting point, re-sitting, and stopping shooting; the resulting video pre-processing is 1280x720 pixels, 30 frames per second;

s13, taking the real falling times of the subject in the time of nearly two years as a label of sample data to form a training data set;

s14, dividing the falling risk into 3 grades: the low risk level corresponds to no fall record within 2 years, the medium risk level corresponds to 1 to 2 fall records within 2 years, and the high risk level corresponds to 3 or more fall records;

s2, performing skeleton modeling: modeling human bodies in video into a skeleton of 13 nodes by using OpenPose, using J ₀ ～J ₁₂ Representing the corresponding joint, specifically includes: j (J) ₀ -head, J ₁ Left shoulder, J ₂ Right shoulder, J ₃ Left elbow, J ₄ -right elbow, J ₅ -left wrist, J ₆ -right wrist, J ₇ Left hip, J ₈ -right hip, J ₉ Left knee, J ₁₀ Right knee, J ₁₁ Left ankle, J ₁₂ -a right ankle;

acquiring a horizontal axis coordinate and a vertical axis coordinate on each frame of 2d image through an OpenPose frame;

reconstructing an absolute coordinate obtained by OpenPose, and constructing a relative coordinate system: in each frame of image, the center of the person, J, is found by the absolute coordinates of the nodes ₁ 、J ₂ 、J ₇ 、J ₈ A person is framed by a rectangular outer frame with fixed proportion, the left lower corner of the outer frame is taken as the origin of coordinates, the direction of the longitudinal axis of the transverse axis is unchanged, and a new coordinate system is established;

s3, video time segmentation based on action characteristics: dividing the video into 7 segments, labeled S ₀ ～S ₆ The method comprises the steps of carrying out a first treatment on the surface of the The total 6 time division points among the 7 fragments are respectively marked as T in sequence ₁ ～T ₆ The whole video start and end time points are respectively denoted by T ₀ and T₇ A representation; wherein S is ₀ Corresponding to sitting and S ₁ Corresponding to standing up, S ₂ Corresponding to forward walking, S ₃ Corresponding to turning around in situ, S ₄ Corresponding to reverse walking, S ₅ Corresponding to turning around and sitting on site, S ₆ Corresponding to the sitting;

s4, judging the time risk according to the time characteristics obtained in the step S3 in a segmentation mode, wherein the specific method comprises the following steps of: the 5 time characteristics are defined as follows:

total time: f (F) ₁ ＝|S ₁ |+|S ₂ |+|S ₃ |+|S ₄ |+|S ₅ |＝T ₆ -T ₁

Time of the rising phase: f (F) ₂ ＝|S ₁ |＝T ₂ -T ₁

Walking phase duration: f (F) ₃ ＝|S ₂ |+|S ₄ |＝T ₅ +T ₃ -T ₄ -T ₂

Turning phase time: f (F) ₄ ＝|S ₃ |＝T ₄ -T ₃

Swivel sitting phase time: f (F) ₅ ＝|S ₅ |＝T ₆ -T ₅

Threshold Th are set for five time features ₁ 、Th ₂ 、Th ₃ 、Th ₄ 、Th ₅ The different time features have different weights for risk assessment, and a risk score weight W is set for each time feature ₁ 、W ₂ 、W ₃ 、W ₄ 、W ₅ Risk score R for temporal features _time ：

wherein ,

risk score R for a temporal feature _time Exceeding a preset integrated time threshold Th _sum Directly recognizing that the risk rating is high without performing subsequent steps; otherwise, executing the step S5;

s5, extracting gesture features of each action stage according to frames from the video of each stage obtained by segmentation in the step S3;

s6, judging the attitude risk according to the attitude characteristics obtained in the step S5;

and S7, carrying out final comprehensive risk judgment according to the time risk and the gesture risk obtained in the steps S4 and S6.

Further, in the step S3, the specific method for video time segmentation based on the action feature is as follows:

s31, constructing the following action characteristics:

junction J ₇ 、J ₈ 、J ₉ and J₁₀ The height difference of (2) is defined as follows:

Y(J ₇ )、Y(J ₈ )、Y(J ₉ )、Y(J ₁₀ ) Respectively represent J ₇ 、J ₈ 、J ₉ and J₁₀ Is the ordinate of (2);

junction J ₁ J of (2) ₂ The horizontal distance difference of (2) is defined as:

F_clip ₂ ＝|X(J ₁ )-X(J ₂ )|

X(J ₁ )、X(J ₂ ) Respectively indicated J ₁ J of (2) ₂ Is the abscissa of (2);

junction J ₇ and J₈ The horizontal distance difference of (2) is defined as:

F_clip ₃ ＝|X(J ₇ )-X(J ₈ )|

X(J ₇ )、X(J ₈ ) Respectively J ₇ and J₈ Is the abscissa of (2);

s32, respectively calculating three action features described in S31 for each frame of image to obtain an F_clip ₁ 、F_clip ₂ 、F_clip ₃ Three discrete sequences, namely, using a binary inflection point search algorithm BinSeg of an L2 loss model to match experience parameters to obtain inflection points of the sequences, and obtaining the stage division of actions according to the inflection points; the method comprises the following steps:

using F_clip ₁ Sequence, adopting inflection point search algorithm to match priori knowledge to judge S ₀ And S is equal to ₁ Is defined by the boundary point T of (2) ₁ The method comprises the steps of carrying out a first treatment on the surface of the Specifically T ₁ Appear in F_clip ₁ At the 1 st inflection point;

using F_clip ₁ Determination S ₁ And S is equal to ₂ Is defined by the boundary point T of (2) ₂ ；T ₂ Appear in F_clip ₁ At the 2 nd inflection point;

using F_clip ₂ Determination S ₂ And S is equal to ₃ Is defined by the boundary point T of (2) ₃ ；T ₃ Appear in F_clip ₂ At the 1 st inflection point;

using F_clip ₃ Determination S ₃ And S is equal to ₄ Is defined by the boundary point T of (2) ₄ ；T ₄ Appear in F_clip ₃ Is at the 3 rd inflection point;

using F_clip ₂ Determination S ₄ And S is equal to ₅ Is defined by the boundary point T of (2) ₅ ；T ₅ Appear in F_clip ₂ At the 4 th inflection point;

using F_clip ₁ Determination S ₅ And S is equal to ₆ Is defined by the boundary point T of (2) ₆ ；T ₆ Appear in F_clip ₁ At the 4 th inflection point of (c).

Further, the specific implementation method of the step S5 is as follows:

s51, lifting-center axis front-rear offset characteristic F ₆ : in the standing process, the central axis deviates from the maximum angle when sitting still, and the trunk central axis is obtained by calculating the coordinates of the left shoulder, the right shoulder, the left hip and the right hip; the specific calculation method is as follows:

firstly, calculating the slope of the central axis of the trunk:

then the slope is converted into an radian system angle, which is represented by a symbol theta;

for each time t _i Calculating the slope according to the above

And converting into radian angle>

Then F is obtained ₆ The following are provided:

wherein T₁ <t _i <T ₂ Corresponding to the moments in all standing processes; θ _sit Mean offset angle for the finger sitting phase;

s52, calculating the mean angle F of the center axis deviation in the turning process ₇ ：

S53, calculating the maximum angle F when the central axis deviates from standing straight in the sitting process ₈ ：

wherein T₅ <t _i <T ₆ ，θ _stand Average deflection angle at the sitting stage;

s54, calculating the height ratio characteristic F of the walking step ₉ : the horizontal movement distance between two places where the ankle of the single foot falls is called a single step length, the single step length of the two feet is averaged in the walking process, and the ratio of the average step length to the height is calculated;

obtaining a landing time point set by extremum through ankle height sequence:

left foot landing time point set:

A _{floor_left} ＝{t _m |T ₂ <t _m <T ₃ or T ₄ <t _m <T ₅ And X' (t) _m ) Is minimum }

t _m For a time point corresponding to the minimum value of the m-th left ankle abscissa in the absolute coordinate system obtained by openPose, X' (t) _m ) Representing t _m Left ankle abscissa in the absolute coordinate system of (2); removing the landing time points of the head and tail incomplete steps, and counting the average 1-step length;

right foot landing time point set:

A _{floor_right} ＝{t _n |T ₂ <t _n <T ₃ or T ₄ <t _n <T ₅ And X' (t) _n ) Is minimum }

t _n The n-th right ankle abscissa in the absolute coordinate system obtained for OpenPose is extremely smallTime point corresponding to the value, X' (t _n ) At t _n Right ankle abscissa in the absolute coordinate system of (2); removing the landing time points of the head and tail incomplete steps, and counting the average 1-step length;

the single step length of the left foot is as follows:

respectively the m+1th left ankle abscissa and the m left ankle abscissa in the absolute coordinate system;

the single step length of the right foot is as follows:

n+1th and n right ankle abscissas in the absolute coordinate system, respectively;

height ratio characteristic F of walking step ₉ The calculation is as follows:

wherein BH represents body height;

s55, calculating left-right offset characteristic F of walking center shaft ₁₀ In the walking process, the horizontal left-right average angle difference when the middle shaft deviates from the standing straight is calculated as follows:

wherein θ_std Center axis angle when standing straight:

s56, amplitude characteristic F of walking swing arm ₁₁ : obtaining vector angle differences formed by the left shoulder, the left elbow, the right shoulder and the right elbow nodes, namely elbow-shoulder vectors when the backward swing is farthest and elbow-shoulder vectors when the forward swing is farthest, wherein the angle differences are the difference;

firstly, respectively calculating the swing arm slopes of a left arm and a right arm:

X'(J _c )、Y'(J _c ) J in absolute coordinate systems obtained for OpenPose respectively _c C=1, …,4;

converting slope into radian angle alpha _{left_arm} And alpha is _{right_arm} ；

Calculating the swing arm angles of the left arm and the right arm in each frame to form a sequence of the swing arm angles of the left arm and the right arm, and obtaining an extremum in the sequence to obtain a maximum angle A of the front swing of the left arm _{af_left} Maximum angle A of left arm backswing _{ab_left} Maximum angle A of front swing of right arm _{af_right} Maximum angle A of right arm backswing _{ab_right} ；

Will be adjacent A _{af_left} And A is a _{ab_left} Aligning and differencing to obtain a single swing arm angle set A of the left arm _{as_left} ；

Will be adjacent A _{af_right} And A is a _{ab_right} Aligning and differencing to obtain a single swing arm angle set A of the right arm _{as_right} ；

Removal A _{as_left} And A is a _{as_right} The first and last data in both sets are then combined with the remaining set A' _{as_left} A' _{as_right} Based on which F is calculated ₁₁ ：

wherein α_m ∈A' _{as_left} ，α _n ∈A' _{as_right} ，|A' _{as_left} |、|A' _{as_right} I respectively represent the set A' _{as_left} 、A' _{as_right} The number of elements in (a);

s57, calculating the amplitude characteristic F of the walking swing leg ₁₂ : obtaining vector angle difference formed by the left hip, the left knee, the right hip and the right knee nodes;

and respectively calculating the swing slope of the left leg and the right leg:

X'(J _c )、Y'(J _c ) J in absolute coordinate systems obtained for OpenPose respectively _c C=7, …,10;

will K _{left_leg} and K_{right_leg} Conversion from slope to radian _{left_leg} And beta _{right_leg} ；

Calculating the swing angles of the left leg and the right leg in each frame to form a sequence of the swing angles of the left leg and the right leg, and obtaining an extremum in the sequence to obtain a maximum swing angle A of the front of the left leg _{lf_left} Maximum angle A of left leg backswing _{lb_left} Maximum angle A of front swing of right leg _{lf_right} Maximum angle A of rear swing of right leg _{lb_right} ；

Will be adjacent A _{lf_left} and A_{lb_left} Aligning and differencing to obtain a left leg swing angle sequence A _{ls_left} ；

Will be adjacent A _{lf_right} and A_{lb_right} Aligning and differencing to obtain a right leg swing angle sequence A _{ls_right} ；

Removal of sequence A _{ls_left} and A_{ls_right} Data of the first and the last of (a) and then in the remaining sequence A' _{ls_left} and A'_{ls_right} Obtaining F ₁₂ The calculation is as follows:

wherein β_m ∈A' _{ls_left} ，β _n ∈A' _{ls_right} ，|A' _{ls_left} |、|A' _{ls_right} I respectively represent the sequence A' _{ls_left} and A'_{ls_right} The number of elements in (a).

Further, the specific implementation method of the step S6 is as follows: will gesture feature F ₆ ～F ₁₂ Inputting the attitude risk value R into a multi-layer perceptron to carry out regression prediction _posture 。

Further, the specific implementation method of the step S7 is as follows: if the time risk R has been obtained in step S4 _time Greater than threshold Th _sum The comprehensive risk judgment directly obtains a result, and the falling risk is high; otherwise, the following determination is made:

setting the weight W by linear fitting _time and W_posture Final risk R _final ＝W _time ·R _time +W _posture ·R _posture ；

Will end risk R _final And a high risk reference threshold Th _danger Stroke risk reference threshold Th _alarm Comparing to be higher than or equal to the high risk reference threshold Th _danger The subject is considered to have a high risk of falling, being greater than or equal to the stroke risk threshold Th _alarm While being below the high risk threshold Th _danger The subject is considered to have a risk of falling well below the risk of falling reference threshold Th _alarm The subject is considered to have a low risk of falling.

The beneficial effects of the invention are as follows: according to the method, the action video is divided into different stages, the time dimension falling risk is predicted based on the time characteristics of each stage, and preliminary falling risk judgment is carried out; then constructing gesture features at different stages based on video segmentation results, inputting the gesture features into a multi-layer perceptron to carry out regression prediction, and obtaining gesture risk prediction results of space dimensions; and finally, combining the risks of two dimensions of time and space to obtain a final falling risk prediction result. According to the invention, the time characteristics and the gesture characteristics of the movement process are combined, the obtained risk assessment result can reflect the real balance and movement capability, and the accuracy of the falling risk prediction result is improved.

Drawings

FIG. 1 is a flow chart of a method for predicting risk of falling over of an elderly person according to the present invention;

FIG. 2 is a schematic diagram of a data acquisition environment according to the present invention;

FIG. 3 is a schematic diagram of the skeletal modeling of the present invention;

FIG. 4 is a schematic diagram of natural partitioning of motion phases according to the present invention;

fig. 5 is a schematic diagram of the present invention for obtaining the phase division of the motion according to the inflection point.

Detailed Description

The technical scheme of the invention is further described below with reference to the accompanying drawings.

As shown in fig. 1, the method for predicting the fall risk of the elderly according to the present invention comprises the following steps:

s1, data acquisition and preprocessing, which comprises the following substeps:

s11, arranging a data acquisition environment according to a TUG standard: a chair with the height of 60cm is opposite to a flat pavement with the length of 3 m; the pixel of the image pickup device is required to be at least 720p and is fixedly arranged at the height of 1.5m, and the connecting line of the image pickup device and the end point of the walkway is perpendicular to the walkway, as shown in fig. 2;

s12, starting shooting when the subject sits at the end, then rising up and advancing, turning back after reaching the end point of the walk, returning to the starting point, re-sitting, and stopping shooting; the resulting video pre-processing is 1280x720 pixels, 30 frames per second;

s13, taking the real falling times of a subject in the time of nearly two years as a label of sample data to form a training data set for parameter fitting;

s14, dividing the falling risk into 3 grades: the low risk level corresponds to no fall record within 2 years, the medium risk level corresponds to 1 to 2 fall records within 2 years, and the high risk level corresponds to 3 or more fall records;

s2, performing skeleton modeling: modeling human bodies in video into a skeleton of 13 nodes by using OpenPose, using J ₀ ～J ₁₂ Representing the corresponding joint of the joint,as shown in fig. 3, the method specifically includes: j (J) ₀ -head, J ₁ Left shoulder, J ₂ Right shoulder, J ₃ Left elbow, J ₄ -right elbow, J ₅ -left wrist, J ₆ -right wrist, J ₇ Left hip, J ₈ -right hip, J ₉ Left knee, J ₁₀ Right knee, J ₁₁ Left ankle, J ₁₂ -a right ankle;

acquiring a horizontal axis coordinate and a vertical axis coordinate on each frame of 2d image through an OpenPose frame; in the following formula, X represents a horizontal coordinate and Y represents a vertical coordinate.

In the human body movement process, the size of projection of a person in an image can be changed along with the movement, and in order to reduce the influence of the change of the size of the person on a characteristic value, the absolute coordinates obtained by OpenPose are reconstructed to construct a relative coordinate system, and the method comprises the following steps: in each frame of image, the center of the person, J, is found by the absolute coordinates of the nodes ₁ 、J ₂ 、J ₇ 、J ₈ A person is framed by a rectangular outer frame with fixed proportion, the left lower corner of the outer frame is taken as the origin of coordinates, the direction of the longitudinal axis of the transverse axis is unchanged, and a new coordinate system is established; the unit length of the new coordinate system is scaled according to the size of the outer frame, namely, the image in the outer frame is scaled to a uniform size. The size of the outer frame is selected according to the strategy that the height (absolute value of the Y-axis coordinate difference) and the width (absolute value of the X-axis coordinate difference) are considered simultaneously, when the height-width ratio is larger than the ratio r, the outer frame is defined by taking the height as a standard, and when the height-width ratio is smaller than r, the outer frame with the fixed aspect ratio is defined by taking the width as a standard, so that the state of curling of the body can be dealt with.

The final image should have the following characteristics that the horizontal axis and the vertical axis of the image obtained by frame selection are almost occupied by the human body when the body is upright, the horizontal axis is almost occupied by the width of the body when the body is curled, and the distance between the human body and the lens does not influence the size of the occupied ratio of the human body in the image. Unless otherwise indicated, the relative coordinate system after reconstruction is applicable.

S3, video time segmentation based on action characteristics: dividing the video into 7 segments, labeled S ₀ ～S ₆ The method comprises the steps of carrying out a first treatment on the surface of the The total 6 time division points among the 7 fragments are respectively marked as T in sequence ₁ ～T ₆ The whole video start and end time points are respectively denoted by T ₀ and T₇ A representation; wherein S is ₀ Corresponding to sitting and S ₁ Corresponding to standing up, S ₂ Corresponding to forward walking, S ₃ Corresponding to turning around in situ, S ₄ Corresponding to reverse walking, S ₅ Corresponding to turning around and sitting on site, S ₆ Corresponding sitting, as shown in fig. 4;

the specific method for video time segmentation based on action characteristics comprises the following steps:

s31, constructing the following action characteristics:

junction J ₇ 、J ₈ 、J ₉ and J₁₀ The height difference of (2) is defined as follows:

Y(J ₇ )、Y(J ₈ )、Y(J ₉ )、Y(J ₁₀ ) Respectively represent J ₇ 、J ₈ 、J ₉ and J₁₀ Is the ordinate of (2);

junction J ₁ J of (2) ₂ The horizontal distance difference of (2) is defined as:

F_clip ₂ ＝|X(J ₁ )-X(J ₂ )|

X(J ₁ )、X(J ₂ ) Respectively indicated J ₁ J of (2) ₂ Is the abscissa of (2);

junction J ₇ and J₈ The horizontal distance difference of (2) is defined as:

F_clip ₃ ＝|X(J ₇ )-X(J ₈ )|

X(J ₇ )、X(J ₈ ) Respectively J ₇ and J₈ Is the abscissa of (2);

s32, respectively calculating three action features described in S31 for each frame of image to obtain an F_clip ₁ 、F_clip ₂ 、F_clip ₃ Three discrete sequences, namely, using a binary inflection point search algorithm BinSeg of an L2 loss model to match experience parameters to obtain inflection points of the sequences, and obtaining the stage division of actions according to the inflection points, as shown in FIG. 5; the method comprises the following steps:

using F_clip ₁ Sequence, adopting inflection point search algorithm to match priori knowledge to judge S ₀ And S is equal to ₁ Is defined by the boundary point T of (2) ₁ The method comprises the steps of carrying out a first treatment on the surface of the Specifically T ₁ Appear in F_clip ₁ At the 1 st inflection point;

using F_clip ₁ Determination S ₁ And S is equal to ₂ Is defined by the boundary point T of (2) ₂ ；T ₂ Appear in F_clip ₁ At the 2 nd inflection point;

using F_clip ₂ Determination S ₂ And S is equal to ₃ Is defined by the boundary point T of (2) ₃ ；T ₃ Appear in F_clip ₂ At the 1 st inflection point;

using F_clip ₃ Determination S ₃ And S is equal to ₄ Is defined by the boundary point T of (2) ₄ ；T ₄ Appear in F_clip ₃ Is at the 3 rd inflection point;

using F_clip ₂ Determination S ₄ And S is equal to ₅ Is defined by the boundary point T of (2) ₅ ；T ₅ Appear in F_clip ₂ At the 4 th inflection point;

using F_clip ₁ Determination S ₅ And S is equal to ₆ Is defined by the boundary point T of (2) ₆ ；T ₆ Appear in F_clip ₁ At the 4 th inflection point of (c).

S4, judging the time risk according to the time characteristics obtained in the step S3 in a segmentation mode, wherein the specific method comprises the following steps of: the 5 time characteristics are defined as follows:

total time: f (F) ₁ ＝|S ₁ |+|S ₂ |+|S ₃ |+|S ₄ |+|S ₅ |＝T ₆ -T ₁

Time of the rising phase: f (F) ₂ ＝|S ₁ |＝T ₂ -T ₁

Walking phase duration: f (F) ₃ ＝|S ₂ |+|S ₄ |＝T ₅ +T ₃ -T ₄ -T ₂

Turning phase time: f (F) ₄ ＝|S ₃ |＝T ₄ -T ₃

Swivel sitting phase time: f (F) ₅ ＝|S ₅ |＝T ₆ -T ₅

Threshold Th are set for five time features ₁ 、Th ₂ 、Th ₃ 、Th ₄ 、Th ₅ The different time features have different weights for risk assessment, and a risk score weight W is set for each time feature ₁ 、W ₂ 、W ₃ 、W ₄ 、W ₅ Risk score R for temporal features _time ：

/>

wherein ,

the threshold and the weight in the above formula are obtained by fitting actual risk data. Risk score R for a temporal feature _time Exceeding a preset integrated time threshold Th _sum Directly recognizing that the risk rating is high without performing subsequent steps; otherwise, executing the step S5; by analysing the actual risk data, a threshold Th is obtained with a maximum degree of differentiation in the dataset _sum 。

S5, extracting gesture features of each action stage according to frames from the video of each stage obtained by segmentation in the step S3;

the specific implementation method comprises the following steps:

s51, lifting-center axis front-rear offset characteristic F ₆ : in the standing process, the central axis deviates from the maximum angle when sitting still, and the trunk central axis is obtained by calculating the coordinates of the left shoulder, the right shoulder, the left hip and the right hip; the specific calculation method is as follows:

firstly, calculating the slope of the central axis of the trunk:

then the slope is converted into an radian system angle, which is represented by a symbol theta;

for each time t _i Calculating the slope according to the above

And converting into radian angle>

Then F is obtained ₆ The following are provided:

wherein T₁ <t _i <T ₂ Corresponds to the time (also called frame, time t) during all the erection processes _i In practice, the unit is a frame, i.e., each frame has a slope calculated and then converted to an angle in radians. Because there are 30 frames per second fixed, the description of the frame sequence and the time series description are equivalent); θ _sit Mean offset angle for the finger sitting phase;

s52, calculating the mean angle F of the center axis deviation in the turning process ₇ ：

S53, calculating the maximum angle F when the central axis deviates from standing straight in the sitting process ₈ ：

wherein T₅ <t _i <T ₆ ，θ _stand Average deflection angle at the sitting stage;

s54, calculating the height ratio characteristic F of the walking step ₉ : the horizontal movement distance between two places where the ankle of the single foot falls is called a single step length, the single step length of the two feet is averaged in the walking process, and the ratio of the average step length to the height is calculated; in the concrete calculation, two minimum step values, namely two incomplete steps of starting and ending, are removed according to the magnitude of the numerical value. The absolute coordinate system obtained here using openPoseFor the coordinate values, X 'and Y' are used.

Obtaining a landing time point set by extremum through ankle height sequence:

left foot landing time point set:

A _{floor_left} ＝{t _m |T ₂ <t _m <T ₃ or T ₄ <t _m <T ₅ And X' (t) _m ) Is minimum }

t _m For a time point corresponding to the minimum value of the m-th left ankle abscissa in the absolute coordinate system obtained by openPose, X' (t) _m ) Representing t _m Left ankle abscissa in the absolute coordinate system of (2); removing the landing time points of the head and tail incomplete steps (when walking starts and ends, the feet are gathered together and do not accord with the definition of single step length, so that the head and tail incomplete steps are removed, and only the two time points of the head and the end are removed), and counting the average 1 step length;

right foot landing time point set:

A _{floor_right} ＝{t _n |T ₂ <t _n <T ₃ or T ₄ <t _n <T ₅ And X' (t) _n ) Is minimum }

t _n For a time point corresponding to the minimum value of the n-th right ankle abscissa in the absolute coordinate system obtained by openPose, X' (t) _n ) At t _n Right ankle abscissa in the absolute coordinate system of (2); removing the landing time points of the head and tail incomplete steps, and counting the average 1-step length;

the single step length of the left foot is as follows:

respectively the m+1th left ankle abscissa and the m left ankle abscissa in the absolute coordinate system;

the single step length of the right foot is as follows:

n+1th and n right ankle abscissas in the absolute coordinate system, respectively;

height ratio characteristic F of walking step ₉ The calculation is as follows:

wherein BH represents body height;

s55, calculating left-right offset characteristic F of walking center shaft ₁₀ In the walking process, the horizontal left-right average angle difference when the middle shaft deviates from the standing straight is calculated as follows:

wherein θ_std Center axis angle when standing straight:

s56, amplitude characteristic F of walking swing arm ₁₁ : obtaining vector angle differences formed by the left shoulder, the left elbow, the right shoulder and the right elbow nodes, namely elbow-shoulder vectors when the backward swing is farthest and elbow-shoulder vectors when the forward swing is farthest, wherein the angle differences are the difference;

firstly, respectively calculating the swing arm slopes of a left arm and a right arm:

X'(J _c )、Y'(J _c ) J in absolute coordinate systems obtained for OpenPose respectively _c C=1, …,4;

converting slope into radian angle alpha _{left_arm} And alpha is _{right_arm} ；

Calculating the swing arm angles of the left arm and the right arm in each frame to form a sequence of the swing arm angles of the left arm and the right arm, and obtaining an extremum in the sequence to obtain a maximum angle A of the front swing of the left arm _{af_left} Maximum angle A of left arm backswing _{ab_left} Maximum angle A of front swing of right arm _{af_right} Maximum angle A of right arm backswing _{ab_right} ；

Will be adjacent A _{af_left} And A is a _{ab_left} Aligning and differencing to obtain a single swing arm angle set A of the left arm _{as_left} ；

Will be adjacent A _{af_right} And A is a _{ab_right} Aligning and differencing to obtain a single swing arm angle set A of the right arm _{as_right} ；

Removal A _{as_left} And A is a _{as_right} The first and last data in both sets (these two non-pure walking actions, therefore removed) are then combined with the remaining set A' _{as_left} A' _{as_right} Based on which F is calculated ₁₁ ：

wherein α_m ∈A' _{as_left} ，α _n ∈A' _{as_right} ，|A' _{as_left} |、|A' _{as_right} I respectively represent the set A' _{as_left} 、A' _{as_right} The number of elements in (a);

s57, calculating the amplitude characteristic F of the walking swing leg ₁₂ : obtaining vector angle difference formed by the left hip, the left knee, the right hip and the right knee nodes;

and respectively calculating the swing slope of the left leg and the right leg:

X'(J _c )、Y'(J _c ) Obtained separately from OpenPoseJ in absolute coordinate system _c C=7, …,10;

will K _{left_leg} and K_{right_leg} Conversion from slope to radian _{left_leg} And beta _{right_leg} ；

Calculating the swing angles of the left leg and the right leg in each frame to form a sequence of the swing angles of the left leg and the right leg, and obtaining an extremum in the sequence to obtain a maximum swing angle A of the front of the left leg _{lf_left} Maximum angle A of left leg backswing _{lb_left} Maximum angle A of front swing of right leg _{lf_right} Maximum angle A of rear swing of right leg _{lb_right} ；

Corresponding in time sequence, adjacent A _{lf_left} and A_{lb_left} Aligning and differencing to obtain a left leg swing angle sequence A _{ls_left} ；

Will be adjacent A _{lf_right} and A_{lb_right} Aligning and differencing to obtain a right leg swing angle sequence A _{ls_right} ；

Removal of sequence A _{ls_left} and A_{ls_right} Data of the first and the last of (a) and then in the remaining sequence A' _{ls_left} and A'_{ls_right} Obtaining F ₁₂ The calculation is as follows:

wherein β_m ∈A' _{ls_left} ，β _n ∈A' _{ls_right} ，|A' _{ls_left} |、|A' _{ls_right} I respectively represent the sequence A' _{ls_left} and A'_{ls_right} The number of elements in (a).

S6, judging the attitude risk according to the attitude characteristics obtained in the step S5; the specific implementation method comprises the following steps: step height ratio F ₉ Swing arm F ₁₁ Swing leg F ₁₂ The larger the description of the better the mobility and balance, while the axis offset-related feature F ₆ 、F ₇ 、F ₈ 、F ₁₀ Smaller indicates better balancing capability. Will gesture feature F ₆ ～F ₁₂ Input toRegression prediction is carried out in the multi-layer perceptron to obtain an attitude risk value R _posture 。

S7, carrying out final comprehensive risk judgment according to the time risk and the gesture risk obtained in the steps S4 and S6; the specific implementation method comprises the following steps: if the time risk R has been obtained in step S4 _time Greater than threshold Th _sum The comprehensive risk judgment directly obtains a result, and the falling risk is high; otherwise, the following determination is made:

setting the weight W by linear fitting _time and W_posture Final risk R _final ＝W _time ·R _time +W _posture ·R _posture ；

Will end risk R _final And a high risk reference threshold Th _danger Stroke risk reference threshold Th _alarm Comparing to be higher than or equal to the high risk reference threshold Th _danger The subject is considered to have a high risk of falling, being greater than or equal to the stroke risk threshold Th _alarm While being below the high risk threshold Th _danger The subject is considered to have a risk of falling well below the risk of falling reference threshold Th _alarm The subject is considered to have a low risk of falling.

Those of ordinary skill in the art will recognize that the embodiments described herein are for the purpose of aiding the reader in understanding the principles of the present invention and should be understood that the scope of the invention is not limited to such specific statements and embodiments. Those of ordinary skill in the art can make various other specific modifications and combinations from the teachings of the present disclosure without departing from the spirit thereof, and such modifications and combinations remain within the scope of the present disclosure.

Claims (5)

1. A method for predicting the risk of falling over of an elderly person, comprising the steps of:

s1, data acquisition and preprocessing, which comprises the following substeps:

s11, arranging a data acquisition environment according to a TUG standard: a chair with the height of 60cm is opposite to a flat pavement with the length of 3 m; the pixel of the image pickup device is required to be at least 720p and is fixedly arranged at the height of 1.5 m;

s12, starting shooting when the subject sits at the end, then rising up and advancing, turning back after reaching the end point of the walk, returning to the starting point, re-sitting, and stopping shooting; the resulting video pre-processing is 1280x720 pixels, 30 frames per second;

s13, taking the real falling times of the subject in the time of nearly two years as a label of sample data to form a training data set;

s14, dividing the falling risk into 3 grades: the low risk level corresponds to no fall record within 2 years, the medium risk level corresponds to 1 to 2 fall records within 2 years, and the high risk level corresponds to 3 or more fall records;

s2, performing skeleton modeling: modeling human bodies in video into a skeleton of 13 nodes by using OpenPose, using J ₀ ～J ₁₂ Representing the corresponding joint, specifically includes: j (J) ₀ -head, J ₁ Left shoulder, J ₂ Right shoulder, J ₃ Left elbow, J ₄ -right elbow, J ₅ -left wrist, J ₆ -right wrist, J ₇ Left hip, J ₈ -right hip, J ₉ Left knee, J ₁₀ Right knee, J ₁₁ Left ankle, J ₁₂ -a right ankle;

acquiring a horizontal axis coordinate and a vertical axis coordinate on each frame of 2d image through an OpenPose frame;

reconstructing an absolute coordinate obtained by OpenPose, and constructing a relative coordinate system: in each frame of image, the center of the person, J, is found by the absolute coordinates of the nodes ₁ 、J ₂ 、J ₇ 、J ₈ A person is framed by a rectangular outer frame with fixed proportion, the left lower corner of the outer frame is taken as the origin of coordinates, the direction of the longitudinal axis of the transverse axis is unchanged, and a new coordinate system is established;

s3, video time segmentation based on action characteristics: dividing the video into 7 segments, labeled S ₀ ～S ₆ The method comprises the steps of carrying out a first treatment on the surface of the The total 6 time division points among the 7 fragments are respectively marked as T in sequence ₁ ～T ₆ The whole video start and end time points are respectively denoted by T ₀ and T₇ A representation; wherein S is ₀ Corresponding to sitting and S ₁ Corresponding to standing up, S ₂ Corresponding to forward walking, S ₃ Corresponding to turning around in situ, S ₄ Corresponding to reverse walking, S ₅ Corresponding to turning around and sitting on site, S ₆ Corresponding to the sitting;

s4, judging the time risk according to the time characteristics obtained in the step S3 in a segmentation mode, wherein the specific method comprises the following steps of: the 5 time characteristics are defined as follows:

total time: f (F) ₁ ＝|S ₁ |+|S ₂ |+|S ₃ |+|S ₄ |+|S ₅ |＝T ₆ -T ₁

Time of the rising phase: f (F) ₂ ＝|S ₁ |＝T ₂ -T ₁

Walking phase duration: f (F) ₃ ＝|S ₂ |+|S ₄ |＝T ₅ +T ₃ -T ₄ -T ₂

Turning phase time: f (F) ₄ ＝|S ₃ |＝T ₄ -T ₃

Swivel sitting phase time: f (F) ₅ ＝|S ₅ |＝T ₆ -T ₅

Threshold Th are set for five time features ₁ 、Th ₂ 、Th ₃ 、Th ₄ 、Th ₅ The different time features have different weights for risk assessment, and a risk score weight W is set for each time feature ₁ 、W ₂ 、W ₃ 、W ₄ 、W ₅ Risk score R for temporal features _time ：

wherein ,

risk score R for a temporal feature _time Exceeding a preset integrated time threshold Th _sum Directly recognizing that the risk rating is high without performing subsequent steps; otherwise, executing the step S5;

s5, extracting gesture features of each action stage according to frames from the video of each stage obtained by segmentation in the step S3;

s6, judging the attitude risk according to the attitude characteristics obtained in the step S5;

and S7, carrying out final comprehensive risk judgment according to the time risk and the gesture risk obtained in the steps S4 and S6.

2. The method for predicting the risk of falling over of the elderly according to claim 1, wherein in the step S3, the specific method for video time segmentation based on the motion features is as follows:

s31, constructing the following action characteristics:

junction J ₇ 、J ₈ 、J ₉ and J₁₀ The height difference of (2) is defined as follows:

Y(J ₇ )、Y(J ₈ )、Y(J ₉ )、Y(J ₁₀ ) Respectively represent J ₇ 、J ₈ 、J ₉ and J₁₀ Is the ordinate of (2);

junction J ₁ J of (2) ₂ The horizontal distance difference of (2) is defined as:

F_clip ₂ ＝|X(J ₁ )-X(J ₂ )|

X(J ₁ )、X(J ₂ ) Respectively indicated J ₁ J of (2) ₂ Is the abscissa of (2);

junction J ₇ and J₈ The horizontal distance difference of (2) is defined as:

F_clip ₃ ＝|X(J ₇ )-X(J ₈ )|

X(J ₇ )、X(J ₈ ) Respectively J ₇ and J₈ Is the abscissa of (2);

s32, respectively calculating three action features described in S31 for each frame of image to obtain an F_clip ₁ 、F_clip ₂ 、F_clip ₃ Three discrete sequences, a binary inflection point search algorithm BinSeg of an L2 loss model is used, and empirical parameters are matched,obtaining inflection points of the sequence, and obtaining the stage division of actions according to the inflection points; the method comprises the following steps:

using F_clip ₁ Sequence, adopting inflection point search algorithm to match priori knowledge to judge S ₀ And S is equal to ₁ Is defined by the boundary point T of (2) ₁ The method comprises the steps of carrying out a first treatment on the surface of the Specifically T ₁ Appear in F_clip ₁ At the 1 st inflection point;

using F_clip ₁ Determination S ₁ And S is equal to ₂ Is defined by the boundary point T of (2) ₂ ；T ₂ Appear in F_clip ₁ At the 2 nd inflection point;

using F_clip ₂ Determination S ₂ And S is equal to ₃ Is defined by the boundary point T of (2) ₃ ；T ₃ Appear in F_clip ₂ At the 1 st inflection point;

using F_clip ₃ Determination S ₃ And S is equal to ₄ Is defined by the boundary point T of (2) ₄ ；T ₄ Appear in F_clip ₃ Is at the 3 rd inflection point;

using F_clip ₂ Determination S ₄ And S is equal to ₅ Is defined by the boundary point T of (2) ₅ ；T ₅ Appear in F_clip ₂ At the 4 th inflection point;

using F_clip ₁ Determination S ₅ And S is equal to ₆ Is defined by the boundary point T of (2) ₆ ；T ₆ Appear in F_clip ₁ At the 4 th inflection point of (c).

3. The method for predicting the risk of falling over of the elderly according to claim 1, wherein the specific implementation method of step S5 is as follows:

s51, lifting-center axis front-rear offset characteristic F ₆ : in the standing process, the central axis deviates from the maximum angle when sitting still, and the trunk central axis is obtained by calculating the coordinates of the left shoulder, the right shoulder, the left hip and the right hip; the specific calculation method is as follows:

firstly, calculating the slope of the central axis of the trunk:

then the slope is converted into an radian system angle, which is represented by a symbol theta;

for each time t _i Calculating the slope according to the above

And converting into radian angle>

Then F is obtained ₆ The following are provided:

wherein T₁ <t _i <T ₂ Corresponding to the moments in all standing processes; θ _sit Mean offset angle for the finger sitting phase;

s52, calculating the mean angle F of the center axis deviation in the turning process ₇ ：

S53, calculating the maximum angle F when the central axis deviates from standing straight in the sitting process ₈ ：

wherein T₅ <t _i <T ₆ ，θ _stand Average deflection angle at the sitting stage;

s54, calculating the height ratio characteristic F of the walking step ₉ : the horizontal movement distance between two places where the ankle of the single foot falls is called a single step length, the single step length of the two feet is averaged in the walking process, and the ratio of the average step length to the height is calculated;

obtaining a landing time point set by extremum through ankle height sequence:

left foot landing time point set:

A _{floor_left} ＝{t _m |T ₂ <t _m <T ₃ or T ₄ <t _m <T ₅ And X' (t) _m ) Is minimum }

t _m For a time point corresponding to the minimum value of the m-th left ankle abscissa in the absolute coordinate system obtained by openPose, X' (t) _m ) Representing t _m Left ankle abscissa in the absolute coordinate system of (2); removing the landing time points of the head and tail incomplete steps, and counting the average 1-step length;

right foot landing time point set:

A _{floor_right} ＝{t _n |T ₂ <t _n <T ₃ or T ₄ <t _n <T ₅ And X' (t) _n ) Is minimum }

t _n For a time point corresponding to the minimum value of the n-th right ankle abscissa in the absolute coordinate system obtained by openPose, X' (t) _n ) At t _n Right ankle abscissa in the absolute coordinate system of (2); removing the landing time points of the head and tail incomplete steps, and counting the average 1-step length;

the single step length of the left foot is as follows:

respectively the m+1th left ankle abscissa and the m left ankle abscissa in the absolute coordinate system;

the single step length of the right foot is as follows:

respectively absolute coordinate systemN+1th and n right ankle abscissas in (a);

height ratio characteristic F of walking step ₉ The calculation is as follows:

wherein BH represents body height;

s55, calculating left-right offset characteristic F of walking center shaft ₁₀ In the walking process, the horizontal left-right average angle difference when the middle shaft deviates from the standing straight is calculated as follows:

wherein θ_std Center axis angle when standing straight:

s56, amplitude characteristic F of walking swing arm ₁₁ : obtaining vector angle differences formed by the left shoulder, the left elbow, the right shoulder and the right elbow nodes, namely elbow-shoulder vectors when the backward swing is farthest and elbow-shoulder vectors when the forward swing is farthest, wherein the angle differences are the difference;

firstly, respectively calculating the swing arm slopes of a left arm and a right arm:

X'(J _c )、Y'(J _c ) J in absolute coordinate systems obtained for OpenPose respectively _c C=1, …,4;

converting slope into radian angle alpha _{left_arm} And alpha is _{right_arm} ；

Calculating the swing arm angles of the left arm and the right arm in each frame to form a sequence of the swing arm angles of the left arm and the right arm, and obtaining an extremum in the sequence to obtain the front left armMaximum angle of swing A _{af_left} Maximum angle A of left arm backswing _{ab_left} Maximum angle A of front swing of right arm _{af_right} Maximum angle A of right arm backswing _{ab_right} ；

Will be adjacent A _{af_left} And A is a _{ab_left} Aligning and differencing to obtain a single swing arm angle set A of the left arm _{as_left} ；

Will be adjacent A _{af_right} And A is a _{ab_right} Aligning and differencing to obtain a single swing arm angle set A of the right arm _{as_right} ；

Removal A _{as_left} And A is a _{as_right} The first and last data in both sets are then combined with the remaining set A' _{as_left} 、A' _{as_right} Based on which F is calculated ₁₁ ：

wherein α_m ∈A' _{as_left} ，α _n ∈A' _{as_right} ，|A' _{as_left} |、|A' _{as_right} I respectively represent the set A' _{as_left} 、A' _{as_right} The number of elements in (a);

s57, calculating the amplitude characteristic F of the walking swing leg ₁₂ : obtaining vector angle difference formed by the left hip, the left knee, the right hip and the right knee nodes;

and respectively calculating the swing slope of the left leg and the right leg:

X'(J _c )、Y'(J _c ) J in absolute coordinate systems obtained for OpenPose respectively _c C=7, …,10;

will K _{left_leg} and K_{right_leg} Conversion from slope to radian _{left_leg} And beta _{right_leg} ；

Calculating left in each frameThe leg swinging angles of the leg and the right leg form a sequence of the left leg swinging angles and the right leg swinging angles, and extreme values are obtained in the sequence to obtain the maximum angle A of the front swing of the left leg _{lf_left} Maximum angle A of left leg backswing _{lb_left} Maximum angle A of front swing of right leg _{lf_right} Maximum angle A of rear swing of right leg _{lb_right} ；

Will be adjacent A _{lf_left} and A_{lb_left} Aligning and differencing to obtain a left leg swing angle sequence A _{ls_left} ；

Will be adjacent A _{lf_right} and A_{lb_right} Aligning and differencing to obtain a right leg swing angle sequence A _{ls_right} ；

Removal of sequence A _{ls_left} and A_{ls_right} Data of the first and the last of (a) and then in the remaining sequence A' _{ls_left} and A'_{ls_right} Obtaining F ₁₂ The calculation is as follows:

wherein β_m ∈A' _{ls_left} ，β _n ∈A' _{ls_right} ，|A' _{ls_left} |、|A' _{ls_right} I respectively represent the sequence A' _{ls_left} and A'_{ls_right} The number of elements in (a).

4. A method for predicting fall risk of elderly people according to claim 3, wherein the specific implementation method of step S6 is as follows: will gesture feature F ₆ ～F ₁₂ Inputting the attitude risk value R into a multi-layer perceptron to carry out regression prediction _posture 。

5. The method for predicting the risk of falling over of the elderly according to claim 1, wherein the specific implementation method of step S7 is as follows: if the time risk R has been obtained in step S4 _time Greater than threshold Th _sum The comprehensive risk judgment directly obtains a result, and the falling risk is high; otherwise, the following determination is made:

setting the weight W by linear fitting _time and W_posture Final risk R _final ＝W _time ·R _time +W _posture ·R _posture ；

Will end risk R _final And a high risk reference threshold Th _danger Stroke risk reference threshold Th _alarm Comparing to be higher than or equal to the high risk reference threshold Th _danger The subject is considered to have a high risk of falling, being greater than or equal to the stroke risk threshold Th _alarm While being below the high risk threshold Th _danger The subject is considered to have a risk of falling well below the risk of falling reference threshold Th _alarm The subject is considered to have a low risk of falling.

Images