CN114694252A - Old people falling risk prediction method - Google Patents

Abstract

The invention discloses a method for predicting falling risk of old people, which comprises the following steps: s1, data acquisition and pretreatment; s2, carrying out skeleton modeling; s3, video time segmentation based on motion characteristics: the video is divided into 7 segments, respectively denoted as S₀～S₆(ii) a The total of 6 time division points among 7 segments are respectively marked as T in sequence₁～T₆(ii) a S4, performing time risk judgment; s5, extracting the attitude characteristics of each action stage according to frames from the video of each stage obtained after the segmentation of the step S3; s6, judging posture risks according to the posture characteristics obtained in the step S5; and S7, performing final comprehensive risk judgment according to the time risk and the posture risk obtained in the steps S4 and S6. According to the invention, the time characteristic and the posture characteristic of the movement process are integrated, the obtained risk evaluation result can reflect the real balance and movement capability, and the accuracy of the falling risk prediction result is improved.

Description

Old people falling risk prediction method

Technical Field

The invention relates to a method for predicting falling risk of old people.

Background

The old people can naturally weaken the physical quality, the functions of bones and muscles and the like, the falling risk is increased, and the falling easily causes the injuries such as fracture and the like. In order to find the falling risk as early as possible and take effective protective measures, the falling risk assessment of the old has important practical significance. The TUG Test, namely the timing standing walking Test, is a relatively mature method for predicting the falling risk of the old people at present, is widely concerned and has proved to be effective.

In recent years, researchers have applied computer technology to the TUG test, i.e., have implemented automation of the TUG test. The computer automation of the TUG test can both retain the utility of the TUG test itself and reduce the direct intervention of professional medical personnel, thereby improving the popularity of the test and also effectively reducing the problem of human error that may occur during operation.

The patent application with the application number of 201911412802.2 discloses a method and a system for evaluating the falling risk of the old, which are used for extracting gait characteristics such as single-leg suspended displacement, knee joint angle, ankle joint angle and the like in the walking process of a person and judging the falling risk.

The existing TUG automation method is limited to the timing characteristic of the TUG, omits the rich motion characteristic and attitude characteristic in the TUG process, and has poor robustness in practical application due to the characteristics and algorithm used for time segmentation. The complete TUG motion process comprises the actions of standing up, walking, turning, sitting down and the like, and the actions can reflect the characteristics of physical strength, balance level and the like of the old from different angles. The 201911412802.2 application mainly focuses on extracting features of a walking stage to perform fall risk judgment, and affects the accuracy of the fall risk judgment.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide the old people falling risk prediction method which integrates the time characteristics and the posture characteristics of the movement process, and the obtained risk assessment result can reflect the real balance and movement capability and can improve the accuracy of the falling risk prediction result.

The purpose of the invention is realized by the following technical scheme: a method for predicting fall risk of an old person comprises the following steps:

s1, data acquisition and preprocessing, including the following substeps:

s11, arranging a data acquisition environment according to the TUG standard: a chair 60cm high, which is opposite to a flat walkway 3m long; the image pickup equipment pixel requires at least 720p and is fixedly placed at the height of 1.5 m;

s12, starting camera shooting when the subject sits up, then rising up and moving forward, turning to and moving back after reaching the end point of the walkway, returning to the starting point and seating again, and stopping camera shooting; the resulting video was preprocessed to 1280x720 pixels, 30 frames per second;

s13, taking the real falling times of the subject in the last two years as the label of sample data to form a training data set;

s14, classifying the fall risk into 3 grades: the low risk level corresponds to no fall record within 2 years, the medium risk level corresponds to 1 to 2 fall records within 2 years, and the high risk level corresponds to 3 fall records or more;

s2, carrying out skeleton modeling: modeling human body in video as skeleton of 13 nodes by using OpenPose, and using J₀～J₁₂Represents a corresponding joint, and specifically comprises: j is a unit of₀Head, J₁Left shoulder, J₂Right shoulder, J₃Left elbow, J₄Right elbow, J₅Left wrist, J₆Right wrist, J₇Left hip, J₈Right hip, J₉Left knee, J₁₀Right knee, J₁₁Left ankle, J₁₂-a right ankle;

obtaining a horizontal axis coordinate and a vertical axis coordinate on each frame of 2d image through an OpenPose framework;

and (3) reconstructing an absolute coordinate obtained by OpenPose, and constructing a relative coordinate system: in each frame of image, the center of the person, i.e. J, is found by the absolute coordinates of the nodes₁、J₂、J₇、J₈The human body is selected by a rectangular outer frame with a fixed proportion, the lower left corner of the outer frame is taken as the origin of coordinates, the direction of the longitudinal axis of the transverse axis is unchanged, and a new coordinate system is established;

s3, video time segmentation based on motion characteristics: the video is divided into 7 segments, respectively denoted as S₀～S₆(ii) a The total of 6 time division points among 7 segments are respectively marked as T in sequence₁～T₆The whole video start and end time points are respectively T₀ and T₇Represents; wherein S is₀Corresponding sitting up and S₁Stand up correspondingly, S₂Corresponding to forward walking, S₃Corresponding to the in-situ turning and S₄Corresponding to the walking in the reverse direction, S₅Corresponding to turning and sitting in place S₆The corresponding end seat;

s4, according to the time characteristics obtained by segmentation in the step S3, time risk judgment is carried out, and the specific method is as follows: the 5 temporal features are defined as follows:

total time: f₁＝|S₁|+|S₂|+|S₃|+|S₄|+|S₅|＝T₆-T₁

Rising stage time: f₂＝|S₁|＝T₂-T₁

Duration of walking phase: f₃＝|S₂|+|S₄|＝T₅+T₃-T₄-T₂

Turning stage time: f₄＝|S₃|＝T₄-T₃

Turning body and sitting down stage time: f₅＝|S₅|＝T₆-T₅

Setting threshold Th for five time characteristics₁、Th₂、Th₃、Th₄、Th₅The risk assessment is weighted differently for different temporal characteristics, and a risk score weight W is set for each temporal characteristic₁、W₂、W₃、W₄、W₅Time-characterized risk score R_time：

wherein ,

risk score R if temporal characteristics_timeExceeds a preset integrated time threshold Th_sumDirectly deem the risk rating high without performing subsequent steps; otherwise, executing step S5;

s5, extracting the attitude characteristics of each action stage according to frames from the video of each stage obtained after the segmentation of the step S3;

s6, judging posture risks according to the posture characteristics obtained in the step S5;

and S7, performing final comprehensive risk judgment according to the time risk and the posture risk obtained in the steps S4 and S6.

Further, in step S3, the specific method of video time segmentation based on motion characteristics is as follows:

s31, constructing the following action characteristics:

node J₇、J₈、J₉ and J₁₀The height difference of (a) is defined as follows:

Y(J₇)、Y(J₈)、Y(J₉)、Y(J₁₀) Respectively represents J₇、J₈、J₉ and J₁₀The ordinate of (a);

node J₁J of (A)₂The horizontal distance difference of (a) is defined as:

F_clip₂＝|X(J₁)-X(J₂)|

X(J₁)、X(J₂) Respectively represent J₁J of (A)₂The abscissa of (a);

node J₇ and J₈The horizontal distance difference of (a) is defined as:

F_clip₃＝|X(J₇)-X(J₈)|

X(J₇)、X(J₈) Are respectively J₇ and J₈The abscissa of (a);

s32, respectively calculating the three action characteristics of S31 for each frame of image to obtain F _ clip₁、F_clip₂、F_clip₃Three discrete sequences, namely using a binary inflection point search algorithm BinSeg of an L2 loss model, matching empirical parameters to obtain inflection points of the sequences, and obtaining stage division of actions according to the inflection points; the method comprises the following specific steps:

using F _ clip₁Sequence, using inflection point search algorithm with prior knowledge to judge S₀And S₁Is a dividing point T₁(ii) a In particular, T₁Appear in F _ clip ₁1 st inflection point;

using F _ clip₁Judgment S₁And S₂Is a dividing point T₂；T₂Appear in F _ clip₁At the 2 nd inflection point;

using F _ clip₂Judgment S₂And S₃Is a dividing point T₃；T₃Appear in F _ clip ₂1 st inflection point;

using F _ clip₃Judgment S₃And S₄Is a dividing point T₄；T₄Appear in F _ clip₃At the 3 rd inflection point;

using F _ clip₂Judgment S₄And S₅Is a dividing point T₅；T₅Appear in F _ clip₂At the 4 th inflection point;

using F _ clip₁Judgment S₅And S₆Is a dividing point T₆；T₆Appear in F _ clip₁At the 4 th inflection point.

Further, the specific implementation method of step S5 is as follows:

s51 rising-middle axle front and back offset characteristic F₆: in the standing process, the central axis deviates from the maximum angle of sitting statically, and the central axis of the trunk is obtained by calculating the coordinates of the left shoulder, the right shoulder, the left hip and the right hip; the specific calculation method is as follows:

firstly, calculating the slope of the central axis of the trunk:

then the slope is converted into a radian system angle which is expressed by a symbol theta;

for each time t_iCalculating the slope according to the above formula

And is turned into an angle of radian

Then obtaining F₆The following were used:

wherein T₁<t_i<T₂Corresponding to all the moments in the standing process; theta_sitMean offset angle of the fingering stage;

s52, calculating the mean deviation angle F of the middle shaft in the turning process₇：

S53, calculating the maximum angle F of the central axis when the central axis deviates from the standing straight state in the sitting process₈：

wherein T₅<t_i<T₆，θ_standMean deviation angle of finger sitting down stage;

s54, calculating walking step length and height ratio characteristic F₉: the horizontal moving distance between two landing points of the single foot ankle is called as a single step length, the single step length of the two feet is averaged in the walking process, and the ratio of the average step length to the height is calculated;

obtaining a set of landing time points by solving an extremum value through an ankle height sequence:

set of landing time points for the left foot:

A_{floor_left}＝{t_m|T₂<t_m<T₃or T₄<t_m<T₅And X' (t)_m) Is a minimum value }

t_mA time point, X' (t), corresponding to the m-th left ankle abscissa minimum value in the absolute coordinate system obtained by OpenPose_m) Represents t_mThe left ankle abscissa in the absolute coordinate system of (1); removing landing time points of incomplete steps at the head and the tail, and counting the average 1-step length;

right foot landing time point set:

A_{floor_right}＝{t_n|T₂<t_n<T₃or T₄<t_n<T₅And X' (t)_n) Is a minimum value }

t_nA time point, X' (t), corresponding to the nth right ankle abscissa minimum value in the absolute coordinate system obtained by OpenPose_n) Is t_nThe abscissa of the right ankle in the absolute coordinate system of (1); removing landing time points of incomplete steps at the head and the tail, and counting the average 1-step length;

the left foot single step size is:

respectively are the m +1 th and m left ankle horizontal coordinates in an absolute coordinate system;

the right foot single step is:

respectively are the abscissa of the (n + 1) th and the n right ankle in an absolute coordinate system;

step length to height ratio characteristic F₉The calculation is as follows:

wherein BH represents body height;

s55, calculating the left and right deviation characteristic F of the walking central axis₁₀That is, during walking, the horizontal left-right average angle difference of the central axis when deviating from the standing straight is calculated as follows:

wherein θ_stdThe angle of the middle shaft when standing straight:

s56 amplitude characteristic F of walking swing arm₁₁: obtaining vector angle difference formed by the left shoulder, the left elbow, the right shoulder and the right elbow node, namely elbow-shoulder vector when the elbow is swung farthest backwards, elbow-shoulder vector when the elbow is swung farthest forwards and angle difference between the elbow and shoulder vector;

firstly, respectively calculating the swing arm slopes of the left arm and the right arm:

X'(J_c)、Y'(J_c) J in absolute coordinate systems respectively obtained for OpenPose_cC 1, …, 4;

converting slope into radian angle alpha_{left_arm}And alpha_{right_arm}；

Calculating the swing arm angles of the left arm and the right arm in each frame to form a sequence of the swing arm angles of the left arm and the right arm, and solving an extreme value in the sequence to obtain the maximum swing angle A of the front swing of the left arm_{af_left}Maximum angle A of left arm back swing_{ab_left}Maximum angle A of front swing of right arm_{af_right}Maximum angle A of right arm backswing_{ab_right}；

Will be adjacent to A_{af_left}And A_{ab_left}Aligning and differentiating to obtain a single swing arm angle set A of the left arm_{as_left}；

Will be adjacent to A_{af_right}And A_{ab_right}Aligning and differentiating to obtain a right arm single swing arm angle set A_{as_right}；

Removing A_{as_left}And A_{as_right}The first and last data in the two sets, then with the remaining set A'_{as_left} A'_{as_right}Based on, calculate F₁₁：

wherein α_m∈A'_{as_left}，α_n∈A'_{as_right}，|A'_{as_left}|、|A'_{as_right}L represents set A 'respectively'_{as_left}、A'_{as_right}The number of middle elements;

s57, calculating walking leg swing amplitude characteristic F₁₂: obtaining the vector angle difference formed by the left hip, the left knee, the right hip and the right knee through the nodes of the left hip, the left knee, the right hip and the right knee;

calculating the leg swinging slopes of the left leg and the right leg respectively:

X'(J_c)、Y'(J_c) J in absolute coordinate systems respectively obtained for OpenPose_cC-7, …, 10;

will K_{left_leg} and K_{right_leg}Conversion from slope to radian angle beta_{left_leg}And beta_{right_leg}；

Calculating the leg swinging angles of the left leg and the right leg in each frame to form a sequence of the leg swinging angles of the left leg and the right leg, and solving an extreme value in the sequence to obtain the maximum front swinging angle A of the left leg_{lf_left}Maximum angle A of left leg back swing_{lb_left}Maximum angle A of front swing of right leg_{lf_right}Maximum angle A of right leg back swing_{lb_right}；

Will be adjacent to A_{lf_left} and A_{lb_left}Aligning and differencing to obtain a left leg swing angle sequence A_{ls_left}；

Will be adjacent to A_{lf_right} and A_{lb_right}Aligning and differentiating to obtain a leg swing angle sequence A of the right leg_{ls_right}；

Removal of sequence A_{ls_left} and A_{ls_right}And then in the remaining sequence A'_{ls_left} and A'_{ls_right}To obtain F₁₂The calculation is as follows:

wherein β_m∈A'_{ls_left}，β_n∈A'_{ls_right}，|A'_{ls_left}|、|A'_{ls_right}L represent the sequence A 'respectively'_{ls_left} and A'_{ls_right}The number of elements in (c).

Further, the specific implementation method of step S6 is as follows: the posture feature F₆～F₁₂Inputting the data into a multilayer perceptron to carry out regression prediction to obtain an attitude risk value R_posture。

Further, the specific implementation method of step S7 is as follows: if the time risk R has been obtained in step S4_timeGreater than threshold Th_sumIf so, the comprehensive risk judgment directly obtains a result, and the falling risk is high risk; otherwise, the following judgment is carried out:

setting the weight W by linear fitting_time and W_postureUltimate risk R_final＝W_time·R_time+W_posture·R_posture；

Will be the final risk R_finalWith a high risk reference threshold Th_dangerMiddle risk reference threshold Th_alarmMaking a comparison higher than or equal to a high risk reference threshold Th_dangerThe subject is considered to have a high risk of falling, higher than or equal to the middle risk threshold Th_alarmBut below a high risk threshold Th_dangerThe subject is considered to have a moderate fall risk, below the moderate risk reference threshold Th_alarmThe test person is considered to have a low fall risk.

The invention has the beneficial effects that: according to the method, the action video is divided into different stages, the time dimension falling risk is predicted based on the time characteristics of each stage, and the primary falling risk judgment is carried out; then constructing attitude characteristics of different stages based on video segmentation results, inputting the attitude characteristics into a multilayer perceptron to perform regression prediction, and obtaining attitude risk prediction results of spatial dimensions; and finally, integrating the risks of two dimensions of time and space to obtain a final falling risk prediction result. According to the invention, the time characteristic and the posture characteristic of the movement process are integrated, the obtained risk evaluation result can reflect the real balance and movement capability, and the accuracy of the falling risk prediction result is improved.

Drawings

Fig. 1 is a flow chart of a fall risk prediction method for elderly people according to the present invention;

FIG. 2 is a schematic diagram of a data acquisition environment of the present invention;

FIG. 3 is a schematic diagram of the skeletal modeling of the present invention;

FIG. 4 is a diagram illustrating natural division of action phases according to the present invention;

FIG. 5 is a diagram illustrating the step division of action according to inflection points in the present invention.

Detailed Description

The technical scheme of the invention is further explained by combining the attached drawings.

As shown in fig. 1, the method for predicting fall risk of an elderly person of the present invention includes the following steps:

s1, data acquisition and preprocessing, including the following substeps:

s11, arranging a data acquisition environment according to the TUG standard: a chair 60cm high, which is opposite to a flat walkway 3m long; the image pickup device pixel is required to be at least 720p, and is fixedly placed at a height of 1.5m, and a connecting line between the image pickup device and the end point of the walkway is vertical to the walkway, as shown in FIG. 2;

s12, starting camera shooting when the subject sits up, then rising up and moving forward, turning to and moving back after reaching the end point of the walkway, returning to the starting point and seating again, and stopping camera shooting; the obtained video is preprocessed to 1280x720 pixels, and 30 frames per second;

s13, taking the real falling times of the subject in the last two years as the label of sample data to form a training data set for parameter fitting;

s14, classifying the fall risk into 3 grades: the low risk level corresponds to no fall record within 2 years, the medium risk level corresponds to 1 to 2 fall records within 2 years, and the high risk level corresponds to 3 fall records or more;

s2, carrying out skeleton modeling: modeling human body in video as skeleton of 13 nodes by using OpenPose, and using J₀～J₁₂Represents the corresponding joint, and specifically includes, as shown in fig. 3: j. the design is a square₀Head, J₁Left shoulder, J₂Right shoulder, J₃Left elbow, J₄Right elbow, J₅Left wrist, J₆Right wrist, J₇Left hip, J₈Right hip, J₉Left knee, J₁₀Right knee, J₁₁Left ankle, J₁₂-a right ankle;

obtaining a horizontal axis coordinate and a vertical axis coordinate on each frame of 2d image through an OpenPose framework; in the following formula, X represents a horizontal coordinate and Y represents a vertical coordinate.

In the process of human body movement, the size of the projection of a human body in an image changes, in order to reduce the influence of the size change of the human image on a characteristic value, the absolute coordinates obtained by OpenPose are reconstructed, and a relative coordinate system is constructed, wherein the method comprises the following steps: in each frame of image, the center of the person, i.e. J, is found by the absolute coordinates of the nodes₁、J₂、J₇、J₈The human body is selected by a rectangular outer frame with a fixed proportion, the lower left corner of the outer frame is taken as the origin of coordinates, the direction of the longitudinal axis of the transverse axis is unchanged, and a new coordinate system is established; and the unit length of the new coordinate system is scaled according to the size of the outer frame, namely, the images in the outer frame are scaled to be uniform in size. The strategy for selecting the size of the outer frame is that the height (absolute value of the difference of the Y-axis coordinates) and the width (absolute value of the difference of the X-axis coordinates) are considered at the same time, when the height-width ratio is larger than a ratio r, the height is used as a standard to demarcate the outer frame, and when the height-width ratio is smaller than the ratio r, the outer frame with a fixed aspect ratio is demarcated by using the width as the standard to correspond to the state that the body is curled.

The finally obtained image should have the characteristics that the vertical axis of the horizontal axis of the image obtained by the frame selection is almost occupied by the human body when the human body is upright, the horizontal axis is almost occupied by the body width when the human body is curled, and the distance between the human body and the lens does not influence the proportion of the human body in the image. The reconstructed relative coordinate system is applicable, if not indicated, in the text.

S3 video time based on motion characteristicsSegmenting: the video is divided into 7 segments, respectively denoted as S₀～S₆(ii) a The total of 6 time division points among 7 segments are respectively marked as T in sequence₁～T₆The whole video start and end time points are respectively T₀ and T₇Represents; wherein S is₀Corresponding sitting up and S₁Stand up correspondingly, S₂Corresponding to forward walking, S₃Corresponding to the in-situ turning, S₄Corresponding to the walking in the reverse direction, S₅Corresponding to turning and sitting in place S₆The corresponding end seat, as shown in fig. 4;

the specific method of video time segmentation based on motion characteristics comprises the following steps:

s31, constructing the following action characteristics:

node J₇、J₈、J₉ and J₁₀The height difference of (a) is defined as follows:

Y(J₇)、Y(J₈)、Y(J₉)、Y(J₁₀) Respectively represents J₇、J₈、J₉ and J₁₀The ordinate of (a);

node J₁J of (A)₂The horizontal distance difference of (a) is defined as:

F_clip₂＝|X(J₁)-X(J₂)|

X(J₁)、X(J₂) Respectively represent J₁J of (A)₂The abscissa of (a);

node J₇ and J₈The horizontal distance difference of (a) is defined as:

F_clip₃＝|X(J₇)-X(J₈)|

X(J₇)、X(J₈) Are respectively J₇ and J₈The abscissa of (a);

s32, respectively calculating the three motion characteristics of S31 for each frame of image to obtain F _ clip₁、F_clip₂、F_clip₃Three discrete sequences, using L2 loss modesThe binary inflection point search algorithm BinSeg of the type is matched with empirical parameters to obtain inflection points of a sequence, and the stage division of actions is obtained according to the inflection points, as shown in fig. 5; the method comprises the following specific steps:

using F _ clip₁Sequence, using inflection point search algorithm with prior knowledge to judge S₀And S₁Is a dividing point T₁(ii) a Specifically, T₁Appear in F _ clip ₁1 st inflection point;

using F _ clip₁Judgment S₁And S₂Is a dividing point T₂；T₂Appear in F _ clip₁At the 2 nd inflection point;

using F _ clip₂Judgment S₂And S₃Is a dividing point T₃；T₃Appear in F _ clip₂At the 1 st inflection point;

using F _ clip₃Judgment S₃And S₄Is a dividing point T₄；T₄Appear in F _ clip₃At the 3 rd inflection point;

using F _ clip₂Judgment S₄And S₅Is a dividing point T₅；T₅Appear in F _ clip₂At the 4 th inflection point;

using F _ clip₁Judgment S₅And S₆Is a demarcation point T₆；T₆Appear in F _ clip₁At the 4 th inflection point.

S4, according to the time characteristics obtained by segmentation in the step S3, time risk judgment is carried out, and the specific method is as follows: the 5 temporal features are defined as follows:

total time: f₁＝|S₁|+|S₂|+|S₃|+|S₄|+|S₅|＝T₆-T₁

Rising stage time: f₂＝|S₁|＝T₂-T₁

Duration of walking phase: f₃＝|S₂|+|S₄|＝T₅+T₃-T₄-T₂

Turn-around phase time: f₄＝|S₃|＝T₄-T₃

Turning body and sitting down stage time: f₅＝|S₅|＝T₆-T₅

Setting threshold Th for five time characteristics₁、Th₂、Th₃、Th₄、Th₅The risk assessment is weighted differently for different temporal characteristics, and a risk score weight W is set for each temporal characteristic₁、W₂、W₃、W₄、W₅Time-characterized risk score R_time：

wherein ,

the threshold and the weight in the above formula are obtained by fitting actual risk data. Risk score R if temporal characteristics_timeExceeds a preset integrated time threshold Th_sumDirectly deem the risk rating high without performing subsequent steps; otherwise, executing step S5; obtaining a threshold Th having a maximum degree of discrimination in the data set by analyzing actual risk data_sum。

S5, extracting the attitude characteristics of each action stage according to frames from the video of each stage obtained after the segmentation of the step S3;

the specific implementation method comprises the following steps:

s51 rising-middle axle front and back deviation characteristic F₆: in the standing process, the central axis deviates from the maximum angle of sitting statically, and the central axis of the trunk is obtained by calculating the coordinates of the left shoulder, the right shoulder, the left hip and the right hip; the specific calculation method is as follows:

firstly, calculating the slope of the central axis of the trunk:

then the slope is converted into a radian system angle which is expressed by a symbol theta;

for each time t_iCalculating the slope according to the above formula

And is turned into an angle of radian

Then F is obtained₆The following were used:

wherein T₁<t_i<T₂Corresponding to all the moments in the process of standing up (also called frames, moment t)_iIn fact, the unit is a frame, that is, each frame calculates the slope and then converts the slope into an angle of radian. Because there are 30 frames per second fixed, the description of the frame sequence and the time series description are equivalent); theta_sitMean offset angle of the fingering stage;

s52, calculating the mean deviation angle F of the middle shaft in the turning process₇：

S53, calculating the maximum angle F of the central axis when the central axis deviates from the standing straight state in the sitting process₈：

wherein T₅<t_i<T₆，θ_standMean deviation angle of finger sitting down stage;

s54, calculating walking step length and height ratio characteristic F₉: the horizontal moving distance between two landing points of the single foot ankle is called as a single step length, the single step length of the two feet is averaged in the walking process, and the ratio of the average step length to the height is calculated; when calculating specificallyAccording to the magnitude of the numerical value, two minimum step values are removed, namely two incomplete steps of starting and stopping at the end point. Here, the absolute coordinate system obtained by openpos is used, and the absolute coordinate symbols are X 'and Y'.

Obtaining a set of landing time points by solving an extreme value through an ankle height sequence:

set of landing time points for the left foot:

A_{floor_left}＝{t_m|T₂<t_m<T₃or T₄<t_m<T₅And X' (t)_m) Is a minimum value }

t_mA time point, X' (t), corresponding to the m-th left ankle abscissa minimum value in the absolute coordinate system obtained by OpenPose_m) Denotes t_mThe left ankle abscissa in the absolute coordinate system of (1); removing landing time points of incomplete steps (when walking starts and finishes, feet are closed, and the definition of single step length is not met, so that the landing time points are removed, only two time points of the start and the end are removed), and counting the average 1-step length;

right foot landing time point set:

A_{floor_right}＝{t_n|T₂<t_n<T₃or T₄<t_n<T₅And X' (t)_n) Is a minimum value }

t_nA time point, X' (t), corresponding to the nth right ankle abscissa minimum value in the absolute coordinate system obtained by OpenPose_n) Is t_nThe abscissa of the right ankle in the absolute coordinate system of (1); removing landing time points of incomplete steps at the head and the tail, and counting the average 1-step length;

the left foot single step size is:

respectively m +1 th in the absolute coordinate systemAnd m left ankle abscissas;

the right foot single step is:

respectively are the abscissa of the (n + 1) th and the n right ankle in an absolute coordinate system;

step length to height ratio characteristic F₉The calculation is as follows:

wherein BH represents body height;

s55, calculating the left and right deviation characteristic F of the walking central axis₁₀That is, during walking, the horizontal left-right average angle difference of the central axis when deviating from the standing straight is calculated as follows:

wherein θ_stdThe angle of the middle shaft when standing straight:

s56 amplitude characteristic F of walking swing arm₁₁: obtaining the vector angle difference formed by the left shoulder, the left elbow, the right shoulder and the right elbow node, namely the elbow-shoulder vector at the farthest position of the back swing and the elbow-shoulder vector at the farthest position of the front swing, and the angle difference of the elbow-shoulder vector and the elbow-shoulder vector;

firstly, respectively calculating the swing arm slopes of the left arm and the right arm:

X'(J_c)、Y'(J_c) J in absolute coordinate systems respectively obtained for OpenPose_cC 1, …, 4;

converting slope into radian angle alpha_{left_arm}And alpha_{right_arm}；

Calculating the swing arm angles of the left arm and the right arm in each frame to form a sequence of the swing arm angles of the left arm and the right arm, and solving an extreme value in the sequence to obtain the maximum swing angle A of the front swing of the left arm_{af_left}Maximum angle A of left arm back swing_{ab_left}Maximum angle A of front swing of right arm_{af_right}Maximum angle A of right arm back swing_{ab_right}；

Will be adjacent to A_{af_left}And A_{ab_left}Aligning and differentiating to obtain a single swing arm angle set A of the left arm_{as_left}；

Will be adjacent to A_{af_right}And A_{ab_right}Aligning and differentiating to obtain a right arm single swing arm angle set A_{as_right}；

Removing A_{as_left}And A_{as_right}The first and last data in the two sets (these two non-pure striding actions, therefore removed) are then grouped by the remaining set A'_{as_left} A'_{as_right}Based on, calculate F₁₁：

wherein α_m∈A'_{as_left}，α_n∈A'_{as_right}，|A'_{as_left}|、|A'_{as_right}L represents set A 'respectively'_{as_left}、A'_{as_right}The number of middle elements;

s57, calculating walking leg swing amplitude characteristic F₁₂: obtaining the vector angle difference formed by the left hip, the left knee, the right hip and the right knee through the nodes of the left hip, the left knee, the right hip and the right knee;

calculating the leg swinging slopes of the left leg and the right leg respectively:

X'(J_c)、Y'(J_c) J in absolute coordinate systems respectively obtained for OpenPose_cC-7, …, 10;

will K_{left_leg} and K_{right_leg}Conversion from slope to radian angle beta_{left_leg}And beta_{right_leg}；

Calculating the leg swinging angles of the left leg and the right leg in each frame to form a sequence of the leg swinging angles of the left leg and the right leg, and solving an extreme value in the sequence to obtain the maximum front swinging angle A of the left leg_{lf_left}Maximum angle A of left leg back swing_{lb_left}Maximum angle A of front swing of right leg_{lf_right}Maximum angle A of right leg back swing_{lb_right}；

Corresponding according to the time sequence, and corresponding to adjacent A_{lf_left} and A_{lb_left}Aligning and differentiating to obtain a left leg swing angle sequence A_{ls_left}；

Will be adjacent to A_{lf_right} and A_{lb_right}Aligning and differencing to obtain a right leg swing angle sequence A_{ls_right}；

Removal of sequence A_{ls_left} and A_{ls_right}And then in the remaining sequence A'_{ls_left} and A'_{ls_right}To obtain F₁₂The calculation is as follows:

wherein β_m∈A'_{ls_left}，β_n∈A'_{ls_right}，|A'_{ls_left}|、|A'_{ls_right}L represent the sequence A 'respectively'_{ls_left} and A'_{ls_right}The number of elements in (c).

S6, judging posture risks according to the posture characteristics obtained in the step S5; the specific implementation method comprises the following steps: step length height ratio F₉And a swing arm F₁₁Swing legs F₁₂The larger the size, the better the movement ability and the balance ability, and the center shaft is deviatedShift correlated feature F₆、F₇、F₈、F₁₀Smaller means better balance ability. The posture feature F₆～F₁₂Inputting the data into a multilayer perceptron to carry out regression prediction to obtain an attitude risk value R_posture。

S7, carrying out final comprehensive risk judgment according to the time risk and the posture risk obtained in the steps S4 and S6; the specific implementation method comprises the following steps: if the time risk R has been obtained in step S4_timeGreater than threshold Th_sumIf so, the comprehensive risk judgment directly obtains a result, and the falling risk is high risk; otherwise, the following judgment is carried out:

setting the weight W by linear fitting_time and W_postureUltimate risk R_final＝W_time·R_time+W_posture·R_posture；

Will end up at risk R_finalWith a high risk reference threshold Th_dangerMiddle risk reference threshold Th_alarmMaking a comparison higher than or equal to a high risk reference threshold Th_dangerThe subject is considered to have a high risk of falling, higher than or equal to the middle risk threshold Th_alarmBut below a high risk threshold Th_dangerThe subject is considered to have a moderate fall risk, below the moderate risk reference threshold Th_alarmThe test person is considered to have a low fall risk.

It will be appreciated by those of ordinary skill in the art that the embodiments described herein are intended to assist the reader in understanding the principles of the invention and are to be construed as being without limitation to such specifically recited embodiments and examples. Those skilled in the art can make various other specific changes and combinations based on the teachings of the present invention without departing from the spirit of the invention, and these changes and combinations are within the scope of the invention.

Claims (5)

1. A method for predicting fall risk of an elderly person, comprising the steps of:

s1, data acquisition and preprocessing, including the following substeps:

s11, arranging a data acquisition environment according to the TUG standard: a chair 60cm high, which is opposite to a flat walkway 3m long; the pixel of the camera equipment is required to be at least 720p and is fixedly placed at the height of 1.5 m;

s12, starting camera shooting when the subject sits up, then rising up and moving forward, turning to and moving back after reaching the end point of the walkway, returning to the starting point and seating again, and stopping camera shooting; the obtained video is preprocessed to 1280x720 pixels, and 30 frames per second;

s13, taking the real falling times of the subject in the last two years as the label of sample data to form a training data set;

s14, classifying the fall risk into 3 grades: the low risk level corresponds to no fall record within 2 years, the medium risk level corresponds to 1 to 2 fall records within 2 years, and the high risk level corresponds to 3 fall records or more;

s2, carrying out skeleton modeling: modeling human body in video as skeleton of 13 nodes by using OpenPose, and using J₀～J₁₂Represents a corresponding joint, and specifically comprises: j. the design is a square₀Head, J₁Left shoulder, J₂Right shoulder, J₃Left elbow, J₄Right elbow, J₅Left wrist, J₆Right wrist, J₇Left hip, J₈Right hip, J₉Left knee, J₁₀Right knee, J₁₁Left ankle, J₁₂-a right ankle;

obtaining a horizontal axis coordinate and a vertical axis coordinate on each frame of 2d image through an OpenPose framework;

and (3) reconstructing an absolute coordinate obtained by OpenPose, and constructing a relative coordinate system: in each frame of image, the center of the person, i.e. J, is found by the absolute coordinates of the nodes₁、J₂、J₇、J₈The human body is selected by a rectangular outer frame with a fixed proportion, the lower left corner of the outer frame is taken as the origin of coordinates, the direction of the longitudinal axis of the transverse axis is unchanged, and a new coordinate system is established;

s3, video time segmentation based on motion characteristics: the video is divided into 7 segments, respectively denoted as S₀～S₆(ii) a The total of 6 time division points among 7 segments are respectively marked as T in sequence₁～T₆At the beginning and end of the entire videoAt intervals with T₀ and T₇Represents; wherein S is₀Corresponding sitting up and S₁Stand up correspondingly, S₂Corresponding to forward walking, S₃Corresponding to the in-situ turning and S₄Corresponding to the walking in the reverse direction, S₅Corresponding to turning and sitting in place S₆The corresponding end seat;

s4, according to the time characteristics obtained by segmentation in the step S3, time risk judgment is carried out, and the specific method is as follows: the 5 temporal characteristics are defined as follows:

total time: f₁＝|S₁|+|S₂|+|S₃|+|S₄|+|S₅|＝T₆-T₁

Rising stage time: f₂＝|S₁|＝T₂-T₁

Duration of walking phase: f₃＝|S₂|+|S₄|＝T₅+T₃-T₄-T₂

Turning stage time: f₄＝|S₃|＝T₄-T₃

Turning body and sitting down stage time: f₅＝|S₅|＝T₆-T₅

Setting threshold Th for five time characteristics₁、Th₂、Th₃、Th₄、Th₅Setting a risk score weight W for each time characteristic₁、W₂、W₃、W₄、W₅Time-characterized risk score R_time：

wherein ,

risk score R if temporal characteristics_timeExceeds a preset integrated time threshold Th_sumThen, thenDirectly deeming the risk rating high without performing subsequent steps; otherwise, executing step S5;

s5, extracting the attitude characteristics of each action stage according to frames from the video of each stage obtained after the segmentation of the step S3;

s6, judging posture risks according to the posture characteristics obtained in the step S5;

and S7, performing final comprehensive risk judgment according to the time risk and the posture risk obtained in the steps S4 and S6.

2. The method for predicting fall risk of the elderly as claimed in claim 1, wherein in step S3, the specific method for video time segmentation based on motion features is as follows:

s31, constructing the following action characteristics:

node J₇、J₈、J₉ and J₁₀The height difference of (a) is defined as follows:

Y(J₇)、Y(J₈)、Y(J₉)、Y(J₁₀) Respectively represents J₇、J₈、J₉ and J₁₀The ordinate of (a);

node J₁J of (A)₂The horizontal distance difference of (a) is defined as:

F_clip₂＝|X(J₁)-X(J₂)|

X(J₁)、X(J₂) Respectively represent J₁J of (A)₂The abscissa of (a);

node J₇ and J₈The horizontal distance difference of (a) is defined as:

F_clip₃＝|X(J₇)-X(J₈)|

X(J₇)、X(J₈) Are respectively J₇ and J₈The abscissa of (a);

s32, respectively calculating the three motion characteristics of S31 for each frame of image to obtainF_clip₁、F_clip₂、F_clip₃Three discrete sequences, namely using a binary inflection point search algorithm BinSeg of an L2 loss model, matching empirical parameters to obtain inflection points of the sequences, and obtaining stage division of actions according to the inflection points; the method comprises the following specific steps:

using F _ clip₁Sequence, using inflection point search algorithm with prior knowledge to judge S₀And S₁Is a dividing point T₁(ii) a Specifically, T₁Appear in F _ clip₁1 st inflection point;

using F _ clip₁Judgment S₁And S₂Is a dividing point T₂；T₂Appear in F _ clip₁At the 2 nd inflection point;

using F _ clip₂Judgment S₂And S₃Is a dividing point T₃；T₃Appear in F _ clip₂1 st inflection point;

using F _ clip₃Judgment S₃And S₄Is a dividing point T₄；T₄Appear in F _ clip₃At the 3 rd inflection point;

using F _ clip₂Judgment S₄And S₅Is a demarcation point T₅；T₅Appear in F _ clip₂At the 4 th inflection point;

using F _ clip₁Judgment S₅And S₆Is a dividing point T₆；T₆Appear in F _ clip₁At the 4 th inflection point.

3. The method for predicting fall risk of the elderly as claimed in claim 1, wherein the step S5 is implemented by:

s51 rising-middle axle front and back deviation characteristic F₆: in the standing process, the central axis deviates from the maximum angle of sitting statically, and the central axis of the trunk is obtained by calculating the coordinates of the left shoulder, the right shoulder, the left hip and the right hip; the specific calculation method is as follows:

firstly, calculating the slope of the central axis of the trunk:

then the slope is converted into a radian system angle which is expressed by a symbol theta;

for each time t_iCalculating the slope according to the above formula

And is turned into an angle of radian

Then F is obtained₆The following were used:

wherein T₁<t_i<T₂Corresponding to all the moments in the standing process; theta_sitMean offset angle of the fingering stage;

s52, calculating the mean deviation angle F of the middle shaft in the turning process₇：

S53, calculating the maximum angle F of the central axis when the central axis deviates from the standing straight state in the sitting process₈：

wherein T₅<t_i<T₆，θ_standMean offset angle at finger sitting down stage;

s54, calculating walking step length and height ratio characteristic F₉: the horizontal moving distance between two landing points of the single foot ankle is called as a single step length, the single step length of the two feet is averaged in the walking process, and the ratio of the average step length to the height is calculated;

obtaining a set of landing time points by solving an extremum value through an ankle height sequence:

set of landing time points for the left foot:

A_{floor_left}＝{t_m|T₂<t_m<T₃or T₄<t_m<T₅And X' (t)_m) Is a minimum value }

t_mA time point, X' (t), corresponding to the m-th left ankle abscissa minimum value in the absolute coordinate system obtained by OpenPose_m) Represents t_mThe left ankle abscissa in the absolute coordinate system of (1); removing landing time points of incomplete steps at the head and the tail, and counting the average 1-step length;

right foot landing time point set:

A_{floor_right}＝{t_n|T₂<t_n<T₃or T₄<t_n<T₅And X' (t)_n) Is a minimum value }

t_nA time point, X' (t), corresponding to the nth right ankle abscissa minimum value in the absolute coordinate system obtained by OpenPose_n) Is t_nThe abscissa of the right ankle in the absolute coordinate system of (1); removing landing time points of incomplete steps at the head and the tail, and counting the average 1-step length;

the left foot single step size is:

respectively are the m +1 th and m left ankle horizontal coordinates in an absolute coordinate system;

the right foot single step is:

respectively are the abscissa of the (n + 1) th and the n right ankle in an absolute coordinate system;

then the walking step length and height ratio characteristic F₉The calculation is as follows:

wherein BH represents body height;

s55, calculating the left and right deviation characteristic F of the walking central axis₁₀That is, during walking, the horizontal left-right average angle difference of the central axis when deviating from the standing straight is calculated as follows:

wherein θ_stdThe angle of the middle shaft when standing straight:

s56 amplitude characteristic F of walking swing arm₁₁: obtaining the vector angle difference formed by the left shoulder, the left elbow, the right shoulder and the right elbow node, namely the elbow-shoulder vector at the farthest position of the back swing and the elbow-shoulder vector at the farthest position of the front swing, and the angle difference of the elbow-shoulder vector and the elbow-shoulder vector;

firstly, respectively calculating the swing arm slopes of the left arm and the right arm:

X'(J_c)、Y'(J_c) J in absolute coordinate systems respectively obtained for OpenPose_cC 1, …, 4;

converting slope into radian angle alpha_{left_arm}And alpha_{right_arm}；

Calculating the swing arm angles of the left arm and the right arm in each frame to form a sequence of the swing arm angles of the left arm and the right arm, and solving an extreme value in the sequence to obtain the maximum swing angle A of the front swing of the left arm_{af_left}Maximum angle A of left arm back swing_{ab_left}Maximum angle A of front swing of right arm_{af_right}Maximum angle A of right arm back swing_{ab_right}；

Will be adjacent to A_{af_left}And A_{ab_left}Aligning and differentiating to obtain a single swing arm angle set A of the left arm_{as_left}；

Will be adjacent to A_{af_right}And A_{ab_right}Aligning and differentiating to obtain a right arm single swing arm angle set A_{as_right}；

Removing A_{as_left}And A_{as_right}The first and last data in the two sets, then with the remaining set A'_{as_left}、A'_{as_right}Based on, calculate F₁₁：

wherein α_m∈A'_{as_left}，α_n∈A'_{as_right}，|A'_{as_left}|、|A'_{as_right}L represents set A 'respectively'_{as_left}、A'_{as_right}The number of middle elements;

s57, calculating walking leg swing amplitude characteristic F₁₂: obtaining the vector angle difference formed by the left hip, the left knee, the right hip and the right knee through the nodes of the left hip, the left knee, the right hip and the right knee;

calculating the leg swinging slopes of the left leg and the right leg respectively:

X'(J_c)、Y'(J_c) J in absolute coordinate systems respectively obtained for OpenPose_cC-7, …, 10;

will K_{left_leg} and K_{right_leg}Conversion from slope to radian angle beta_{left_leg}And beta_{right_leg}；

Calculating the leg swinging angles of the left leg and the right leg in each frame to form a sequence of the leg swinging angles of the left leg and the right leg, and solving an extreme value in the sequence to obtain the maximum front swinging angle A of the left leg_{lf_left}Maximum angle A of left leg back swing_{lb_left}Maximum angle A of front swing of right leg_{lf_right}Maximum angle A of right leg back swing_{lb_right}；

Will be adjacent to A_{lf_left} and A_{lb_left}Aligning and differencing to obtain a left leg swing angle sequence A_{ls_left}；

Will be adjacent to A_{lf_right} and A_{lb_right}Aligning and differentiating to obtain a leg swing angle sequence A of the right leg_{ls_right}；

Removal of sequence A_{ls_left} and A_{ls_right}And then in the remaining sequence A'_{ls_left} and A'_{ls_right}To obtain F₁₂The calculation is as follows:

wherein β_m∈A'_{ls_left}，β_n∈A'_{ls_right}，|A'_{ls_left}|、|A'_{ls_right}L represent the sequences A 'respectively'_{ls_left} and A'_{ls_right}The number of elements in (c).

4. The method for predicting fall risk of the elderly as claimed in claim 3, wherein the step S6 is implemented by: the posture feature F₆～F₁₂Inputting the data into a multilayer perceptron to carry out regression prediction to obtain an attitude risk value R_posture。

5. The method for predicting fall risk of the elderly as claimed in claim 1, wherein the step S7 is implemented by: if it has already been done in step S4Get the time risk R_timeGreater than threshold Th_sumIf so, the comprehensive risk judgment directly obtains a result, and the falling risk is high risk; otherwise, the following judgment is carried out:

setting the weight W by linear fitting_time and W_postureUltimate risk R_final＝W_time·R_time+W_posture·R_posture；

Will end up at risk R_finalWith a high risk reference threshold Th_dangerMiddle risk reference threshold Th_alarmMaking a comparison higher than or equal to a high risk reference threshold Th_dangerThe subject is considered to have a high risk of falling, higher than or equal to the middle risk threshold Th_alarmBut below a high risk threshold Th_dangerThe subject is considered to have a moderate fall risk, below the moderate risk reference threshold Th_alarmThe test person is considered to have a low fall risk.

Images