CN113398556A - Push-up identification method and system - Google Patents

Abstract

The invention discloses a push-up recognition method and system, wherein the method includes a geometric feature extraction step and an evaluation step, and the evaluation step is specifically: detecting whether the posture is standard based on the angle feature, and updating the angle evaluation mark when it is determined that the posture is not standard; The speed feature detects whether the movement direction is reversed; when the movement direction is reversed, it detects whether the action is qualified based on the distance feature, and when it is determined that the action is unqualified, the distance evaluation flag is updated; When the discriminating directions are consistent, it is detected whether there is a discriminant mark, and when it does not exist, the discriminant mark is recorded, and when it exists, the evaluation result is output based on the obtained evaluation mark, and the evaluation mark is reset. The present invention automatically recognizes each complete push-up in the motion image sequence by identifying the identification mark, and automatically evaluates whether the complete push-up is standard by evaluating the angle evaluation mark and the distance evaluation mark.

Description

Push-up identification method and system

technical field

The invention relates to the field of image processing, in particular to a push-up recognition method and system.

Background technique

Push-ups are the most basic bodyweight training method, which can enhance core stability, strengthen the chest, triceps and shoulders, and improve mobility through the shoulder joint, and there is no need for additional fitness equipment during the training process. Implement home workouts.

But push-ups are one of the easiest training items to do wrong. When the push-up movements are not standard, the purpose of exercise cannot be achieved. Today, fitness guidance apps such as keep can only show users the standard push-up posture, but cannot monitor the user's movements.

SUMMARY OF THE INVENTION

The present invention provides a push-up recognition technology in view of the shortcomings of the prior art that only standard push-ups are displayed for users and cannot supervise users to exercise according to the standard.

In order to solve the above-mentioned technical problems, the present invention is solved by the following technical solutions:

A push-up identification method, comprising the following steps:

Obtaining several frames of moving images arranged according to time, and extracting geometric features of each moving image, the geometric features include angle features, distance features and speed features;

Based on the geometric features, each moving image is evaluated in turn, and the corresponding evaluation result is output, and each moving image is evaluated according to the following steps:

Detecting whether the posture is standard based on the angle feature, and updating the angle evaluation indicator when it is determined that the posture is not standard;

Detecting whether the movement direction is reversed based on the speed feature;

When the motion direction does not change, evaluate the geometric features of the next frame of motion images;

When the direction of motion is reversed:

Detecting whether the action is qualified based on the distance feature, and updating the distance evaluation indicator when it is determined that the action is not qualified;

Comparing the movement direction with a preset discrimination direction;

When the motion direction is inconsistent with the preset discrimination direction, detect the geometric feature of the next frame of motion image;

When the movement direction is consistent with the preset discrimination direction, it is detected whether there is a discriminant mark, when it does not exist, the discrimination mark is recorded, and when it exists, the evaluation result is output based on the obtained angle evaluation mark and distance evaluation mark, and repeat The angle evaluation flag and the distance evaluation flag are set.

As an implementation:

Bone point extraction is performed on each moving image to obtain corresponding bone point data, where the bone point data includes the type of the bone point and the corresponding three-dimensional coordinates.

As an embodiment, the angle feature includes the included angle of the knee bone point and the included angle of the hip bone point;

Comparing the included angle of the knee bone points with a preset knee angle threshold, and comparing the hip bone point angle with the preset hip angle threshold, when the knee bone point angle is smaller than the knee The included angle threshold, or when the included angle of the hip bone point is smaller than the hip included angle threshold, it is determined that the posture is not standard, and the angle evaluation flag is updated.

As an implementation:

The method of extracting distance features is:

Based on the preset distance evaluation type, extract the three-dimensional coordinates of the corresponding skeleton point from the skeleton point data, and calculate the distance between the skeleton point and the ground to obtain the corresponding distance feature;

Based on the distance feature to detect whether the action is qualified, when it is judged that the action is not qualified, the method for updating the distance evaluation flag is:

The motion direction is obtained based on the speed feature, and a preset distance threshold is extracted based on the motion direction. When the distance feature fails to match the preset distance threshold, it is determined that the action is not standard, and the distance evaluation indicator is updated.

As an embodiment, the speed feature includes the speed vector of each skeleton point, and the extraction method of the speed vector is as follows:

Extract the skeleton point data corresponding to the kth frame of motion image and the k+1th frame of motion image, and calculate and obtain the velocity vector of each skeleton point corresponding to the kth frame of motion image.

As an implementation:

The velocity feature also includes the velocity gradient vector of each skeleton point, and the velocity gradient vector is obtained by derivation of the corresponding velocity vector with respect to time;

The geometric feature also includes the angle between the elbow bone points. Before detecting whether the motion direction is flipped based on the speed feature, the following steps are also included:

Determine whether the acceleration direction is flipped based on the velocity gradient vector, and when flipped, compare the angle between the elbow bone points with a preset elbow angle threshold, and when the elbow bone point angle exceeds the elbow When the angle threshold is exceeded, the angle evaluation flag is updated.

As an embodiment, after outputting the evaluation result based on the obtained angle evaluation mark and distance evaluation mark, it also includes an evaluation verification step, and the specific steps are:

Extract the time point corresponding to the current frame, and use the time point as the end time point of the current push-up action and the start time point of the next push-up action;

Extract the start time point of the current push-up action, and extract the speed features and distance features of all moving images between the start time point and the end time point of the current push-up action, and based on the preset key parts, from the speed features Extracting key point speed features, forming action frames based on the corresponding key point speed features and distance features, and obtaining a first action frame sequence;

Extracting velocity features in which the direction of the velocity vector or velocity gradient vector is reversed from the first action frame sequence, to obtain a second action frame sequence, where the second action frame sequence includes 5 action frames sorted by time;

The second action frame sequence is input into a pre-built recognition model, and the recognition model outputs standard or non-standard recognition results.

As an embodiment, the steps of constructing the recognition model are:

Obtain a sample moving image sequence, which includes several sample moving images sorted by time;

extracting skeleton points on the sample moving image sequence to obtain a sample skeleton point sequence;

Perform geometric feature extraction on the sample skeleton point sequence to obtain a first sample frame sequence, where the first sample frame sequence includes several sample frames arranged in chronological order, and each sample frame includes key point velocity features and distance features ;

Extracting sample frames whose motion direction or acceleration direction changes from the first sample frame sequence to obtain a second sample frame sequence;

Splitting the second sample frame sequence into several sub-sample frame sequences, each sub-sample frame sequence includes 5 frame sample frames;

Label each sub-sample frame with a sample label, where the sample label is used to indicate whether the push-up action corresponding to the sub-sample frame sequence is standard

The recognition model is obtained by training the sub-sample frame sequence and the sample label.

As an implementation:

The recognition model is an HMM model, an LSTM-FCN model or an SVM model.

The present invention also proposes a push-up identification system, comprising:

The feature extraction module is used to obtain several frames of moving images arranged in time, and extract the geometric features of each moving image, the geometric features include angle features, distance features and speed features

an evaluation module, configured to evaluate actions in each moving image in turn based on the geometric features, and output corresponding evaluation results;

The evaluation module includes an angle determination unit, a flip determination unit, a distance evaluation unit, a direction comparison unit, a discrimination mark detection unit and an evaluation unit;

The angle judgment unit is used to detect whether the posture is standard based on the angle feature, and when it is determined that the posture is not standard, update the angle evaluation mark, and the angle assessment mark is used to indicate whether the posture is standard;

The flip judgment unit is used to detect whether the motion direction flips based on the speed feature;

A distance evaluation unit, configured to detect whether the action is qualified based on the distance feature when the direction of movement is reversed, and when it is determined that the action is unqualified, update the distance assessment mark, and the distance assessment mark is used to indicate whether the action is qualified;

a direction comparison unit, used for comparing the movement direction with a preset discrimination direction;

a discrimination mark detection unit, used for detecting whether a discrimination mark already exists when the movement direction is consistent with a preset discrimination direction;

The evaluation unit is used to record the identification mark when the identification mark does not exist, and is also used to output the evaluation result based on the obtained angle evaluation mark and the distance evaluation mark when the identification mark exists, and clear the angle evaluation mark and the distance evaluation mark logo.

The present invention has significant technical effects due to the adoption of the above technical solutions:

The present invention can automatically identify each complete push-up in the motion image sequence through the design of the discrimination mark, and realizes the evaluation of the posture and movement in a complete push-up action cycle by designing the angle evaluation mark and the distance evaluation mark. and feedback to urge users to make corrections.

The present invention uses HMM model (Hidden Markov Model) to model the push-up process through the design of the evaluation and verification steps, and further identifies the detected complete push-up action, so as to improve the accuracy of the push-up evaluation.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

Fig. 1 is the identification flow schematic diagram of a kind of push-up identification method of the present invention;

Fig. 2 is the module connection schematic diagram of a kind of push-up identification system of the present invention;

FIG. 3 is a schematic diagram of module connections of the evaluation module 200 in FIG. 2 .

Detailed ways

The present invention will be further described in detail below in conjunction with the examples. The following examples are to explain the present invention and the present invention is not limited to the following examples.

Embodiment 1, a kind of push-up identification method, comprises the following steps:

S100, acquiring several frames of moving images arranged according to time, and extracting geometric features of each moving image, where the geometric features include distance features and speed features;

S200, evaluating each moving image in turn based on the geometric features, and outputting corresponding evaluation results;

As shown in Figure 1, each moving image is evaluated according to the following steps:

S210. Detect whether the posture is standard:

Detect whether the posture is standard based on the angle feature, and when it is determined that the posture is not standard, update the angle evaluation identifier, and go to step S220;

In this embodiment, the angle evaluation flag is a variable flag1, and its initial value is zero. When it is determined that the posture is not standard based on the angle feature, the value of flag1 is updated to 1.

S220. Whether the movement direction is reversed:

Detecting whether the movement direction is reversed based on the speed feature;

When no flip occurs, end the evaluation of the current moving image, obtain the geometric features of the next frame of moving image, and repeat this step;

When the movement direction is reversed, go to step S230;

S230. Check whether the action is qualified:

Detect whether the action is qualified based on the distance feature, and when it is determined that the action is unqualified, update the distance evaluation flag, and go to step S40;

In the present embodiment, the angle evaluation is marked as variable flag2, and its initial value is zero, and when the action is determined to be unqualified based on the distance feature, the value of flag2 is updated to 1;

If the direction of movement is reversed, it means that the user bends his elbows to the lowest position or straightens his elbows to the highest position. Therefore, in this embodiment, the distance between the body and the ground is evaluated when the direction of movement is reversed to evaluate the push-up action. Whether it is in place and qualified.

S240. Whether it is a judgment point:

Acquire a preset discrimination direction, and compare the movement direction with the preset discrimination direction;

Those skilled in the art can set the discrimination direction according to actual needs, and the discrimination direction is used to indicate the starting direction of the push-up action;

The push-up movement includes descending action and push-up action. In this embodiment, the action direction of the descending action is set as the discriminating direction, so when it is detected that the descending action will be performed, it can be determined that a new push-up action will be started. This is used as a discriminant point to detect the complete movement of push-ups;

When the movement direction is inconsistent with the preset discrimination direction, it means that the current frame is not the discriminant point in the current push-up action cycle. End the evaluation of the current moving image, obtain the geometric features of the next frame of moving image, and enter Step S210;

When the movement direction is consistent with the preset discrimination direction, it means that a new push-up action will be performed, and the current frame is the discrimination point, and at this time, step S250 is entered;

S250. Is the push-up action complete?

Check whether there is a discriminant mark;

When it does not exist, record the identification mark;

When present, an evaluation result is output based on the obtained angle evaluation flag and distance evaluation flag, and the angle evaluation flag and distance evaluation flag are reset.

The evaluation results may include, for example, standard evaluation results and counting results. As shown in FIG. 1 , when the current push-up action has a non-standard posture or an unqualified action, the non-standard standard evaluation result is output, and the current push-up action is not technically performed. , otherwise output the standard standard evaluation result and count the current push-up action.

The identification mark is initially empty. When the push-up is started, the user performs a descending action. At this time, the movement direction changes, and the movement direction is consistent with the preset identification direction. There is no identification mark before. It is to record the start of the push-up action and record the identification mark. ;

After completing a push-up, the user performs the descending action again, and an existing judgment mark will be detected, which means that the user has completed a complete push-up and will proceed to the next push-up. Therefore, at this time, the obtained evaluation mark is first extracted to judge the push-up. Whether it is standard or not, and feedback the corresponding evaluation results to the user to remind the user to pay attention to correcting the push-up action, and then clear the evaluation flag to facilitate the evaluation of the next push-up action.

The complete push-up routine includes:

The initial state, that is, the preparation state, at this time, the hands are on the ground and the body is kept in a straight line;

In the first movement process, bend the elbows and let the chest go down to the ground. At this time, the movement direction is downward, and the speed of the body's descending process is to accelerate first and then decelerate;

The lowest state, when the chest drops to the ground, and the hips are at the same height as the rest of the body;

In the second movement process, push the body upwards and return to the ready state. During this process, the movement direction is upward, and the speed of the body is also accelerated first and then decelerated during the ascent;

The final state is the initial state of the next push-up.

It can be seen from the above that push-ups are a dynamic movement process, not a static posture. When users perform push-up training, it will be difficult for users to check whether the movements are standard, and the purpose of exercise cannot be achieved. The push-up identification method disclosed in this embodiment, As long as the user shoots a motion video of the push-up as required, the push-up recognition can be performed based on each video frame of the motion video as a moving image, and each complete push-up can be automatically recognized, and the posture and action of completing the push-up can be performed. Evaluation and feedback to urge users to make corrections.

Further, the specific steps of extracting the geometric features of each moving image in step S100 are:

S110, performing skeleton point extraction on each moving image to obtain corresponding skeleton point data, where the skeleton point data includes the type of skeleton point and corresponding three-dimensional coordinates;

Specifically:

S111, extracting 2D skeleton points of each moving image based on an existing published OpenPose model;

S112, repairing the 2D skeleton points of each moving image to obtain corresponding skeleton point data, where the skeleton point data includes the type of the skeleton point and corresponding three-dimensional coordinates;

Since it is difficult for the OpenPose model to find the skeleton points of the occluded part, and the extraction effect of the skeleton points is easily affected by the environment, when performing push-up training, the overlapping part of the body is as high as 50%, and the accuracy of the 2D skeleton points extracted by the OpenPose model is low. will affect the accuracy of push-up recognition.

Therefore, in this embodiment, the obtained skeleton points are repaired and converted into three-dimensional data, so as to solve the above-mentioned defects.

Those skilled in the art can choose any of the existing published repair methods to repair 2D skeleton points according to the actual situation, such as using the existing 2D to 3D conversion module for repair, and using the optical flow method to form a vector diagram for repair , Linear repair can also be used to obtain the position of each bone point in its two adjacent frames (the previous frame and the next frame) 2D bone point data, as well as the velocity characteristics corresponding to the bone point, and complete the bone point. point repair, which is not specifically limited in this embodiment.

In this embodiment, the positions of the neck, shoulders, elbows, wrists, both sides of the hips, knees and feet of the human body in the moving image are extracted as bone points;

S120, constructing a geometric feature based on the skeleton point data, and obtaining a geometric feature corresponding to the moving image;

Those skilled in the art can increase or decrease the position of the bone points to be extracted according to actual needs, and set the angle features, distance features and speed features to be extracted according to the actual situation, and only need to make the angle features reflect the human body posture. , The distance feature can reflect the distance between the human body and the ground, and the speed feature can reflect the direction of the human body movement.

Further, the specific steps of constructing geometric features based on the skeleton point data in step S120 are:

S121, extracting angle features:

Angle features include the included angle of knee bone points and the included angle of hip bone points;

The angle between the above-mentioned knee bone points is the angle formed by the line connecting the foot bone point and the knee bone point, and the line connecting the knee bone point and the hip bone point, and two line segments;

The angle between the above-mentioned hip bone points is the line connecting the knee bone point and the hip bone point, the line connecting the hip bone point and the shoulder bone point, and the angle formed by the two line segments; it can be seen from the above that the above-mentioned angle is the corresponding three bones Inferior or straight angle formed by points.

In push-up training, the human body needs to keep a straight line from the shoulders to the ankles. In this embodiment, the posture of the human body in each frame of moving images is judged during the exercise process by designing the angle between the knee bone point and the hip bone point. Is it standard.

S122, extracting distance features:

Based on the preset distance evaluation type, extract the three-dimensional coordinates of the corresponding skeleton point from the skeleton point data, and calculate the distance between the skeleton point and the ground to obtain the corresponding distance feature;

Those skilled in the art can set the distance evaluation type by themselves according to actual needs. In this embodiment, the distance evaluation type is shoulder, that is, the three-dimensional coordinates of the bone point corresponding to the shoulder are extracted from the obtained bone point data, and the shoulder is calculated. The distance to the ground, which is used as the distance feature.

S123. Extracting speed features:

The velocity feature includes the velocity vector and velocity gradient vector of each skeleton point;

The extraction method of the velocity feature corresponding to the kth frame of the moving image is:

Extract the skeletal point data corresponding to the k-th moving image and the k+1-th moving image, and calculate the velocity vector of each bone point, where k is greater than 0 and less than m, where m is the total number of moving images.

的计算公式为：The three-dimensional coordinates of a bone point are (x, y, z), the three-dimensional coordinates corresponding to the bone point in the next frame are (x', y', z'), and the time interval between two frames is Δt, and the bone point corresponds to the velocity vector of

The calculation formula is:

判断运动方向，如基于对应速度矢量

中

的正负确定运动方向和运动方向的翻转情况，或当

的方向发生变化，且

与

中任意一个的方向发生变化时，判定运动方向发生翻转，再以

的方向确定运动方向。In practical applications, by specifying one or more bone points, the velocity vector based on the specified bone points

Determine the direction of motion, such as based on the corresponding velocity vector

middle

The positive and negative of , determine the direction of motion and the inversion of the direction of motion, or when

changes in direction, and

and

When the direction of any one of the

The direction of the motion determines the direction of movement.

由0转为正值或负值时，仍旧判定其方向发生变化。Note that when

When changing from 0 to positive or negative value, it is still judged that its direction has changed.

Further, in step S210, it is detected whether the posture is standard based on the angle feature, and when it is determined that the posture is not standard, the specific steps of updating the angle evaluation mark are as follows:

Comparing the included angle of the knee bone points with a preset knee angle threshold, and comparing the hip bone point angle with the preset hip angle threshold, when the knee bone point angle is smaller than the knee The included angle threshold, or when the included angle of the hip bone point is smaller than the hip included angle threshold, it is determined that the posture is not standard, and the angle evaluation flag is updated.

Those skilled in the art can set the knee angle threshold and the hip angle threshold by themselves according to the actual situation. In this embodiment, the knee angle threshold and the hip angle threshold are both set to 150°.

Those skilled in the art can set the update method of the angle evaluation mark by themselves according to actual needs, such as:

There is only one angle evaluation identifier, and its initial value is 0. When the angle between the knee bone points is less than the knee angle threshold, or the hip bone point angle is less than the hip angle threshold, it is updated to 1. When it is detected in step S250 that the angle evaluation flag is 1, it means that in the whole push-up cycle, the user's posture is not standard, so that corresponding feedback is given to the user.

In the angle feature, each included angle feature has an angle evaluation sign corresponding to it, and the initial value of each angle evaluation sign is 0, and when the included angle is less than the preset included angle threshold, the corresponding angle evaluation sign is updated to 1, In step S250, each angle evaluation flag is traversed, and based on the angle evaluation flag with a value of 1, feedback is given to the user that the user's posture is not standard and existing problems in the push-up cycle, such as the hips not keeping a straight line with the body.

Further, in step S230, based on the distance feature to detect whether the action is qualified, when it is determined that the action is not qualified, the specific steps of updating the distance evaluation mark are:

The movement direction is obtained based on the speed feature, a preset distance threshold is extracted based on the movement direction, and when the distance feature fails to match the preset distance threshold, the distance evaluation identifier is updated.

In this embodiment, based on the three-dimensional coordinates of the skeleton point in the next frame, the velocity vector of the corresponding skeleton point in the current frame is calculated, so the velocity vector is used to indicate the movement direction of the action;

That is, when the user is in the initial state or the final state, its movement direction is downward, and when the user is in the lowest state, its movement direction is upward;

In this embodiment, the distance threshold includes a first distance threshold and a second distance threshold, wherein the first distance threshold is greater than the second distance threshold, and when the movement direction is descending, the first distance threshold is extracted; otherwise, the second distance threshold is extracted; When the distance between the skeleton point and the ground is less than the first distance threshold, it is determined that the action is not standard, and the distance evaluation threshold is updated;

When the distance between the shoulder bone point and the ground is greater than the second distance threshold, it is determined that the action is not standard, and the distance evaluation threshold is updated;

Those skilled in the art can set the first distance threshold and the second distance threshold by themselves according to actual needs. For example, the distance from the shoulder to the ground when the user starts doing push-ups can be used as the first distance threshold, and the length of the user's arm can also be obtained. The set weight coefficient is calculated to obtain the first distance threshold.

Those skilled in the art can set the update method of the distance evaluation mark by themselves according to actual needs, such as:

There is only one distance evaluation indicator, and its initial value is 0. When the distance feature fails to match with the preset distance threshold, it is updated to 1. When step S250 detects that the distance evaluation indicator is 1, it indicates that the entire push-up cycle is In the case of non-standard user actions, corresponding feedback is given to the user.

Each movement direction corresponds to a distance evaluation mark, and the initial value of each distance assessment mark is 0. When the distance feature fails to match with the corresponding distance threshold, the corresponding distance assessment mark is updated to 1, and each distance assessment mark is traversed in step S250, Based on the distance evaluation flag with a value of 1, it is fed back to the user that the user's actions are not standard and existing problems in the push-up cycle, and the above push is not in place.

Further, the velocity feature also includes the velocity gradient vector of each skeleton point, and the velocity gradient vector is obtained by derivation of the corresponding velocity vector with respect to time;

The velocity gradient vector Δv corresponding to this bone point is:

表示该骨骼点所对应的速度矢量，Δt表示计算速度矢量时所采用的间隔时间。in,

Represents the velocity vector corresponding to the skeleton point, and Δt represents the interval time used to calculate the velocity vector.

In this embodiment, the velocity vector and the velocity gradient vector are both three-dimensional vectors. The velocity gradient vector is used to represent the gradient of the velocity vector corresponding to the skeleton point, and the velocity gradient vector is used to identify the moving image frame corresponding to the intermediate state. During the motion process or the second motion process, the direction of the motion acceleration is reversed, that is, the frame in which the positive and negative directions of the velocity gradient vector z direction are converted.

Further, the geometric feature also includes the included angle of the elbow bone point;

The angle between the elbow skeleton points is the connection line between the shoulder skeleton point and the elbow skeleton point, the connection line between the elbow skeleton point and the wrist skeleton point, and the angle formed by the two connection lines.

Further, step S210 also includes a step of updating the angle evaluation identifier based on the angle between the elbow bone points, and the specific steps are:

Determine whether the acceleration direction is flipped based on the velocity gradient vector, and when flipped, compare the angle between the elbow bone points with a preset elbow angle threshold, and when the elbow bone point angle exceeds the elbow When the angle threshold is exceeded, the angle evaluation flag is updated.

When the value of the corresponding velocity gradient vector in the z direction changes from positive to negative or from negative to positive, it is determined that the direction of the acceleration is reversed, and the corresponding moving image is in an intermediate state. Detection of bending angle to detect whether the posture of the elbow is standard when it is used for descending or pushing up.

Further, the evaluation result in step S250 includes the standard evaluation result;

Determine whether the posture is standard or not based on the angle evaluation mark, and obtain the corresponding posture evaluation result;

Judging whether the action is standard based on the distance evaluation mark, and obtaining the corresponding action evaluation result;

generating corresponding standard evaluation results based on the posture evaluation results and the motion evaluation results;

Embodiment 2, after step S250 in Embodiment 1, the described evaluation verification step is added, that is, the evaluation result is output based on the obtained angle evaluation sign and distance evaluation sign, and after the angle evaluation sign and the distance evaluation sign, increase To the evaluation verification step of described evaluation result, all the other are equal to embodiment 1;

The evaluation and verification steps are as follows:

S310, extract the time point corresponding to the current frame, and use the time point as the end time point of the current push-up action and the start time point of the next push-up action;

S320, extracting the start time point of the current push-up action, and extracting the speed features and distance features of all moving images between the start time point and the end time point of the current push-up action, and based on preset key parts, from the speed Extracting key point speed features from the features, forming action frames based on the corresponding key point speed features and distance features, and obtaining a first action frame sequence;

In this embodiment, the key parts are shoulders, hips, knees, elbows and wrists, and those skilled in the art can designate key parts according to actual needs;

Extract the velocity vector and velocity gradient vector of each key part corresponding to the skeleton point, and obtain the corresponding key point velocity feature;

The speed feature and distance feature of the key points corresponding to the same moving image are taken as the corresponding action frame.

S330, extracting a velocity feature in which the direction of the velocity vector or velocity gradient vector is reversed from the first action frame sequence, to obtain a second action frame sequence, where the second action frame sequence includes 5 action frames sorted by time;

When the push-up action is in the initial state, the lowest state and the final state, its velocity vector changes in the positive and negative directions of the z direction. In this embodiment, it is determined that the direction of the velocity vector is reversed, so as to obtain the velocity characteristics corresponding to these three states; When the action is in the intermediate state, the corresponding velocity gradient vector is reversed in the z direction. In this embodiment, it is determined that the direction of the acceleration is reversed, so as to obtain the speed characteristic corresponding to this state. A complete push-up action contains two intermediate states. , so two corresponding velocity characteristics are obtained;

Based on the time sequence information of the moving images corresponding to each speed feature, the speed features are arranged in chronological order to obtain the order indicating the initial state, the intermediate state (falling action), the lowest state, the intermediate state (upward action), and the final state. , to indicate the action frame sequence of the complete push-up action, that is, the second action frame sequence.

S340. Input the second action frame sequence into a pre-built recognition model, and the recognition model outputs a standard or non-standard recognition result.

The recognition model is a binary classification model.

Those skilled in the art can verify each evaluation result according to actual needs, verify when the evaluation result is not standard, or verify when the evaluation result is standard, obtain the corresponding identification result, count and feedback based on the obtained identification result, It is not specifically limited in this embodiment.

Through the design of the evaluation and verification steps in this embodiment, the machine learning technology can be used to further identify the push-up action, so as to improve the accuracy of the action standard evaluation and provide the user with more accurate training guidance.

Further, the steps of constructing the recognition model are:

Obtain a sample moving image sequence, which includes several sample moving images sorted by time;

extracting skeleton points on the sample moving image sequence to obtain a sample skeleton point sequence;

Perform geometric feature extraction on the sample skeleton point sequence to obtain a first sample frame sequence, where the first sample frame sequence includes several sample frames arranged in chronological order, and each sample frame includes key point velocity features and distance features ;

Splitting the second sample frame sequence into several sub-sample frame sequences, each sub-sample frame sequence includes 5 frame sample frames;

Each sub-sample frame is marked with a sample label, the sample label is used to indicate whether the push-up action corresponding to the sub-sample frame sequence is standard, and the sample label can be standard, non-standard, or the state corresponding to each sample frame in the sub-sample frame sequence. Including the initial/final state, the first intermediate state, the lowest state, the second intermediate state and the initial/final state;

The recognition model is obtained by training the sub-sample frame sequence and the sample label.

Those skilled in the art can use the obtained sample action frame sequences and corresponding sample labels to perform model training according to the existing conventional model training steps to obtain a corresponding recognition model. The specific training steps are not described in detail in this embodiment.

further:

The recognition model is an existing published HMM model (hidden Markov model), LSTM-FCN model (long short-term memory full convolutional neural network model) or SVM model (support vector machine).

further:

The recognition model is an HMM model, and the Baum_Welch algorithm is used to perform parameter learning on the HMM model to maximize the output possibility for an observation sequence, and the parameter learning steps are as follows:

E. Calculate hidden layer parameters:

The hidden layer parameters include α _t (j), β _t (i), γ _t (i), P(O ₁ , O ₂ ,..., O _T |λ);

The calculation formula of parameter α _t (i) is as follows:

Among them, t represents the t-th action frame, i represents the motion state, N represents the number of motion states, O _t represents the observation value corresponding to the t-th action frame, and the observation value is the feature data, including the key point speed feature and distance feature , a _ji represents the probability of transitioning from state q _j to q _i , which is a matrix, representing the probability of each state transitioning to state q _i .

The calculation formula of parameter β _t (i) is as follows:

Among them, a _ij represents the probability of _{transitioning from state qi to q j ,} which is a matrix and represents the probability of each state transitioning to state q _j .

The calculation formula of parameter γ _t (i) is as follows:

Among them, α _T (i) represents the value of α _t (i) when t=T.

The calculation formula of the parameter P(O ₁ , ..., O _T |λ) is as follows:

Among them, λ is the parameter of the HMM model, which is estimated by the maximum likelihood method.

The principle is as follows:

In the training process, the input sub-sample frame sequence is taken as the observation sequence O ₁ , O ₂ , ..., O _T , which contains 5 frames of feature data, and each frame of feature data is equivalent to an observation value;

At the same time, each sample frame indicates a state, which is the initial/final state, the first intermediate state, the lowest state, the second intermediate state, and the initial/state, that is, 5 motion states (q ₁ , q ₂ , ... , q _n ), each state has an output distribution d _i (O) and a transition probability distribution a _ij ; d _i (O) represents the probability of generating the corresponding observation O under the state q _i , and a _ij represents the state determined by the state The probability that q _i goes to q _j .

The parameter of the hidden Markov model to be maximized is preset to be λ, and the hidden layer variable X _t represents the state (q ₁ , q ₂ , . . . , q _n ) of the HMM model at time t, and the conditional probability P(O ₁ , O ₂ ,..., O _T |λ) indicates the degree of agreement between the event and the calculated HMM model. The higher the degree of agreement (probability value), the higher the probability of correct action. The large likelihood method estimates the model parameter λ;

Then use γ _t (i)=P(X _t = q _i |O ₁ , O ₂ ,..., O _T , λ) to represent the distribution of the hidden layer state sequence, through the given observation sequence O ₁ , O _{2 , . _ _} The sequence of the push-up movements is the same, that is, the sequence is the initial/final state, the first intermediate state, the lowest state, the second intermediate state, and the initial/final state. make calculations to the algorithm;

The result is obtained by the variables α _t (i)=P(O ₁ , O ₂ , . . . , O _t , X _t =q _i |λ) and the variables β _t (i)=P(O _t , O _t+1 , .. ., O _T |X _t =q _i , λ).

M. Update the parameter λ of the HMM model to maximize the output probability given the hidden layer parameters.

In actual use, the input second action frame sequence is taken as the observation sequence O, the state sequence I with the largest P(I|O) is obtained by calculation, and the state sequence I is compared with the preset push-up action sequence, if the same Then judge its standard, otherwise judge its non-standard.

Embodiment 3. A push-up recognition system, as shown in Figure 2, includes:

The feature extraction module 100 is used for acquiring several frames of moving images arranged according to time, and extracting the geometric features of each moving image, the geometric features include angle features, distance features and speed features

The evaluation module 200 is used to evaluate the actions in each moving image in turn based on the geometric features, and output corresponding evaluation results;

As shown in FIG. 3 , the evaluation module 200 includes an angle determination unit 210, a flip determination unit 220, a distance evaluation unit 230, a direction comparison unit 240, a discriminant identification detection unit 250 and an evaluation unit 260;

The angle judging unit 210 is configured to detect whether the posture is standard based on the angle feature, and when it is determined that the posture is not standard, update the angle evaluation mark, and the angle assessment mark is used to indicate whether the posture is standard;

The flip judging unit 220 is configured to detect whether the motion direction flips based on the speed feature;

The distance evaluation unit 230 is used to detect whether the action is qualified based on the distance feature when the direction of movement is reversed, and when it is determined that the action is unqualified, update the distance assessment mark, and the distance assessment mark is used to indicate whether the action is qualified;

a direction comparison unit 240, configured to compare the movement direction with a preset discrimination direction;

The discrimination mark detection unit 250 is used to detect whether there is a discrimination mark when the movement direction is consistent with the preset discrimination direction;

The evaluation unit 260 is used to record the identification mark when the identification mark does not exist, and is also used to output the evaluation result based on the obtained angle evaluation mark and the distance evaluation mark when the identification mark exists, and clear the angle evaluation mark and the distance. Evaluation ID.

Further, the feature extraction module 100 is configured to perform skeleton point extraction on each moving image to obtain corresponding skeleton point data, where the skeleton point data includes the type of skeleton point and corresponding three-dimensional coordinates.

In this embodiment, the feature extraction module 100 includes a skeleton point extraction unit, a repair unit and a feature extraction unit;

The skeleton point extraction unit is used to extract the 2D skeleton points of each moving image based on the existing OpenPose model;

The repairing unit is used to repair the 2D skeleton points of each moving image, and obtain corresponding skeleton point data, where the skeleton point data includes the type of the skeleton point and the corresponding three-dimensional coordinates;

The feature extraction unit constructs geometric features based on the skeleton point data, and obtains geometric features corresponding to the moving image, which includes a first extraction subunit for extracting angle features, and a second extraction subunit for extracting distance features. subunit and a third extraction subunit for extracting velocity features.

Further, a verification module 300 is also included, and the verification module 300 includes:

a time configuration unit, used to extract the time point corresponding to the current frame, and use the time point as the end time point of the current push-up action and the start time point of the next push-up action;

The action frame construction unit is used to extract the starting time point of the current push-up action, and extract the speed features and distance features of all moving images between the starting time point and the end time point of the current push-up action, and based on the preset key parts , extracting key point speed features from the speed features, forming action frames based on the corresponding key point speed features and distance features, and obtaining a first action frame sequence;

An action frame extraction unit, configured to extract the velocity feature of the reversed direction of the velocity vector or velocity gradient vector from the first action frame sequence, to obtain a second action frame sequence, the second action frame sequence includes 5 frames by time Sorted action frames;

The recognition unit is configured to input the second action frame sequence into a pre-built recognition model, and the recognition model outputs a standard or non-standard recognition result.

As for the apparatus embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for related parts.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments may be referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing terminal equipment to produce a machine that causes the instructions to be executed by the processor of the computer or other programmable data processing terminal equipment Means are created for implementing the functions specified in the flow or flows of the flowcharts and/or the blocks or blocks of the block diagrams.

These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal equipment to operate in a particular manner, such that the instructions stored in the computer readable memory result in an article of manufacture comprising instruction means, the The instruction means implement the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing terminal equipment, so that a series of operational steps are performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby executing on the computer or other programmable terminal equipment The instructions executed on the above provide steps for implementing the functions specified in the flowchart or blocks and/or the block or blocks of the block diagrams.

It should be noted:

Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases "one embodiment" or "an embodiment" in various places throughout the specification are not necessarily all referring to the same embodiment.

Although the preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of the present invention.

In addition, it should be noted that, in the specific embodiments described in this specification, the shapes and names of parts and components thereof may be different. All equivalent or simple changes made according to the structures, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or supplements to the described specific embodiments or substitute in similar manners, as long as they do not deviate from the structure of the present invention or go beyond the scope defined by the claims, All should belong to the protection scope of the present invention.

Claims (10)

1. A push-up identification method is characterized by comprising the following steps:

acquiring a plurality of moving images arranged according to time, and extracting geometric features of each moving image, wherein the geometric features comprise angle features, distance features and speed features;

and sequentially evaluating the moving images based on the geometric features, and outputting corresponding evaluation results, wherein the moving images are evaluated according to the following steps:

detecting whether the posture is standard or not based on the angle features, and updating the angle evaluation identifier when the posture is judged to be not standard;

detecting whether the movement direction is overturned based on the speed characteristics;

when the motion direction is not changed, evaluating the geometrical characteristics of the next frame of motion image;

when the motion direction is reversed:

detecting whether the action is qualified or not based on the distance characteristics, and updating the distance evaluation identifier when the action is judged to be unqualified;

comparing the motion direction with a preset judgment direction;

when the motion direction is inconsistent with a preset judgment direction, detecting the geometric characteristics of the next frame of motion image;

and when the movement direction is consistent with the preset judgment direction, detecting whether a judgment mark exists or not, when the judgment mark does not exist, recording the judgment mark, when the judgment mark exists, outputting an evaluation result based on the obtained angle evaluation mark and the distance evaluation mark, and resetting the angle evaluation mark and the distance evaluation mark.

2. The push-up identification method according to claim 1, characterized in that:

and extracting the bone points of each moving image to obtain corresponding bone point data, wherein the bone point data comprises the types of the bone points and the corresponding three-dimensional coordinates.

3. The push-up identification method according to claim 2, wherein the angular features include knee bone point angles and hip bone point angles;

comparing the knee skeleton point included angle with a preset knee included angle threshold value, comparing the hip skeleton point included angle with a preset hip included angle threshold value, judging that the posture is not standard when the knee skeleton point included angle is smaller than the knee included angle threshold value or the hip skeleton point included angle is smaller than the hip included angle threshold value, and updating the angle evaluation identification.

4. The push-up recognition method according to claim 2, characterized in that:

the method for extracting the distance features comprises the following steps:

extracting three-dimensional coordinates of corresponding bone points from the bone point data based on a preset distance evaluation type, and calculating the distance between the bone points and the ground to obtain corresponding distance characteristics;

whether the action is qualified or not is detected based on the distance characteristics, and when the action is judged to be unqualified, the method for updating the distance evaluation identifier comprises the following steps:

and obtaining a movement direction based on the speed characteristics, extracting a preset distance threshold value based on the movement direction, judging that the action is not standard when the distance characteristics are failed to be matched with the preset distance threshold value, and updating the distance evaluation identifier.

5. The push-up identification method according to any one of claims 2 to 4, wherein the velocity features include velocity vectors of respective bone points, and the velocity vectors are extracted by:

and extracting the bone point data corresponding to the k frame of moving image and the k +1 frame of moving image, and calculating to obtain the speed vector of each bone point corresponding to the k frame of moving image.

6. The push-up recognition method according to claim 5, characterized in that:

the velocity features further comprise velocity gradient vectors for each bone point, the velocity gradient vectors being derived from the corresponding velocity vectors over time;

the geometric characteristics further comprise an included angle of an elbow bone point, and before detecting whether the motion direction is overturned based on the speed characteristics, the method further comprises the following steps:

judging whether the acceleration direction is overturned or not based on the velocity gradient vector, comparing the elbow skeleton point included angle with a preset elbow included angle threshold value when the acceleration direction is overturned, and updating the angle evaluation identification when the elbow skeleton point included angle exceeds the elbow included angle threshold value.

7. The push-up recognition method according to claim 6, wherein after outputting the evaluation result based on the obtained angle evaluation identifier and distance evaluation identifier, the method further comprises an evaluation verification step, and the specific steps are as follows:

extracting a time point corresponding to the current frame, and taking the time point as an ending time point of the current push-up action and an initial time point of the next push-up action;

extracting a starting time point of a current push-up action, extracting speed characteristics and distance characteristics of all moving images between the starting time point and an ending time point of the current push-up action, extracting key point speed characteristics from the speed characteristics based on a preset key part, forming an action frame based on the corresponding key point speed characteristics and distance characteristics, and obtaining a first action frame sequence;

extracting a speed feature with the direction of a speed vector or a speed gradient vector reversed from the first action frame sequence to obtain a second action frame sequence, wherein the second action frame sequence comprises 5 action frames ordered in time;

and inputting the second action frame sequence into a pre-constructed recognition model, and outputting a standard or non-standard recognition result by the recognition model.

8. The push-up recognition method according to claim 7, wherein the recognition model is constructed by the steps of:

acquiring a sample moving image sequence which comprises a plurality of sample moving images which are ordered according to time;

extracting skeleton points of the sample motion image sequence to obtain a sample skeleton point sequence;

extracting geometric features of the sample skeleton point sequence to obtain a first sample frame sequence, wherein the first sample frame sequence comprises a plurality of sample frames which are arranged according to a time sequence, and all the sample frames comprise a key point speed feature and a distance feature;

extracting sample frames with changed motion direction or acceleration direction from the first sample frame sequence to obtain a second sample frame sequence;

splitting the second sample frame sequence into a plurality of sub-sample frame sequences, wherein each sub-sample frame sequence comprises 5 sample frames;

labeling each sub-sample frame with a sample label, wherein the sample label is used for indicating whether the push-up action corresponding to the sub-sample frame sequence is standard or not

And training by using the sub-sample frame sequence and the sample label to obtain a recognition model.

9. The push-up recognition method according to claim 8, characterized in that:

the recognition model is an HMM model, an LSTM-FCN model or an SVM model.

10. A push-up identification system, comprising:

a feature extraction module for acquiring a plurality of frames of moving images arranged according to time and extracting geometric features of each moving image, wherein the geometric features comprise angle features, distance features and speed features

The evaluation module is used for sequentially evaluating the actions in the motion images based on the geometric characteristics and outputting corresponding evaluation results;

the evaluation module comprises an angle judgment unit, a turning judgment unit, a distance evaluation unit, a direction comparison unit, a judgment identifier detection unit and an evaluation unit;

the angle judging unit is used for detecting whether the gesture is standard or not based on the angle features, and updating an angle evaluation identifier when the gesture is judged to be not standard, wherein the angle evaluation identifier is used for indicating whether the gesture is standard or not;

the turnover judging unit is used for detecting whether the motion direction is turned over or not based on the speed characteristics;

the distance evaluation unit is used for detecting whether the action is qualified or not based on the distance characteristics when the movement direction is overturned, and updating a distance evaluation identifier when the action is judged to be unqualified, wherein the distance evaluation identifier is used for indicating whether the action is qualified or not;

the direction comparison unit is used for comparing the motion direction with a preset judgment direction;

a discrimination identifier detection unit for detecting whether a discrimination identifier exists when the movement direction is consistent with a preset discrimination direction;

and the evaluation unit is used for recording the judgment identification when the judgment identification does not exist, and is also used for outputting an evaluation result based on the obtained angle evaluation identification and distance evaluation identification and emptying the angle evaluation identification and the distance evaluation identification when the judgment identification exists.

Images