CN113505735B - Human body key point stabilization method based on hierarchical filtering - Google Patents
Human body key point stabilization method based on hierarchical filtering Download PDFInfo
- Publication number
- CN113505735B CN113505735B CN202110846716.3A CN202110846716A CN113505735B CN 113505735 B CN113505735 B CN 113505735B CN 202110846716 A CN202110846716 A CN 202110846716A CN 113505735 B CN113505735 B CN 113505735B
- Authority
- CN
- China
- Prior art keywords
- human body
- key point
- filtering
- level
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a human body key point stabilization method based on hierarchical filtering, which comprises the following steps: s1, determining the geometric gravity center of a human body; s2, constructing a filtering coefficient database based on the determined geometric center of gravity of the human body; s3, based on the constructed filtering coefficient database, the coordinates of key points of the human body are determined through hierarchical filtering, and then the stability of the key points is achieved. The invention effectively improves the jitter phenomenon when the key points of the human body are positioned, and ensures that the identified key points are still stable when the human body slightly shakes; the smooth key point identification can be still maintained under the condition that the human body movement is obvious; the method enables the human body key point recognition technology to be widely applied to the fields of medical health and the like with high requirements on stable recognition precision, for example, when the human body anatomical structure is mapped, the mapping is basically stable and motionless, and when a user performs rehabilitation exercise actions, action evaluation feedback can be provided in a stable flow.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a human body key point stabilizing method based on hierarchical filtering.
Background
The method is a significant research direction in the field of computer vision when detecting key points of human bodies, is mainly used for identifying and analyzing the actions and postures of human bodies, and has huge application scenes in the fields of security monitoring, sports fitness, rehabilitation training, electronic games, 3D fitting and the like. Many teams have performed a lot of research work on human body key point recognition, wherein the open source project openPose of Kanezukun university is representative, the openPose comprises positioning of faces, hands and joints of human bodies, estimation of motion gestures, facial expressions, arm movements and the like can be achieved, and the model can recognize gestures of multiple people and has good robustness.
However, when we use openPose et al human body gesture recognition model to locate key points of human body, body movement can cause unstable jitter condition of locating points of some human body flexible parts, and influence the locating effect and the presentation effect of other algorithm functions based on openPose et al model. For fields requiring high-precision human body action gesture recognition effects, such as rehabilitation training evaluation and ultrasonic probe guidance in the medical field, the jitters bring serious consequences, and once the recognition effects are poor, misjudgment of human body conditions, poor rehabilitation effect evaluation and ultrasonic guidance deviation are caused, so that the problem is solved with great practical significance.
Disclosure of Invention
Aiming at the defects in the prior art, the human body key point stabilizing method based on hierarchical filtering solves the problems that the existing human body key point positioning effect is poor and jitter is easy to occur.
In order to achieve the aim of the invention, the invention adopts the following technical scheme: a human body key point stabilization method based on hierarchical filtering comprises the following steps:
s1, determining the geometric gravity center of a human body;
s2, constructing a filtering coefficient database based on the determined geometric center of gravity of the human body;
s3, based on the constructed filtering coefficient database, the coordinates of key points of the human body are determined through hierarchical filtering, and then the stability of the key points is achieved.
Further, the step S1 specifically includes:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of a human body through OpenPose, and forming a link between two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the mass centers of all links according to the percentage of the mass centers of all links to the proximal end;
s14, calculating the position (X, Y) of the geometrical center of gravity of the human body in a rectangular coordinate system based on the position coordinates of the mass centers of all links;
wherein, the calculation formula is:
wherein P is i Is the gravity of the ith link, X i Is the barycenter abscissa of the ith link, Y i The gravity center ordinate of the ith link is P, the gravity of the human body is P, and X and Y are the abscissas and ordinates of the gravity center of the human body respectively.
Further, the step S2 specifically includes:
s21, classifying the identified key points of the human body and the geometric gravity center of the human body;
s22, setting corresponding filter coefficients for the key points of each level of the division, and constructing a filter coefficient database.
Further, in the step S21, the identified key points of the human body include 1 to 13 corresponding to the head, the right shoulder, the right elbow, the right hand head, the left shoulder, the left elbow, the left hand head, the right waist, the right knee, the right foot head, the left waist, the left knee and the left foot head;
dividing the geometric gravity center of the human body into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second level keypoints;
4, 7, 10 and 13 are divided into third level keypoints.
Further, the step S22 specifically includes:
a1, determining the distance between adjacent pixels in a human body picture;
a2, calculating the inter-frame change of the target value based on the determined pixel distance;
a3, setting standard filter coefficients corresponding to key points of each level and interframe space difference grading reference values;
and A4, determining a change rule of a filtering coefficient of the key point under the corresponding level based on the calculated inter-frame change of the target value and the inter-frame difference grading reference value, and constructing a filtering coefficient database by combining the standard filtering coefficient.
Further, the method for determining the adjacent pixel pitch in the step A1 specifically includes:
selecting two points with known and fixed intermediate distances of a human body picture, and determining the distance between adjacent pixels according to the pixel difference of the two points in the y direction;
the inter-frame change of the target value in the step A2 is the product of the pixel displacement and the adjacent pixel spacing;
in the step A3, the standard filter coefficient w of the zero-order key point 0 0.3, the interframe space difference grade reference value of which is 10 mm; standard filter coefficient w of first-stage key point 1 0.4, and the interframe space difference grade reference value is 15 mm;standard filter coefficient w of second level key point 2 0.5, and the interframe space difference reference value is 20 mm; standard filter coefficient w of third level key point 3 0.6, and the interframe space difference reference value is 25 mm;
in the step A4, for the zero-order key point, when the difference between the two frame coordinate distances exceeds 10 mm, the corresponding filter coefficient is increased by 0.04 every 5 mm; for the first-stage key points, determining the average value of the distances from the first-stage key points to the center of gravity, and when the difference between the average values between two frames exceeds 15 millimeters, increasing the corresponding filter coefficient by 0.06 every 5 millimeters; for the second-stage key points, determining the average value of the distances from the second-stage key points to the center of gravity, and when the difference between the average values of the distances between two frames exceeds 20 millimeters, increasing the corresponding filter coefficient by 0.08 every 5 millimeters; for the third level key point, determining the average value of the distance from the third level key point to the center of gravity, and when the difference between the average values of the distances between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
Further, the step S3 specifically includes:
in the continuous video frame processing process, vector distances between two human body key points are sequentially determined according to key point level division, step-by-step filtering is performed according to filter coefficients in a filter coefficient database, and all the filtered key point coordinates are output, so that key point stability is realized.
Further, the method for sequentially determining the vector distance between two human body key points, performing step-by-step filtering according to the filter coefficients in the filter coefficient database and outputting all the filtered key point coordinates comprises the following specific steps:
t1, determining the geometric gravity center O of a human body;
t2, calculating first-stage key point coordinates (p 1 ,p 2 ,p 5 ,p 8 ,p 11 ) Distance to center of gravity O (d 1 o1 ,d1 o2 ,d1 o5 ,d1 o8 ,d1 o11 ) And performing first-order filtering to obtain the distance (d 1 'from the updated first-order key point to the center of gravity' o1 ,d1' o2 ,d1' o5 ,d1' o8 ,d1' o11 )；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4, according to the centers of gravity O 'and (d 1' o1 ,d1' o2 ,d1' o5 ,d1' o8 ,d1' o11 ) Calculates an updated first level keypoint coordinate (p' 1 ,p' 2 ,p' 5 ,p' 8 ,p' 11 )；
T5, calculating the coordinates (p) 3 ,p 6 ,p 9 ,p 12 ) To the coordinates (p 'of the updated first level keypoints' 2 ,p' 5 ,p' 8 ,p' 11 ) Corresponding distance, and performing secondary filtering to obtain the distance (d2 'from the updated second-stage key point to the corresponding first key point' 23 ,d2' 56 ,d2' 89 ,d2' 1112 )；
T6, for updated first level keypoint coordinates (p' 1 ,p' 2 ,p' 5 ,p' 8 ,p' 11 ) Performing primary filtering to obtain stable first-stage key point coordinates (p' 1 ,p” 2 ,p” 5 ,p” 8 ,p” 11 )；
T7, according to (p' 2 ,p” 5 ,p” 8 ,p” 11 ) And (d2' 23 ,d2' 56 ,d2' 89 ,d2' 1112 ) Calculates the updated second level keypoint coordinates (p' 3 ,p' 6 ,p' 9 ,p' 12 )；
T8, calculating the third level key point coordinates (p 4 ,p 7 ,p 10 ,p 13 ) To the updated second level keypoint coordinates (p' 3 ,p' 6 ,p' 9 ,p' 12 ) Corresponding distance (d 3 34 ,d3 67 ,d3 910 ,d3 1213 ) And three-stage filtering to obtain the distance (d3 'from the updated third-stage key point to the corresponding second-stage key point' 34 ,d3' 67 ,d3' 910 ,d3' 1213 )；
T9, for the updated second level keypoints (p' 3 ,p' 6 ,p' 9 ,p' 12 ) Performing secondary filteringObtaining stable second-level key point coordinates (p' 3 ,p” 6 ,p” 9 ,p” 12 )；
T10, according to (p' 3 ,p” 6 ,p” 9 ,p” 12 ) And (d3' 34 ,d3' 67 ,d3' 910 ,d3' 1213 ) Calculates the updated third level keypoint coordinates (p' 4 ,p' 7 ,p' 10 ,p' 13 )；
T11, for the updated third level keypoint coordinates (p' 4 ,p' 7 ,p' 10 ,p' 13 ) Three-stage filtering to obtain stable third-stage key point coordinate (p' 4 ,p” 7 ,p” 10 ,p” 13 )；
T12, outputting all stable key point coordinates (p' 1 ,p” 2 ,p” 3 ,p” 4 ,p” 5 ,p” 6 ,p” 7 ,p” 8 ,p” 9 ,p” 10 ,p” 11 ,p” 12 ,p” 13 )。
Further, the formulas of the distance filtering in the steps T2, T5 and T8 and the key point filtering in the steps T3, T6, T9 and T11 are as follows:
N rn ＝r n-1 ×(1-w)+r n ×w
wherein r is n Is the actual coordinates of the key points of the nth frame, r n-1 Is the actual coordinates of the key points of the N-1 th frame, N rn And w is the filter coefficient of the key point of the current level in the filter coefficient database for the coordinates after the key point of the nth frame is filtered.
The beneficial effects of the invention are as follows:
(1) The invention effectively improves the jitter phenomenon when the key points of the human body are positioned, and ensures that the identified key points are still stable when the human body slightly shakes;
(2) The method can still keep smooth key point identification under the condition of obvious human body movement;
(3) The method of the invention enables the human body key point recognition technology to be widely applied in the fields of medical health and the like with higher requirements on stable recognition precision, for example, when the human body anatomy structure is mapped, the mapping is basically stable and motionless, and when the user performs rehabilitation exercise action, the action evaluation feedback can be provided in a stable flow.
Drawings
Fig. 1 is a flowchart of a human body key point stabilization method based on hierarchical filtering.
Fig. 2 is a schematic diagram of a 18-point model key point label based on openPose provided by the invention.
Fig. 3 is a schematic diagram of a method for calculating a distance between key points according to the present invention.
Fig. 4 is a flowchart of a method for solving key points by hierarchical filtering provided by the invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
As shown in fig. 1, a human body key point stabilization method based on hierarchical filtering includes the following steps:
s1, determining the geometric gravity center of a human body;
s2, constructing a filtering coefficient database based on the determined geometric center of gravity of the human body;
s3, based on the constructed filtering coefficient database, the coordinates of key points of the human body are determined through hierarchical filtering, and then the stability of the key points is achieved.
In step S1 of the present embodiment, if a plane parallel force system has a resultant force according to the valirung theorem, the moment of the resultant force on any point is equal to the algebraic sum of the moments of all points of the force system on the same point. Applying the theory to biology, the human body can be simplified into 13 rigid bodies by combining with OpenPose, each link of the divided human body is regarded as homogeneous, and the position (X, Y) of the geometric center of gravity of the human body in a rectangular coordinate system is calculated by the following formula;
wherein P is i Is the gravity of the ith link, X i Is the barycenter abscissa of the ith link, Y i The gravity center ordinate of the ith link is P, the gravity of the human body is P, and X and Y are the abscissas and ordinates of the gravity center of the human body respectively.
Therefore, the step S1 specifically includes:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of a human body through OpenPose, and forming a link between two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the mass centers of all links according to the percentage of the mass centers of all links to the proximal end;
s14, calculating the position (X, Y) of the geometrical center of gravity of the human body in a rectangular coordinate system based on the position coordinates of the mass centers of all links.
In step S2 of this embodiment, considering that the flexibility of the head, the trunk, the elbow, the ankle, and other parts is different in the human body movement process, it is necessary to rank the key points of the human body, and the key points of different ranks correspond to the filter coefficients of different ranks. Based on this, the step S2 specifically includes:
s21, classifying the identified key points of the human body and the geometric gravity center of the human body;
s22, setting corresponding filter coefficients for the key points of each level of the division, and constructing a filter coefficient database.
In the step S21, taking the 18-point model of openPose as an example, the key points obtained by human body gesture recognition are shown in fig. 2, and since the flexibility of the head key points is not high, the filtering processing is not performed in the present invention, the identified human body key points include 1-13 corresponding to the head, the right shoulder, the right elbow, the right hand head, the left shoulder, the left elbow, the left hand head, the right waist, the right knee, the right foot head, the left waist, the left knee and the left foot head respectively;
dividing the geometric gravity center of the human body into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second level keypoints;
4, 7, 10 and 13 are divided into third level keypoints.
Because the center of gravity point in the key of the human body can well embody the position information and is convenient to establish connection with the first-stage key point, the accuracy of positioning the key point of the human body can be increased by adding the filtering of the center of gravity coordinate in the filtering process.
We set the filter coefficients based on the flexibility of the key points. The gravity center has strong stability and the filter coefficient is minimum; the first-stage key point has stronger stability and smaller filter coefficient; the second-stage key points have strong flexibility and larger filter coefficients; the third level key point has the strongest flexibility and the largest filter coefficient. This can be achieved by: w (w) 0 <w 1 <w 2 <w 3 。
The step S22 specifically includes:
a1, determining the distance between adjacent pixels in a human body picture;
a2, calculating the inter-frame change of the target value based on the determined pixel distance; a3, setting standard filter coefficients corresponding to key points of each level and interframe space difference grading reference values;
and A4, determining a change rule of a filtering coefficient of the key point under the corresponding level based on the calculated inter-frame change of the target value and the inter-frame difference grading reference value, and constructing a filtering coefficient database by combining the standard filtering coefficient.
The method for determining the distance between adjacent pixels in the step A1 specifically includes:
selecting two points with known and fixed intermediate distances of a human body picture, and determining the distance between adjacent pixels according to the pixel difference of the two points in the y direction; for example, the distance from the tip of the nose to the midpoint of the ankle on both sides is measured, assuming that their vertical distance is 1600 mm, the actual size in units of mm, i.e., a single pixel value, is deduced from the difference in pixel values of these two points in the y-direction.
The target values in the step A2 refer to selected variable values, such as x, y coordinates of the key point and coordinate-based reference variables, such as distance d;
the inter-frame variation of the target value in the step A2 is the product of the pixel displacement and the adjacent pixel spacing;
specifically, the image is composed of a plurality of pixels, and the relative positions of the pixels with respect to the whole picture are approximately regarded as unchanged in a very short time. For example, a pixel representing the nose tip in a picture may be considered stationary for a very short period of time relative to the picture, but may be displaced for a long period of time. Pixel displacement is the difference in the number of pixels in the next frame relative to the previous frame at the point of the nose tip in the figure. For example, the image is 1024x768, the adjacent pixel pitch is 0.1mm, the pixel at the nose tip is (800,400) in this application, but by the next frame, if the pixel shift is 3, the pixel becomes (803,400), the inter-frame variation is 0.3mm.
In the above step A3, the standard filter coefficient w of the zero-order key point 0 0.3, the interframe space difference grade reference value of which is 10 mm; standard filter coefficient w of first-stage key point 1 0.4, and the interframe space difference grade reference value is 15 mm; standard filter coefficient w of second level key point 2 0.5, and the interframe space difference reference value is 20 mm; standard filter coefficient w of third level key point 3 0.6, and the interframe space difference reference value is 25 mm;
in the step A4, for the zero-order key point, when the difference between the two frame coordinate distances exceeds 10 mm, the corresponding filter coefficient is increased by 0.04 every 5 mm; for the first-stage key points, determining the average value of the distances from the first-stage key points to the center of gravity, and when the difference between the average values between two frames exceeds 15 millimeters, increasing the corresponding filter coefficient by 0.06 every 5 millimeters; for the second-stage key points, determining the average value of the distances from the second-stage key points to the center of gravity, and when the difference between the average values of the distances between two frames exceeds 20 millimeters, increasing the corresponding filter coefficient by 0.08 every 5 millimeters; for the third level key point, determining the average value of the distance from the third level key point to the center of gravity, and when the difference between the average values of the distances between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
The step S3 of this embodiment specifically includes:
in the continuous video frame processing process, vector distances between two human body key points are sequentially determined according to key point level division, step-by-step filtering is performed according to filter coefficients in a filter coefficient database, and all the filtered key point coordinates are output, so that key point stability is realized.
Specifically, in the hierarchical filtering process, coordinates of key points are represented by vectors in the embodiment, so that distances are calculated more conveniently and filtering is performed; therefore, as shown in fig. 4, the method for sequentially determining the vector distance between two human body key points, performing step-by-step filtering according to the filter coefficients in the filter coefficient database, and outputting the coordinates of all the filtered key points specifically includes:
t1, determining the geometric gravity center O of a human body;
t2, calculating first-stage key point coordinates (p 1 ,p 2 ,p 5 ,p 8 ,p 11 ) Distance to center of gravity O (d 1 o1 ,d1 o2 ,d1 o5 ,d1 o8 ,d1 o11 ) And performing first-order filtering to obtain the distance (d 1 'from the updated first-order key point to the center of gravity' o1 ,d1' o2 ,d1' o5 ,d1' o8 ,d1' o11 )；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4, according to the centers of gravity O 'and (d 1' o1 ,d1' o2 ,d1' o5 ,d1' o8 ,d1' o11 ) Calculates an updated first level keypoint coordinate (p' 1 ,p' 2 ,p' 5 ,p' 8 ,p' 11 )；
T5, calculating the coordinates (p) 3 ,p 6 ,p 9 ,p 12 ) To a greater extentCoordinates of the new first level keypoints (p' 2 ,p' 5 ,p' 8 ,p' 11 ) Corresponding distance, and performing secondary filtering to obtain the distance (d2 'from the updated second-stage key point to the corresponding first key point' 23 ,d2' 56 ,d2' 89 ,d2' 1112 )；
T6, for updated first level keypoint coordinates (p' 1 ,p' 2 ,p' 5 ,p' 8 ,p' 11 ) Performing primary filtering to obtain stable first-stage key point coordinates (p' 1 ,p” 2 ,p” 5 ,p” 8 ,p” 11 )；
T7, according to (p' 2 ,p” 5 ,p” 8 ,p” 11 ) And (d2' 23 ,d2' 56 ,d2' 89 ,d2' 1112 ) Calculates the updated second level keypoint coordinates (p' 3 ,p' 6 ,p' 9 ,p' 12 )；
T8, calculating the third level key point coordinates (p 4 ,p 7 ,p 10 ,p 13 ) To the updated second level keypoint coordinates (p' 3 ,p' 6 ,p' 9 ,p' 12 ) Corresponding distance (d 3 34 ,d3 67 ,d3 910 ,d3 1213 ) And three-stage filtering to obtain the distance (d3 'from the updated third-stage key point to the corresponding second-stage key point' 34 ,d3' 67 ,d3' 910 ,d3' 1213 )；
T9, for the updated second level keypoints (p' 3 ,p' 6 ,p' 9 ,p' 12 ) Performing secondary filtering to obtain stable second-stage key point coordinates (p' 3 ,p” 6 ,p” 9 ,p” 12 )；
T10, according to (p' 3 ,p” 6 ,p” 9 ,p” 12 ) And (d3' 34 ,d3' 67 ,d3' 910 ,d3' 1213 ) Calculates the updated third level keypoint coordinates (p' 4 ,p' 7 ,p' 10 ,p' 13 )；
T11, for the updated third level keypoint coordinates (p' 4 ,p' 7 ,p' 10 ,p' 13 ) Three-stage filtering to obtain stable third-stage key point coordinate (p' 4 ,p” 7 ,p” 10 ,p” 13 )；
T12, outputting all stable key point coordinates (p' 1 ,p” 2 ,p” 3 ,p” 4 ,p” 5 ,p” 6 ,p” 7 ,p” 8 ,p” 9 ,p” 10 ,p” 11 ,p” 12 ,p” 13 )。
In the above process, taking the key point 5 and the key point 6 in fig. 2 as examples, the coordinates of the two points are p 5 ＝x 5 +iy 5 、p 6 ＝x 6 +iy 6 According to the property theorem of the vector, the vector of the distance between two points can be obtained to be expressed as d2 56 ＝p 5 -p 6 As shown in fig. 3, the dotted line between the key points 5 to 6 is a two-point distance vector;
the formulas of the distance filtering in the steps T2, T5 and T8 and the key point filtering in the steps T3, T6, T9 and T11 are as follows:
N rn ＝r n-1 ×(1-w)+r n ×w
wherein r is n Is the actual coordinates of the key points of the nth frame, r n-1 Is the actual coordinates of the key points of the N-1 th frame, N rn And w is the filter coefficient of the key point of the current level in the filter coefficient database for the coordinates after the key point of the nth frame is filtered.
In this embodiment, the priority of the center of gravity is highest, and then the first level, the second level and the third level of key points are sequentially arranged, coordinate filtering of the key points of the first level is first performed during processing, then filtering of the distance between the key points of the first level and the key points of the second level is performed, and the coordinates after the filtering of the key points of the second level can be obtained by subtracting the distance vector from the coordinate vector after the filtering.
Claims (5)
1. The human body key point stabilizing method based on hierarchical filtering is characterized by comprising the following steps of:
s1, determining the geometric gravity center of a human body;
s2, constructing a filtering coefficient database based on the determined geometric center of gravity of the human body;
s3, determining coordinates of key points of a human body based on the constructed filtering coefficient database through hierarchical filtering, and further achieving key point stabilization;
the step S1 specifically comprises the following steps:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of a human body through OpenPose, and forming a link between two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the mass centers of all links according to the percentage of the mass centers of all links to the proximal end;
s14, calculating the position (X, Y) of the geometrical center of gravity of the human body in a rectangular coordinate system based on the position coordinates of the mass centers of all links;
wherein, the calculation formula is:
wherein P is i Is the gravity of the ith link, X i Is the barycenter abscissa of the ith link, Y i The gravity center ordinate of the ith link is P, the gravity of a human body is P, and X and Y are the abscissas and ordinates of the gravity center of the human body respectively;
the step S2 specifically comprises the following steps:
s21, classifying the identified key points of the human body and the geometric gravity center of the human body;
s22, setting corresponding filter coefficients for the key points of each divided level, and constructing a filter coefficient database;
the step S22 specifically includes:
a1, determining the distance between adjacent pixels in a human body picture;
a2, calculating the inter-frame change of the target value based on the determined pixel distance;
a3, setting standard filter coefficients corresponding to key points of each level and interframe space difference grading reference values;
a4, determining a change rule of a filtering coefficient of the key point under the corresponding level based on the calculated inter-frame change of the target value and the inter-frame difference grading reference value, and constructing a filtering coefficient database by combining the standard filtering coefficient;
the step S3 specifically comprises the following steps:
in the continuous video frame processing process, vector distances between two human body key points are sequentially determined according to key point level division, step-by-step filtering is performed according to filter coefficients in a filter coefficient database, and all the filtered key point coordinates are output, so that key point stability is realized.
2. The method for stabilizing human body key points based on hierarchical filtering according to claim 1, wherein in the step S21, the identified human body key points include 1 to 13 corresponding to the head, the right shoulder, the right elbow, the right hand head, the left shoulder, the left elbow, the left hand head, the right waist, the right knee, the right foot head, the left waist, the left knee and the left foot head, respectively;
dividing the geometric gravity center of the human body into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second level keypoints;
4, 7, 10 and 13 are divided into third level keypoints.
3. The human body key point stabilizing method based on hierarchical filtering according to claim 1, wherein the method for determining the adjacent pixel distance in the step A1 specifically comprises the following steps:
selecting two points with known and fixed intermediate distances of a human body picture, and determining the distance between adjacent pixels according to the pixel difference of the two points in the y direction;
the inter-frame change of the target value in the step A2 is the product of the pixel displacement and the adjacent pixel spacing;
in the step A3, the standard filter coefficient w of the zero-order key point 0 0.3, the interframe space difference grade reference value of which is 10 mm; standard filter coefficient w of first-stage key point 1 0.4, and the interframe space difference grade reference value is 15 mm; standard filter coefficient w of second level key point 2 0.5, and the interframe space difference reference value is 20 mm; standard filter coefficient w of third level key point 3 0.6, and the interframe space difference reference value is 25 mm;
in the step A4, for the zero-order key point, when the difference between the two frame coordinate distances exceeds 10 mm, the corresponding filter coefficient is increased by 0.04 every 5 mm; for the first-stage key points, determining the average value of the distances from the first-stage key points to the center of gravity, and when the difference between the average values between two frames exceeds 15 millimeters, increasing the corresponding filter coefficient by 0.06 every 5 millimeters; for the second-stage key points, determining the average value of the distances from the second-stage key points to the center of gravity, and when the difference between the average values of the distances between two frames exceeds 20 millimeters, increasing the corresponding filter coefficient by 0.08 every 5 millimeters; for the third level key point, determining the average value of the distance from the third level key point to the center of gravity, and when the difference between the average values of the distances between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
4. The human body key point stabilizing method based on hierarchical filtering according to claim 1, wherein the method for sequentially determining the vector distance between two human body key points, performing step-by-step filtering according to the filter coefficients in the filter coefficient database, and outputting all the filtered key point coordinates comprises the following specific steps:
t1, determining the geometric gravity center O of a human body;
t2, calculating first-stage key point coordinates (p 1 ,p 2 ,p 5 ,p 8 ,p 11 ) Distance to center of gravity O (d 1 o1 ,d1 o2 ,d1 o5 ,d1 o8 ,d1 o11 ) And performing first-order filtering to obtain the distance (d 1 'from the first-order key point to the center of gravity' o1 ,d1' o2 ,d1' o5 ,d1' o8 ,d1' o11 )；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4, according to the centers of gravity O 'and (d 1' o1 ,d1' o2 ,d1' o5 ,d1' o8 ,d1' o11 ) Calculates an updated first level keypoint coordinate (p' 1 ,p' 2 ,p' 5 ,p' 8 ,p' 11 )；
T5, calculating the coordinates (p) 3 ,p 6 ,p 9 ,p 12 ) To the coordinates (p 'of the updated first level keypoints' 1 ,p' 2 ,p' 5 ,p' 8 ,p' 11 ) Key point (p' 2 ,p' 5 ,p' 8 ,p' 11 ) Corresponding distance, and performing secondary filtering to obtain distance (d2 'from the second level key point to the corresponding first key point' 23 ,d2' 56 ,d2' 89 ,d2' 1112 )；
T6, for updated first level keypoint coordinates (p' 1 ,p' 2 ,p' 5 ,p' 8 ,p' 11 ) Performing primary filtering to obtain stable first-stage key point coordinates (p' 1 ,p” 2 ,p” 5 ,p” 8 ,p” 11 )；
T7, according to (p' 2 ,p” 5 ,p” 8 ,p” 11 ) And (d2' 23 ,d2' 56 ,d2' 89 ,d2' 1112 ) Calculates the updated second level keypoint coordinates (p' 3 ,p' 6 ,p' 9 ,p' 12 )；
T8, calculating the third level key point coordinates (p 4 ,p 7 ,p 10 ,p 13 ) To the updated second level keypoint coordinates (p' 3 ,p' 6 ,p' 9 ,p' 12 ) Corresponding distance (d 3 34 ,d3 67 ,d3 910 ,d3 1213 ) And three-stage filtering to obtain the distance (d3 'from the third-stage key point to the corresponding second-stage key point' 34 ,d3' 67 ,d3' 910 ,d3' 1213 )；
T9, for the updated second level keypoints (p' 3 ,p' 6 ,p' 9 ,p' 12 ) Performing secondary filtering to obtain stable second-stage key point coordinates (p' 3 ,p” 6 ,p” 9 ,p” 12 )；
T10, according to (p' 3 ,p” 6 ,p” 9 ,p” 12 ) And (d3' 34 ,d3' 67 ,d3' 910 ,d3' 1213 ) Calculates the updated third level keypoint coordinates (p' 4 ,p' 7 ,p' 10 ,p' 13 )；
T11, for the updated third level keypoint coordinates (p' 4 ,p' 7 ,p' 10 ,p' 13 ) Three-stage filtering to obtain stable third-stage key point coordinate (p' 4 ,p” 7 ,p” 10 ,p” 13 )；
T12, outputting all stable key point coordinates (p' 1 ,p” 2 ,p” 3 ,p” 4 ,p” 5 ,p” 6 ,p” 7 ,p” 8 ,p” 9 ,p” 10 ,p” 11 ,p” 12 ,p” 13 )。
5. The hierarchical filtering based human keypoint stabilization method according to claim 4, wherein the formulas of the distance filtering in the steps T2, T5 and T8 and the keypoint filtering in the steps T3, T6, T9 and T11 are:
N rn ＝r n-1 ×(1-w)+r n ×w
wherein r is n Is the actual coordinates of the key points of the nth frame, r n-1 Is the actual coordinates of the key points of the N-1 th frame, N rn And w is the filter coefficient of the key point of the current level in the filter coefficient database for the coordinates after the key point of the nth frame is filtered.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2021105764153 | 2021-05-26 | ||
| CN202110576415 | 2021-05-26 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113505735A CN113505735A (en) | 2021-10-15 |
| CN113505735B true CN113505735B (en) | 2023-05-02 |
Family
ID=78014094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110846716.3A Active CN113505735B (en) | 2021-05-26 | 2021-07-26 | Human body key point stabilization method based on hierarchical filtering |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113505735B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114267086A (en) * | 2021-12-30 | 2022-04-01 | 西南石油大学 | Execution quality evaluation method for complex continuous motion in motion |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
| CN110210323A (en) * | 2019-05-09 | 2019-09-06 | 浙江大学 | A kind of drowned behavior online recognition method based on machine vision |
| CN110287891A (en) * | 2019-06-26 | 2019-09-27 | 北京字节跳动网络技术有限公司 | Gestural control method, device and electronic equipment based on human body key point |
| CN110287923A (en) * | 2019-06-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | Human body attitude acquisition methods, device, computer equipment and storage medium |
| CN111242982A (en) * | 2020-01-02 | 2020-06-05 | 浙江工业大学 | Human body target tracking method based on progressive Kalman filtering |
| CN112163564A (en) * | 2020-10-26 | 2021-01-01 | 燕山大学 | Tumble prejudging method based on human body key point behavior identification and LSTM (least Square TM) |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI662514B (en) * | 2018-09-13 | 2019-06-11 | 緯創資通股份有限公司 | Falling detection method and electronic system using the same |
-
2021
- 2021-07-26 CN CN202110846716.3A patent/CN113505735B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
| CN110210323A (en) * | 2019-05-09 | 2019-09-06 | 浙江大学 | A kind of drowned behavior online recognition method based on machine vision |
| CN110287891A (en) * | 2019-06-26 | 2019-09-27 | 北京字节跳动网络技术有限公司 | Gestural control method, device and electronic equipment based on human body key point |
| CN110287923A (en) * | 2019-06-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | Human body attitude acquisition methods, device, computer equipment and storage medium |
| CN111242982A (en) * | 2020-01-02 | 2020-06-05 | 浙江工业大学 | Human body target tracking method based on progressive Kalman filtering |
| CN112163564A (en) * | 2020-10-26 | 2021-01-01 | 燕山大学 | Tumble prejudging method based on human body key point behavior identification and LSTM (least Square TM) |
Non-Patent Citations (4)
| Title |
|---|
| Alexandros Makris et al..Robust 3D Human Pose Estimation Guided by Filtered Subsets of Body Keypoints.《2019 16th International Conference on Machine Vision Applications (MVA)》.2019,1-6. * |
| Weiming Chen et al..Fall Detection Based on Key Points of Human-Skeleton Using OpenPose.《Symmetry》.2020,第12卷(第5期),1-17. * |
| 张欣等.基于相关滤波和关键点的目标跟踪算法.《电视技术》.2017,第41卷(第Z2期),147-151+163. * |
| 郑恬.基于视频的人体检测、跟踪和三维重建.《中国优秀硕士学位论文全文数据库 (信息科技辑)》.2020,(第1期),I138-2147. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113505735A (en) | 2021-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Qiao et al. | Real-time human gesture grading based on OpenPose | |
| Patrona et al. | Motion analysis: Action detection, recognition and evaluation based on motion capture data | |
| CN106650687B (en) | Posture correction method based on depth information and skeleton information | |
| Anilkumar et al. | Pose estimated yoga monitoring system | |
| CN113255522B (en) | Personalized motion attitude estimation and analysis method and system based on time consistency | |
| Fieraru et al. | Learning complex 3D human self-contact | |
| CN113505735B (en) | Human body key point stabilization method based on hierarchical filtering | |
| Kanase et al. | Pose estimation and correcting exercise posture | |
| Yang et al. | Human exercise posture analysis based on pose estimation | |
| Agarwal et al. | FitMe: a fitness application for accurate pose estimation using deep learning | |
| Pardos et al. | Automated Posture Analysis for the Assessment of Sports Exercises | |
| Li et al. | A new fingertip detection and tracking algorithm and its application on writing-in-the-air system | |
| Bhamidipati et al. | Robust intelligent posture estimation for an ai gym trainer using mediapipe and opencv | |
| Zhou et al. | Skeleton-based human keypoints detection and action similarity assessment for fitness assistance | |
| Shen et al. | Design of OpenPose-based of exercise assistant system with instructor-user synchronization for self-practice dynamic yoga | |
| Solongontuya et al. | Novel side pose classification model of stretching gestures using three-layer LSTM | |
| CN112602100B (en) | Action analysis device, action analysis method, storage medium, and action analysis system | |
| CN116189301A (en) | Standing long jump motion standardability assessment method based on attitude estimation | |
| Wang et al. | A six-degree-of-freedom virtual mouse based on hand gestures | |
| Zheng et al. | A Skeleton-based Rehabilitation Exercise Assessment System with Rotation Invariance | |
| Otberdout et al. | Hand pose estimation based on deep learning depth map for hand gesture recognition | |
| CN115205737A (en) | Real-time motion counting method and system based on Transformer model | |
| CN115223240A (en) | Motion real-time counting method and system based on dynamic time warping algorithm | |
| CN115346640A (en) | Intelligent monitoring method and system for closed-loop feedback of functional rehabilitation training | |
| CN115205750A (en) | Motion real-time counting method and system based on deep learning model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |