CN113505735A - Human body key point stabilizing method based on hierarchical filtering - Google Patents
Human body key point stabilizing method based on hierarchical filtering Download PDFInfo
- Publication number
- CN113505735A CN113505735A CN202110846716.3A CN202110846716A CN113505735A CN 113505735 A CN113505735 A CN 113505735A CN 202110846716 A CN202110846716 A CN 202110846716A CN 113505735 A CN113505735 A CN 113505735A
- Authority
- CN
- China
- Prior art keywords
- human body
- key points
- level
- key point
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000000087 stabilizing effect Effects 0.000 title claims abstract description 17
- 230000005484 gravity Effects 0.000 claims abstract description 61
- 230000008569 process Effects 0.000 claims abstract description 10
- 239000013598 vector Substances 0.000 claims description 12
- 230000008859 change Effects 0.000 claims description 6
- 210000003127 knee Anatomy 0.000 claims description 6
- 238000006073 displacement reaction Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 2
- 210000003484 anatomy Anatomy 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000036541 health Effects 0.000 abstract description 2
- 238000013507 mapping Methods 0.000 abstract 2
- 230000000694 effects Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 4
- 210000002683 foot Anatomy 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 3
- 210000003423 ankle Anatomy 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了一种基于分级滤波的人体关键点稳定方法，包括：S1、确定人体几何重心；S2、基于确定的人体几何重心，构建滤波系数数据库；S3、基于构建的滤波系数数据库，分级滤波确定人体关键点坐标，进而实现关键点稳定。本发明有效地改善人体关键点定位时的抖动现象，保证在人体轻微抖动时，识别的关键点仍然保持稳定；能够在人体运动较为明显的情况下，仍然保持流畅的关键点识别；本方法使得人体关键点识别技术能够在对稳定识别精度要求较高的医疗健康等领域得到更广泛的应用，例如在人体解剖结构贴图时，贴图基本稳定不动，在用户做康复运动动作时，能够流程稳定的提供动作评估反馈。
The invention discloses a method for stabilizing key points of human body based on hierarchical filtering, comprising: S1, determining the geometric center of gravity of the human body; S2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body; S3, based on the constructed filter coefficient database, hierarchical filtering Determine the coordinates of the key points of the human body, and then achieve the stability of the key points. The present invention effectively improves the shaking phenomenon during the positioning of key points of the human body, and ensures that when the human body shakes slightly, the identified key points are still stable; in the case of obvious human body movements, the key point identification can still be maintained smoothly; the method enables Human key point recognition technology can be more widely used in medical and health fields that require high stable recognition accuracy. For example, when mapping the human anatomy, the mapping is basically stable, and when the user performs rehabilitation exercises, the process can be stable. to provide feedback on movement assessments.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a human body key point stabilizing method based on hierarchical filtering.
Background
The human body key point detection is a significant research direction in the field of computer vision, is mainly used for human body action posture recognition and analysis, and has huge application scenes in the fields of security monitoring, sports fitness, rehabilitation training, electronic games, 3D fitting and the like. A lot of research works are carried out on human body key point recognition by a plurality of teams, wherein the typical example is openpos which is an open source project of the university of Chimeilong in the card, openpos contains positioning of a face, a hand and each joint of a human body, estimation of action posture, facial expression, arm movement and the like can be realized, the model can carry out posture recognition on a plurality of people, and the model has good robustness.
However, when the human body posture recognition model such as openpos is used for positioning key points of the human body, the body movement can cause unstable jitter of positioning points of some flexible parts of the human body, and the positioning effect and the presentation effect of other algorithm functions established on the basis of the openpos model and the like are influenced. For the fields requiring high-precision human action and posture recognition effects, such as rehabilitation training evaluation and ultrasonic probe guidance in the medical field, the jitters bring serious consequences, and once the recognition effect is poor, the human condition misjudgment, poor rehabilitation effect evaluation and ultrasonic guidance deviation are caused, so that the method has great practical significance for solving the problem.
Disclosure of Invention
Aiming at the defects in the prior art, the method for stabilizing the human body key points based on the hierarchical filtering solves the problems that the existing human body key points are poor in positioning effect and prone to shaking.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that: a human body key point stabilizing method based on hierarchical filtering comprises the following steps:
s1, determining the geometric center of gravity of the human body;
s2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body;
and S3, based on the constructed filter coefficient database, determining the coordinates of the key points of the human body by hierarchical filtering, and further realizing the stability of the key points.
Further, the step S1 is specifically:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of the human body through OpenPose, and forming a link between every two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the centroid of each link according to the percentage of the centroid of each link to the proximal end;
s14, calculating the position (X, Y) of the geometric center of gravity of the human body in a rectangular coordinate system based on the position coordinates of the centroid of each link;
wherein, the calculation formula is:
in the formula, PiIs the gravity of the ith link, XiIs the barycentric abscissa, Y, of the ith linkiIs the longitudinal coordinate of the gravity center of the ith link, P is the gravity of the human body, and X and Y are the transverse and longitudinal coordinates of the gravity center of the human body respectively.
Further, the step S2 is specifically:
s21, carrying out level division on the identified human body key points and the human body geometric gravity center;
and S22, setting corresponding filter coefficients for the key points of each divided level, and constructing a filter coefficient database.
Further, in the step S21, the identified key points of the human body include 1 to 13 corresponding to the head, the right shoulder, the right elbow, the right hand head, the left shoulder, the left elbow, the left hand head, the right waist, the right knee, the right foot head, the left waist, the left knee and the left foot head, respectively;
wherein, the geometric center of gravity of the human body is divided into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second-level key points;
4, 7, 10 and 13 are divided into third level keypoints.
Further, the step S22 is specifically:
a1, determining the distance between adjacent pixels in the human body picture;
a2, calculating the frame-to-frame variation of the target value based on the determined pixel spacing;
a3, setting standard filter coefficients and inter-frame distance difference grading reference values corresponding to the key points of each level;
a4, determining the change rule of the filter coefficient of the key point under the corresponding level based on the calculated inter-frame change and inter-frame distance difference grade reference value of the target value, and constructing a filter coefficient database by combining the standard filter coefficient.
Further, the method for determining the distance between adjacent pixels in the step a1 specifically includes:
selecting two points with known and fixed middle distances of the human body pictures, and determining the distance between adjacent pixels according to the pixel difference of the points in the y direction;
the interframe variation of the target value in the step A2 is the product of the pixel displacement and the adjacent pixel spacing;
in the step A3, the standard filter coefficient w of the zero-level key point00.3, and the inter-frame distance difference level reference value is 10 mm; standard filter coefficient w of first-stage key point10.4, and the inter-frame distance difference level reference value is 15 mm; standard filter coefficient w of second-stage key point20.5, and the reference value of the difference of the distance between frames is 20 mm; standard filter coefficient w of third-stage key point30.6, and the reference value of the difference of the distances between frames is 25 mm;
in the step a4, for the zero-level keypoint, when the difference between the two frame coordinate distances exceeds 10 mm, for each 5 mm increase, the corresponding filter coefficient increases by 0.04; for the first-level key points, determining the average value of the distances from the first-level key points to the gravity center, and when the difference between the average values of the distances between two frames exceeds 15 mm, increasing the corresponding filter coefficient by 0.06 every 5 mm; for the second-level key points, determining the average distance values from the second-level key points to the gravity center, and when the difference between the average distance values between two frames exceeds 20 mm, increasing the corresponding filter coefficient by 0.08 every 5 mm; for the third-level key points, determining the distance average value from the third-level key points to the gravity center, and when the difference between the distance average values between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
Further, the step S3 is specifically:
in the continuous video frame processing process, the vector distance between two human body key points is determined in sequence according to the key point grade division, filtering is carried out step by step according to the filtering coefficients in the filtering coefficient database, and coordinates of all the filtered key points are output, so that the key point stability is realized.
Further, the method for sequentially determining the vector distance between two human body key points, performing step-by-step filtering according to the filter coefficients in the filter coefficient database, and outputting the coordinates of all the filtered key points specifically comprises the following steps:
t1, determining the geometric center of gravity O of the human body;
t2, calculating the coordinates (p) of the first-level key points respectively1,p2,p5,p8,p11) Distance to center of gravity O (d 1)o1,d1o2,d1o5,d1o8,d1o11) And performing primary filtering to obtain the updated distance (d1 ') from the first-stage key point to the gravity center'o1,d1'o2,d1'o5,d1'o8,d1'o11)；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4 according to centers of gravity O 'and (d1'o1,d1'o2,d1'o5,d1'o8,d1'o11) Calculating updated primary keypoint coordinates (p'1,p'2,p'5,p'8,p'11)；
T5, respectively calculating the coordinates (p) of the second-stage key points3,p6,p9,p12) To the updated coordinates (p ') of the first level keypoints'2,p'5,p'8,p'11) Corresponding distance is obtained, secondary filtering is carried out, and the distance (d2 ') from the updated second-stage key point to the corresponding first key point is obtained'23,d2'56,d2'89,d2'1112)；
T6, pair of updated Primary Key Point coordinates (p'1,p'2,p'5,p'8,p'11) Performing primary filtering to obtain stable first-stage key point coordinates (p) "1,p”2,p”5,p”8,p”11)；
T7, according to (p "2,p”5,p”8,p”11) And (d2'23,d2'56,d2'89,d2'1112) Calculating updated second level keypoint coordinates (p'3,p'6,p'9,p'12)；
T8, calculating the coordinate (p) of the key point of the third level respectively4,p7,p10,p13) To updated second level keypoint coordinates (p'3,p'6,p'9,p'12) Corresponding distance (d 3)34,d367,d3910,d31213) And go through three stagesFiltering to obtain the distance (d3 ') from the updated third-level key point to the corresponding second-level key point'34,d3'67,d3'910,d3'1213)；
T9, pair of updated second level Key points (p'3,p'6,p'9,p'12) Secondary filtering is carried out to obtain stable secondary key point coordinates (p) "3,p”6,p”9,p”12)；
T10, according to (p "3,p”6,p”9,p”12) And (d3'34,d3'67,d3'910,d3'1213) Calculating updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13)；
T11, pair of updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13) Carrying out three-level filtering to obtain stable third-level key point coordinates (p) "4,p”7,p”10,p”13)；
T12, outputting all stable key point coordinates (p "1,p”2,p”3,p”4,p”5,p”6,p”7,p”8,p”9,p”10,p”11,p”12,p”13)。
Further, the formulas of the distance filtering in the steps T2, T5 and T8 and the keypoint filtering in the steps T3, T6, T9 and T11 are:
Nrn＝rn-1×(1-w)+rn×w
in the formula, rnIs the actual coordinate of the key point of the nth frame, rn-1Is the actual coordinate of the key point of the (N-1) th frame, NrnAnd W is the filtered coordinate of the key point of the nth frame, and W is the filter coefficient of the key point of the current level in the filter coefficient database.
The invention has the beneficial effects that:
(1) the method effectively improves the shaking phenomenon when the key points of the human body are positioned, and ensures that the identified key points still keep stable when the human body shakes slightly;
(2) the method can still keep smooth key point identification under the condition of relatively obvious human body movement;
(3) the method of the invention enables the human body key point identification technology to be widely applied in the fields of medical health and the like with higher requirements on stable identification precision, for example, when a human body anatomical structure is pasted, the pasting is basically stable and motionless, and when a user does rehabilitation exercise, the action evaluation feedback can be provided stably in the process.
Drawings
Fig. 1 is a flowchart of a human body key point stabilizing method based on hierarchical filtering provided by the present invention.
FIG. 2 is a schematic diagram of the 18-point model key point labels based on OpenPose according to the present invention.
Fig. 3 is a schematic diagram of a method for calculating a distance between key points according to the present invention.
Fig. 4 is a flowchart of a method for solving key points by hierarchical filtering according to the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
As shown in fig. 1, a method for stabilizing human body key points based on hierarchical filtering includes the following steps:
s1, determining the geometric center of gravity of the human body;
s2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body;
and S3, based on the constructed filter coefficient database, determining the coordinates of the key points of the human body by hierarchical filtering, and further realizing the stability of the key points.
In step S1 of the present embodiment, according to the law of varez, if there is a resultant force in a plane-parallel force system, the moment of the resultant force at any point is equal to the algebraic sum of the moments of all points of the force system at the same point. The theory is applied to biology, a human body can be simplified into 13 rigid bodies by combining OpenPose, each link of the divided human body is considered to be homogeneous, and the position (X, Y) of the geometric center of gravity of the human body in a rectangular coordinate system is calculated by the following formula;
in the formula, PiIs the gravity of the ith link, XiIs the barycentric abscissa, Y, of the ith linkiIs the longitudinal coordinate of the gravity center of the ith link, P is the gravity of the human body, and X and Y are the transverse and longitudinal coordinates of the gravity center of the human body respectively.
Therefore, the step S1 is specifically:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of the human body through OpenPose, and forming a link between every two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the centroid of each link according to the percentage of the centroid of each link to the proximal end;
and S14, calculating the position (X, Y) of the geometric center of gravity of the human body in the rectangular coordinate system based on the position coordinates of the centroid of each link.
In step S2 of this embodiment, in consideration of different flexibility of the head, the trunk, the elbows, the ankles, and the like during the movement of the human body, the key points of the human body need to be classified into different levels, and the key points of different levels correspond to the filter coefficients of different levels. Based on this, the step S2 is specifically:
s21, carrying out level division on the identified human body key points and the human body geometric gravity center;
and S22, setting corresponding filter coefficients for the key points of each divided level, and constructing a filter coefficient database.
In the step S21, taking an 18-point openpos model as an example, the key points obtained by human posture recognition are as shown in fig. 2, since the flexibility of the key points of the head is not high, the filtering process is not performed in the present invention, and the recognized key points of the human body include 1 to 13, which respectively correspond to the head, the right shoulder, the right elbow, the right hand head, the left shoulder, the left elbow, the left hand head, the right waist, the right knee, the right foot head, the left waist, the left knee and the left foot head;
wherein, the geometric center of gravity of the human body is divided into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second-level key points;
4, 7, 10 and 13 are divided into third level keypoints.
Because the key center of gravity of the human body can well reflect the position information and is convenient to establish the relation with the first-stage key points, the filtering of the center of gravity coordinate is added in the filtering process, so that the positioning accuracy of the key points of the human body can be improved.
We set the filter coefficients according to the flexibility of the key points. The stability of the center of gravity is strong, and the filter coefficient is minimum; the first-stage key points have high stability and small filter coefficients; the second-stage key points have high flexibility and high filter coefficients; the third-stage key point has the strongest flexibility and the largest filtering coefficient. This gives: w is a0<w1<w2<w3。
The step S22 is specifically:
a1, determining the distance between adjacent pixels in the human body picture;
a2, calculating the frame-to-frame variation of the target value based on the determined pixel spacing; a3, setting standard filter coefficients and inter-frame distance difference grading reference values corresponding to the key points of each level;
a4, determining the change rule of the filter coefficient of the key point under the corresponding level based on the calculated inter-frame change and inter-frame distance difference grade reference value of the target value, and constructing a filter coefficient database by combining the standard filter coefficient.
The method for determining the distance between adjacent pixels in the step a1 specifically includes:
selecting two points with known and fixed middle distances of the human body pictures, and determining the distance between adjacent pixels according to the pixel difference of the points in the y direction; for example, the distance from the tip of the nose to the midpoint of the two lateral ankles is measured, and assuming that their vertical distance is 1600 mm, the pixel pitch in mm, i.e., the actual size of a single pixel value, is deduced from the difference in pixel values of these two points in the y-direction.
The target value in step a2 refers to the selected variable value, such as the x and y coordinates of the key point and the coordinate-based reference variable, such as the distance d;
the inter-frame variation of the target value in the step a2 is the product of the pixel displacement and the adjacent pixel pitch;
specifically, the image is composed of a plurality of pixel points, and the relative positions of the pixel points relative to the whole picture are approximately regarded as unchanged in a very short time. For example, a pixel point in a picture representing the tip of a nose may be considered stationary for a very short time, but may be displaced for a long time. The pixel displacement is the difference of the number of pixel points of the point of the nose tip in the image in the next frame relative to the previous frame. For example, the image is 1024x768, the adjacent pixel pitch is 0.1mm, the pixel at the tip of the nose in this application is (800,400), but by the next frame, if the pixel displacement is 3, the pixel becomes (803,400), and the frame-to-frame variation is 0.3 mm.
In the above step A3, the standard filter coefficient w of the zero-level key point00.3, and the inter-frame distance difference level reference value is 10 mm; standard filter coefficient w of first-stage key point10.4, and the inter-frame distance difference level reference value is 15 mm; standard filter coefficient w of second-stage key point20.5, and the reference value of the difference of the distance between frames is 20 mm; standard filter coefficient w of third-stage key point30.6, and the reference value of the difference of the distances between frames is 25 mm;
in the step a4, for the zero-level keypoint, when the difference between the two frame coordinate distances exceeds 10 mm, for each 5 mm increase, the corresponding filter coefficient increases by 0.04; for the first-level key points, determining the average value of the distances from the first-level key points to the gravity center, and when the difference between the average values of the distances between two frames exceeds 15 mm, increasing the corresponding filter coefficient by 0.06 every 5 mm; for the second-level key points, determining the average distance values from the second-level key points to the gravity center, and when the difference between the average distance values between two frames exceeds 20 mm, increasing the corresponding filter coefficient by 0.08 every 5 mm; for the third-level key points, determining the distance average value from the third-level key points to the gravity center, and when the difference between the distance average values between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
Step S3 of this embodiment specifically includes:
in the continuous video frame processing process, the vector distance between two human body key points is determined in sequence according to the key point grade division, filtering is carried out step by step according to the filtering coefficients in the filtering coefficient database, and coordinates of all the filtered key points are output, so that the key point stability is realized.
Specifically, in the hierarchical filtering process, the coordinates of the key points are expressed by vectors in this embodiment, so that the distance is more conveniently calculated and the filtering process is performed; therefore, as shown in fig. 4, the method for sequentially determining the vector distance between two human body key points, performing progressive filtering according to the filter coefficients in the filter coefficient database, and outputting the coordinates of all the filtered key points specifically includes:
t1, determining the geometric center of gravity O of the human body;
t2, calculating the coordinates (p) of the first-level key points respectively1,p2,p5,p8,p11) Distance to center of gravity O (d 1)o1,d1o2,d1o5,d1o8,d1o11) And performing primary filtering to obtain the updated distance (d1 ') from the first-stage key point to the gravity center'o1,d1'o2,d1'o5,d1'o8,d1'o11)；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4 according to centers of gravity O 'and (d1'o1,d1'o2,d1'o5,d1'o8,d1'o11) Calculating updated primary keypoint coordinates (p'1,p'2,p'5,p'8,p'11)；
T5, respectively calculating the coordinates (p) of the second-stage key points3,p6,p9,p12) To the updated coordinates (p ') of the first level keypoints'2,p'5,p'8,p'11) Corresponding distance is obtained, secondary filtering is carried out, and the distance (d2 ') from the updated second-stage key point to the corresponding first key point is obtained'23,d2'56,d2'89,d2'1112)；
T6, pair of updated Primary Key Point coordinates (p'1,p'2,p'5,p'8,p'11) Performing primary filtering to obtain stable first-stage key point coordinates (p) "1,p”2,p”5,p”8,p”11)；
T7, according to (p "2,p”5,p”8,p”11) And (d2'23,d2'56,d2'89,d2'1112) Calculating updated second level keypoint coordinates (p'3,p'6,p'9,p'12)；
T8, calculating the coordinate (p) of the key point of the third level respectively4,p7,p10,p13) To updated second level keypoint coordinates (p'3,p'6,p'9,p'12) Corresponding distance (d 3)34,d367,d3910,d31213) And carrying out three-stage filtering to obtain the distance (d3 ') from the updated third-stage key point to the corresponding second-stage key point'34,d3'67,d3'910,d3'1213)；
T9, pair of updated second level Key points (p'3,p'6,p'9,p'12) Secondary filtering is carried out to obtain stable secondary key point coordinates (p) "3,p”6,p”9,p”12)；
T10, according to (p "3,p”6,p”9,p”12) And (d3'34,d3'67,d3'910,d3'1213) Calculating updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13)；
T11, pair of updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13) Carrying out three-level filtering to obtain stable third-level key point coordinates (p) "4,p”7,p”10,p”13)；
T12, outputting all stable key point coordinates (p "1,p”2,p”3,p”4,p”5,p”6,p”7,p”8,p”9,p”10,p”11,p”12,p”13)。
In the above process, taking the key point 5 and the key point 6 in fig. 2 as examples, the coordinates of the two points are p5＝x5+iy5、p6＝x6+iy6From the theorem of the nature of the vector, a vector representation d2, which can obtain the distance between two points56＝p5-p6As shown in fig. 3, the dashed line between the key points 5 and 6 is a two-point distance vector;
the formulas of the distance filtering in the above steps T2, T5, and T8 and the keypoint filtering in the above steps T3, T6, T9, and T11 are:
Nrn＝rn-1×(1-w)+rn×w
in the formula, rnIs the actual coordinate of the key point of the nth frame, rn-1Is the actual coordinate of the key point of the (N-1) th frame, NrnAnd W is the filtered coordinate of the key point of the nth frame, and W is the filter coefficient of the key point of the current level in the filter coefficient database.
In this embodiment, the gravity center has the highest priority, and then the first-level, second-level, and third-level key points are sequentially provided, during processing, the coordinate filtering of the key point of the current level is performed first, then the filtering of the distance between the key point of the current level and the key point of the next level is performed, and the filtered coordinate vector is subtracted from the distance vector, so that the coordinate after the filtering update of the key point of the next level can be obtained.
Claims (9)
1. A human body key point stabilizing method based on hierarchical filtering is characterized by comprising the following steps:
s1, determining the geometric center of gravity of the human body;
s2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body;
and S3, based on the constructed filter coefficient database, determining the coordinates of the key points of the human body by hierarchical filtering, and further realizing the stability of the key points.
2. The method for stabilizing human body key points based on hierarchical filtering according to claim 1, wherein the step S1 specifically comprises:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of the human body through OpenPose, and forming a link between every two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the centroid of each link according to the percentage of the centroid of each link to the proximal end;
s14, calculating the position (X, Y) of the geometric center of gravity of the human body in a rectangular coordinate system based on the position coordinates of the centroid of each link;
wherein, the calculation formula is:
in the formula, PiIs the gravity of the ith link, XiIs the barycentric abscissa, Y, of the ith linkiIs the longitudinal coordinate of the gravity center of the ith link, P is the gravity of the human body, and X and Y are the transverse and longitudinal coordinates of the gravity center of the human body respectively.
3. The method for stabilizing human body key points based on hierarchical filtering according to claim 2, wherein the step S2 specifically comprises:
s21, carrying out level division on the identified human body key points and the human body geometric gravity center;
and S22, setting corresponding filter coefficients for the key points of each divided level, and constructing a filter coefficient database.
4. The method for stabilizing human body key points based on hierarchical filtering according to claim 3, wherein the human body key points identified in step S21 include 1-13 corresponding to head, right shoulder, right elbow, right hand head, left shoulder, left elbow, left hand head, right waist, right knee, right foot head, left waist, left knee and left foot head, respectively;
wherein, the geometric center of gravity of the human body is divided into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second-level key points;
4, 7, 10 and 13 are divided into third level keypoints.
5. The method for stabilizing human body key points based on hierarchical filtering according to claim 4, wherein the step S22 specifically comprises:
a1, determining the distance between adjacent pixels in the human body picture;
a2, calculating the frame-to-frame variation of the target value based on the determined pixel spacing;
a3, setting standard filter coefficients and inter-frame distance difference grading reference values corresponding to the key points of each level;
a4, determining the change rule of the filter coefficient of the key point under the corresponding level based on the calculated inter-frame change and inter-frame distance difference grade reference value of the target value, and constructing a filter coefficient database by combining the standard filter coefficient.
6. The method for stabilizing human body key points based on hierarchical filtering according to claim 5, wherein the method for determining the distance between adjacent pixels in the step A1 is specifically as follows:
selecting two points with known and fixed middle distances of the human body pictures, and determining the distance between adjacent pixels according to the pixel difference of the points in the y direction;
the interframe variation of the target value in the step A2 is the product of the pixel displacement and the adjacent pixel spacing;
in the step A3, the standard filter coefficient w of the zero-level key point00.3, and the inter-frame distance difference level reference value is 10 mm; standard filter coefficient w of first-stage key point10.4, and the inter-frame distance difference level reference value is 15 mm; standard filter coefficient w of second-stage key point20.5, and the reference value of the difference of the distance between frames is 20 mm; standard filter coefficient w of third-stage key point30.6, and the reference value of the difference of the distances between frames is 25 mm;
in the step a4, for the zero-level keypoint, when the difference between the two frame coordinate distances exceeds 10 mm, for each 5 mm increase, the corresponding filter coefficient increases by 0.04; for the first-level key points, determining the average value of the distances from the first-level key points to the gravity center, and when the difference between the average values of the distances between two frames exceeds 15 mm, increasing the corresponding filter coefficient by 0.06 every 5 mm; for the second-level key points, determining the average distance values from the second-level key points to the gravity center, and when the difference between the average distance values between two frames exceeds 20 mm, increasing the corresponding filter coefficient by 0.08 every 5 mm; for the third-level key points, determining the distance average value from the third-level key points to the gravity center, and when the difference between the distance average values between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
7. The method for stabilizing human body key points based on hierarchical filtering according to claim 5, wherein the step S3 specifically comprises:
in the continuous video frame processing process, the vector distance between two human body key points is determined in sequence according to the key point grade division, filtering is carried out step by step according to the filtering coefficients in the filtering coefficient database, and coordinates of all the filtered key points are output, so that the key point stability is realized.
8. The method for stabilizing human body key points based on hierarchical filtering according to claim 7, wherein the method for sequentially determining the vector distance between two human body key points, performing the hierarchical filtering according to the filter coefficients in the filter coefficient database, and outputting the coordinates of all the filtered key points specifically comprises:
t1, determining the geometric center of gravity O of the human body;
t2, calculating the coordinates (p) of the first-level key points respectively1,p2,p5,p8,p11) Distance to center of gravity O (d 1)o1,d1o2,d1o5,d1o8,d1o11) And performing primary filtering to obtain the updated distance (d1 ') from the first-stage key point to the gravity center'o1,d1'o2,d1'o5,d1'o8,d1'o11)；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4 according to centers of gravity O 'and (d1'o1,d1'o2,d1'o5,d1'o8,d1'o11) Calculating updated primary keypoint coordinates (p'1,p'2,p'5,p'8,p'11)；
T5, respectively calculating the coordinates (p) of the second-stage key points3,p6,p9,p12) To the updated coordinates (p ') of the first level keypoints'2,p'5,p'8,p'11) Corresponding distance is obtained, secondary filtering is carried out, and the distance (d2 ') from the updated second-stage key point to the corresponding first key point is obtained'23,d2'56,d2'89,d2'1112)；
T6, pair of updated Primary Key Point coordinates (p'1,p'2,p'5,p'8,p'11) Performing primary filtering to obtain stable first-stage key point coordinates (p) "1,p”2,p”5,p”8,p”11)；
T7, according to (p "2,p”5,p”8,p”11) And (d2'23,d2'56,d2'89,d2'1112) Calculating updated second level keypoint coordinates (p'3,p'6,p'9,p'12)；
T8, calculating the coordinate (p) of the key point of the third level respectively4,p7,p10,p13) To updated second level keypoint coordinates (p'3,p'6,p'9,p'12) Corresponding distance (d 3)34,d367,d3910,d31213) And carrying out three-stage filtering to obtain the distance (d3 ') from the updated third-stage key point to the corresponding second-stage key point'34,d3'67,d3'910,d3'1213)；
T9, pair of updated second level Key points (p'3,p'6,p'9,p'12) Secondary filtering is carried out to obtain stable secondary key point coordinates (p) "3,p”6,p”9,p”12)；
T10, according to (p "3,p”6,p”9,p”12) And (d3'34,d3'67,d3'910,d3'1213) Calculating updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13)；
T11, pair of updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13) Carrying out three-level filtering to obtain stable third-level key point coordinates (p) "4,p”7,p”10,p”13)；
T12, outputting all stable key point coordinates (p "1,p”2,p”3,p”4,p”5,p”6,p”7,p”8,p”9,p”10,p”11,p”12,p”13)。
9. The hierarchical filtering-based human body keypoint stabilizing method according to claim 8, wherein the formulas of the distance filtering in steps T2, T5 and T8 and the keypoint filtering in steps T3, T6, T9 and T11 are:
Nrn＝rn-1×(1-w)+rn×w
in the formula, rnIs the actual coordinate of the key point of the nth frame, rn-1Is the actual coordinate of the key point of the (N-1) th frame, NrnAnd W is the filtered coordinate of the key point of the nth frame, and W is the filter coefficient of the key point of the current level in the filter coefficient database.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110576415 | 2021-05-26 | ||
| CN2021105764153 | 2021-05-26 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113505735A true CN113505735A (en) | 2021-10-15 |
| CN113505735B CN113505735B (en) | 2023-05-02 |
Family
ID=78014094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110846716.3A Active CN113505735B (en) | 2021-05-26 | 2021-07-26 | Human body key point stabilization method based on hierarchical filtering |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113505735B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114267086A (en) * | 2021-12-30 | 2022-04-01 | 西南石油大学 | A method for evaluating the execution quality of complex continuous movements in sports |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
| CN110210323A (en) * | 2019-05-09 | 2019-09-06 | 浙江大学 | A kind of drowned behavior online recognition method based on machine vision |
| CN110287923A (en) * | 2019-06-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | Human body attitude acquisition methods, device, computer equipment and storage medium |
| CN110287891A (en) * | 2019-06-26 | 2019-09-27 | 北京字节跳动网络技术有限公司 | Gestural control method, device and electronic equipment based on human body key point |
| US20200090484A1 (en) * | 2018-09-13 | 2020-03-19 | Wistron Corporation | Falling detection method and electronic system using the same |
| CN111242982A (en) * | 2020-01-02 | 2020-06-05 | 浙江工业大学 | A Human Target Tracking Method Based on Progressive Kalman Filtering |
| CN112163564A (en) * | 2020-10-26 | 2021-01-01 | 燕山大学 | Fall prediction method based on human key point behavior recognition and LSTM |
-
2021
- 2021-07-26 CN CN202110846716.3A patent/CN113505735B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
| US20200090484A1 (en) * | 2018-09-13 | 2020-03-19 | Wistron Corporation | Falling detection method and electronic system using the same |
| CN110210323A (en) * | 2019-05-09 | 2019-09-06 | 浙江大学 | A kind of drowned behavior online recognition method based on machine vision |
| CN110287891A (en) * | 2019-06-26 | 2019-09-27 | 北京字节跳动网络技术有限公司 | Gestural control method, device and electronic equipment based on human body key point |
| CN110287923A (en) * | 2019-06-29 | 2019-09-27 | 腾讯科技（深圳）有限公司 | Human body attitude acquisition methods, device, computer equipment and storage medium |
| CN111242982A (en) * | 2020-01-02 | 2020-06-05 | 浙江工业大学 | A Human Target Tracking Method Based on Progressive Kalman Filtering |
| CN112163564A (en) * | 2020-10-26 | 2021-01-01 | 燕山大学 | Fall prediction method based on human key point behavior recognition and LSTM |
Non-Patent Citations (4)
| Title |
|---|
| ALEXANDROS MAKRIS ET AL.: "Robust 3D Human Pose Estimation Guided by Filtered Subsets of Body Keypoints" * |
| WEIMING CHEN ET AL.: "Fall Detection Based on Key Points of Human-Skeleton Using OpenPose" * |
| 张欣等: "基于相关滤波和关键点的目标跟踪算法" * |
| 郑恬: "基于视频的人体检测、跟踪和三维重建" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114267086A (en) * | 2021-12-30 | 2022-04-01 | 西南石油大学 | A method for evaluating the execution quality of complex continuous movements in sports |
| CN114267086B (en) * | 2021-12-30 | 2024-08-09 | 西南石油大学 | Execution quality evaluation method for complex continuous actions in motion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113505735B (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111144217B (en) | Motion evaluation method based on human body three-dimensional joint point detection | |
| CN108597578B (en) | A Human Motion Assessment Method Based on Two-dimensional Skeleton Sequences | |
| Qiao et al. | Real-time human gesture grading based on OpenPose | |
| CN106650687B (en) | Posture correction method based on depth information and skeleton information | |
| CN111931804B (en) | An automatic scoring method for human action based on RGBD camera | |
| CN113255522B (en) | Personalized motion attitude estimation and analysis method and system based on time consistency | |
| CN104035557B (en) | Kinect action identification method based on joint activeness | |
| US20200089958A1 (en) | Image recognition method and apparatus, electronic device, and readable storage medium | |
| CN106066996A (en) | The local feature method for expressing of human action and in the application of Activity recognition | |
| Bhamidipati et al. | Robust intelligent posture estimation for an ai gym trainer using mediapipe and opencv | |
| CN111860157B (en) | Motion analysis method, device, equipment and storage medium | |
| WO2021051526A1 (en) | Multi-view 3d human pose estimation method and related apparatus | |
| CN112001217A (en) | Multi-person human pose estimation algorithm based on deep learning | |
| Dedhia et al. | Pose estimation and virtual gym assistant using MediaPipe and machine learning | |
| He et al. | A New Kinect‐Based Posture Recognition Method in Physical Sports Training Based on Urban Data | |
| Pardos et al. | Automated posture analysis for the assessment of sports exercises | |
| Zheng et al. | A skeleton-based rehabilitation exercise assessment system with rotation invariance | |
| CN116543458A (en) | A Method of Attitude Analysis and Assessment Based on Period Length Segment Analysis | |
| CN113505735A (en) | Human body key point stabilizing method based on hierarchical filtering | |
| Shen et al. | Design of OpenPose-based of exercise assistant system with instructor-user synchronization for self-practice dynamic yoga | |
| CN108392207A (en) | A kind of action identification method based on posture label | |
| CN114724253A (en) | A motion process management method, device, computer equipment and storage medium | |
| Aarthy et al. | Yoga pose detection and identification using mediapipe and openpose model | |
| Liu et al. | Deep learning-based standardized evaluation and human pose estimation: A novel approach to motion perception | |
| CN115620392A (en) | Action counting method, device, medium and fitness equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |