CN113505735A - Human body key point stabilizing method based on hierarchical filtering - Google Patents

Human body key point stabilizing method based on hierarchical filtering Download PDF

Info

Publication number
CN113505735A
CN113505735A CN202110846716.3A CN202110846716A CN113505735A CN 113505735 A CN113505735 A CN 113505735A CN 202110846716 A CN202110846716 A CN 202110846716A CN 113505735 A CN113505735 A CN 113505735A
Authority
CN
China
Prior art keywords
human body
key points
level
key point
distance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110846716.3A
Other languages
Chinese (zh)
Other versions
CN113505735B (en
Inventor
张新颖
陈安成
胡峰颖
严伟成
王权泳
吴哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Publication of CN113505735A publication Critical patent/CN113505735A/en
Application granted granted Critical
Publication of CN113505735B publication Critical patent/CN113505735B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a human body key point stabilizing method based on hierarchical filtering, which comprises the following steps: s1, determining the geometric center of gravity of the human body; s2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body; and S3, based on the constructed filter coefficient database, determining the coordinates of the key points of the human body by hierarchical filtering, and further realizing the stability of the key points. The method effectively improves the shaking phenomenon when the key points of the human body are positioned, and ensures that the identified key points still keep stable when the human body shakes slightly; the method can still keep smooth key point identification under the condition of obvious human body movement; the method enables the human body key point identification technology to be widely applied in the fields of medical health and the like with higher requirements on stable identification precision, for example, when a human body anatomical structure is pasted, the pasting is basically stable and motionless, and when a user does rehabilitation exercise, action evaluation feedback can be stably provided in the process.

Description

Human body key point stabilizing method based on hierarchical filtering
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a human body key point stabilizing method based on hierarchical filtering.
Background
The human body key point detection is a significant research direction in the field of computer vision, is mainly used for human body action posture recognition and analysis, and has huge application scenes in the fields of security monitoring, sports fitness, rehabilitation training, electronic games, 3D fitting and the like. A lot of research works are carried out on human body key point recognition by a plurality of teams, wherein the typical example is openpos which is an open source project of the university of Chimeilong in the card, openpos contains positioning of a face, a hand and each joint of a human body, estimation of action posture, facial expression, arm movement and the like can be realized, the model can carry out posture recognition on a plurality of people, and the model has good robustness.
However, when the human body posture recognition model such as openpos is used for positioning key points of the human body, the body movement can cause unstable jitter of positioning points of some flexible parts of the human body, and the positioning effect and the presentation effect of other algorithm functions established on the basis of the openpos model and the like are influenced. For the fields requiring high-precision human action and posture recognition effects, such as rehabilitation training evaluation and ultrasonic probe guidance in the medical field, the jitters bring serious consequences, and once the recognition effect is poor, the human condition misjudgment, poor rehabilitation effect evaluation and ultrasonic guidance deviation are caused, so that the method has great practical significance for solving the problem.
Disclosure of Invention
Aiming at the defects in the prior art, the method for stabilizing the human body key points based on the hierarchical filtering solves the problems that the existing human body key points are poor in positioning effect and prone to shaking.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that: a human body key point stabilizing method based on hierarchical filtering comprises the following steps:
s1, determining the geometric center of gravity of the human body;
s2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body;
and S3, based on the constructed filter coefficient database, determining the coordinates of the key points of the human body by hierarchical filtering, and further realizing the stability of the key points.
Further, the step S1 is specifically:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of the human body through OpenPose, and forming a link between every two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the centroid of each link according to the percentage of the centroid of each link to the proximal end;
s14, calculating the position (X, Y) of the geometric center of gravity of the human body in a rectangular coordinate system based on the position coordinates of the centroid of each link;
wherein, the calculation formula is:
Figure BDA0003180854780000021
Figure BDA0003180854780000022
in the formula, PiIs the gravity of the ith link, XiIs the barycentric abscissa, Y, of the ith linkiIs the longitudinal coordinate of the gravity center of the ith link, P is the gravity of the human body, and X and Y are the transverse and longitudinal coordinates of the gravity center of the human body respectively.
Further, the step S2 is specifically:
s21, carrying out level division on the identified human body key points and the human body geometric gravity center;
and S22, setting corresponding filter coefficients for the key points of each divided level, and constructing a filter coefficient database.
Further, in the step S21, the identified key points of the human body include 1 to 13 corresponding to the head, the right shoulder, the right elbow, the right hand head, the left shoulder, the left elbow, the left hand head, the right waist, the right knee, the right foot head, the left waist, the left knee and the left foot head, respectively;
wherein, the geometric center of gravity of the human body is divided into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second-level key points;
4, 7, 10 and 13 are divided into third level keypoints.
Further, the step S22 is specifically:
a1, determining the distance between adjacent pixels in the human body picture;
a2, calculating the frame-to-frame variation of the target value based on the determined pixel spacing;
a3, setting standard filter coefficients and inter-frame distance difference grading reference values corresponding to the key points of each level;
a4, determining the change rule of the filter coefficient of the key point under the corresponding level based on the calculated inter-frame change and inter-frame distance difference grade reference value of the target value, and constructing a filter coefficient database by combining the standard filter coefficient.
Further, the method for determining the distance between adjacent pixels in the step a1 specifically includes:
selecting two points with known and fixed middle distances of the human body pictures, and determining the distance between adjacent pixels according to the pixel difference of the points in the y direction;
the interframe variation of the target value in the step A2 is the product of the pixel displacement and the adjacent pixel spacing;
in the step A3, the standard filter coefficient w of the zero-level key point00.3, and the inter-frame distance difference level reference value is 10 mm; standard filter coefficient w of first-stage key point10.4, and the inter-frame distance difference level reference value is 15 mm; standard filter coefficient w of second-stage key point20.5, and the reference value of the difference of the distance between frames is 20 mm; standard filter coefficient w of third-stage key point30.6, and the reference value of the difference of the distances between frames is 25 mm;
in the step a4, for the zero-level keypoint, when the difference between the two frame coordinate distances exceeds 10 mm, for each 5 mm increase, the corresponding filter coefficient increases by 0.04; for the first-level key points, determining the average value of the distances from the first-level key points to the gravity center, and when the difference between the average values of the distances between two frames exceeds 15 mm, increasing the corresponding filter coefficient by 0.06 every 5 mm; for the second-level key points, determining the average distance values from the second-level key points to the gravity center, and when the difference between the average distance values between two frames exceeds 20 mm, increasing the corresponding filter coefficient by 0.08 every 5 mm; for the third-level key points, determining the distance average value from the third-level key points to the gravity center, and when the difference between the distance average values between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
Further, the step S3 is specifically:
in the continuous video frame processing process, the vector distance between two human body key points is determined in sequence according to the key point grade division, filtering is carried out step by step according to the filtering coefficients in the filtering coefficient database, and coordinates of all the filtered key points are output, so that the key point stability is realized.
Further, the method for sequentially determining the vector distance between two human body key points, performing step-by-step filtering according to the filter coefficients in the filter coefficient database, and outputting the coordinates of all the filtered key points specifically comprises the following steps:
t1, determining the geometric center of gravity O of the human body;
t2, calculating the coordinates (p) of the first-level key points respectively1,p2,p5,p8,p11) Distance to center of gravity O (d 1)o1,d1o2,d1o5,d1o8,d1o11) And performing primary filtering to obtain the updated distance (d1 ') from the first-stage key point to the gravity center'o1,d1'o2,d1'o5,d1'o8,d1'o11)；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4 according to centers of gravity O 'and (d1'o1,d1'o2,d1'o5,d1'o8,d1'o11) Calculating updated primary keypoint coordinates (p'1,p'2,p'5,p'8,p'11)；
T5, respectively calculating the coordinates (p) of the second-stage key points3,p6,p9,p12) To the updated coordinates (p ') of the first level keypoints'2,p'5,p'8,p'11) Corresponding distance is obtained, secondary filtering is carried out, and the distance (d2 ') from the updated second-stage key point to the corresponding first key point is obtained'23,d2'56,d2'89,d2'1112)；
T6, pair of updated Primary Key Point coordinates (p'1,p'2,p'5,p'8,p'11) Performing primary filtering to obtain stable first-stage key point coordinates (p) "1,p”2,p”5,p”8,p”11)；
T7, according to (p "2,p”5,p”8,p”11) And (d2'23,d2'56,d2'89,d2'1112) Calculating updated second level keypoint coordinates (p'3,p'6,p'9,p'12)；
T8, calculating the coordinate (p) of the key point of the third level respectively4,p7,p10,p13) To updated second level keypoint coordinates (p'3,p'6,p'9,p'12) Corresponding distance (d 3)34,d367,d3910,d31213) And go through three stagesFiltering to obtain the distance (d3 ') from the updated third-level key point to the corresponding second-level key point'34,d3'67,d3'910,d3'1213)；
T9, pair of updated second level Key points (p'3,p'6,p'9,p'12) Secondary filtering is carried out to obtain stable secondary key point coordinates (p) "3,p”6,p”9,p”12)；
T10, according to (p "3,p”6,p”9,p”12) And (d3'34,d3'67,d3'910,d3'1213) Calculating updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13)；
T11, pair of updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13) Carrying out three-level filtering to obtain stable third-level key point coordinates (p) "4,p”7,p”10,p”13)；
T12, outputting all stable key point coordinates (p "1,p”2,p”3,p”4,p”5,p”6,p”7,p”8,p”9,p”10,p”11,p”12,p”13)。
Further, the formulas of the distance filtering in the steps T2, T5 and T8 and the keypoint filtering in the steps T3, T6, T9 and T11 are:
Nrn＝rn-1×(1-w)+rn×w
in the formula, rnIs the actual coordinate of the key point of the nth frame, rn-1Is the actual coordinate of the key point of the (N-1) th frame, NrnAnd W is the filtered coordinate of the key point of the nth frame, and W is the filter coefficient of the key point of the current level in the filter coefficient database.
The invention has the beneficial effects that:
(1) the method effectively improves the shaking phenomenon when the key points of the human body are positioned, and ensures that the identified key points still keep stable when the human body shakes slightly;
(2) the method can still keep smooth key point identification under the condition of relatively obvious human body movement;
(3) the method of the invention enables the human body key point identification technology to be widely applied in the fields of medical health and the like with higher requirements on stable identification precision, for example, when a human body anatomical structure is pasted, the pasting is basically stable and motionless, and when a user does rehabilitation exercise, the action evaluation feedback can be provided stably in the process.
Drawings
Fig. 1 is a flowchart of a human body key point stabilizing method based on hierarchical filtering provided by the present invention.
FIG. 2 is a schematic diagram of the 18-point model key point labels based on OpenPose according to the present invention.
Fig. 3 is a schematic diagram of a method for calculating a distance between key points according to the present invention.
Fig. 4 is a flowchart of a method for solving key points by hierarchical filtering according to the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
As shown in fig. 1, a method for stabilizing human body key points based on hierarchical filtering includes the following steps:
s1, determining the geometric center of gravity of the human body;
s2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body;
and S3, based on the constructed filter coefficient database, determining the coordinates of the key points of the human body by hierarchical filtering, and further realizing the stability of the key points.
In step S1 of the present embodiment, according to the law of varez, if there is a resultant force in a plane-parallel force system, the moment of the resultant force at any point is equal to the algebraic sum of the moments of all points of the force system at the same point. The theory is applied to biology, a human body can be simplified into 13 rigid bodies by combining OpenPose, each link of the divided human body is considered to be homogeneous, and the position (X, Y) of the geometric center of gravity of the human body in a rectangular coordinate system is calculated by the following formula;
Figure BDA0003180854780000071
Figure BDA0003180854780000072
in the formula, PiIs the gravity of the ith link, XiIs the barycentric abscissa, Y, of the ith linkiIs the longitudinal coordinate of the gravity center of the ith link, P is the gravity of the human body, and X and Y are the transverse and longitudinal coordinates of the gravity center of the human body respectively.
Therefore, the step S1 is specifically:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of the human body through OpenPose, and forming a link between every two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the centroid of each link according to the percentage of the centroid of each link to the proximal end;
and S14, calculating the position (X, Y) of the geometric center of gravity of the human body in the rectangular coordinate system based on the position coordinates of the centroid of each link.
In step S2 of this embodiment, in consideration of different flexibility of the head, the trunk, the elbows, the ankles, and the like during the movement of the human body, the key points of the human body need to be classified into different levels, and the key points of different levels correspond to the filter coefficients of different levels. Based on this, the step S2 is specifically:
s21, carrying out level division on the identified human body key points and the human body geometric gravity center;
and S22, setting corresponding filter coefficients for the key points of each divided level, and constructing a filter coefficient database.
In the step S21, taking an 18-point openpos model as an example, the key points obtained by human posture recognition are as shown in fig. 2, since the flexibility of the key points of the head is not high, the filtering process is not performed in the present invention, and the recognized key points of the human body include 1 to 13, which respectively correspond to the head, the right shoulder, the right elbow, the right hand head, the left shoulder, the left elbow, the left hand head, the right waist, the right knee, the right foot head, the left waist, the left knee and the left foot head;
wherein, the geometric center of gravity of the human body is divided into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second-level key points;
4, 7, 10 and 13 are divided into third level keypoints.
Because the key center of gravity of the human body can well reflect the position information and is convenient to establish the relation with the first-stage key points, the filtering of the center of gravity coordinate is added in the filtering process, so that the positioning accuracy of the key points of the human body can be improved.
We set the filter coefficients according to the flexibility of the key points. The stability of the center of gravity is strong, and the filter coefficient is minimum; the first-stage key points have high stability and small filter coefficients; the second-stage key points have high flexibility and high filter coefficients; the third-stage key point has the strongest flexibility and the largest filtering coefficient. This gives: w is a0<w1<w2<w3。
The step S22 is specifically:
a1, determining the distance between adjacent pixels in the human body picture;
a2, calculating the frame-to-frame variation of the target value based on the determined pixel spacing; a3, setting standard filter coefficients and inter-frame distance difference grading reference values corresponding to the key points of each level;
a4, determining the change rule of the filter coefficient of the key point under the corresponding level based on the calculated inter-frame change and inter-frame distance difference grade reference value of the target value, and constructing a filter coefficient database by combining the standard filter coefficient.
The method for determining the distance between adjacent pixels in the step a1 specifically includes:
selecting two points with known and fixed middle distances of the human body pictures, and determining the distance between adjacent pixels according to the pixel difference of the points in the y direction; for example, the distance from the tip of the nose to the midpoint of the two lateral ankles is measured, and assuming that their vertical distance is 1600 mm, the pixel pitch in mm, i.e., the actual size of a single pixel value, is deduced from the difference in pixel values of these two points in the y-direction.
The target value in step a2 refers to the selected variable value, such as the x and y coordinates of the key point and the coordinate-based reference variable, such as the distance d;
the inter-frame variation of the target value in the step a2 is the product of the pixel displacement and the adjacent pixel pitch;
specifically, the image is composed of a plurality of pixel points, and the relative positions of the pixel points relative to the whole picture are approximately regarded as unchanged in a very short time. For example, a pixel point in a picture representing the tip of a nose may be considered stationary for a very short time, but may be displaced for a long time. The pixel displacement is the difference of the number of pixel points of the point of the nose tip in the image in the next frame relative to the previous frame. For example, the image is 1024x768, the adjacent pixel pitch is 0.1mm, the pixel at the tip of the nose in this application is (800,400), but by the next frame, if the pixel displacement is 3, the pixel becomes (803,400), and the frame-to-frame variation is 0.3 mm.
In the above step A3, the standard filter coefficient w of the zero-level key point00.3, and the inter-frame distance difference level reference value is 10 mm; standard filter coefficient w of first-stage key point10.4, and the inter-frame distance difference level reference value is 15 mm; standard filter coefficient w of second-stage key point20.5, and the reference value of the difference of the distance between frames is 20 mm; standard filter coefficient w of third-stage key point30.6, and the reference value of the difference of the distances between frames is 25 mm;
in the step a4, for the zero-level keypoint, when the difference between the two frame coordinate distances exceeds 10 mm, for each 5 mm increase, the corresponding filter coefficient increases by 0.04; for the first-level key points, determining the average value of the distances from the first-level key points to the gravity center, and when the difference between the average values of the distances between two frames exceeds 15 mm, increasing the corresponding filter coefficient by 0.06 every 5 mm; for the second-level key points, determining the average distance values from the second-level key points to the gravity center, and when the difference between the average distance values between two frames exceeds 20 mm, increasing the corresponding filter coefficient by 0.08 every 5 mm; for the third-level key points, determining the distance average value from the third-level key points to the gravity center, and when the difference between the distance average values between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
Step S3 of this embodiment specifically includes:
in the continuous video frame processing process, the vector distance between two human body key points is determined in sequence according to the key point grade division, filtering is carried out step by step according to the filtering coefficients in the filtering coefficient database, and coordinates of all the filtered key points are output, so that the key point stability is realized.
Specifically, in the hierarchical filtering process, the coordinates of the key points are expressed by vectors in this embodiment, so that the distance is more conveniently calculated and the filtering process is performed; therefore, as shown in fig. 4, the method for sequentially determining the vector distance between two human body key points, performing progressive filtering according to the filter coefficients in the filter coefficient database, and outputting the coordinates of all the filtered key points specifically includes:
t1, determining the geometric center of gravity O of the human body;
t2, calculating the coordinates (p) of the first-level key points respectively1,p2,p5,p8,p11) Distance to center of gravity O (d 1)o1,d1o2,d1o5,d1o8,d1o11) And performing primary filtering to obtain the updated distance (d1 ') from the first-stage key point to the gravity center'o1,d1'o2,d1'o5,d1'o8,d1'o11)；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4 according to centers of gravity O 'and (d1'o1,d1'o2,d1'o5,d1'o8,d1'o11) Calculating updated primary keypoint coordinates (p'1,p'2,p'5,p'8,p'11)；
T5, respectively calculating the coordinates (p) of the second-stage key points3,p6,p9,p12) To the updated coordinates (p ') of the first level keypoints'2,p'5,p'8,p'11) Corresponding distance is obtained, secondary filtering is carried out, and the distance (d2 ') from the updated second-stage key point to the corresponding first key point is obtained'23,d2'56,d2'89,d2'1112)；
T6, pair of updated Primary Key Point coordinates (p'1,p'2,p'5,p'8,p'11) Performing primary filtering to obtain stable first-stage key point coordinates (p) "1,p”2,p”5,p”8,p”11)；
T7, according to (p "2,p”5,p”8,p”11) And (d2'23,d2'56,d2'89,d2'1112) Calculating updated second level keypoint coordinates (p'3,p'6,p'9,p'12)；
T8, calculating the coordinate (p) of the key point of the third level respectively4,p7,p10,p13) To updated second level keypoint coordinates (p'3,p'6,p'9,p'12) Corresponding distance (d 3)34,d367,d3910,d31213) And carrying out three-stage filtering to obtain the distance (d3 ') from the updated third-stage key point to the corresponding second-stage key point'34,d3'67,d3'910,d3'1213)；
T9, pair of updated second level Key points (p'3,p'6,p'9,p'12) Secondary filtering is carried out to obtain stable secondary key point coordinates (p) "3,p”6,p”9,p”12)；
T10, according to (p "3,p”6,p”9,p”12) And (d3'34,d3'67,d3'910,d3'1213) Calculating updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13)；
T11, pair of updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13) Carrying out three-level filtering to obtain stable third-level key point coordinates (p) "4,p”7,p”10,p”13)；
T12, outputting all stable key point coordinates (p "1,p”2,p”3,p”4,p”5,p”6,p”7,p”8,p”9,p”10,p”11,p”12,p”13)。
In the above process, taking the key point 5 and the key point 6 in fig. 2 as examples, the coordinates of the two points are p5＝x5+iy5、p6＝x6+iy6From the theorem of the nature of the vector, a vector representation d2, which can obtain the distance between two points56＝p5-p6As shown in fig. 3, the dashed line between the key points 5 and 6 is a two-point distance vector;
the formulas of the distance filtering in the above steps T2, T5, and T8 and the keypoint filtering in the above steps T3, T6, T9, and T11 are:
Nrn＝rn-1×(1-w)+rn×w
in the formula, rnIs the actual coordinate of the key point of the nth frame, rn-1Is the actual coordinate of the key point of the (N-1) th frame, NrnAnd W is the filtered coordinate of the key point of the nth frame, and W is the filter coefficient of the key point of the current level in the filter coefficient database.
In this embodiment, the gravity center has the highest priority, and then the first-level, second-level, and third-level key points are sequentially provided, during processing, the coordinate filtering of the key point of the current level is performed first, then the filtering of the distance between the key point of the current level and the key point of the next level is performed, and the filtered coordinate vector is subtracted from the distance vector, so that the coordinate after the filtering update of the key point of the next level can be obtained.

Claims (9)

1. A human body key point stabilizing method based on hierarchical filtering is characterized by comprising the following steps:
s1, determining the geometric center of gravity of the human body;
s2, constructing a filter coefficient database based on the determined geometric center of gravity of the human body;
and S3, based on the constructed filter coefficient database, determining the coordinates of the key points of the human body by hierarchical filtering, and further realizing the stability of the key points.
2. The method for stabilizing human body key points based on hierarchical filtering according to claim 1, wherein the step S1 specifically comprises:
s11, placing the human body picture to be processed in a rectangular coordinate system;
s12, identifying key points of the human body through OpenPose, and forming a link between every two adjacent key points to obtain a stick-shaped figure of the human body;
s13, determining the position coordinates of the centroid of each link according to the percentage of the centroid of each link to the proximal end;
s14, calculating the position (X, Y) of the geometric center of gravity of the human body in a rectangular coordinate system based on the position coordinates of the centroid of each link;
wherein, the calculation formula is:
Figure FDA0003180854770000011
Figure FDA0003180854770000012
in the formula, PiIs the gravity of the ith link, XiIs the barycentric abscissa, Y, of the ith linkiIs the longitudinal coordinate of the gravity center of the ith link, P is the gravity of the human body, and X and Y are the transverse and longitudinal coordinates of the gravity center of the human body respectively.
3. The method for stabilizing human body key points based on hierarchical filtering according to claim 2, wherein the step S2 specifically comprises:
s21, carrying out level division on the identified human body key points and the human body geometric gravity center;
and S22, setting corresponding filter coefficients for the key points of each divided level, and constructing a filter coefficient database.
4. The method for stabilizing human body key points based on hierarchical filtering according to claim 3, wherein the human body key points identified in step S21 include 1-13 corresponding to head, right shoulder, right elbow, right hand head, left shoulder, left elbow, left hand head, right waist, right knee, right foot head, left waist, left knee and left foot head, respectively;
wherein, the geometric center of gravity of the human body is divided into zero-order key points;
dividing 1, 2, 5, 8 and 11 into first-level key points;
dividing 3, 6, 9 and 12 into second-level key points;
4, 7, 10 and 13 are divided into third level keypoints.
5. The method for stabilizing human body key points based on hierarchical filtering according to claim 4, wherein the step S22 specifically comprises:
a1, determining the distance between adjacent pixels in the human body picture;
a2, calculating the frame-to-frame variation of the target value based on the determined pixel spacing;
a3, setting standard filter coefficients and inter-frame distance difference grading reference values corresponding to the key points of each level;
a4, determining the change rule of the filter coefficient of the key point under the corresponding level based on the calculated inter-frame change and inter-frame distance difference grade reference value of the target value, and constructing a filter coefficient database by combining the standard filter coefficient.
6. The method for stabilizing human body key points based on hierarchical filtering according to claim 5, wherein the method for determining the distance between adjacent pixels in the step A1 is specifically as follows:
selecting two points with known and fixed middle distances of the human body pictures, and determining the distance between adjacent pixels according to the pixel difference of the points in the y direction;
the interframe variation of the target value in the step A2 is the product of the pixel displacement and the adjacent pixel spacing;
in the step A3, the standard filter coefficient w of the zero-level key point00.3, and the inter-frame distance difference level reference value is 10 mm; standard filter coefficient w of first-stage key point10.4, and the inter-frame distance difference level reference value is 15 mm; standard filter coefficient w of second-stage key point20.5, and the reference value of the difference of the distance between frames is 20 mm; standard filter coefficient w of third-stage key point30.6, and the reference value of the difference of the distances between frames is 25 mm;
in the step a4, for the zero-level keypoint, when the difference between the two frame coordinate distances exceeds 10 mm, for each 5 mm increase, the corresponding filter coefficient increases by 0.04; for the first-level key points, determining the average value of the distances from the first-level key points to the gravity center, and when the difference between the average values of the distances between two frames exceeds 15 mm, increasing the corresponding filter coefficient by 0.06 every 5 mm; for the second-level key points, determining the average distance values from the second-level key points to the gravity center, and when the difference between the average distance values between two frames exceeds 20 mm, increasing the corresponding filter coefficient by 0.08 every 5 mm; for the third-level key points, determining the distance average value from the third-level key points to the gravity center, and when the difference between the distance average values between two frames exceeds 25 mm, increasing the corresponding filter coefficient by 0.1 every 5 mm.
7. The method for stabilizing human body key points based on hierarchical filtering according to claim 5, wherein the step S3 specifically comprises:
in the continuous video frame processing process, the vector distance between two human body key points is determined in sequence according to the key point grade division, filtering is carried out step by step according to the filtering coefficients in the filtering coefficient database, and coordinates of all the filtered key points are output, so that the key point stability is realized.
8. The method for stabilizing human body key points based on hierarchical filtering according to claim 7, wherein the method for sequentially determining the vector distance between two human body key points, performing the hierarchical filtering according to the filter coefficients in the filter coefficient database, and outputting the coordinates of all the filtered key points specifically comprises:
t1, determining the geometric center of gravity O of the human body;
t2, calculating the coordinates (p) of the first-level key points respectively1,p2,p5,p8,p11) Distance to center of gravity O (d 1)o1,d1o2,d1o5,d1o8,d1o11) And performing primary filtering to obtain the updated distance (d1 ') from the first-stage key point to the gravity center'o1,d1'o2,d1'o5,d1'o8,d1'o11)；
T3, carrying out zero-order filtering on the gravity center O to obtain a new gravity center O';
t4 according to centers of gravity O 'and (d1'o1,d1'o2,d1'o5,d1'o8,d1'o11) Calculating updated primary keypoint coordinates (p'1,p'2,p'5,p'8,p'11)；
T5, respectively calculating the coordinates (p) of the second-stage key points3,p6,p9,p12) To the updated coordinates (p ') of the first level keypoints'2,p'5,p'8,p'11) Corresponding distance is obtained, secondary filtering is carried out, and the distance (d2 ') from the updated second-stage key point to the corresponding first key point is obtained'23,d2'56,d2'89,d2'1112)；
T6, pair of updated Primary Key Point coordinates (p'1,p'2,p'5,p'8,p'11) Performing primary filtering to obtain stable first-stage key point coordinates (p) "1,p”2,p”5,p”8,p”11)；
T7, according to (p "2,p”5,p”8,p”11) And (d2'23,d2'56,d2'89,d2'1112) Calculating updated second level keypoint coordinates (p'3,p'6,p'9,p'12)；
T8, calculating the coordinate (p) of the key point of the third level respectively4,p7,p10,p13) To updated second level keypoint coordinates (p'3,p'6,p'9,p'12) Corresponding distance (d 3)34,d367,d3910,d31213) And carrying out three-stage filtering to obtain the distance (d3 ') from the updated third-stage key point to the corresponding second-stage key point'34,d3'67,d3'910,d3'1213)；
T9, pair of updated second level Key points (p'3,p'6,p'9,p'12) Secondary filtering is carried out to obtain stable secondary key point coordinates (p) "3,p”6,p”9,p”12)；
T10, according to (p "3,p”6,p”9,p”12) And (d3'34,d3'67,d3'910,d3'1213) Calculating updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13)；
T11, pair of updated tertiary keypoint coordinates (p'4,p'7,p'10,p'13) Carrying out three-level filtering to obtain stable third-level key point coordinates (p) "4,p”7,p”10,p”13)；
T12, outputting all stable key point coordinates (p "1,p”2,p”3,p”4,p”5,p”6,p”7,p”8,p”9,p”10,p”11,p”12,p”13)。
9. The hierarchical filtering-based human body keypoint stabilizing method according to claim 8, wherein the formulas of the distance filtering in steps T2, T5 and T8 and the keypoint filtering in steps T3, T6, T9 and T11 are:
Nrn＝rn-1×(1-w)+rn×w
in the formula, rnIs the actual coordinate of the key point of the nth frame, rn-1Is the actual coordinate of the key point of the (N-1) th frame, NrnAnd W is the filtered coordinate of the key point of the nth frame, and W is the filter coefficient of the key point of the current level in the filter coefficient database.
CN202110846716.3A 2021-05-26 2021-07-26 Human body key point stabilization method based on hierarchical filtering Active CN113505735B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110576415 2021-05-26
CN2021105764153 2021-05-26

Publications (2)

Publication Number Publication Date
CN113505735A true CN113505735A (en) 2021-10-15
CN113505735B CN113505735B (en) 2023-05-02

Family

ID=78014094

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110846716.3A Active CN113505735B (en) 2021-05-26 2021-07-26 Human body key point stabilization method based on hierarchical filtering

Country Status (1)

Country Link
CN (1) CN113505735B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114267086A (en) * 2021-12-30 2022-04-01 西南石油大学 Execution quality evaluation method for complex continuous motion in motion

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109003301A (en) * 2018-07-06 2018-12-14 东南大学 A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect
CN110210323A (en) * 2019-05-09 2019-09-06 浙江大学 A kind of drowned behavior online recognition method based on machine vision
CN110287923A (en) * 2019-06-29 2019-09-27 腾讯科技（深圳）有限公司 Human body attitude acquisition methods, device, computer equipment and storage medium
CN110287891A (en) * 2019-06-26 2019-09-27 北京字节跳动网络技术有限公司 Gestural control method, device and electronic equipment based on human body key point
US20200090484A1 (en) * 2018-09-13 2020-03-19 Wistron Corporation Falling detection method and electronic system using the same
CN111242982A (en) * 2020-01-02 2020-06-05 浙江工业大学 Human body target tracking method based on progressive Kalman filtering
CN112163564A (en) * 2020-10-26 2021-01-01 燕山大学 Tumble prejudging method based on human body key point behavior identification and LSTM (least Square TM)

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109003301A (en) * 2018-07-06 2018-12-14 东南大学 A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect
US20200090484A1 (en) * 2018-09-13 2020-03-19 Wistron Corporation Falling detection method and electronic system using the same
CN110210323A (en) * 2019-05-09 2019-09-06 浙江大学 A kind of drowned behavior online recognition method based on machine vision
CN110287891A (en) * 2019-06-26 2019-09-27 北京字节跳动网络技术有限公司 Gestural control method, device and electronic equipment based on human body key point
CN110287923A (en) * 2019-06-29 2019-09-27 腾讯科技（深圳）有限公司 Human body attitude acquisition methods, device, computer equipment and storage medium
CN111242982A (en) * 2020-01-02 2020-06-05 浙江工业大学 Human body target tracking method based on progressive Kalman filtering
CN112163564A (en) * 2020-10-26 2021-01-01 燕山大学 Tumble prejudging method based on human body key point behavior identification and LSTM (least Square TM)

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
ALEXANDROS MAKRIS ET AL.: "Robust 3D Human Pose Estimation Guided by Filtered Subsets of Body Keypoints" *
WEIMING CHEN ET AL.: "Fall Detection Based on Key Points of Human-Skeleton Using OpenPose" *
张欣等: "基于相关滤波和关键点的目标跟踪算法" *
郑恬: "基于视频的人体检测、跟踪和三维重建" *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114267086A (en) * 2021-12-30 2022-04-01 西南石油大学 Execution quality evaluation method for complex continuous motion in motion
CN114267086B (en) * 2021-12-30 2024-08-09 西南石油大学 Execution quality evaluation method for complex continuous actions in motion

Also Published As

Publication number Publication date
CN113505735B (en) 2023-05-02

Similar Documents

Publication Publication Date Title
CN108597578B (en) Human motion assessment method based on two-dimensional skeleton sequence
CN106650687B (en) Posture correction method based on depth information and skeleton information
Patrona et al. Motion analysis: Action detection, recognition and evaluation based on motion capture data
US11417095B2 (en) Image recognition method and apparatus, electronic device, and readable storage medium using an update on body extraction parameter and alignment parameter
CN109308459A (en) Gesture estimation method based on finger attention model and key point topological model
CN112597933B (en) Action scoring method, device and readable storage medium
WO2021051526A1 (en) Multi-view 3d human pose estimation method and related apparatus
CN111597975B (en) Personnel action detection method and device and electronic equipment
CN113255522B (en) Personalized motion attitude estimation and analysis method and system based on time consistency
He et al. A New Kinect‐Based Posture Recognition Method in Physical Sports Training Based on Urban Data
CN113505735A (en) Human body key point stabilizing method based on hierarchical filtering
Agarwal et al. FitMe: a fitness application for accurate pose estimation using deep learning
Shen et al. Design of OpenPose-based of exercise assistant system with instructor-user synchronization for self-practice dynamic yoga
Dedhia et al. Pose Estimation and Virtual Gym Assistant Using MediaPipe and Machine Learning
CN115346640B (en) Intelligent monitoring method and system for closed-loop feedback of functional rehabilitation training
CN115205750B (en) Motion real-time counting method and system based on deep learning model
CN115205737B (en) Motion real-time counting method and system based on transducer model
CN115223240B (en) Motion real-time counting method and system based on dynamic time warping algorithm
CN116189301A (en) Standing long jump motion standardability assessment method based on attitude estimation
Liu et al. Deep Learning-Based Standardized Evaluation and Human Pose Estimation: A Novel Approach to Motion Perception.
CN118212683A (en) Three-dimensional gesture estimation method and head-mounted display device
CN114724253A (en) Exercise process management method and device, computer equipment and storage medium
Kumar et al. Estimation of Human Posture Using Convolutional Neural Network Using Web Architecture
Hagimoto et al. A Dynamic Finger Character Recognition Method Using Landmark Behavior Rule Base
CN114882581B (en) Data enhancement and human body action recognition method for noise skeleton data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant