CN104715493B - A kind of method of movement human Attitude estimation - Google Patents

Abstract

The invention discloses a method for estimating the posture of a moving human body, which can accurately locate the skeleton points of the human body, effectively obtain the more expressive features of the three-dimensional moving human body, and make the construction more effective and simple. It includes: (1) using the median filter method to preprocess the depth image data, and using the Dijkstra algorithm based on geodesic distance to calibrate the human body pixels; (2) based on the K-means clustering algorithm Based on the regional feature point extraction algorithm, the number of clusters in each class is determined to be 3, and 32 posture features are extracted to represent different human postures; (3) In the training phase, the skeleton point position labeling information is obtained through the PoserPro2012 software, and the synthetic The posture features of 300 frames of virtual human are marked with standard skeleton points. Through the posture feature points of training samples and standard skeleton points, the linear regression model of posture features and skeleton points is calculated to obtain the mapping relationship between posture features and standard skeleton points. .

Description

A Method for Pose Estimation of Moving Human Body

technical field

The invention belongs to the technical field of computer vision and pattern recognition, and in particular relates to a method for gesture estimation of a moving human body.

Background technique

Human pose estimation is an important research direction in the field of computer vision. In the past ten years, the problem of automatically recognizing human poses in image and video sequences has been a research hotspot in the field of computer vision. The main reason for human body pose estimation to become the focus of research is the rapid development of electronic equipment and the huge application market generated by it. Effective processing and understanding of human activities in data will have a profound impact on the development of human society. The purpose of sports human behavior analysis is to describe, recognize and understand human actions, human-human and human-environment interactions, which can be used in intelligent video surveillance, virtual reality, security, advanced human-computer interaction and content-based images. It has a wide range of application backgrounds in storage and retrieval.

There are two dimensions of human body pose representation: two-dimensional and three-dimensional. Two-dimensional human body posture refers to a description of the distribution of human body joints on the two-dimensional plane of the image. The human body posture estimation of traditional two-dimensional images is greatly affected by surrounding environmental factors (clothes color, light) and occlusion. At the same time, there is a lack of pixel Spatial location information. Among the current human pose estimation methods for two-dimensional images, the methods based on graph structure models and their improvements occupy an overwhelming advantage. The graph structure model uses the graph model structure to represent the connection between components. The graph structure model divides the human body into multiple rigid parts (head, torso, a pair of upper arms, a pair of lower arms, a pair of thighs, a pair of calves, etc.), and each part is represented by a rectangular frame; the two parts are separated by The joints are connected (as shown in Figure 1a). A rectangular box corresponding to a human body part can be represented as a vector

L=(x,y,r,s,w) (1)

Where (x, y) represents the center position of the rectangle, r represents the angle of the rectangle relative to the vertical direction, s represents the length of the rectangle, and w represents the width of the rectangle. The human tree diagram model (as shown in Figure 1b) can be represented as an undirected graph

G=(V,E) (2)

Where E is the set of all edges in the graph, each element in the vertex set V={v ₁ ,v ₂ ,v ₃ ,...,v _n } is the corresponding rigid part of the human body, if two human body parts v _i If it is connected with v _j , there is an edge (v _i , v _j )∈E. Human pose estimation based on graphical models requires the representation of design features, part detection, and handling complex topologies. Therefore, the method based on the graphical model is not a very efficient method for human pose estimation. For the two-dimensional human pose estimation problem, Alexander Toshev et al. proposed a DNN-based human pose estimation method in 2014, which formalized the human pose estimation problem as a regression problem of joint points. Human pose estimation based on deep convolutional neural network has obvious defects: it is only for RGB images, no depth data is used, and the network structure used by this method is very complex (as shown in Figure 2), and the training efficiency is low.

At present, 3D human pose estimation methods can be roughly divided into two categories. One is the method based on geodesic distance and optical flow represented by the method proposed by Loren Arthur Schwarz et al. The defect of this type of algorithm is that it is limited to the secondary skeleton points (elbow, knee, neck, shoulder, crotch). The positioning adopts the proportional positioning method, so the effect is not very ideal for people of different body types. At the same time, the application of optical flow requires a large amount of calculation and it is difficult to meet the occasions with high real-time requirements. Another type of method is a clustering-based method, represented by the random forest-based clustering method proposed by Jamie Shotton. In this type of method, each skeleton point is the regression value of all pixels, and the model is complex and requires a large number of training samples. , through a large number of supervised training processes, it is possible to ideally determine the weight of the pixel for each skeleton point.

Contents of the invention

The problem solved by the technology of the present invention is to overcome the deficiencies of the prior art and provide a method for estimating the pose of a moving human body, which can accurately locate the skeleton points of the human body, effectively obtain more expressive features in the three-dimensional moving human body, and construct more To be effective and simple.

Technical solution of the present invention is: the method for this motion human body attitude estimation, comprises the following steps:

(1) The depth image data is preprocessed by the median filter method, and the human body pixels are calibrated by the Dijkstra algorithm based on the geodesic distance;

(2) Based on the regional feature point extraction algorithm of the K-means clustering algorithm, the number of clusters in each class is determined to be 3, and 32 posture features are extracted to represent different human postures;

(3) In the training phase, the skeleton point position labeling information was obtained through Poser Pro 2012 software, and the posture features of 300 frames of virtual human were synthesized and the standard skeleton points were marked. The posture features and skeleton were calculated through the posture feature points and standard skeleton points of the training samples Point linear regression model in order to obtain the mapping relationship between pose features and standard skeleton points.

The invention can very accurately locate the first-level skeleton points (limbs and head) of the human body through the single source point Dijkstra algorithm based on the geodesic distance. In order to locate the secondary skeleton points of the human body more conveniently and effectively (neck, elbow, knee joint, hip), etc., a feature extraction method based on clustering is proposed. This feature selection method seeks an efficient feature and skeleton The mapping relationship between the points makes the method have higher processing efficiency and can accurately locate the secondary skeleton points of the human body. This method starts from the way of clustering, which can effectively obtain the more expressive features of the three-dimensional moving human body, and the construction is more effective and simple.

Description of drawings

Figure 1a is the human body structure, and Figure 1b is the graph structure model.

Figure 2 is a schematic diagram of the DNN model structure.

detailed description

The method for estimating the posture of a moving human body includes the following steps:

(1) The depth image data is preprocessed by the median filter method, and the human body pixels are calibrated by the Dijkstra algorithm based on the geodesic distance;

(2) Based on the regional feature point extraction algorithm of the K-means clustering algorithm, the number of clusters in each class is determined to be 3, and 32 posture features are extracted to represent different human postures;

(3) In the training phase, the skeleton point position labeling information was obtained through Poser Pro 2012 software, and the posture features of 300 frames of virtual human were synthesized and the standard skeleton points were marked. The posture features and skeleton were calculated through the posture feature points and standard skeleton points of the training samples Point linear regression model in order to obtain the mapping relationship between pose features and standard skeleton points.

The invention can very accurately locate the first-level skeleton points (limbs and head) of the human body through the single source point Dijkstra algorithm based on the geodesic distance. In order to locate the secondary skeleton points of the human body more conveniently and effectively (neck, elbow, knee joint, hip), etc., a feature extraction method based on clustering is proposed. This feature selection method seeks an efficient feature and skeleton The mapping relationship between the points makes the method have higher processing efficiency and can accurately locate the secondary skeleton points of the human body. This method starts from the way of clustering, which can effectively obtain the more expressive features of the three-dimensional moving human body, and the construction is more effective and simple.

Preferably, the preprocessing operations of the step (1) include smoothing, denoising, alignment and background removal to obtain the human body pixels in the depth image, and use the Dijkstra algorithm based on geodesic distance to carry out part calibration on the human body pixels: Based on the pixel representation of the graph structure, using the single source point Dijkstra algorithm based on the geodesic distance, the geometric center point of the pixel of the human body is used as the source point for calculation, and the positions of the left and right hands, left and right feet, and the extremity points of the head are marked ; In the single-source point Dijkstra algorithm, the initialization of the adjacency matrix with adjacent edges satisfies the formula (3):

(x _ij , x _kt )∈V _t ×V _t , ||x _ij -x _kt || ₂ <δ∧|ik|≤1∧|jt|≤1 (3)

where x _ij represents the depth value of the pixel at position (i, j) in the image, V _t is the set of human body pixels, and δ represents the statistical experience value of connected pixels; the five limbs of the human body are calculated by the single-source point Dijkstra algorithm Endpoints, using the nearest neighbor classification algorithm based on geodesic distance, with extremity endpoints and geometric centers as the centroids, to achieve human body area division; there are 6 types of pattern categories represent the extremities and the torso of the human body, respectively, where is the trunk part, according to the formulas (4), (9) to get

in The subscript i means class, c means The geometric center point pixel in the class,

||xy|| _geodesic represents the geodesic distance between pixels.

Preferably, the feature point extraction in the step (2) is based on the K-means clustering algorithm of the human body region:

Pose feature extraction: Using the K-means clustering algorithm based on the human body part area, the five human body parts obtained Inside, the K-means clustering algorithm with three clusters is used to extract the cluster center location features of these five human body parts.

Dimensions of features: at the five acral point sites Inside, the K-means clustering algorithm with three clusters is used to obtain 15 human body posture feature points; in order to obtain the global description of human body posture, the geometric center point of human body pixels is added to the human body posture feature points as the global description feature , the number of feature points of the human body posture is 16, the feature points include their two-dimensional coordinate values, and the feature dimension is 32.

Preferably, in said step (3), a sparse linear projection matrix B is obtained according to formulas (5)-(7), so that the error of predicting Y from X is small

Where X={x ₁ ,...,x _n } represents the feature point sample set of n samples obtained by clustering, each

Sample x _i ={x _i1 , x _i2 ... x _im }, where i∈{1,2,3...,n}, m is the number of extracted features 32.

Y={y ₁ ,...,y _n } means n samples composed of skeleton point labels corresponding to training samples, each sample y _i ={y _i1 ,y _i2 ...y _it }, where i∈ {1,2,3...,n}, t is the number of standard skeleton point coordinates of human body posture.

In order to ensure the rotation invariance of the subspace distance measure, a rotation matrix R is added to the model, and the problem of extracting human skeleton points is formalized as a rotation sparse regression problem. Preferably, in the step (3), according to the formula (8), the extraction of human skeleton points is formalized as a rotation sparse regression problem solution,

The invention is applied to the depth image data obtained by Kinect2 and the virtual data exported by Poser Pro 2012, and has achieved obvious effects. In the experiment, a 640×480 RGBD image is selected, the collection environment is indoor, and the collection light is fluorescent lamp.

In the synthetic data test set Poser Pro 2012, a very high accuracy has been achieved. 100 frames of depth image data were tested, and the root mean square error (RMS) and maximum error (Max) of the synthetic data were counted. The values are in pixels. . RMS=10.1305 The maximum joint Euclidean distance error is 37.3363, it can be seen that the synthetic data has a good sense of reality and accuracy. Compared with the random forest method proposed by Jamie Shotton et al., the present invention has achieved ideal improvement in both RMS and MAX standards.

On the actual depth image test data obtained by Kinect2, this method also achieved ideal results.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention in any form. Any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention are still within the scope of this invention. The protection scope of the technical solution of the invention.

Claims (4)

1. A kind of 1. method of movement human Attitude estimation, it is characterised in that：Comprise the following steps：

   (1) pretreatment operation is carried out to depth image data using the method for medium filtering, using the Di Jie based on geodesic distance Si Tela algorithms carry out position demarcation to human body pixel；

   (2) the provincial characteristics point extraction algorithm based on K- means clustering algorithms, it is determined that the cluster number in each class is 3, carry 32 dimension posture features are taken to characterize different human body attitudes；

   (3) skeletal point position markup information is obtained by the softwares of Poser Pro 2012 in the training stage, synthesizes 300 frame visual humans Posture feature and be labelled with standard skeletal point, by the posture feature point of training sample and standard skeletal point, it is special to calculate posture Linear regression model (LRM) of the sign point with skeletal point, to obtain the mapping relations between posture feature and standard skeletal point；

   The pretreatment operation of the step (1) includes smooth, denoising, alignment and background and rejects the human body picture obtained in depth image Element, position demarcation is carried out to human body pixel using based on the Dijkstra's algorithm of geodesic distance：Pixel table based on graph structure Show, using single source point Dijkstra's algorithm based on geodesic distance, using the geometric center point of human body pixel as source point, carry out Calculate, indicate the limb endpoint location of the right-hand man of human body, left and right pin and head；Adjoining is initialized in single source point Dijkstra's algorithm The construction that matrix has adjacent side meets formula (3)：

   (x_ij, x_kt)∈V_t×V_t, | | x_ij-x_kt||₂＜ δ ∧ | i-k |≤1 ∧ | j-t |≤1 (3)

   Wherein x_ijRepresent the pixel depth value in (i, j) position, V in image_tIt is human body pixel set, δ represents connected pixel Statistics empirical value；5 acra points of human body are asked for by single source point Dijkstra's algorithm, using based on the nearest of geodesic distance Adjacent sorting algorithm, using acra point and geometric center as the class heart, realize that human region divides；Pixel classification shares 6 classesI= 1,2 ..., 6, human body acra point position and trunk are represented respectively, whereinFor metastomium, obtained according to formula (4), (9)

   <mrow> <msub> <mi>g</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mrow> <mi>k</mi> <mi>t</mi> </mrow> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> <mi>k</mi> </munder> <mo>|</mo> <mo>|</mo> <msub> <mi>x</mi> <mrow> <mi>k</mi> <mi>t</mi> </mrow> </msub> <mo>-</mo> <msubsup> <mi>x</mi> <mi>i</mi> <mi>c</mi> </msubsup> <mo>|</mo> <msub> <mo>|</mo> <mrow> <mi>g</mi> <mi>e</mi> <mi>o</mi> <mi>d</mi> <mi>e</mi> <mi>s</mi> <mi>i</mi> <mi>c</mi> </mrow> </msub> <mo>,</mo> <mi>i</mi> <mo>=</mo> <mn>1</mn> <mo>,</mo> <mo>...</mo> <mo>,</mo> <mn>6</mn> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>4</mn> <mo>)</mo> </mrow> </mrow>

   <mrow> <msub> <mover> <mi>&amp;omega;</mi> <mo>&amp;OverBar;</mo> </mover> <mi>j</mi> </msub> <mo>=</mo> <mo>{</mo> <msub> <mi>x</mi> <mrow> <mi>k</mi> <mi>t</mi> </mrow> </msub> <mo>:</mo> <msub> <mi>g</mi> <mi>j</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mrow> <mi>k</mi> <mi>t</mi> </mrow> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mrow> <mi>m</mi> <mi>i</mi> <mi>n</mi> </mrow> <mrow> <mn>1</mn> <mo>&amp;le;</mo> <mi>i</mi> <mo>&amp;le;</mo> <mn>6</mn> </mrow> </munder> <msub> <mi>g</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mrow> <mi>k</mi> <mi>t</mi> </mrow> </msub> <mo>)</mo> </mrow> <mo>}</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>9</mn> <mo>)</mo> </mrow> </mrow>

   WhereinSubscript i representClass, c are representedGeometric center point pixel in class,

   ||x-y||_geodesicRepresent the geodesic distance between pixel.
2. 2. the method for movement human Attitude estimation according to claim 1, it is characterised in that：Spy in the step (2) Sign point extraction is the K- means clustering algorithms based on human region：

   Posture feature extracts：Using the K- means clustering algorithms based on human body region, in five obtained human bodies In j=1 ..., 5, cluster number is used to calculate cluster centre position to this five human bodies for three K- means clustering algorithms Put,

   Characteristic dimension：At five acra point positionsIn j=1 ..., 5, cluster number is used to be calculated for three K- mean clusters Method, obtain 15 human body posture feature points；In order to obtain the global description of human body attitude, people is added in human body attitude characteristic point The geometric center point of volumetric pixel is 16 as global description's feature, human body attitude characteristic point, and characteristic point includes its two-dimensional coordinate value, Characteristic dimension is 32.
3. 3. the method for movement human Attitude estimation according to claim 2, it is characterised in that：Basis in the step (3) Formula (5)-(7) obtain a sparse linear projection matrix B, to minimize the prediction for mapping to obtain by projection matrix B by X The error of characteristic point position and fact characteristic point position Y is object function

   <mrow> <mi>X</mi> <mo>=</mo> <mfenced open = "(" close = ")"> <mtable> <mtr> <mtd> <msub> <mi>x</mi> <mn>11</mn> </msub> </mtd> <mtd> <msub> <mi>x</mi> <mn>12</mn> </msub> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <msub> <mi>x</mi> <mrow> <mn>1</mn> <mi>m</mi> </mrow> </msub> </mtd> </mtr> <mtr> <mtd> <msub> <mi>x</mi> <mn>21</mn> </msub> </mtd> <mtd> <msub> <mi>x</mi> <mn>22</mn> </msub> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <msub> <mi>x</mi> <mrow> <mn>2</mn> <mi>m</mi> </mrow> </msub> </mtd> </mtr> <mtr> <mtd> <mo>...</mo> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <mo>...</mo> </mtd> </mtr> <mtr> <mtd> <msub> <mi>x</mi> <mrow> <mi>n</mi> <mn>1</mn> </mrow> </msub> </mtd> <mtd> <msub> <mi>x</mi> <mrow> <mi>n</mi> <mn>2</mn> </mrow> </msub> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <msub> <mi>x</mi> <mrow> <mi>n</mi> <mi>m</mi> </mrow> </msub> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>5</mn> <mo>)</mo> </mrow> </mrow>

   <mrow> <mi>Y</mi> <mo>=</mo> <mfenced open = "(" close = ")"> <mtable> <mtr> <mtd> <msub> <mi>y</mi> <mn>11</mn> </msub> </mtd> <mtd> <msub> <mi>y</mi> <mn>12</mn> </msub> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <msub> <mi>y</mi> <mrow> <mn>1</mn> <mi>t</mi> </mrow> </msub> </mtd> </mtr> <mtr> <mtd> <msub> <mi>y</mi> <mn>21</mn> </msub> </mtd> <mtd> <msub> <mi>y</mi> <mn>22</mn> </msub> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <msub> <mi>y</mi> <mrow> <mn>2</mn> <mi>t</mi> </mrow> </msub> </mtd> </mtr> <mtr> <mtd> <mo>...</mo> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <mo>...</mo> </mtd> </mtr> <mtr> <mtd> <msub> <mi>y</mi> <mrow> <mi>n</mi> <mn>1</mn> </mrow> </msub> </mtd> <mtd> <msub> <mi>y</mi> <mrow> <mi>n</mi> <mn>2</mn> </mrow> </msub> </mtd> <mtd> <mo>...</mo> </mtd> <mtd> <msub> <mi>y</mi> <mrow> <mi>n</mi> <mi>t</mi> </mrow> </msub> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>6</mn> <mo>)</mo> </mrow> </mrow>

   Wherein X={ x₁..., x_nRepresent to cluster the characteristic point sample set of n obtained sample, each sample x_i={ x_i1, x_i2...x_im, wherein i ∈ { 1,2,3..., n }, m=32 be extraction Characteristic Number, Y={ y₁..., y_nRepresent training sample N sample of corresponding skeletal point label composition, each sample y_i={ y_i1, y_i2...y_it, wherein i ∈ 1,2,3..., N }, t is the number of the standard skeleton point coordinates of human body attitude.
4. 4. the method for movement human Attitude estimation according to claim 3, it is characterised in that：Basis in the step (3) Formula (8) turns to the solution of rotation sparse regression problem by human skeleton point form is extracted,