CN115393754A - A 3D Human Pose Recognition Method Based on Angle Vector Calculation and Neural Network Improvement - Google Patents

Abstract

The invention discloses a three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement, which comprises the steps of firstly carrying out frame extraction on an input original video file, and then extracting a video frame into a two-dimensional human body posture through a neural network OpenPose; secondly, the human body posture is converted into a three-dimensional human body posture by using an angular vector calculation method, so that the matching speed is greatly improved, and meanwhile, the three-dimensional human body posture which accords with human engineering is calculated according to the two-dimensional posture, so that the recognition precision is improved; finally, mapping the skeleton model to the original video frame, and forming a new human skeleton model file which can be used for secondary processing; the processing process is simple and quick, the step of the opera video fusion is simplified, and the opera video fusion is quicker, clearer and more harmonious.

Description

A 3D Human Pose Recognition Based on Angle Vector Calculation and Neural Network Improvement method

technical field

The invention belongs to the technical field of computer graphics processing, in particular to a three-dimensional human body posture recognition method based on angle vector calculation and neural network improvement.

Background technique

In 3D human pose recognition, the digital generation of virtual characters mainly uses related technologies of deep learning such as pose estimation. The basic idea is to use a certain geometric model or structure to represent the structure and shape of the object, and by extracting some object features, establish a correspondence between the model and the image, and then realize the estimation of the object's spatial attitude by geometric or other methods . The model used here may be a simple geometric body, such as a plane, a cylinder, or a certain geometric structure, or a three-dimensional model obtained by laser scanning or other methods.

Learning-based methods generally use global observation features, which can ensure that the algorithm has better robustness. However, the pose estimation accuracy of this class of methods largely depends on the adequacy of training. In order to obtain the corresponding relationship between two-dimensional observation and three-dimensional attitude more accurately, it is necessary to obtain enough dense samples to learn decision rules and regression functions. Generally speaking, the number of samples required increases exponentially with the dimension of the state space. For high-dimensional state spaces, it is actually impossible to obtain the dense sampling required for accurate estimation. Therefore, it is difficult to obtain dense sampling and ensure the accuracy and continuity of estimation, which is the fundamental difficulty that cannot be overcome by learning-based attitude estimation methods. Object recognition is a key technology. If the face cannot be recognized correctly, the human body positioning, and if the background cannot be recognized correctly, the human body may be distorted. Object recognition is a basic research in the field of computer vision. Its task is to identify what object is in the image and report the position and orientation of the object in the scene represented by the image.

At present, most of the existing 3D pose recognition methods need to produce good results in a specific environment or under a complex neural network model, which requires high material requirements, cumbersome processing, and long processing time, making it difficult to achieve real-time application effects. .

Contents of the invention

In order to solve the problems existing in the prior art, the object of the present invention is to provide a three-dimensional human body posture recognition method based on angle vector calculation and neural network improvement, the processing process is simple and fast, and the recognition accuracy is improved; the fusion of opera videos is faster , clearer and more harmonious.

In order to achieve the above object, the present invention adopts the following technical solutions to achieve:

A three-dimensional human gesture recognition method based on angle vector calculation and neural network improvement, comprising the following steps:

Step 1: Input the original video file and decompose it into video frames;

Step 2: Input the video frame in step 1 into the OpenPose algorithm for two-dimensional pose extraction;

Step 3: Normalize the two-dimensional pose extracted in step 2;

Step 4: Restore the normalized two-dimensional human body posture in step 3 to a three-dimensional human body posture through the angle vector calculation formula;

Step 5: By doing the above step 3 for each connected joint point of the human body, the three-dimensional coordinate positions of all relevant nodes are obtained; finally, all relevant nodes are connected into a tree-structured human skeleton feature map;

Step 6: Map the human skeleton feature map obtained in step 5 back to the original video file;

Step 7: Regenerate the 3D human pose video from the bone feature map processed in step 6.

Further, in the step 2, the video frames are normalized, and the pixel value of each frame is set to 1920×1080.

Further, in the step 2, the two-dimensional posture is normalized, and the method of obtaining the normalization matrix R is: take the π of the plane where the head joint point is located as the normal vector, and after rotation, use e _z as the normal vector , the rotation vector is A=(a _x , a _y , a _z ), and the rotation angle is θ; because π is perpendicular to the two-dimensional plane composed of other joint points, the principle that the product of the normal vector and any plane vector is 0 is used To get (π _x , π _y , π _z ), let π′, e _z ′ be unit normal vectors. The calculation method of the normalization process is the following formula:

The final R obtained is the normalization matrix, and the normalization matrix R is multiplied by the original two-dimensional coordinate matrix to obtain the two-dimensional human body posture corrected to face the screen;

Further, the angle vector calculation formula used in the step 4 is:

Compared with the prior art, the present invention has the following technical effects:

The invention discloses a three-dimensional human body posture recognition method based on angle vector calculation and neural network improvement. First, frame extraction is performed on an input original video file, and then the video frame is extracted into a two-dimensional human body posture through neural network OpenPose; secondly, using The angle vector calculation method converts the human body posture into a three-dimensional human body posture, which greatly improves the matching speed. At the same time, the ergonomic three-dimensional human body posture is calculated according to the two-dimensional posture, which improves the recognition accuracy; finally, the skeleton model Mapped to the original video frame, and form a new human skeleton model file, which can be used for secondary processing. The processing process is simple and fast, which simplifies the steps of opera video fusion and makes the opera video fusion faster, clearer and more harmonious.

The two-dimensional pose is normalized, and the tilted two-dimensional human pose is changed into a two-dimensional human pose facing the camera through normalization, so as to correctly obtain the clip between joint points in the three-dimensional human pose transformation. horn.

The invention conducts research on the three-dimensional human body pose recognition method in real scenes, and conducts research on problems such as model timing shake, environmental object occlusion, unclear character images, and detection speed encountered when rebuilding the model. Aiming at the problem of time series jitter and detection of unclear images of people, a two-stage pose recognition model is proposed, which uses the dependency relationship between temporal convolutional neural network and time frame sequence, and combines the information between the front and rear frames to effectively eliminate the time series jitter and blurred images of people , and the use of angle vector calculations to reconstruct 3D human poses helps to solve the problems of joint point occlusion and unreasonable generated poses.

Description of drawings

Fig. 1 is a flow chart of the present invention;

Fig. 2 is a normalized schematic diagram of the present invention;

Fig. 3 is angle vector calculation figure of the present invention;

Fig. 4 is a schematic diagram of gesture recognition of the present invention;

Detailed ways

The specific content of the present invention will be further explained in detail below in conjunction with the examples, but it is not intended to limit the present invention.

As shown in Figure 1, the present invention provides a three-dimensional human body gesture recognition method based on angle vector calculation and neural network improvement, which specifically includes the following steps:

Step 1: Input the original video file and decompose it into video frames;

Step 2: Input the video frame in step 1 into the OpenPose algorithm for two-dimensional pose extraction;

Step 3: Normalize the two-dimensional pose extracted in step 2, as shown in Figure 2;

The method of obtaining the normalization matrix R is: take the π of the plane where the head joint point is located as the normal vector, and after rotation, it becomes the plane with e _z as the normal vector, and the rotation vector is A=(a _x , a _y , a _z ), and the rotation angle is θ. Because π is perpendicular to the two-dimensional plane composed of other joint points, (π _x , π _y , π _z ) can be obtained by using the principle that the product of the normal vector and any vector on the plane is 0. Let π′, e′ _z be the unit normal vector, then formulas (1) and (2) can be obtained:

From this, the rotation matrix R can be obtained as:

Step 4: Restore the normalized two-dimensional human body posture in step 3 to a three-dimensional human body posture through the angle vector calculation formula;

After obtaining the frontal coordinates of the image, it is assumed that the limb does not tilt backward. The lengths of all interconnected joint points are then measured. Next, take the left shoulder joint point (1, 2) and the left arm joint point (2, 3) as examples to make a three-dimensional gesture recognition example, as shown in Figure 3.

For a two-dimensional plane, the angle θ between the joint point (1, 2) and the joint point (2, 3) is known, and can be obtained by calculating the angle between the vectors L ₁₂ and L ₂₃ out, as shown in Figure 3(a).

When joint point 3 is leaning forward, L ₁₂ remains unchanged, and the obtained two-dimensional output is actually L′ ₂₃ , where 3′ represents the mapping of joint point 3 after forward leaning. The length of L' ₂₃ and the length of L ₂₃ can be obtained according to the image distance between two frames. Thus it can be calculated that the angle between L' ₂₃ and L ₂₃ is θ', as shown in Figure 3(b), θ' can be calculated by formula (6):

Step 5: By performing the above operations on each connected joint point, the three-dimensional coordinate positions of all related nodes can be obtained. Finally, connect all relevant nodes into a tree-structured human skeleton feature map;

Step 6: Map the bone feature map obtained in step 5 back to the original video file;

Step 7: Regenerate the 3D human pose video from the bone feature map processed in step 6, as shown in Figure 4.

Claims (4)

1. A three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement is characterized by comprising the following steps:

step 1: inputting an original video file and decomposing the original video file into video frames;

and 2, step: inputting the video frame in the step 1 into an OpenPose algorithm for two-dimensional posture extraction;

and step 3: normalizing the two-dimensional posture extracted in the step 2;

and 4, step 4: restoring the two-dimensional human body posture normalized in the step 3 into a three-dimensional human body posture through an angular quantity calculation formula;

and 5: 3, acquiring the three-dimensional coordinate positions of all the joint points by performing the step 3 on each connected joint point of the human body; finally, connecting all the joint points into a human skeleton characteristic diagram with a tree structure;

step 6: mapping the human skeleton characteristic diagram obtained in the step 5 back to the original video file;

and 7: and (6) regenerating the three-dimensional human body posture video from the bone characteristic diagram processed in the step 6.

2. The method for three-dimensional human body posture recognition based on angular vector calculation and neural network improvement of claim 1, characterized in that: in the step 2, the video frames are normalized, and the pixel value of each frame is set to 1920 × 1080.

3. The method for recognizing the three-dimensional human body posture based on the angular vector calculation and the neural network improvement according to claim 1, wherein the two-dimensional posture is normalized in the step 2, and the obtaining method of the normalization matrix R is as follows: the pi of the plane of the head joint point is taken as a normal vector, and the pi is taken as e after rotation _z Is the plane of normal vector, and the rotation vector is A = (a) _x ,a _y ,a _z ) The rotation angle is theta; because pi is perpendicular to a two-dimensional plane formed by other joint points, the method is obtained by using the principle that the product of a normal vector and a plane arbitrary vector is 0 (pi) _x ,π _y ,π _z ) Let pi' e _z ' is a unit normal vector. The calculation method of the normalization process is as follows:

the finally obtained R is a normalization matrix, and the normalization matrix R is multiplied by the original two-dimensional coordinate matrix to obtain the two-dimensional human body posture which is corrected to be over against the screen;

4. the method for three-dimensional human body posture recognition based on angular vector calculation and neural network improvement of claim 1, characterized in that: the angle quantity calculation formula used in the step 4 is as follows:

Images