CN115393754A - Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement - Google Patents

Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement Download PDF

Info

Publication number
CN115393754A
CN115393754A CN202210739379.2A CN202210739379A CN115393754A CN 115393754 A CN115393754 A CN 115393754A CN 202210739379 A CN202210739379 A CN 202210739379A CN 115393754 A CN115393754 A CN 115393754A
Authority
CN
China
Prior art keywords
human body
dimensional
body posture
dimensional human
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210739379.2A
Other languages
Chinese (zh)
Inventor
邹嘉祥
刘喆
常欣雨
田琦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwest University
Original Assignee
Northwest University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwest University filed Critical Northwest University
Priority to CN202210739379.2A priority Critical patent/CN115393754A/en
Publication of CN115393754A publication Critical patent/CN115393754A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Computational Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Algebra (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement, which comprises the steps of firstly carrying out frame extraction on an input original video file, and then extracting a video frame into a two-dimensional human body posture through a neural network OpenPose; secondly, the human body posture is converted into a three-dimensional human body posture by using an angular vector calculation method, so that the matching speed is greatly improved, and meanwhile, the three-dimensional human body posture which accords with human engineering is calculated according to the two-dimensional posture, so that the recognition precision is improved; finally, mapping the skeleton model to the original video frame, and forming a new human skeleton model file which can be used for secondary processing; the processing process is simple and quick, the step of the opera video fusion is simplified, and the opera video fusion is quicker, clearer and more harmonious.

Description

Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement
Technical Field
The invention belongs to the technical field of computer graphic processing, and particularly relates to a three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement.
Background
In the three-dimensional human body posture recognition, the digitalized generation virtual character mainly uses the related technology of deep learning such as posture estimation and the like. The basic idea is to use a certain geometric model or structure to represent the structure and shape of an object, and to establish a corresponding relationship between the model and an image by extracting certain object features, and then to realize estimation of the spatial attitude of the object by geometric or other methods. The model used here may be either a simple geometric form, such as a plane, a cylinder, or some geometry, or a three-dimensional model obtained by laser scanning or other methods.
The learning-based method generally adopts global observation features, and can ensure that the algorithm has better robustness. However, the accuracy of pose estimation in this class of methods depends largely on the sufficiency of the training. To obtain the correspondence between the two-dimensional observations and the three-dimensional poses relatively accurately, samples that are dense enough must be obtained to learn the decision rules and regression functions. While the number of samples required generally increases exponentially with the dimension of the state space, for high dimensional state spaces it is virtually impossible to obtain the dense sampling required for accurate estimation. Therefore, it is difficult to obtain dense samples and to ensure the accuracy and continuity of estimation, which is a fundamental difficulty that the learning-based attitude estimation method cannot overcome. Object recognition is a key technology, and if a human face cannot be correctly recognized, a human body is positioned, and if a background cannot be correctly recognized, the human body may be distorted. Object recognition is a fundamental research in the field of computer vision, whose task is to identify what object is in an image and to report the position and orientation of this object in the scene represented by the image.
At present, most of the existing three-dimensional gesture recognition methods can generate a good effect only under a specific environment or a complex neural network model, the requirements on materials are high, the processing process is complicated, the processing time is long, and the effect of real-time application is difficult to achieve.
Disclosure of Invention
In order to solve the problems in the prior art, the invention aims to provide a three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement, the processing process is simple and quick, and the recognition precision is improved; the method ensures that the opera video is fused more quickly, clearly and harmoniously.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
a three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement comprises the following steps:
step 1: inputting an original video file and decomposing the original video file into video frames;
and 2, step: inputting the video frame in the step 1 into an OpenPose algorithm for two-dimensional posture extraction;
and step 3: normalizing the two-dimensional posture extracted in the step 2;
and 4, step 4: restoring the two-dimensional human body posture normalized in the step 3 into a three-dimensional human body posture through an angular quantity calculation formula;
and 5: 3, acquiring the three-dimensional coordinate positions of all the joint points by performing the step 3 on each connected joint point of the human body; finally, connecting all the joint points into a human skeleton characteristic diagram with a tree structure;
step 6: mapping the human skeleton characteristic diagram obtained in the step 5 back to the original video file;
and 7: and (6) regenerating the three-dimensional human body posture video from the bone characteristic diagram processed in the step 6.
Further, the video frames in step 2 are normalized, and the pixel value of each frame is set to 1920 × 1080.
Further, in the step 2, the two-dimensional posture is normalized, and the method for obtaining the normalization matrix R is as follows: taking pi of the plane where the head joint point is located as a normal vector, and taking e after rotation z Is the plane of normal vector, and the rotation vector is A = (a) x ，a y ，a z ) The rotation angle is theta; because pi is perpendicular to a two-dimensional plane formed by other joint points, the (pi) is obtained by using the principle that the product of a normal vector and an arbitrary vector of the plane is 0 x ，π y ，π z ) Let pi' e z ' is a unit normal vector. The calculation method of the normalization process is as follows:
Figure BDA0003717111490000021
Figure BDA0003717111490000022
Figure BDA0003717111490000023
the finally obtained R is a normalization matrix, and the normalization matrix R is multiplied by the original two-dimensional coordinate matrix to obtain the two-dimensional human body posture which is corrected to be over against the screen;
Figure BDA0003717111490000031
Figure BDA0003717111490000032
further, the angle quantity calculation formula used in step 4 is as follows:
Figure BDA0003717111490000033
compared with the prior art, the invention has the following technical effects:
the invention discloses a three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement, which comprises the steps of firstly carrying out frame extraction on an input original video file, and then extracting a video frame into a two-dimensional human body posture through a neural network OpenPose; secondly, the human body posture is converted into a three-dimensional human body posture by using an angular vector calculation method, so that the matching speed is greatly improved, and meanwhile, the three-dimensional human body posture which accords with human engineering is calculated according to the two-dimensional posture, so that the recognition precision is improved; finally, the skeleton model is mapped to the original video frame, and a new human skeleton model file is formed and can be used for secondary processing. The processing process is simple and quick, the step of the opera video fusion is simplified, and the opera video fusion is quicker, clearer and more harmonious.
And carrying out normalization processing on the two-dimensional postures, and changing the inclined two-dimensional human body posture into a two-dimensional human body posture which is over against the lens through the normalization processing so as to correctly acquire an included angle between the joint points in the three-dimensional human body posture conversion.
According to the method for recognizing the three-dimensional human body posture of the real scene, the problems of model time sequence jitter, environmental object shielding, unclear figure images, detection speed and the like in model reconstruction are researched. Aiming at the problems of time sequence jitter and unclear image detection of a person, a two-stage posture recognition model is provided, the problems of time sequence jitter and image blurring of the person are effectively eliminated by utilizing the dependency relationship between a time convolution neural network and a time frame sequence and combining information between front and back frames, and the three-dimensional human body posture is reconstructed by adopting angular vector calculation, so that the problems of joint point shielding and unreasonable generated posture are solved.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a schematic diagram of the normalization of the present invention;
FIG. 3 is an angle quantity calculation diagram of the present invention;
FIG. 4 is a schematic of gesture recognition of the present invention;
Detailed Description
The following examples are given to illustrate the present invention in further detail, but are not intended to limit the scope of the present invention.
As shown in fig. 1, the present invention provides a three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement, which specifically comprises the following steps:
step 1: inputting an original video file and decomposing the original video file into video frames;
step 2: inputting the video frame in the step 1 into an OpenPose algorithm to extract a two-dimensional gesture;
and step 3: normalizing the two-dimensional posture extracted in the step 2, as shown in fig. 2;
the method for acquiring the normalization matrix R comprises the following steps: taking pi of the plane where the head joint point is located as a normal vector, and taking e after rotation z Is the plane of normal vector, and the rotation vector is A = (a) x ，a y ，a z ) The rotation angle is θ. Because pi is perpendicular to a two-dimensional plane formed by other joint points, the (pi) can be obtained by using the principle that the product of a normal vector and any vector of the plane is 0 x ，π y ，π z ). Let pi ', e' z As a unit normal vector, equations (1) and (2) can be obtained:
Figure BDA0003717111490000041
Figure BDA0003717111490000042
from this, the rotation matrix R can be obtained as:
Figure BDA0003717111490000043
Figure BDA0003717111490000044
Figure BDA0003717111490000051
and 4, step 4: restoring the two-dimensional human body posture normalized in the step 3 into a three-dimensional human body posture by an angular quantity calculation formula;
after obtaining the image frontal coordinates, it is assumed that the limb does not tilt backwards. The length of all interconnected joints is then measured. Next, taking the left shoulder joint points (1, 2) and the left arm joint points (2, 3) as examples, an example of three-dimensional posture recognition is made, as shown in fig. 3.
For a two-dimensional plane, the angle θ between the joint point (1, 2) and the joint point (2, 3), which is known, can be calculated by calculating the vector L 12 And L 23 The angle therebetween is obtained as shown in fig. 3 (a).
When the node 3 is anteverted, L 12 While remaining unchanged, the resulting two-dimensional output is actually L' 23 Where 3' represents the mapping of the anteversion posterior joint point 3. L 'can be obtained according to image distance between two frames' 23 Length and L of 23 Length of (d). From this L 'can be calculated' 23 And L 23 The included angle therebetween is θ ', as shown in fig. 3 (b), θ' can be calculated by equation (6):
Figure BDA0003717111490000052
and 5: the three-dimensional coordinate positions of all the joint points can be obtained by performing the operation on each connected joint point. Finally, connecting all the joint points into a human skeleton characteristic diagram with a tree structure;
step 6: mapping the bone characteristic graph obtained in the step 5 back to the original video file;
and 7: and (4) regenerating the bone feature map processed in the step 6 into a three-dimensional human body posture video, as shown in fig. 4.

Claims (4)

1. A three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement is characterized by comprising the following steps:
step 1: inputting an original video file and decomposing the original video file into video frames;
and 2, step: inputting the video frame in the step 1 into an OpenPose algorithm for two-dimensional posture extraction;
and step 3: normalizing the two-dimensional posture extracted in the step 2;
and 4, step 4: restoring the two-dimensional human body posture normalized in the step 3 into a three-dimensional human body posture through an angular quantity calculation formula;
and 5: 3, acquiring the three-dimensional coordinate positions of all the joint points by performing the step 3 on each connected joint point of the human body; finally, connecting all the joint points into a human skeleton characteristic diagram with a tree structure;
step 6: mapping the human skeleton characteristic diagram obtained in the step 5 back to the original video file;
and 7: and (6) regenerating the three-dimensional human body posture video from the bone characteristic diagram processed in the step 6.
2. The method for three-dimensional human body posture recognition based on angular vector calculation and neural network improvement of claim 1, characterized in that: in the step 2, the video frames are normalized, and the pixel value of each frame is set to 1920 × 1080.
3. The method for recognizing the three-dimensional human body posture based on the angular vector calculation and the neural network improvement according to claim 1, wherein the two-dimensional posture is normalized in the step 2, and the obtaining method of the normalization matrix R is as follows: the pi of the plane of the head joint point is taken as a normal vector, and the pi is taken as e after rotation z Is the plane of normal vector, and the rotation vector is A = (a) x ,a y ,a z ) The rotation angle is theta; because pi is perpendicular to a two-dimensional plane formed by other joint points, the method is obtained by using the principle that the product of a normal vector and a plane arbitrary vector is 0 (pi) x ,π y ,π z ) Let pi' e z ' is a unit normal vector. The calculation method of the normalization process is as follows:
Figure FDA0003717111480000011
Figure FDA0003717111480000012
Figure FDA0003717111480000013
the finally obtained R is a normalization matrix, and the normalization matrix R is multiplied by the original two-dimensional coordinate matrix to obtain the two-dimensional human body posture which is corrected to be over against the screen;
Figure FDA0003717111480000021
Figure FDA0003717111480000022
4. the method for three-dimensional human body posture recognition based on angular vector calculation and neural network improvement of claim 1, characterized in that: the angle quantity calculation formula used in the step 4 is as follows:
Figure FDA0003717111480000023
CN202210739379.2A 2022-06-28 2022-06-28 Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement Pending CN115393754A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210739379.2A CN115393754A (en) 2022-06-28 2022-06-28 Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210739379.2A CN115393754A (en) 2022-06-28 2022-06-28 Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement

Publications (1)

Publication Number Publication Date
CN115393754A true CN115393754A (en) 2022-11-25

Family

ID=84116636

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210739379.2A Pending CN115393754A (en) 2022-06-28 2022-06-28 Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement

Country Status (1)

Country Link
CN (1) CN115393754A (en)

Similar Documents

Publication Publication Date Title
Zhang et al. Interacting two-hand 3d pose and shape reconstruction from single color image
CN109636831B (en) Method for estimating three-dimensional human body posture and hand information
Yu et al. Bodyfusion: Real-time capture of human motion and surface geometry using a single depth camera
Yamany et al. Surface signatures: an orientation independent free-form surface representation scheme for the purpose of objects registration and matching
JP7015152B2 (en) Processing equipment, methods and programs related to key point data
Wang et al. Sequential 3D human pose and shape estimation from point clouds
CN109829972B (en) Three-dimensional human standard skeleton extraction method for continuous frame point cloud
CN107240129A (en) Object and indoor small scene based on RGB D camera datas recover and modeling method
CN109325995B (en) Low-resolution multi-view hand reconstruction method based on hand parameter model
CN112530019A (en) Three-dimensional human body reconstruction method and device, computer equipment and storage medium
Li et al. Hybrik-x: Hybrid analytical-neural inverse kinematics for whole-body mesh recovery
Lee et al. Im2hands: Learning attentive implicit representation of interacting two-hand shapes
Cao et al. Human posture recognition using skeleton and depth information
CN116342621A (en) Geometric parameter identification method and system based on three-dimensional reconstruction of space moving target
Li et al. Image-guided human reconstruction via multi-scale graph transformation networks
Xu et al. 3D joints estimation of the human body in single-frame point cloud
Chen et al. 3D reconstruction of unstructured objects using information from multiple sensors
WO2022018811A1 (en) Three-dimensional posture of subject estimation device, three-dimensional posture estimation method, and program
CN117218192A (en) Weak texture object pose estimation method based on deep learning and synthetic data
Fan et al. Full face-and-head 3D model with photorealistic texture
CN112435345B (en) Human body three-dimensional measurement method and system based on deep learning
CN114548224B (en) 2D human body pose generation method and device for strong interaction human body motion
CN115393754A (en) Three-dimensional human body posture recognition method based on angular vector calculation and neural network improvement
Jin et al. DOPE++: 6D pose estimation algorithm for weakly textured objects based on deep neural networks
CN107341476A (en) A kind of unsupervised manikin construction method based on system-computed principle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination