CN110544302A - Human body action reconstruction system and method based on multi-view vision and action training system - Google Patents
Human body action reconstruction system and method based on multi-view vision and action training system Download PDFInfo
- Publication number
- CN110544302A CN110544302A CN201910844266.7A CN201910844266A CN110544302A CN 110544302 A CN110544302 A CN 110544302A CN 201910844266 A CN201910844266 A CN 201910844266A CN 110544302 A CN110544302 A CN 110544302A
- Authority
- CN
- China
- Prior art keywords
- human body
- dimensional
- action
- motion
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009471 action Effects 0.000 title claims abstract description 138
- 238000012549 training Methods 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000033001 locomotion Effects 0.000 claims description 143
- 238000003384 imaging method Methods 0.000 claims description 54
- 238000011156 evaluation Methods 0.000 claims description 39
- 239000013598 vector Substances 0.000 claims description 21
- 239000011159 matrix material Substances 0.000 claims description 17
- 230000003287 optical effect Effects 0.000 claims description 17
- 238000012937 correction Methods 0.000 claims description 15
- 238000013519 translation Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 11
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 230000033764 rhythmic process Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 8
- 230000006872 improvement Effects 0.000 description 6
- 238000013500 data storage Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000003550 marker Substances 0.000 description 3
- 230000036544 posture Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Graphics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
the application discloses a human body action reconstruction system, a human body action reconstruction method and an action training system based on multi-view vision.
Description
Technical Field
The application relates to the technical field of motion recognition, in particular to a human motion reconstruction system and method based on multi-view vision and a motion training system.
background
with the development of artificial intelligence, the human body gesture recognition technology makes an important breakthrough on the basis of a larger data set and strong computing power available in the big data era.
the existing three-dimensional human body motion recognition method is to capture the three-dimensional motion of a target human body by using an optical capture technology to perform motion recognition, and a user wears a special optical capture suit with an optical marker or a corresponding detection sensor to acquire a corresponding marker position to generate a spatial relative position, so that a human body three-dimensional model is constructed to perform motion recognition. The method for capturing the three-dimensional motion of the target human body by using the optical capturing technology to perform motion recognition needs a user to wear a special optical capturing garment or assemble a corresponding sensor, is troublesome to wear, has large load, causes inconvenience to motion training, and affects user experience.
Disclosure of Invention
The application aims to provide a human body action reconstruction system and method based on multi-view vision and an action training system, and the system and the method are used for solving the technical problems that in the existing three-dimensional human body action recognition method, a user needs to wear a specially-made optical capture garment or assemble a corresponding sensor, the wearing is troublesome, the load of the user is large, the action training is inconvenient, and the user experience is influenced.
The application provides a human action system of rebuilding based on many meshes vision in the first aspect, includes:
The camera calibration module is used for calibrating monocular cameras according to the collected calibration point data, and the number of the monocular cameras is at least two;
the action acquisition module is used for acquiring action image sequences of the target human body acquired by all the monocular cameras and sending the action image sequences to the two-dimensional human body action recognition module;
the two-dimensional human body motion recognition module is used for recognizing key human body joint parts in the motion image sequence, extracting two-dimensional joint points which belong to the same target human body in each frame of the motion image sequence so as to reconstruct a human body skeleton of the target human body, and storing and sending the human body skeleton information of each frame to the three-dimensional motion reconstruction module;
the three-dimensional action reconstruction module is used for restoring the real position of the two-dimensional joint point of the target human body in a three-dimensional space based on the human body skeleton information in each frame, and reconstructing the three-dimensional action of the target human body.
Optionally, the method further includes: an imaging point error correction module;
The imaging point error correction module is used for correcting the pixel coordinates of the two-dimensional joint points in the two-dimensional human body action recognition module when the monocular camera deviates, so that the two-dimensional human body action recognition module reconstructs the human body skeleton of the target human body according to the two-dimensional joint points after the pixel coordinates are corrected, and stores and sends the human body skeleton information of each frame to the three-dimensional action reconstruction module.
optionally, the three-dimensional motion reconstruction module specifically includes:
the first solving submodule is used for solving a first real three-dimensional coordinate of a projection imaging point, which is vertically projected on an imaging plane of the monocular camera with the focus of the monocular camera, in a preset three-dimensional coordinate system based on parameters of the monocular camera after the monocular camera is calibrated;
The second solving submodule is used for solving a rotation matrix and a translation vector of the first real three-dimensional coordinate based on the first real three-dimensional coordinate and the imaging plane parameter of the monocular camera;
A third solving submodule, configured to solve a second real three-dimensional coordinate of the two-dimensional joint point on the imaging plane in the preset three-dimensional coordinate system based on the rotation matrix and the translation vector;
And the joint point reconstruction submodule is used for reconstructing the three-dimensional motion of the target human body by taking the three-dimensional coordinate point with the minimum distance obtained by the optical center coordinates of all the monocular cameras and the corresponding second real three-dimensional coordinate straight line in the preset three-dimensional coordinate system as the real three-dimensional joint point of the two-dimensional joint point in the preset three-dimensional coordinate system.
Optionally, the joint sub-module is specifically configured to:
And solving straight lines formed by the optical center coordinates of all the monocular cameras and the corresponding second real three-dimensional coordinates based on an overdetermined equation set least square method to obtain a three-dimensional coordinate point with the minimum distance, taking the three-dimensional coordinate point as a real three-dimensional joint point of the two-dimensional joint point in the preset three-dimensional coordinate system, and reconstructing the three-dimensional motion of the target human body.
optionally, the two-dimensional human body motion recognition module is specifically configured to:
and identifying key human body joint parts in the action image sequence based on a preset convolutional neural network, extracting 25 joint points belonging to the same target human body in each frame of the action image sequence so as to reconstruct the human body skeleton of the target human body, and storing and sending the human body skeleton information of each frame to a three-dimensional action reconstruction module.
The second aspect of the present application further provides a three-dimensional human body motion reconstruction method, including:
performing monocular camera calibration based on the collected calibration point data, wherein the number of the monocular cameras is at least two;
acquiring action image sequences of a target human body shot by at least two monocular cameras;
Identifying key body joint parts of the human body in the action image sequence, and extracting two-dimensional joint points which belong to the same target human body in each frame of the action image sequence so as to reconstruct the human body skeleton of the target human body and obtain the human body skeleton information of each frame;
And restoring the real position of the target human body in the three-dimensional space of the joint point based on the human body skeleton information in each frame, and reconstructing the three-dimensional action of the target human body.
optionally, the restoring the real position of the target human body in the three-dimensional space based on the human body skeleton information in each frame to reconstruct the three-dimensional motion of the target human body specifically includes:
Based on monocular camera parameters after the monocular camera is calibrated, solving a first real three-dimensional coordinate of a projection imaging point which is vertically projected on an imaging plane of the monocular camera with a focus of the monocular camera in a preset three-dimensional coordinate system;
Solving a rotation matrix and a translation vector of the first real three-dimensional coordinate based on the first real three-dimensional coordinate and imaging plane parameters of the monocular camera;
solving a second real three-dimensional coordinate of the two-dimensional joint point on the imaging plane in the preset three-dimensional coordinate system based on the rotation matrix and the translation vector;
And in the preset three-dimensional coordinate system, taking the three-dimensional coordinate point with the minimum distance between the optical center coordinates of all the monocular cameras and the corresponding second real three-dimensional coordinate straight line as the real three-dimensional joint point of the two-dimensional joint point in the preset three-dimensional coordinate system, and reconstructing the three-dimensional motion of the target human body.
A third aspect of the present application provides a motion training system, including any one of the three-dimensional human motion reconstruction systems of the first aspect, further including a motion evaluation module;
and the action evaluation module is used for comparing the three-dimensional action of the target human body acquired from the three-dimensional action reconstruction module with a preset standard action in a difference mode and outputting an action evaluation result corresponding to the difference comparison result.
Optionally, the action evaluation module is specifically configured to:
Performing joint point combination angle similarity judgment, average curvature comparison of joint point combination motion tracks and human body joint point combination motion quantity comparison on the three-dimensional motion of the target human body acquired from the three-dimensional motion reconstruction module and a preset standard motion;
Acquiring a first action evaluation result corresponding to the result of the joint point combination angle similarity judgment, a second action evaluation result corresponding to the result of the average curvature comparison of the joint point combination motion trail and a third action evaluation result corresponding to the result of the human body joint point combination motion quantity comparison;
And performing weighting processing on the first action evaluation result, the second action evaluation result and the third action evaluation result, and outputting action evaluation results obtained after weighting processing.
Optionally, the method further includes: a music rhythm integrating degree module;
The music rhythm integrating module is used for extracting the music characteristics of the music on broadcasting based on an audio extraction algorithm, judging the matching degree of the human body actions of the target human body in continuous frames and the preset standard actions in the series of actions matching the beat by taking the beat as a unit, and outputting a beat matching result.
the application provides a human action system of rebuilding based on many meshes vision, includes: the camera calibration module is used for calibrating the monocular cameras according to the collected calibration point data, and the number of the monocular cameras is at least two; the motion acquisition module is used for acquiring motion image sequences of the target human body acquired by all the monocular cameras and sending the motion image sequences to the two-dimensional human body motion recognition module; the two-dimensional human body action recognition module is used for recognizing key limb joint parts of the human body in the action image sequence, extracting two-dimensional joint points which belong to the same target human body in each frame of the action image sequence so as to reconstruct the human body skeleton of the target human body, and storing and sending the human body skeleton information of each frame to the three-dimensional action reconstruction module; and the three-dimensional action reconstruction module is used for restoring the real position of the two-dimensional joint point of the target human body in the three-dimensional space based on the human body skeleton information in each frame and reconstructing the three-dimensional action of the target human body.
According to the human body motion reconstruction based on the multi-view vision, the corrected motion image of the target human body is captured based on at least two monocular cameras, two-dimensional joint points of the human body on the two-dimensional image are identified for the motion image, then the three-dimensional motion of the target human body is reconstructed based on the two-dimensional joint points of the human body, a user does not need to wear a special optical capturing garment or assemble a corresponding sensor, and the technical problems that the existing human body three-dimensional motion identification method of the depth camera has high requirements on application scenes and has low identification accuracy in complex application scenes are solved.
Drawings
in order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
fig. 1 is a schematic structural diagram of an embodiment of a human motion reconstruction system based on multi-view vision according to an embodiment of the present application;
Fig. 2 is another schematic structural diagram of a human motion reconstruction system based on multi-view vision according to an embodiment of the present disclosure;
Fig. 3 is a schematic flowchart of an embodiment of a three-dimensional human body motion reconstruction method according to an embodiment of the present application;
fig. 4 is another schematic flow chart of an embodiment of a three-dimensional human body motion reconstruction method according to an embodiment of the present application
fig. 5 is a schematic structural diagram of an action training system provided in an embodiment of the present application;
Fig. 6 is an image point error correction diagram of an image point error correction module according to an embodiment of the present disclosure;
Fig. 7 is a three-dimensional space theoretical layout diagram of three monocular cameras (A, B, C in the figure) provided in the embodiment of the present application.
Detailed Description
in order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
referring to fig. 1, fig. 1 is a schematic structural diagram of a human body motion reconstruction system based on multi-view vision according to an embodiment of the present application, where the human body motion reconstruction system based on multi-view vision according to the embodiment of the present application includes:
and the camera calibration module 101 is configured to perform monocular camera calibration according to the collected calibration point data, where the number of the monocular cameras is at least two.
It should be noted that, in the embodiment of the present application, the motion image of the target human body is obtained by at least two monocular cameras, the monocular cameras used for obtaining the motion image of the target human body in the system need to perform calibration processing, and all the monocular cameras perform the acquisition of the calibration points, which may be checkerboard calibration points or two-dimensional code calibration points, where the calibration method of the calibration points is not limited. The calibration point acquisition only needs to be acquired when the monocular camera is used for the first time, the calibration point acquisition is not needed to be repeated subsequently, and the acquired calibration point data is stored in the storage module so as to be called again subsequently. And calibrating the monocular camera according to the calibration point data to obtain calibration parameters of the monocular camera.
In the embodiment of the application, the number of the images collected by each monocular camera is 25 images of the calibration point when the calibration point data is collected, and the calibration precision and stability can be improved by calibrating each monocular camera.
the action acquisition module 102 is configured to acquire action image sequences of the target human body acquired by all monocular cameras, and send the action image sequences to the two-dimensional human body action recognition module 103.
it should be noted that the motion image sequence of the target human body is acquired by a calibrated monocular camera, and the motion image sequence is sent to the two-dimensional human body motion recognition module 103 for two-dimensional image processing.
The two-dimensional human body motion recognition module 103 is configured to recognize key limb joint parts of a human body in a motion image sequence, extract two-dimensional joint points belonging to the same target human body in each frame of the motion image sequence, so as to reconstruct a human body skeleton of the target human body, store human body skeleton information of each frame, and send the human body skeleton information to the three-dimensional motion reconstruction module 104.
It should be noted that, the joint points belonging to the same person in each frame of the motion image sequence are extracted to form a complete human body skeleton, and the skeleton information of each frame is cached in the database by using a json-format file and further provided to the three-dimensional motion reconstruction module 104 for use. In the embodiment of the application, the two-dimensional human body motion recognition module 103 adopts an open-source multi-person two-dimensional posture neural network openposition to realize two-dimensional human body motion recognition to reconstruct a human body skeleton of a target human body.
and the three-dimensional action reconstruction module 104 is configured to restore the real position of the target human body in the three-dimensional space based on the human body skeleton information in each frame, and reconstruct the three-dimensional action of the target human body.
it should be noted that the motion image sequence of the target human body captured by the monocular camera after correction is input into the two-dimensional human body motion recognition module 103 to obtain skeleton information represented by two-dimensional joint point information, and the three-dimensional motion reconstruction module 104 restores the human body skeleton information in each frame to the real position of the target human body in the three-dimensional space at the joint point to reconstruct the three-dimensional motion of the target human body.
according to the three-dimensional human body motion reconstruction system provided by the embodiment of the application, the corrected motion image of the target human body is captured based on at least two monocular cameras, human body two-dimensional joint points on the two-dimensional image are identified for the motion image, then the three-dimensional motion of the target human body is reconstructed based on the human body two-dimensional joint points, a user does not need to wear a special optical capturing garment or assemble a corresponding sensor, and the technical problems that the existing depth camera human body three-dimensional motion identification method is high in requirement on an application scene, and low in identification accuracy rate in a complex application scene are solved.
As a further improvement of the human motion reconstruction system based on multi-view vision in the embodiment of the present application, as shown in fig. 2, the human motion reconstruction system based on multi-view vision in the embodiment of the present application further includes: an imaging point error correction module 105;
And the imaging point error correction module 105 is used for correcting the pixel coordinates of the two-dimensional joint points in the two-dimensional human body action recognition module when the monocular camera deviates, so that the two-dimensional human body action recognition module reconstructs a human body skeleton of a target human body according to the two-dimensional joint points after the pixel coordinates are corrected, and stores and sends the human body skeleton information of each frame to the three-dimensional action reconstruction module.
it should be noted that, as shown in fig. 6, fig. 6 is a schematic diagram illustrating an imaging point error correction of an imaging point error correction module, in an actual application scenario, in the installation and use process of a monocular camera, there is a high probability that a shake or a shift of the monocular camera may occur, which may cause an imaging point error, thereby affecting a recognition result, in order to solve the technical problem, in the embodiment of the present application, an imaging point error correction module 105 is provided, and when the monocular camera shifts, pixel coordinates of a two-dimensional joint point in a two-dimensional human body motion recognition module 103 are corrected, so that the two-dimensional human body motion recognition module 103 reconstructs a human body skeleton of a target human body according to the two-dimensional joint point after correcting the pixel coordinates, and stores and sends human body skeleton information of each frame to a three-dimensional motion reconstruction module 104. The operation of the imaging point error correction module 105 is explained below with reference to fig. 6:
The imaging surface EFHG is an original imaging surface (namely a normal imaging surface before shaking or shifting) of the monocular camera A, the imaging surface BCJK is an imaging surface after shaking or shifting of the monocular camera A, the known mark point D can be identified through an image identification algorithm carried by opencv, the pixel coordinates of an image of the mark point D before and after shifting are respectively (ui, vi) and (ui ', vi '), and the pixel coordinates (uw ', vw ') of the error imaging point can be corrected to (uw, vw ') by performing the following operation according to the calibrated focal length f of the monocular camera. The operation formula is as follows:
wherein, the delta theta is the elevation angle difference and the direction angle difference of the imaging point of the marking point D on the spherical coordinate system before and after the imaging surface is deviated; theta 'and theta' are the elevation angle and the direction angle of any error imaging point on the spherical coordinate system after the imaging plane is deviated respectively.
according to the formula, the error correction can be carried out on the pixel coordinate of any imaging point in the monocular camera A, and the technical problem that the identification result is poor due to imaging point errors caused by shaking or deviation of the monocular camera is solved.
As a further improvement, the two-dimensional human body motion recognition module 103 in the embodiment of the present application is specifically configured to:
And identifying key body joint parts of the human body in the action image sequence based on a preset convolutional neural network, extracting 25 joint points belonging to the same target human body in each frame of the action image sequence so as to reconstruct the human body skeleton of the target human body, and storing and sending the human body skeleton information of each frame to the three-dimensional action reconstruction module 104.
it should be noted that the preset convolutional neural network matches the key limb joint position of the body in the two-dimensional image with the corresponding individual by using a non-parameter characterization method Part affinity fields and reconstructs the human body joint skeleton. The main idea is to use the bottom-up analysis step of the greedy algorithm to achieve high accuracy and real-time, and the body part location and association are performed on two branches simultaneously. The positions of 25 joint points of a two-dimensional image human body can be identified, and the method has high precision and real-time speed.
The method for recognizing the two-dimensional human body posture by adopting the preset convolutional neural network comprises the following steps: inputting a two-dimensional human body image video sequence with the pixel scale of w multiplied by h, positioning key points of individuals appearing in each frame of the image video sequence, predicting by utilizing a CNN neural network model with a double-branch multi-stage architecture, predicting a group of confidence maps S of human body parts and a group of two-dimensional limb vector fields J (each body part has a corresponding confidence map, each human body limb corresponds to a vector) by a first branch of the CNN, analyzing the confidence maps and the PAF through greedy reasoning and outputting 2D skeleton key point information of each person on each frame of the image video sequence, wherein the total number of the 2D skeleton key point information is 25 human body joint points
As a further improvement, the three-dimensional motion reconstruction module 104 in the embodiment of the present application specifically includes:
The first solving submodule 1041 is configured to solve, based on monocular camera parameters after the monocular camera is calibrated, a first real three-dimensional coordinate of a projection imaging point, in a preset three-dimensional coordinate system, which is projected on an imaging plane of the monocular camera perpendicularly to a focal point of the monocular camera.
The second solving submodule 1042 is configured to solve a rotation matrix and a translation vector of the first real three-dimensional coordinate based on the first real three-dimensional coordinate and the imaging plane parameter of the monocular camera.
and a third solving submodule 1043, configured to solve a second real three-dimensional coordinate of the two-dimensional joint point on the imaging plane in the preset three-dimensional coordinate system based on the rotation matrix and the translation vector.
And the joint point reconstruction submodule 1044 is configured to reconstruct a three-dimensional motion of the target human body by using the three-dimensional coordinate points, which are obtained by taking the optical center coordinates of all monocular cameras and the corresponding second real three-dimensional coordinate straight lines with the minimum distance, as real three-dimensional joint points of the two-dimensional joint points in the preset three-dimensional coordinate system.
Specifically, the joint point sub-module 1044 is specifically configured to:
and solving straight lines formed by the optical center coordinates of all monocular cameras and the corresponding second real three-dimensional coordinates based on an overdetermined equation set least square method to obtain a three-dimensional coordinate point with the minimum distance, taking the three-dimensional coordinate point as a real three-dimensional joint point of the two-dimensional joint point in a preset three-dimensional coordinate system, and reconstructing the three-dimensional action of the target human body.
it should be noted that fig. 7 is a three-dimensional space theoretical layout diagram of three monocular cameras (A, B, C in the drawing) provided in the embodiment of the present application, and for simplification of description, only A, B two monocular cameras are taken as objects of study in the embodiment of the present application, and by calibrating the monocular cameras, it is possible to know the focal distances f (in centimeters) of the A, B two monocular cameras and the two-dimensional image coordinates (in pixels) of the two focal points (I, N), and the coordinates of the focal points I, N of the A, B two monocular cameras are I (ui, vi), N (un, vn), respectively.
Taking monocular camera a as an example, the real three-dimensional coordinate of the solution focus I is (xi, yi, zi), D is the intersection point of the extension line from point a to focus I and the extension line from point B to focus N, and then the distance from point a to point D can be expressed as:
the relationship between the vector formed by the point A and the point D and the vector formed by the point A and the point I is as follows:
The above formula can be modified into:
Therefore, the real three-dimensional coordinates (xi, yi, zi) of the point I and the real three-dimensional coordinates (xe ', ye', ze ') and (xf', yf ', zf') of the imaging points E ', F' of the other two marker points on the imaging surface EFHG of the monocular camera B can be obtained according to the above relations.
Solving a rotation matrix a and a translation vector T of the monocular camera, and obtaining the rotation matrix a and the translation vector T according to the determined coordinates of the point I, the point A, the point E 'and the point F', which can be expressed as:
The rotation matrix a and the translational vector t are respectively:
Solving A, B for the true three-dimensional coordinates of imaging point W, V on the imaging plane:
substituting for the monocular camera coordinate system coordinates (uw, vw, f) of the point W, the coordinates (xw, yw, zw) of the point W in the true three-dimensional coordinate system can be found by the following equation:
similarly, for B monocular camera, the corresponding transformation matrix B may also be found, and the true three-dimensional coordinates (xv, yv, zv) of point V may be obtained. After the rotation matrix a and the translation vector t of each of the two monocular cameras are obtained, the pixel coordinates of the monocular camera coordinates of the imaging point of any point in the space on the corresponding imaging surface can be converted into real three-dimensional coordinates.
taking two cameras A and B as research objects, intersecting a point T in space through straight lines AT and BT, and solving the real three-dimensional coordinates (x, y and z) of any point T in space through the following equations:
It is known that two monocular cameras can find the three-dimensional coordinates of a point in space. However, in the process of the application, the BT can not be guaranteed to be handed over to the point T AT the AT due to the problem of monocular camera manufacturing. In order to reduce errors, a method of adding a monocular camera can be adopted to improve the accuracy of the solution. The least square solution between the straight lines can be solved by using an overdetermined equation set least square method through a plurality of non-intersecting straight lines in the space, the three-dimensional coordinate point corresponding to the minimum value is obtained by the sum of the distance between the point and each straight line, and the three-dimensional coordinate point can be used as a real three-dimensional joint point of the two-dimensional joint point in a preset three-dimensional coordinate system to reconstruct the three-dimensional motion of the target human body.
For easy understanding, please refer to fig. 3, an embodiment of a three-dimensional human motion reconstruction method is further provided in the present application, including:
And step S1, calibrating the monocular cameras based on the collected calibration point data, wherein the number of the monocular cameras is at least two.
and step S2, acquiring a motion image sequence of the target human body shot by at least two monocular cameras.
And step S3, identifying key limb joint parts of the human body in the action image sequence, and extracting two-dimensional joint points belonging to the same target human body in each frame of the action image sequence so as to reconstruct the human body skeleton of the target human body and obtain the human body skeleton information of each frame.
And step S4, restoring the real position of the target human body in the three-dimensional space based on the human body skeleton information in each frame, and reconstructing the three-dimensional action of the target human body.
As a further improvement, please refer to fig. 4, step S4 specifically includes the following steps:
And step S41, based on the monocular camera parameters after the monocular camera is calibrated, solving a first real three-dimensional coordinate of a projection imaging point which is vertically projected on an imaging plane of the monocular camera with the focus of the monocular camera in a preset three-dimensional coordinate system.
and step S42, based on the first real three-dimensional coordinate and the imaging plane parameter of the monocular camera, solving a rotation matrix and a translation vector of the first real three-dimensional coordinate.
And step S43, solving a second real three-dimensional coordinate of the two-dimensional joint point on the imaging plane in the preset three-dimensional coordinate system based on the rotation matrix and the translation vector.
and step S44, in the preset three-dimensional coordinate system, taking the three-dimensional coordinate point with the minimum distance between the optical center coordinates of all monocular cameras and the corresponding second real three-dimensional coordinate straight line as the real three-dimensional joint point of the two-dimensional joint point in the preset three-dimensional coordinate system, and reconstructing the three-dimensional motion of the target human body.
for easy understanding, please refer to fig. 3, an embodiment of a motion training system is provided in the present application, which includes a camera calibration module 101, a motion acquisition module 102, a two-dimensional human motion recognition module 103, a three-dimensional motion reconstruction module 104, and an imaging point error correction module 105 in the foregoing embodiments, and further includes a motion evaluation module 106;
And the action evaluation module 106 is used for performing difference comparison on the three-dimensional action of the target human body acquired from the three-dimensional action reconstruction module and a preset standard action and outputting an action evaluation result corresponding to the difference comparison result.
specifically, the action evaluation module 106 is configured to:
performing joint point combination angle similarity judgment, average curvature comparison of joint point combination motion tracks and human body joint point combination motion quantity comparison on the three-dimensional motion of the target human body acquired from the three-dimensional motion reconstruction module and a preset standard motion;
acquiring a first action evaluation result corresponding to the result of joint point combination angle similarity judgment, a second action evaluation result corresponding to the result of average curvature comparison of joint point combination motion tracks and a third action evaluation result corresponding to the result of human body joint point combination motion quantity comparison;
and performing weighting processing on the first action evaluation result, the second action evaluation result and the third action evaluation result, and outputting the action evaluation result obtained after the weighting processing.
after obtaining the three-dimensional dance movement of the human body of the user, the difference evaluation is performed on the movement and the standard movement, and the basis of the difference evaluation is divided into the following parts:
(1) And judging the similarity of the joint point combination angles. The evaluation method for angle similarity analysis can derive the cosine of the corresponding joint angle and then convert the cosine into the angle theta by using the Euclidean dot product formula (A and B are two vectors formed by three joint points, namely the three joint points are combined into a group of joint point combination), so that a user can visually see the difference conveniently
And the average difference in a set of joint point combinations in a set of consecutive movements can be expressed as:
Compared with the standard motion data, the corresponding joint angle is generally considered to be good when the angle is less than 10 degrees, the angle is generally considered to be greater than 10 degrees and less than 20 degrees, some errors are caused, and motion errors are judged when the angle exceeds 20 degrees.
(2) And comparing the average curvatures of the joint point combined motion tracks. The human body movement is a dynamic movement, and the integral movement degree of the joints can influence the aesthetic property of the dance. And judging the similarity of the motion tracks by calculating the average curvature of the combined motion track of the joint points of all parts of the human body and comparing the average curvature with the average curvature of the combined motion track of the standard action joint points. Firstly, a curve is fitted to the motion track of the joint points, the space distance of the movement of the joint points between two frames is defined as deltas, the corresponding rotating angle is delta alpha, the average curvature of the motion of each joint point is represented, and the average curvature of the combination of the joint points can be represented as
The similarity of the combined motion tracks of the two corresponding joint points can be compared through the judged sizes.
(3) and (4) comparing the combined motion amount of the human body joint points. The target human body motion (such as dance motion) is the motion of the whole human body, and it is necessary to judge the swing amplitude of the whole human body in addition to the evaluation of the condition of a single joint. Because each human body is different and is a unified standard, the distance between two shoulders of the human body is firstly adopted to carry out normalization processing on each node of the human body, then the front and back movement amounts of the normalized joints are added to obtain the human body movement amount of the human body at a certain moment, and then the human body movement amount is compared with the standard action.
firstly, because human bodies have different degrees of difference, the three-dimensional space coordinate distance of the shoulders of the human bodies is adopted to carry out normalized data processing on the coordinates of each joint point of the human bodies
x*＝(x-x)/(x-x)
y*＝(y-y)/(y-y)
z*＝(z-z)/(z-z)
secondly, after normalization, the sum of the normalized relative coordinate displacement of each joint point before and after a certain corresponding time interval calculation interval of the human body is Wp, and the total displacement W of the joint point combination formed by the joint points is taken as the movement amount of the joint point combination
comparing the human body exercise amount at a certain moment with the standard action, and judging the difference between the two actions through delta W.
The resulting values of the three parts are weighted, but at the same time the quality level determined for each body part depends on the difficulty of the exercise. And setting a percentage range suitable for the quality grade result to obtain a final dance action score, and visualizing the three parts of data on a display screen through a central processing unit and providing the data to a user.
as a further improvement, the action training system in the embodiment of the present application further includes: a music tempo fitness module 107;
And the music rhythm integrating module 107 is used for extracting the music characteristics of the music on broadcasting based on an audio extraction algorithm, judging the matching degree of the human body actions of the target human body in continuous frames and the preset standard actions in the series of actions matching the beat by taking the beat as a unit, and outputting a beat matching result.
it should be noted that, the music features of the dance music are extracted by using an audio extraction algorithm, and whether the motion of the user in the continuous frames is consistent with the series of motions of the standard motion in the beat is determined by taking the beat as a unit, so as to determine whether the beat is correct. If the first half part is divided into the partial shot scores and recorded, the system resets the beat alignment to match with the actual action of the user and provide accurate action data comparison for the action evaluation module 106. The beat scores and the training reports are generated by combining the action evaluation module 106 and the music rhythm integrating module 107 to be visually displayed to the user.
A data storage module 109 can also be arranged, and in the data storage module 109, a user can select to store the scoring and training report and the training recording and comparison video data to a hard disk, or upload the scoring and training report and the training recording and comparison video data to a cloud platform through a network partially so that a recommendation algorithm can be matched with the personal dance variety and training pattern of the user. Meanwhile, the data storage module 109 also stores camera parameters for human body three-dimensional motion reconstruction.
As a further improvement, the action training system in the embodiment of the present application further includes: a health daemon module 108;
The health guarding module 108 is used for calculating the physical quality degree, the proper dancing style and the current training exercise amount of the user through a cloud platform algorithm according to the user information of the target human body, recommending the most proper action set of the user according to the calculation result in a personalized high-quality mode, avoiding the action that the training intensity of the user is too weak or too high, recording the training time and the training intensity of the user every time, and prompting the user to have a rest through voice broadcasting and/or displaying a pop window mode.
It should be noted that the mode switching module 110 may also be configured to: the module provides mode selection for dance training. Three functions can be selected under the module, namely a learning module, a grading mode and a special training mode. Wherein, the learning module and the scoring mode can be carried out by a plurality of persons at the same time, and the special training mode is defaulted to be a single mode
1) the learning mode can be used for downloading standard dance videos recorded by professional dancers in advance from the cloud platform to perform simulated learning, the standard videos can provide a front version and a mirror inversion version for users to select and display to a display, and meanwhile, the system can divide learning steps and mark difficult and important actions to the users according to differences of standard actions or music rhythms so as to reinforce the learning.
2) and the scoring mode can call a dance evaluation module, which evaluates the fitness of the dance action and the music beat of the user and visually displays the fitness to the user in a text special effect mode.
3) The special training module judges the dance defects of the users according to the series course learning grade storage data of the users through the cloud platform, and provides corresponding local special training according to the defects.
The action training system that provides in the embodiment of this application, based on binocular stereovision principle, the monocular camera need not to add all the other expensive sensors, only need can reach higher precision after the camera is markd, be close to Kinect depth camera's recognition accuracy basically, it is lower to realize the cost, do not worry camera shake simultaneously, the problem of complicated application scene, the user can be according to the position of demand dynamic change camera in order to obtain better human observation angle, even can not influence its precision in mixed and disorderly room based on parallax triangulation's binocular formation of image, adaptability is stronger. Meanwhile, the dancer does not need to wear any sensing equipment or special clothes for collecting human body actions, so that the dancer can more conveniently stretch and dance postures, and the weight is reduced. Meanwhile, the method can also be expanded to adapt to the recognition of multi-person dance actions.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (10)
1. A human motion reconstruction system based on multi-view vision is characterized by comprising:
The camera calibration module is used for calibrating monocular cameras according to the collected calibration point data, and the number of the monocular cameras is at least two;
The action acquisition module is used for acquiring action image sequences of the target human body acquired by all the monocular cameras and sending the action image sequences to the two-dimensional human body action recognition module;
the two-dimensional human body motion recognition module is used for recognizing key human body joint parts in the motion image sequence, extracting two-dimensional joint points which belong to the same target human body in each frame of the motion image sequence so as to reconstruct a human body skeleton of the target human body, and storing and sending the human body skeleton information of each frame to the three-dimensional motion reconstruction module;
the three-dimensional action reconstruction module is used for restoring the real position of the two-dimensional joint point of the target human body in a three-dimensional space based on the human body skeleton information in each frame, and reconstructing the three-dimensional action of the target human body.
2. the system of claim 1, further comprising: an imaging point error correction module;
the imaging point error correction module is used for correcting the pixel coordinates of the two-dimensional joint points in the two-dimensional human body action recognition module when the monocular camera deviates, so that the two-dimensional human body action recognition module reconstructs the human body skeleton of the target human body according to the two-dimensional joint points after the pixel coordinates are corrected, and stores and sends the human body skeleton information of each frame to the three-dimensional action reconstruction module.
3. The system for human motion reconstruction based on multi-view vision according to claim 1, wherein the three-dimensional motion reconstruction module specifically comprises:
the first solving submodule is used for solving a first real three-dimensional coordinate of a projection imaging point, which is vertically projected on an imaging plane of the monocular camera with the focus of the monocular camera, in a preset three-dimensional coordinate system based on parameters of the monocular camera after the monocular camera is calibrated;
The second solving submodule is used for solving a rotation matrix and a translation vector of the first real three-dimensional coordinate based on the first real three-dimensional coordinate and the imaging plane parameter of the monocular camera;
A third solving submodule, configured to solve a second real three-dimensional coordinate of the two-dimensional joint point on the imaging plane in the preset three-dimensional coordinate system based on the rotation matrix and the translation vector;
And the joint point reconstruction submodule is used for reconstructing the three-dimensional motion of the target human body by taking the three-dimensional coordinate point with the minimum distance obtained by the optical center coordinates of all the monocular cameras and the corresponding second real three-dimensional coordinate straight line in the preset three-dimensional coordinate system as the real three-dimensional joint point of the two-dimensional joint point in the preset three-dimensional coordinate system.
4. The system of claim 3, wherein the joint sub-module is configured to:
and solving straight lines formed by the optical center coordinates of all the monocular cameras and the corresponding second real three-dimensional coordinates based on an overdetermined equation set least square method to obtain a three-dimensional coordinate point with the minimum distance, taking the three-dimensional coordinate point as a real three-dimensional joint point of the two-dimensional joint point in the preset three-dimensional coordinate system, and reconstructing the three-dimensional motion of the target human body.
5. the system according to claim 1, wherein the two-dimensional human motion recognition module is specifically configured to:
and identifying key human body joint parts in the action image sequence based on a preset convolutional neural network, extracting 25 joint points belonging to the same target human body in each frame of the action image sequence so as to reconstruct the human body skeleton of the target human body, and storing and sending the human body skeleton information of each frame to a three-dimensional action reconstruction module.
6. a human body action reconstruction method based on multi-view vision is characterized by comprising the following steps:
performing monocular camera calibration based on the collected calibration point data, wherein the number of the monocular cameras is at least two;
acquiring action image sequences of a target human body shot by at least two monocular cameras;
identifying key body joint parts of the human body in the action image sequence, and extracting two-dimensional joint points which belong to the same target human body in each frame of the action image sequence so as to reconstruct the human body skeleton of the target human body and obtain the human body skeleton information of each frame;
and restoring the real position of the target human body in the three-dimensional space of the joint point based on the human body skeleton information in each frame, and reconstructing the three-dimensional action of the target human body.
7. The method according to claim 6, wherein the reconstructing the three-dimensional motion of the target human body by restoring the real position of the target human body in the three-dimensional space at the joint point based on the human skeleton information in each frame specifically comprises:
Based on monocular camera parameters after the monocular camera is calibrated, solving a first real three-dimensional coordinate of a projection imaging point which is vertically projected on an imaging plane of the monocular camera with a focus of the monocular camera in a preset three-dimensional coordinate system;
solving a rotation matrix and a translation vector of the first real three-dimensional coordinate based on the first real three-dimensional coordinate and imaging plane parameters of the monocular camera;
Solving a second real three-dimensional coordinate of the two-dimensional joint point on the imaging plane in the preset three-dimensional coordinate system based on the rotation matrix and the translation vector;
and in the preset three-dimensional coordinate system, taking the three-dimensional coordinate point with the minimum distance between the optical center coordinates of all the monocular cameras and the corresponding second real three-dimensional coordinate straight line as the real three-dimensional joint point of the two-dimensional joint point in the preset three-dimensional coordinate system, and reconstructing the three-dimensional motion of the target human body.
8. A motion training system comprising the multi-vision based human motion reconstruction system of any one of claims 1-5, further comprising a motion evaluation module;
and the action evaluation module is used for comparing the three-dimensional action of the target human body acquired from the three-dimensional action reconstruction module with a preset standard action in a difference mode and outputting an action evaluation result corresponding to the difference comparison result.
9. the action training system of claim 8, wherein the action assessment module is specifically configured to:
performing joint point combination angle similarity judgment, average curvature comparison of joint point combination motion tracks and human body joint point combination motion quantity comparison on the three-dimensional motion of the target human body acquired from the three-dimensional motion reconstruction module and a preset standard motion;
acquiring a first action evaluation result corresponding to the result of the joint point combination angle similarity judgment, a second action evaluation result corresponding to the result of the average curvature comparison of the joint point combination motion trail and a third action evaluation result corresponding to the result of the human body joint point combination motion quantity comparison;
And performing weighting processing on the first action evaluation result, the second action evaluation result and the third action evaluation result, and outputting action evaluation results obtained after weighting processing.
10. the action training system of claim 9, further comprising: a music rhythm integrating degree module;
the music rhythm integrating module is used for extracting the music characteristics of the music on broadcasting based on an audio extraction algorithm, judging the matching degree of the human body actions of the target human body in continuous frames and the preset standard actions in the series of actions matching the beat by taking the beat as a unit, and outputting a beat matching result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910844266.7A CN110544302A (en) | 2019-09-06 | 2019-09-06 | Human body action reconstruction system and method based on multi-view vision and action training system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910844266.7A CN110544302A (en) | 2019-09-06 | 2019-09-06 | Human body action reconstruction system and method based on multi-view vision and action training system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110544302A true CN110544302A (en) | 2019-12-06 |
Family
ID=68712830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910844266.7A Pending CN110544302A (en) | 2019-09-06 | 2019-09-06 | Human body action reconstruction system and method based on multi-view vision and action training system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110544302A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027522A (en) * | 2019-12-30 | 2020-04-17 | 华通科技有限公司 | Bird detection positioning system based on deep learning |
| CN111028912A (en) * | 2019-12-23 | 2020-04-17 | 北京深测科技有限公司 | Environment-assisted life monitoring method and system |
| CN111105486A (en) * | 2019-12-07 | 2020-05-05 | 东南大学 | Multi-view-angle dynamic three-dimensional reconstruction method for mice |
| CN112200041A (en) * | 2020-09-29 | 2021-01-08 | Oppo（重庆）智能科技有限公司 | Video motion recognition method and device, storage medium and electronic equipment |
| CN112365578A (en) * | 2020-11-11 | 2021-02-12 | 杭州勾勾科技有限公司 | Three-dimensional human body model reconstruction system and method based on double cameras |
| CN113421286A (en) * | 2021-07-12 | 2021-09-21 | 北京未来天远科技开发有限公司 | Motion capture system and method |
| CN113989928A (en) * | 2021-10-27 | 2022-01-28 | 南京硅基智能科技有限公司 | Motion capturing and redirecting method |
| CN114463850A (en) * | 2022-02-08 | 2022-05-10 | 南京科源视觉技术有限公司 | Human body action recognition system suitable for multiple application scenes |
| WO2022241583A1 (en) * | 2021-05-15 | 2022-11-24 | 电子科技大学 | Family scenario motion capture method based on multi-target video |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346829A (en) * | 2013-07-29 | 2015-02-11 | 中国农业机械化科学研究院 | Three-dimensional color reconstruction system and method based on PMD (photonic mixer device) cameras and photographing head |
| CN106204656A (en) * | 2016-07-21 | 2016-12-07 | 中国科学院遥感与数字地球研究所 | Target based on video and three-dimensional spatial information location and tracking system and method |
| CN106503605A (en) * | 2015-09-01 | 2017-03-15 | 南京理工大学 | Human body target recognition methods based on stereovision technique |
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
-
2019
- 2019-09-06 CN CN201910844266.7A patent/CN110544302A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346829A (en) * | 2013-07-29 | 2015-02-11 | 中国农业机械化科学研究院 | Three-dimensional color reconstruction system and method based on PMD (photonic mixer device) cameras and photographing head |
| CN106503605A (en) * | 2015-09-01 | 2017-03-15 | 南京理工大学 | Human body target recognition methods based on stereovision technique |
| CN106204656A (en) * | 2016-07-21 | 2016-12-07 | 中国科学院遥感与数字地球研究所 | Target based on video and three-dimensional spatial information location and tracking system and method |
| CN109003301A (en) * | 2018-07-06 | 2018-12-14 | 东南大学 | A kind of estimation method of human posture and rehabilitation training system based on OpenPose and Kinect |
Non-Patent Citations (3)
| Title |
|---|
| 李琳等: "二维和三维视觉传感集成系统联合标定方法", 《仪器仪表学报》 * |
| 陈学梅: "基于人体三维姿态的动作评价系统", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 魏玮等: "人体运动分析综述", 《计算机系统应用》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111105486A (en) * | 2019-12-07 | 2020-05-05 | 东南大学 | Multi-view-angle dynamic three-dimensional reconstruction method for mice |
| CN111028912A (en) * | 2019-12-23 | 2020-04-17 | 北京深测科技有限公司 | Environment-assisted life monitoring method and system |
| CN111027522B (en) * | 2019-12-30 | 2023-09-01 | 华通科技有限公司 | Bird detection positioning system based on deep learning |
| CN111027522A (en) * | 2019-12-30 | 2020-04-17 | 华通科技有限公司 | Bird detection positioning system based on deep learning |
| CN112200041B (en) * | 2020-09-29 | 2022-08-02 | Oppo（重庆）智能科技有限公司 | Video motion recognition method and device, storage medium and electronic equipment |
| CN112200041A (en) * | 2020-09-29 | 2021-01-08 | Oppo（重庆）智能科技有限公司 | Video motion recognition method and device, storage medium and electronic equipment |
| CN112365578A (en) * | 2020-11-11 | 2021-02-12 | 杭州勾勾科技有限公司 | Three-dimensional human body model reconstruction system and method based on double cameras |
| WO2022241583A1 (en) * | 2021-05-15 | 2022-11-24 | 电子科技大学 | Family scenario motion capture method based on multi-target video |
| CN113421286A (en) * | 2021-07-12 | 2021-09-21 | 北京未来天远科技开发有限公司 | Motion capture system and method |
| CN113421286B (en) * | 2021-07-12 | 2024-01-02 | 北京未来天远科技开发有限公司 | Motion capturing system and method |
| CN113989928A (en) * | 2021-10-27 | 2022-01-28 | 南京硅基智能科技有限公司 | Motion capturing and redirecting method |
| CN113989928B (en) * | 2021-10-27 | 2023-09-05 | 南京硅基智能科技有限公司 | Motion capturing and redirecting method |
| CN114463850A (en) * | 2022-02-08 | 2022-05-10 | 南京科源视觉技术有限公司 | Human body action recognition system suitable for multiple application scenes |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110544301A (en) | Three-dimensional human body action reconstruction system, method and action training system | |
| CN110544302A (en) | Human body action reconstruction system and method based on multi-view vision and action training system | |
| CN110321754B (en) | Human motion posture correction method and system based on computer vision | |
| CN111881887A (en) | Multi-camera-based motion attitude monitoring and guiding method and device | |
| JP4473754B2 (en) | Virtual fitting device | |
| KR101424942B1 (en) | A system and method for 3D space-dimension based image processing | |
| JP7427188B2 (en) | 3D pose acquisition method and device | |
| CN113706699B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN110448870B (en) | Human body posture training method | |
| CN109919141A (en) | A kind of recognition methods again of the pedestrian based on skeleton pose | |
| CN112911393B (en) | Method, device, terminal and storage medium for identifying part | |
| CN111951325B (en) | Pose tracking method, pose tracking device and electronic equipment | |
| JP5795250B2 (en) | Subject posture estimation device and video drawing device | |
| CN110717391A (en) | Height measuring method, system, device and medium based on video image | |
| CN113989928B (en) | Motion capturing and redirecting method | |
| CN110941996A (en) | Target and track augmented reality method and system based on generation of countermeasure network | |
| US20210035326A1 (en) | Human pose estimation system | |
| CN111832386A (en) | Method and device for estimating human body posture and computer readable medium | |
| CN111435550A (en) | Image processing method and apparatus, image device, and storage medium | |
| Chen et al. | Camera networks for healthcare, teleimmersion, and surveillance | |
| CN105225270B (en) | A kind of information processing method and electronic equipment | |
| CN107016730A (en) | The device that a kind of virtual reality is merged with real scene | |
| Zou et al. | Automatic reconstruction of 3D human motion pose from uncalibrated monocular video sequences based on markerless human motion tracking | |
| JP5503510B2 (en) | Posture estimation apparatus and posture estimation program | |
| CN106981100A (en) | The device that a kind of virtual reality is merged with real scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191206 |
|
| RJ01 | Rejection of invention patent application after publication |