CN110599540A - Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera - Google Patents
Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera Download PDFInfo
- Publication number
- CN110599540A CN110599540A CN201910716814.8A CN201910716814A CN110599540A CN 110599540 A CN110599540 A CN 110599540A CN 201910716814 A CN201910716814 A CN 201910716814A CN 110599540 A CN110599540 A CN 110599540A
- Authority
- CN
- China
- Prior art keywords
- human body
- dimensional
- camera
- parameters
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Abstract
The invention discloses a real-time three-dimensional human body shape and posture reconstruction method and a device under a multi-view camera, wherein the method comprises the following steps: enclosing a capturing area by a plurality of camera frames, and calibrating camera internal parameters and camera external parameters of a plurality of cameras by a camera calibration method; collecting human body images in a capture area through a plurality of calibrated cameras and processing the human body images to enable the human body images to be transcoded into RGB images, detecting the RGB images by utilizing a pre-trained convolutional neural network, generating human body joint two-dimensional coordinate information under each visual angle, and triangulating the human body joint two-dimensional coordinate information to obtain human body three-dimensional joint coordinate information; and optimizing the posture parameters and the shape parameters in the preset human body model by utilizing the coordinate information of the human body three-dimensional joint, and then optimizing and stabilizing the optimized preset human body model by time domain optimization to obtain a human body three-dimensional reconstruction model. The method utilizes deep learning to complete estimation of human body postures, and can fit and render human body models of multiple persons in real time in a test environment.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a real-time three-dimensional human body shape and posture reconstruction method and device under a multi-view camera.
Background
With the improvement of the computing power of the computer and the continuous iteration of the graphic card, the deep learning technology is rapidly developed, and the field of computer vision is greatly promoted. The current reconstruction technology is mainly divided into two types, one is to use a common RGB camera to obtain depth information through multi-view feature point matching and triangulation, the other is to directly use a depth camera to obtain a depth map for reconstruction, for example, a new iPhone X issued by apple Inc. carries a depth camera to complete face reconstruction, and the technology is pushed to the consumption field.
However, compared with the RGB camera, the depth camera has the disadvantages of large interference by ambient light, limited depth detection distance, high price, and the like, so the RGB camera with high popularity rate has a greater potential for human body reconstruction, and can be mainly applied to the fields of virtual fitting, CG games, and the like. Most of the traditional methods for human body reconstruction adopt methods of wearing sensors or green screen segmentation and the like, which have high requirements on environment or require huge calculation amount and cannot meet the real-time requirement.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, an object of the present invention is to provide a method for reconstructing a three-dimensional human body shape and posture in real time under a multi-view camera, which combines a deep learning technique to estimate a human body shape, and can reconstruct a human body model of multiple people.
The invention also aims to provide a real-time three-dimensional human body shape and posture reconstruction device under the multi-view camera.
In order to achieve the above object, an embodiment of the present invention provides a method for reconstructing a body shape and a posture of a real-time three-dimensional human body under a multi-view camera, including:
enclosing a plurality of camera frames to form a capture area, and calibrating camera internal parameters and camera external parameters of the plurality of cameras by a camera calibration method;
collecting human body images in the capture area through a plurality of calibrated cameras, processing the human body images to enable the human body images to be transcoded into RGB images, detecting the RGB images by utilizing a pre-trained convolutional neural network to generate human body joint two-dimensional coordinate information under each view angle, and triangulating the human body joint two-dimensional information to obtain human body three-dimensional joint coordinate information;
and optimizing the posture parameters and the shape parameters in the preset human body model by using the human body three-dimensional joint coordinate information, and then optimizing and stabilizing the optimized preset human body model by time domain optimization to obtain a human body three-dimensional reconstruction model.
According to the real-time three-dimensional human body shape and posture reconstruction method under the multi-view camera, the human body image is collected on the human body model and converted into the RGB image, the two-dimensional joint coordinate information is generated through detection of the convolutional neural network and is triangulated to generate the three-dimensional joint coordinate information, the human body three-dimensional reconstruction model is obtained by optimizing and presetting the human body model according to the three-dimensional joint coordinate information, and the human body three-dimensional reconstruction is achieved.
In addition, the real-time three-dimensional human body shape and posture reconstruction method under the multi-view camera according to the above embodiment of the present invention may further have the following additional technical features:
further, in an embodiment of the present invention, the calibrating the camera internal parameters and the camera external parameters of the plurality of cameras includes:
and calibrating the internal parameters and the external parameters of the camera by a checkerboard calibration method.
Further, in an embodiment of the present invention, the human body images are collected by the calibrated multiple cameras, transmitted to a collection card of a host through a PCIe interface, transcoded by a cuda program, and scaled into a three-channel RGB matrix form, so as to obtain the RGB images.
Further, in an embodiment of the present invention, the preset human body model is an open-source linear model SMPL, and the linear model SMPL is driven by a posture parameter and a shape parameter and constrained by the three-dimensional joint coordinate information of the human body;
and adding a Gaussian mixture model into the linear model SMPL to restrict the rotation of limbs so as to enable the human body posture after three-dimensional reconstruction to be a human body reasonable posture.
Further, in one embodiment of the present invention, the RGB image is corrected by gamma correction; and optimizing the preset human body model through human body contour information.
In order to achieve the above object, an embodiment of the present invention provides a real-time three-dimensional human body shape and posture reconstruction apparatus under a multi-view camera, including:
the camera calibration system comprises a building module, a calibration module and a calibration module, wherein the building module is used for enclosing a plurality of camera frames into a capture area and calibrating camera internal parameters and camera external parameters of a plurality of cameras by a camera calibration method;
the processing module is used for acquiring a human body image in the capturing area through the calibrated cameras, processing the human body image to enable the human body image to be transcoded into an RGB image, detecting the RGB image by using a pre-trained convolutional neural network to generate human body joint two-dimensional coordinate information under each visual angle, and triangulating the human body joint two-dimensional information to obtain human body three-dimensional joint coordinate information;
and the reconstruction module is used for optimizing the posture parameters and the shape parameters in the preset human body model by utilizing the human body three-dimensional joint coordinate information and then stabilizing the optimized preset human body model through time domain optimization to obtain the human body three-dimensional reconstruction model.
According to the real-time three-dimensional human body shape and posture reconstruction device under the multi-view camera, the human body image is collected on the human body model and converted into the RGB image, the two-dimensional joint coordinate information is generated through detection of the convolutional neural network and is triangulated to generate the three-dimensional joint coordinate information, the human body three-dimensional reconstruction model is obtained by optimizing and presetting the human body model according to the three-dimensional joint coordinate information, and the human body three-dimensional reconstruction is achieved.
In addition, the real-time three-dimensional human body shape and posture reconstruction device under the multi-view camera according to the above embodiment of the present invention may further have the following additional technical features:
further, in an embodiment of the present invention, the calibrating the camera internal parameters and the camera external parameters of the plurality of cameras includes:
and calibrating the internal parameters and the external parameters of the camera by a checkerboard calibration method.
Further, in an embodiment of the present invention, the human body images are collected by the calibrated multiple cameras, transmitted to a collection card of a host through a PCIe interface, transcoded by a cuda program, and scaled into a three-channel RGB matrix form, so as to obtain the RGB images.
Further, in an embodiment of the present invention, the preset human body model is an open-source linear model SMPL, and the linear model SMPL is driven by a posture parameter and a shape parameter and constrained by the three-dimensional joint coordinate information of the human body;
and adding a Gaussian mixture model into the linear model SMPL to restrict the rotation of limbs so as to enable the human body posture after three-dimensional reconstruction to be a human body reasonable posture.
Further, in one embodiment of the present invention, the RGB image is corrected by gamma correction; and optimizing the preset human body model through human body contour information.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow chart of a real-time three-dimensional human body shape and posture reconstruction method under a multi-view camera according to an embodiment of the invention;
FIG. 2 is a flow chart of a real-time three-dimensional human body shape and posture reconstruction method under a multi-view camera according to an embodiment of the invention;
fig. 3 is a schematic structural diagram of a real-time three-dimensional human body shape and posture reconstruction device under a multi-view camera according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The following describes a real-time three-dimensional human body shape and posture reconstruction method and device under a multi-view camera according to an embodiment of the invention with reference to the accompanying drawings.
First, a real-time three-dimensional human body shape and posture reconstruction method under a multi-view camera according to an embodiment of the present invention will be described with reference to the accompanying drawings.
Fig. 1 is a flowchart of a real-time three-dimensional human body shape and posture reconstruction method under a multi-view camera according to an embodiment of the invention.
As shown in fig. 1, the real-time three-dimensional human body shape and posture reconstruction method under the multi-view camera includes the following steps:
in step S101, a plurality of camera frames are surrounded by a capture area, and camera internal parameters and camera external parameters of the plurality of cameras are calibrated by a camera calibration method.
Specifically, four industrial cameras are erected on a test site, the height from the ground is about 1.2m, the distance between the cameras is about 3-5 m, and a rectangular capture area is defined. In practical application, the number of cameras, the capturing area and the setting parameters can be set according to requirements.
Further, the camera parameters are calibrated, and as a way, a chessboard calibration method can be adopted to calibrate the camera.
Calibrating camera parameters, wherein the parameters to be calibrated comprise camera internal parameters and camera external parameters, and calibrating the camera internal parameters by using a checkerboard. For each camera, about 20 pieces of checkerboards in different handheld postures need to be photographed, and then a matlab calibration tool box is called to calibrate the internal reference, wherein calibration parameters comprise the focal length, distortion parameters and the like of the camera. And after calibrating the internal reference, continuously calibrating the external reference of the camera, similarly calibrating by using a checkerboard and a matlab toolbox, and if the precision requirement is higher, paving a texture-rich material on the center of the scene and using a photoscan to perform auxiliary calibration.
In step S102, a human body image is collected in a capture area by a plurality of calibrated cameras, the human body image is processed to be transcoded into an RGB image, the RGB image is detected by using a pre-trained convolutional neural network and human body joint two-dimensional coordinate information at each view angle is generated, and the human body joint two-dimensional information is triangulated to obtain human body three-dimensional joint coordinate information.
Further, in an embodiment of the present invention, the calibrated multiple cameras are used to collect human body images, the human body images are transmitted to the capture card of the host through the PCIe interface, and the human body images are transcoded by the cuda program and scaled into a three-channel RGB matrix form, so as to obtain RGB images.
It is understood that the quality of the image can be improved by gamma correction.
Specifically, as shown in fig. 2, the plurality of cameras respectively use the acquired three-channel images to complete monocular human body posture estimation, and may use open-source work openposition, alphaposition, position-proxy network, and the like.
A heat point map (heatmap) and a joint affinity map (paf) of each joint of the human body can be obtained through single-purpose human body posture estimation, and each joint coordinate can be obtained by carrying out non-maximum value inhibition on the joint heat point map.
And constraining the monocular detection result by using polar constraint of multi-view information to obtain two-dimensional joint coordinate information of the human body under each view angle, and triangulating the coordinate of the matched joint under each view angle by using camera parameters to obtain three-dimensional joint coordinate information of each joint.
In step S103, the posture parameters and the shape parameters in the preset human body model are optimized by using the coordinate information of the human body three-dimensional joint, and the time domain optimization stabilizes the optimized preset human body model to obtain a human body three-dimensional reconstruction model.
Further, the posture parameters and the shape parameters of the human body model SMPL are optimized by utilizing the three-dimensional joint coordinate information, and the result tends to be stable and jitter is eliminated through time domain optimization.
Specifically, a human body model is fitted by using human body three-dimensional joint coordinate information, wherein an open source model of the human body model, namely a preset human body model is an open source linear model SMPL, joint coordinates defined by the SMPL are not identical to definitions given by an attitude detector (openposition), coordinate conversion needs to be completed through a regression matrix, and the regression matrix needs to be obtained by utilizing a data set for training in advance.
Furthermore, parameter estimation can be performed by using the obtained three-dimensional joint coordinate information as a constraint, but since the joint coordinates cannot constrain the rotation of the limb, some distorted postures may occur, and therefore a gaussian mixture model is required to be added for constraint. The Gaussian mixture model represents reasonable posture distribution of the human body and needs to be trained in advance.
Further, the shape parameters cannot be well estimated only by the joint coordinates, so that further human body contour information needs to be added for optimization, and the shape parameters tend to be stable through time domain optimization.
According to the real-time three-dimensional human body shape and posture reconstruction method under the multi-view camera provided by the embodiment of the invention, the human body image is collected on the human body model, the human body image is converted into the RGB image, the two-dimensional joint coordinate information is generated by detecting through the convolutional neural network and is triangulated to generate the three-dimensional joint coordinate information, the preset human body model is optimized by utilizing the three-dimensional joint coordinate information to obtain the human body three-dimensional reconstruction model, and the human body three-dimensional reconstruction is realized.
Next, a real-time three-dimensional human body shape and posture reconstruction device under a multi-view camera according to an embodiment of the present invention is described with reference to the drawings.
Fig. 3 is a real-time three-dimensional human body shape and posture reconstruction apparatus under a multi-view camera according to an embodiment of the present invention.
As shown in fig. 3, the real-time three-dimensional human body shape and posture reconstruction apparatus under the multi-view camera includes: a building module 100, a processing module 200 and a reconstruction module 300.
The module 100 is constructed for enclosing a plurality of camera frames to form a capture area, and calibrating camera internal parameters and camera external parameters of a plurality of cameras by a camera calibration method.
The processing module 200 is configured to collect a human body image in a capture area through the calibrated multiple cameras, process the human body image to convert the human body image into an RGB image, detect the RGB image by using a pre-trained convolutional neural network, generate human body joint two-dimensional coordinate information at each view angle, and triangulate the human body joint two-dimensional information to obtain human body three-dimensional joint coordinate information.
And the reconstruction module 300 is configured to optimize the posture parameters and the shape parameters in the preset human body model by using the coordinate information of the human body three-dimensional joint, and then stabilize the optimized preset human body model by time domain optimization to obtain a human body three-dimensional reconstruction model.
Further, in an embodiment of the present invention, calibrating the camera internal parameters and the camera external parameters of the plurality of cameras includes:
calibrating the camera internal parameters and the camera external parameters by a checkerboard calibration method.
Further, in an embodiment of the present invention, the calibrated multiple cameras are used to collect human body images, the human body images are transmitted to the capture card of the host through the PCIe interface, and the human body images are transcoded by the cuda program and scaled into a three-channel RGB matrix form, so as to obtain RGB images.
Further, in one embodiment of the invention, the human body model is preset as an open-source linear model SMPL, and the linear model SMPL is driven by attitude parameters and shape parameters and constrained by the coordinate information of the three-dimensional joints of the human body;
and adding a Gaussian mixture model into the linear model SMPL to restrict the rotation of limbs so as to enable the human body posture after three-dimensional reconstruction to be a human body reasonable posture.
Further, in one embodiment of the present invention, the RGB image is corrected by gamma correction;
and optimizing the preset human body model through the human body contour information.
It should be noted that the explanation of the embodiment of the real-time three-dimensional human body shape and posture reconstruction method under the multi-view camera is also applicable to the apparatus of the embodiment, and is not repeated herein.
According to the real-time three-dimensional human body shape and posture reconstruction device under the multi-view camera, the human body image is collected on the human body model, the human body image is converted into the RGB image, the two-dimensional joint coordinate information is generated by detection through the convolutional neural network and is triangulated to generate the three-dimensional joint coordinate information, the preset human body model is optimized through the three-dimensional joint coordinate information to obtain the human body three-dimensional reconstruction model, and the human body three-dimensional reconstruction is achieved.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A real-time three-dimensional human body shape and posture reconstruction method under a multi-view camera is characterized by comprising the following steps:
enclosing a plurality of camera frames to form a capture area, and calibrating camera internal parameters and camera external parameters of the plurality of cameras by a camera calibration method;
collecting human body images in the capture area through a plurality of calibrated cameras, processing the human body images to enable the human body images to be transcoded into RGB images, detecting the RGB images by utilizing a pre-trained convolutional neural network to generate human body joint two-dimensional coordinate information under each view angle, and triangulating the human body joint two-dimensional information to obtain human body three-dimensional joint coordinate information;
and optimizing the posture parameters and the shape parameters in the preset human body model by using the human body three-dimensional joint coordinate information, and then optimizing and stabilizing the optimized preset human body model by time domain optimization to obtain a human body three-dimensional reconstruction model.
2. The method of claim 1, wherein the calibrating the camera internal parameters and the camera external parameters of the plurality of cameras comprises:
and calibrating the internal parameters and the external parameters of the camera by a checkerboard calibration method.
3. The method of claim 1,
the human body images are collected through the calibrated cameras, transmitted to a collection card of a host through a PCIe interface, transcoded through a cuda program and scaled into a three-channel RGB matrix form, and the RGB images are obtained.
4. The method of claim 1,
the preset human body model is an open-source linear model SMPL, and the linear model SMPL is driven by attitude parameters and shape parameters and constrained by the coordinate information of the human body three-dimensional joint;
and adding a Gaussian mixture model into the linear model SMPL to restrict the rotation of limbs so as to enable the human body posture after three-dimensional reconstruction to be a human body reasonable posture.
5. The method of claim 1,
correcting the RGB image through gamma correction;
and optimizing the preset human body model through human body contour information.
6. A real-time three-dimensional human body shape and posture reconstruction device under a multi-view camera is characterized by comprising:
the camera calibration system comprises a building module, a calibration module and a calibration module, wherein the building module is used for enclosing a plurality of camera frames into a capture area and calibrating camera internal parameters and camera external parameters of a plurality of cameras by a camera calibration method;
the processing module is used for acquiring a human body image in the capturing area through the calibrated cameras, processing the human body image to enable the human body image to be transcoded into an RGB image, detecting the RGB image by using a pre-trained convolutional neural network to generate human body joint two-dimensional coordinate information under each visual angle, and triangulating the human body joint two-dimensional information to obtain human body three-dimensional joint coordinate information;
and the reconstruction module is used for optimizing the posture parameters and the shape parameters in the preset human body model by utilizing the human body three-dimensional joint coordinate information and then stabilizing the optimized preset human body model through time domain optimization to obtain the human body three-dimensional reconstruction model.
7. The apparatus of claim 6, wherein the calibrating the camera internal reference and the camera external reference of the plurality of cameras comprises:
and calibrating the internal parameters and the external parameters of the camera by a checkerboard calibration method.
8. The apparatus of claim 6,
the human body images are collected through the calibrated cameras, transmitted to a collection card of a host through a PCIe interface, transcoded through a cuda program and scaled into a three-channel RGB matrix form, and the RGB images are obtained.
9. The apparatus of claim 6,
the preset human body model is an open-source linear model SMPL, and the linear model SMPL is driven by attitude parameters and shape parameters and constrained by the coordinate information of the human body three-dimensional joint;
and adding a Gaussian mixture model into the linear model SMPL to restrict the rotation of limbs so as to enable the human body posture after three-dimensional reconstruction to be a human body reasonable posture.
10. The apparatus of claim 6,
correcting the RGB image through gamma correction;
and optimizing the preset human body model through human body contour information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910716814.8A CN110599540B (en) | 2019-08-05 | 2019-08-05 | Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910716814.8A CN110599540B (en) | 2019-08-05 | 2019-08-05 | Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110599540A true CN110599540A (en) | 2019-12-20 |
| CN110599540B CN110599540B (en) | 2022-06-17 |
Family
ID=68853608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910716814.8A Active CN110599540B (en) | 2019-08-05 | 2019-08-05 | Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110599540B (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111047690A (en) * | 2019-12-23 | 2020-04-21 | 北京奇艺世纪科技有限公司 | Model construction method and apparatus, storage medium, and electronic apparatus |
| CN111445475A (en) * | 2020-02-26 | 2020-07-24 | 清华大学 | Human body model attitude prediction method based on inverse dynamics |
| CN111462302A (en) * | 2020-03-05 | 2020-07-28 | 清华大学 | Multi-view human body dynamic three-dimensional reconstruction method and system based on depth coding network |
| CN111489392A (en) * | 2020-03-30 | 2020-08-04 | 清华大学 | Single target human motion posture capturing method and system in multi-person environment |
| CN111583386A (en) * | 2020-04-20 | 2020-08-25 | 清华大学 | Multi-view human body posture reconstruction method based on label propagation algorithm |
| CN111932678A (en) * | 2020-08-13 | 2020-11-13 | 北京未澜科技有限公司 | Multi-view real-time human motion, gesture, expression and texture reconstruction system |
| CN112070815A (en) * | 2020-09-07 | 2020-12-11 | 哈尔滨工业大学（威海） | Automatic slimming method based on human body outer contour deformation |
| CN112099330A (en) * | 2020-08-31 | 2020-12-18 | 中国人民解放军军事科学院国防科技创新研究院 | Holographic human body reconstruction method based on external camera and wearable display control equipment |
| CN112419479A (en) * | 2020-11-10 | 2021-02-26 | 广州二元科技有限公司 | Body type data calculation method based on weight, height and body image |
| CN112837406A (en) * | 2021-01-11 | 2021-05-25 | 聚好看科技股份有限公司 | Three-dimensional reconstruction method, device and system |
| CN112907631A (en) * | 2021-02-20 | 2021-06-04 | 北京未澜科技有限公司 | Multi-RGB camera real-time human body motion capture system introducing feedback mechanism |
| CN113284249A (en) * | 2021-06-11 | 2021-08-20 | 清华大学深圳国际研究生院 | Multi-view three-dimensional human body reconstruction method and system based on graph neural network |
| CN113658303A (en) * | 2021-06-29 | 2021-11-16 | 清华大学 | Monocular vision-based virtual human generation method and device |
| CN114373018A (en) * | 2021-12-06 | 2022-04-19 | 聚好看科技股份有限公司 | Real-time driving method, device and equipment |
| CN116403275A (en) * | 2023-03-14 | 2023-07-07 | 南京航空航天大学 | Method and system for detecting personnel advancing posture in closed space based on multi-vision |
| TWI822473B (en) * | 2022-11-18 | 2023-11-11 | 財團法人資訊工業策進會 | Method, processing control device and system of providing movement improvement information |
| CN117726907A (en) * | 2024-02-06 | 2024-03-19 | 之江实验室 | Training method of modeling model, three-dimensional human modeling method and device |
| CN117745978A (en) * | 2024-02-20 | 2024-03-22 | 四川大学华西医院 | Simulation quality control method, equipment and medium based on human body three-dimensional reconstruction algorithm |
| CN117745978B (en) * | 2024-02-20 | 2024-04-30 | 四川大学华西医院 | Simulation quality control method, equipment and medium based on human body three-dimensional reconstruction algorithm |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101916456A (en) * | 2010-08-11 | 2010-12-15 | 李浩民 | Method for producing personalized three-dimensional cartoon |
| CN107945269A (en) * | 2017-12-26 | 2018-04-20 | 清华大学 | Complicated dynamic human body object three-dimensional rebuilding method and system based on multi-view point video |
| CN108053469A (en) * | 2017-12-26 | 2018-05-18 | 清华大学 | Complicated dynamic scene human body three-dimensional method for reconstructing and device under various visual angles camera |
| CN109859296A (en) * | 2019-02-01 | 2019-06-07 | 腾讯科技（深圳）有限公司 | Training method, server and the storage medium of SMPL parametric prediction model |
-
2019
- 2019-08-05 CN CN201910716814.8A patent/CN110599540B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101916456A (en) * | 2010-08-11 | 2010-12-15 | 李浩民 | Method for producing personalized three-dimensional cartoon |
| CN107945269A (en) * | 2017-12-26 | 2018-04-20 | 清华大学 | Complicated dynamic human body object three-dimensional rebuilding method and system based on multi-view point video |
| CN108053469A (en) * | 2017-12-26 | 2018-05-18 | 清华大学 | Complicated dynamic scene human body three-dimensional method for reconstructing and device under various visual angles camera |
| CN109859296A (en) * | 2019-02-01 | 2019-06-07 | 腾讯科技（深圳）有限公司 | Training method, server and the storage medium of SMPL parametric prediction model |
Non-Patent Citations (3)
| Title |
|---|
| ANDREA BIGNOLI ET AL.: ""Multi-View Stereo 3D Edge Reconstruction"", 《ARXIV》 * |
| KUN LI ET AL.: ""Shape and Pose Estimation for Closely Interacting Persons Using Multi-view Images"", 《PACIFIC GRAPHICS 2018》 * |
| 孟影: ""多视角动态三维重建技术研究与实现"", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》 * |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111047690B (en) * | 2019-12-23 | 2023-12-15 | 北京奇艺世纪科技有限公司 | Model construction method and device, storage medium and electronic device |
| CN111047690A (en) * | 2019-12-23 | 2020-04-21 | 北京奇艺世纪科技有限公司 | Model construction method and apparatus, storage medium, and electronic apparatus |
| CN111445475A (en) * | 2020-02-26 | 2020-07-24 | 清华大学 | Human body model attitude prediction method based on inverse dynamics |
| CN111445475B (en) * | 2020-02-26 | 2022-06-24 | 清华大学 | Human model attitude prediction method based on reverse dynamics |
| CN111462302A (en) * | 2020-03-05 | 2020-07-28 | 清华大学 | Multi-view human body dynamic three-dimensional reconstruction method and system based on depth coding network |
| CN111462302B (en) * | 2020-03-05 | 2022-06-03 | 清华大学 | Multi-view human body dynamic three-dimensional reconstruction method and system based on depth coding network |
| CN111489392A (en) * | 2020-03-30 | 2020-08-04 | 清华大学 | Single target human motion posture capturing method and system in multi-person environment |
| CN111489392B (en) * | 2020-03-30 | 2022-09-09 | 清华大学 | Single target human motion posture capturing method and system in multi-person environment |
| CN111583386A (en) * | 2020-04-20 | 2020-08-25 | 清华大学 | Multi-view human body posture reconstruction method based on label propagation algorithm |
| CN111583386B (en) * | 2020-04-20 | 2022-07-05 | 清华大学 | Multi-view human body posture reconstruction method based on label propagation algorithm |
| CN111932678A (en) * | 2020-08-13 | 2020-11-13 | 北京未澜科技有限公司 | Multi-view real-time human motion, gesture, expression and texture reconstruction system |
| CN112099330A (en) * | 2020-08-31 | 2020-12-18 | 中国人民解放军军事科学院国防科技创新研究院 | Holographic human body reconstruction method based on external camera and wearable display control equipment |
| CN112070815B (en) * | 2020-09-07 | 2024-01-05 | 哈尔滨工业大学（威海） | Automatic weight reducing method based on human body outline deformation |
| CN112070815A (en) * | 2020-09-07 | 2020-12-11 | 哈尔滨工业大学（威海） | Automatic slimming method based on human body outer contour deformation |
| CN112419479A (en) * | 2020-11-10 | 2021-02-26 | 广州二元科技有限公司 | Body type data calculation method based on weight, height and body image |
| CN112837406A (en) * | 2021-01-11 | 2021-05-25 | 聚好看科技股份有限公司 | Three-dimensional reconstruction method, device and system |
| CN112837406B (en) * | 2021-01-11 | 2023-03-14 | 聚好看科技股份有限公司 | Three-dimensional reconstruction method, device and system |
| CN112907631A (en) * | 2021-02-20 | 2021-06-04 | 北京未澜科技有限公司 | Multi-RGB camera real-time human body motion capture system introducing feedback mechanism |
| CN113284249B (en) * | 2021-06-11 | 2022-05-10 | 清华大学深圳国际研究生院 | Multi-view three-dimensional human body reconstruction method and system based on graph neural network |
| CN113284249A (en) * | 2021-06-11 | 2021-08-20 | 清华大学深圳国际研究生院 | Multi-view three-dimensional human body reconstruction method and system based on graph neural network |
| CN113658303A (en) * | 2021-06-29 | 2021-11-16 | 清华大学 | Monocular vision-based virtual human generation method and device |
| CN114373018A (en) * | 2021-12-06 | 2022-04-19 | 聚好看科技股份有限公司 | Real-time driving method, device and equipment |
| TWI822473B (en) * | 2022-11-18 | 2023-11-11 | 財團法人資訊工業策進會 | Method, processing control device and system of providing movement improvement information |
| CN116403275A (en) * | 2023-03-14 | 2023-07-07 | 南京航空航天大学 | Method and system for detecting personnel advancing posture in closed space based on multi-vision |
| CN117726907A (en) * | 2024-02-06 | 2024-03-19 | 之江实验室 | Training method of modeling model, three-dimensional human modeling method and device |
| CN117726907B (en) * | 2024-02-06 | 2024-04-30 | 之江实验室 | Training method of modeling model, three-dimensional human modeling method and device |
| CN117745978A (en) * | 2024-02-20 | 2024-03-22 | 四川大学华西医院 | Simulation quality control method, equipment and medium based on human body three-dimensional reconstruction algorithm |
| CN117745978B (en) * | 2024-02-20 | 2024-04-30 | 四川大学华西医院 | Simulation quality control method, equipment and medium based on human body three-dimensional reconstruction algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110599540B (en) | 2022-06-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110599540B (en) | Real-time three-dimensional human body shape and posture reconstruction method and device under multi-viewpoint camera | |
| CN111932678B (en) | Multi-view real-time human motion, gesture, expression and texture reconstruction system | |
| CN109448090B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN112907631B (en) | Multi-RGB camera real-time human body motion capture system introducing feedback mechanism | |
| WO2019219012A1 (en) | Three-dimensional reconstruction method and device uniting rigid motion and non-rigid deformation | |
| US20200334842A1 (en) | Methods, devices and computer program products for global bundle adjustment of 3d images | |
| CN102072706B (en) | Multi-camera positioning and tracking method and system | |
| CN104537616A (en) | Correction method for fisheye image distortion | |
| CN108053373A (en) | One kind is based on deep learning model fisheye image correcting method | |
| CN111862299A (en) | Human body three-dimensional model construction method and device, robot and storage medium | |
| CN109325995B (en) | Low-resolution multi-view hand reconstruction method based on hand parameter model | |
| WO2015197026A1 (en) | Method, apparatus and terminal for acquiring sign data of target object | |
| CN110782498B (en) | Rapid universal calibration method for visual sensing network | |
| WO2023273093A1 (en) | Human body three-dimensional model acquisition method and apparatus, intelligent terminal, and storage medium | |
| CN104318604A (en) | 3D image stitching method and apparatus | |
| CN111996883B (en) | Method for detecting width of road surface | |
| CN105865462A (en) | Three dimensional SLAM method based on events with depth enhanced vision sensor | |
| CN106203361A (en) | A kind of robotic tracking's method and apparatus | |
| CN105374067A (en) | Three-dimensional reconstruction method based on PAL cameras and reconstruction system thereof | |
| CN111583386B (en) | Multi-view human body posture reconstruction method based on label propagation algorithm | |
| CN115457176A (en) | Image generation method and device, electronic equipment and storage medium | |
| CN111047678B (en) | Three-dimensional face acquisition device and method | |
| CN103260008B (en) | A kind of image position is to the projection conversion method of physical location | |
| CN114119987A (en) | Feature extraction and descriptor generation method and system based on convolutional neural network | |
| CN112102504A (en) | Three-dimensional scene and two-dimensional image mixing method based on mixed reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221221 Address after: Room 3346, Floor 3, International Innovation Expo Center, No. 267, Kejiyuan Road, Baiyang Street, Qiantang District, Hangzhou, Zhejiang 310020 Patentee after: Hangzhou Xinchangyuan Technology Co.,Ltd. Address before: 100084 Tsinghua Yuan, Beijing, Haidian District Patentee before: TSINGHUA University |