CN108711185A - Joint rigid moves and the three-dimensional rebuilding method and device of non-rigid shape deformations - Google Patents
Joint rigid moves and the three-dimensional rebuilding method and device of non-rigid shape deformations Download PDFInfo
- Publication number
- CN108711185A CN108711185A CN201810460091.5A CN201810460091A CN108711185A CN 108711185 A CN108711185 A CN 108711185A CN 201810460091 A CN201810460091 A CN 201810460091A CN 108711185 A CN108711185 A CN 108711185A
- Authority
- CN
- China
- Prior art keywords
- rigid
- model
- dimensional
- point cloud
- vertex
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
Abstract
The invention discloses the three-dimensional rebuilding methods and device of a kind of joint rigid movement and non-rigid shape deformations, wherein method includes：Shooting based on depth camera is carried out to target object and obtains individual depth image；Three-dimensional framework extraction is carried out to depth point cloud by three-dimensional framework extraction algorithm；Obtain the matching double points between three-dimensional point cloud and reconstruction model vertex；Energy function is established according to matching double points and three-dimensional framework information, and solves the non-rigid motion evolution parameter and optimization object matrix parameter on each vertex on reconstruction model；GPU Optimization Solutions are carried out to energy function, to obtain the non-rigid shape deformations of each surface vertices, and the reconstruction threedimensional model of former frame are carried out by deformation according to solving result so that deformation model is aligned with present frame three-dimensional point cloud；The updated model for obtaining present frame, to enter the iteration of next frame.This method can effectively improve the real-time, robustness and accuracy of reconstruction, and autgmentability is strong, simple easily to realize.
Description
Technical field
The present invention relates to computer vision and computer graphics techniques field, more particularly to a kind of joint rigid movement and
The three-dimensional rebuilding method and device of non-rigid shape deformations.
Background technology
Dynamic object three-dimensional reconstruction is the Important Problems of computer graphics and computer vision field.The dynamic of high quality
Object threedimensional model, such as human body, animal, face, human hand etc. have in fields such as video display amusement, physical game, virtual realities
It is widely applied foreground and important application value.But the acquisition of high quality threedimensional model usually relies on expensive laser
Scanner or polyphaser array system realize that, although precision is higher, also significantly there is some disadvantages：First, it sweeps
Object is required to keep absolute rest during retouching, small movement may result in scanning result, and there are apparent errors；Second, it makes
It is false expensive, it is difficult to spread in general public daily life, often it is applied to major company or national statistics department.Third, speed
Slowly, it often rebuilds a threedimensional model and needs at least 10 minutes time to a few hours, rebuild the cost of dynamic model sequence more
Greatly.
Object is obtained from technical standpoint or existing method for reconstructing concentrates the rigid motion information for formerly solving object
Approach, and then rebuild non-rigid surface's movable information.But this method for reconstructing needs the key frame for obtaining object in advance three-dimensional
Model.On the other hand, although the method for reconstructing on the existing surface of dynamic fusion frame by frame can realize the dynamic three-dimensional reconstruction of no template,
But non-rigid surface's deformation method is only used only, the robustness for tracking reconstruction is low.
Invention content
The present invention is directed to solve at least some of the technical problems in related technologies.
For this purpose, an object of the present invention is to provide a kind of three-dimensional reconstruction sides of joint rigid movement and non-rigid shape deformations
Method, this method can effectively improve the real-time, robustness and accuracy of reconstruction, and autgmentability is strong, simple easily to realize.
It is another object of the present invention to propose a kind of three-dimensional reconstruction apparatus of joint rigid movement and non-rigid shape deformations.
In order to achieve the above objectives, one aspect of the present invention embodiment proposes a kind of movement of joint rigid and non-rigid shape deformations
Three-dimensional rebuilding method includes the following steps：Shooting based on depth camera is carried out to target object, to obtain individual depth map
Picture；Three-dimensional framework extraction is carried out to depth point cloud by three-dimensional framework extraction algorithm；Individual described depth image is transformed to three
Dimension point cloud, and obtain the matching double points between the three-dimensional point cloud and reconstruction model vertex；According to the matching double points and three-dimensional
Framework information establishes energy function, and solves the non-rigid motion evolution parameter on each vertex on the reconstruction model simultaneously
Optimization object matrix parameter；It is excellent that GPU (Graphics Processing Unit, graphics processor) is carried out to the energy function
Change and solve, to obtain the non-rigid shape deformations of each surface vertices, is carried out the reconstruction threedimensional model of former frame according to solving result
Deformation so that deformation model is aligned with present frame three-dimensional point cloud；Present frame three-dimensional point cloud and the deformation model are merged, with
The updated model for obtaining present frame, to enter the iteration of next frame.
The three-dimensional rebuilding method of the joint rigid movement and non-rigid shape deformations of the embodiment of the present invention, by non-rigid right in real time
Neat method, frame by frame merge dynamic object surface three dimension information, are robustly tracked to realize, realize without first frame key frame
The real-time dynamic three-dimensional reconstruction of robustness under the conditions of three-dimensional template, so as to effectively improve the real-time of reconstruction, robustness and
Accuracy, autgmentability is strong, simple easily to realize.
In addition, the three-dimensional rebuilding method of joint rigid movement according to the above embodiment of the present invention and non-rigid shape deformations may be used also
With with following additional technical characteristic：
It is further, in one embodiment of the invention, described that individual described depth image is transformed to three-dimensional point cloud,
Further comprise：Individual described depth image is projected in three dimensions by the internal reference matrix of depth camera, to generate
State three-dimensional point cloud, wherein depth map projection formula is：
Wherein, u, v are pixel coordinate, and d (u, v) is the depth value at the position pixel (u, v) on depth image,It is described
The internal reference matrix of depth camera.
Further, in one embodiment of the invention, the energy function is：
Et=λnEn+λsEs+λjEj+λgEg+λbEb,
Wherein, EtFor total energy quantifier, EnFor non-rigid surface's deformation bound term, EsFor rigid backbone kinematic constraint item, EjFor
Rigid backbone identifies bound term, EgFor local stiffness kinematic constraint item, λn、λs、λjAnd λgRespectively correspond to the power of each bound term
Weight coefficient.
Further, in one embodiment of the invention, wherein
Wherein, uiIndicate the position coordinates of three-dimensional point cloud in same matching double points, ciIndicate i-th in matching double points set
A element, in non-rigid surface's deformation bound termWithThe model vertices after non-rigid shape deformations drive are indicated respectively
Coordinate and its normal direction, in the rigid backbone kinematic constraint itemWithThe mould after object skeleton motion drives is indicated respectively
Type apex coordinate and its normal direction,WithRespectively represent the model vertices coordinate after being driven by target rigid motion and by three-dimensional bone
Frame estimates the model vertices coordinate after obtained movement driving, and in the local stiffness kinematic constraint item, i indicates on model the
I vertex,Indicate the set of the neighbouring vertices on model around i-th of vertex,WithIt respectively represents known non-rigid
Movement is to model surface vertex viAnd vjDriving effect,WithRole of delegate is in viAnd vjOn non-rigid motion it is same
When act on vjOn evolution effect.
Further, in one embodiment of the invention, it is moved according to surface non-rigid shape deformations and object rigid backbone
Driving model vertex, wherein calculation formula is：
Wherein,To act on vertex viDeformation matrix, including rotation and translation two parts；For the deformation
The rotating part of matrix；For opposite vertexes viThere is the set of the bone of driving effect；αi,jFor j-th of bone pair, i-th of model top
The weight of the driving effect of point, indicates power of the bone to the vertex driving effect；TbjBecome for the movement of j-th of bone itself
Shape matrix, rot (Tbj) be the deformation matrix rotating part.
In order to achieve the above objectives, another aspect of the present invention embodiment proposes a kind of movement of joint rigid and non-rigid shape deformations
Three-dimensional reconstruction apparatus, including：Taking module, for carrying out the shooting based on depth camera to target object, to obtain individual
Depth image；Extraction module, for carrying out three-dimensional framework extraction to depth point cloud by three-dimensional framework extraction algorithm；Match mould
Individual described depth image is transformed to three-dimensional point cloud, and obtains between the three-dimensional point cloud and reconstruction model vertex by block
With point pair；Module is resolved, for establishing energy function according to the matching double points and three-dimensional framework information, and solves the reconstruction
The non-rigid motion evolution parameter and optimization object matrix parameter on each vertex on model；Module is solved, for institute
It states energy function and carries out GPU Optimization Solutions, to obtain the non-rigid shape deformations of each surface vertices, and will be previous according to solving result
The reconstruction threedimensional model of frame carries out deformation so that deformation model is aligned with present frame three-dimensional point cloud；Model modification module is used
In fusion present frame three-dimensional point cloud and the deformation model, to obtain the updated model of present frame, to enter next frame
Iteration.
The three-dimensional reconstruction apparatus of the joint rigid movement and non-rigid shape deformations of the embodiment of the present invention, by non-rigid right in real time
Neat method, frame by frame merge dynamic object surface three dimension information, are robustly tracked to realize, realize without first frame key frame
The real-time dynamic three-dimensional reconstruction of robustness under the conditions of three-dimensional template, so as to effectively improve the real-time of reconstruction, robustness and
Accuracy, autgmentability is strong, simple easily to realize.
In addition, the three-dimensional reconstruction apparatus of joint rigid movement according to the above embodiment of the present invention and non-rigid shape deformations may be used also
With with following additional technical characteristic：
Further, in one embodiment of the invention, the matching module is further used for through depth camera
Internal reference matrix projects to individual described depth image in three dimensions, to generate the three-dimensional point cloud, wherein depth map projects
Formula is：
Wherein, u, v are pixel coordinate, and d (u, v) is the depth value at the position pixel (u, v) on depth image,It is described
The internal reference matrix of depth camera.
Further, in one embodiment of the invention, the energy function is：
Et=λnEn+λsEs+λjEj+λgEg+λbEb,
Wherein, EtFor total energy quantifier, EnFor non-rigid surface's deformation bound term, EsFor rigid backbone kinematic constraint item, EjFor
Rigid backbone identifies bound term, EgFor local stiffness kinematic constraint item, λn、λs、λjAnd λgRespectively correspond to the power of each bound term
Weight coefficient.
Further, in one embodiment of the invention, wherein
Wherein, uiIndicate the position coordinates of three-dimensional point cloud in same matching double points, ciIndicate i-th in matching double points set
A element, in non-rigid surface's deformation bound termWithThe model top after non-rigid shape deformations drive is indicated respectively
Point coordinates and its normal direction, in the rigid backbone kinematic constraint itemWithIt indicates respectively after object skeleton motion drives
Model vertices coordinate and its normal direction,WithRespectively represent the model vertices coordinate after being driven by target rigid motion and by three
Dimension skeleton estimates the model vertices coordinate after obtained movement driving, and in the local stiffness kinematic constraint item, i indicates model
Upper i-th of vertex,Indicate the set of the neighbouring vertices on model around i-th of vertex,WithIt respectively represents known non-
Rigid motion is to model surface vertex viAnd vjDriving effect,WithRole of delegate is in viAnd vjOn non-rigid fortune
It moves while acting on vjOn evolution effect.
Further, in one embodiment of the invention, it is moved according to surface non-rigid shape deformations and object rigid backbone
Driving model vertex, wherein calculation formula is：
Wherein,To act on vertex viDeformation matrix, including rotation and translation two parts；For the deformation
The rotating part of matrix；For opposite vertexes viThere is the set of the bone of driving effect；αi,jFor j-th of bone pair, i-th of model top
The weight of the driving effect of point, indicates power of the bone to the vertex driving effect；TbjBecome for the movement of j-th of bone itself
Shape matrix, rot (Tbj) be the deformation matrix rotating part.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description
Obviously, or practice through the invention is recognized.
Description of the drawings
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments
Obviously and it is readily appreciated that, wherein：
Fig. 1 is the stream of the three-dimensional rebuilding method according to the movement of the joint rigid of one embodiment of the invention and non-rigid shape deformations
Cheng Tu；
Fig. 2 is the three-dimensional rebuilding method according to the joint rigid movement and non-rigid shape deformations of one specific embodiment of the present invention
Flow chart；
Fig. 3 is the knot of the three-dimensional reconstruction apparatus according to the movement of the joint rigid of one embodiment of the invention and non-rigid shape deformations
Structure schematic diagram.
Specific implementation mode
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and is not considered as limiting the invention.
The three-dimensional of the joint rigid proposed according to embodiments of the present invention movement and non-rigid shape deformations is described with reference to the accompanying drawings
Method for reconstructing and device describe the joint rigid proposed according to embodiments of the present invention movement and non-rigid shape with reference to the accompanying drawings first
The three-dimensional rebuilding method of change.
Fig. 1 is the flow of the joint rigid movement of one embodiment of the invention and the three-dimensional rebuilding method of non-rigid shape deformations
Figure.
As shown in Figure 1, the three-dimensional rebuilding method of joint rigid movement and non-rigid shape deformations includes the following steps：
In step S101, the shooting based on depth camera is carried out to target object, to obtain individual depth image.
It is understood that as shown in Fig. 2, real-time video frame per second depth point cloud obtains, to dynamic object progress depth map
Shooting is to obtain depth point cloud frame by frame.Specifically, dynamic object is shot using depth camera, obtains individual continuous depth
Spend image sequence.Individual depth image is transformed to one group of three-dimensional point cloud.
In step s 102, three-dimensional framework extraction is carried out to depth point cloud by three-dimensional framework extraction algorithm.
It is understood that as shown in Fig. 2, by skeleton recognizer progress 3D skeletal extractions, pass through existing skeleton
The Three-dimensional Rigidity framework information of recognizer extracting object present frame.For example, realizing that object three-dimensional framework is stolen by KinectSDK
It takes.
In step s 103, individual depth image is transformed to three-dimensional point cloud, and obtains three-dimensional point cloud and reconstruction model top
Matching double points between point.
It is understood that as shown in Fig. 2, establish threedimensional model and cloud matching double points, calculate present frame three-dimensional point cloud with
Matching double points between reconstruction model vertex.
Further, in one embodiment of the invention, individual depth image is transformed to three-dimensional point cloud, further wrapped
It includes：Individual depth image is projected in three dimensions by the internal reference matrix of depth camera, to generate three-dimensional point cloud, wherein
Depth map projection formula is：
Wherein, u, v are pixel coordinate, and d (u, v) is the depth value at the position pixel (u, v) on depth image,For depth
The internal reference matrix of camera.
It is understood that shooting object to obtain depth image by depth camera, depth map is transformed to one group
Depth map is projected in three dimensions based on the internal reference matrix of depth camera calibration and is generated one group of three-dimensional point cloud by three-dimensional point cloud.
Depth map projection formula be：
Wherein, u, v are pixel coordinate, and d (u, v) is the depth value at the position pixel (u, v) on depth image,For depth
Camera internal reference matrix.
Specifically, obtaining the internal reference matrix of depth camera, depth map is projected in three dimensions according to internal reference matrix
It is transformed to one group of three-dimensional point cloud.Wherein, the formula of transformation is：Wherein, u, v are pixel coordinate, d
(u, v) is the depth value at the position pixel (u, v) on depth image,For depth camera internal reference matrix.Obtaining matching double points
Aspect is projected on the vertex of threedimensional model on depth image to obtain matching double points using camera projection formula.
Further, in one embodiment of the invention, it is moved according to surface non-rigid shape deformations and object rigid backbone
Driving model vertex, wherein calculation formula is：
Wherein,To act on vertex viDeformation matrix, including rotation and translation two parts；For the deformation
The rotating part of matrix；For opposite vertexes viThere is the set of the bone of driving effect；αi,jFor j-th of bone pair, i-th of model top
The weight of the driving effect of point, indicates power of the bone to the vertex driving effect；TbjBecome for the movement of j-th of bone itself
Shape matrix, rot (Tbj) be the deformation matrix rotating part.
In step S104, energy function is established according to matching double points and three-dimensional framework information, and solve on reconstruction model
The non-rigid motion evolution parameter and optimization object matrix parameter on each vertex.
It is understood that energy function is established, it is three-dimensional according to present frame matching double points information, and the present frame of extraction
Rigid backbone information, establishes energy function.
For example, using single depth camera, such as optical depth in Microsoft's Kinect depth cameras, IphoneX depth cameras, ratio difficult to understand
Camera etc. is spent, dynamic scene is shot, acquisition real-time deep image data (video frame rate, 20 frames/it is more than the second) be transmitted to
On computer, the three-dimensional geometric information of dynamic object is calculated in real time by computer, rebuild the object threedimensional model under identical frame per second
And export the three-dimensional framework information of object.
Further, in one embodiment of the invention, energy function is：
Et=λnEn+λsEs+λjEj+λgEg+λbEb,
Wherein, EtFor total energy quantifier, EnFor non-rigid surface's deformation bound term, EsFor rigid backbone kinematic constraint item, EjFor
Rigid backbone identifies bound term, EgFor local stiffness kinematic constraint item, λn、λs、λjAnd λgRespectively correspond to the power of each bound term
Weight coefficient.
Further, in one embodiment of the invention, wherein
Wherein, uiIndicate the position coordinates of three-dimensional point cloud in same matching double points, ciIndicate i-th in matching double points set
A element, in non-rigid surface's deformation bound termWithThe model vertices coordinate after non-rigid shape deformations drive is indicated respectively
And its normal direction, in rigid backbone kinematic constraint itemWithIndicate that the model vertices after object skeleton motion drives are sat respectively
Mark and its normal direction,WithIt respectively represents the model vertices coordinate after being driven by target rigid motion and estimates institute with by three-dimensional framework
Model vertices coordinate after obtained movement driving, in local stiffness kinematic constraint item, i indicates i-th of vertex on model,Indicate the set of the neighbouring vertices on model around i-th of vertex,WithRespectively represent known non-rigid motion pair
Model surface vertex viAnd vjDriving effect,WithRole of delegate is in viAnd vjOn non-rigid motion act on simultaneously
In vjOn evolution effect.
Specifically, having used rigid motion bound term E simultaneouslysWith non-rigid motion bound term EnCarry out object movement
Optimization Solution, while single depth image having been used to carry out object rigid backbone bound term EjThe rigid motion of solution is carried out about
Beam.
(1) surface non-rigid restraint EnThe three-dimensional point cloud for ensureing the model after non-rigid shape deformations and being obtained from depth map
It is aligned as far as possible；WithModel vertices coordinate and its normal direction after non-rigid shape deformations drive are indicated respectively,With
The model vertices coordinate after object skeleton motion drives and its normal direction are indicated respectively.
(2) rigid backbone kinematic constraint item EsEnsure the model after skeleton motion drives rigid deformation with from depth map
The three-dimensional point cloud of acquisition is aligned as far as possible.
(3) in rigid backbone movement and non-rigid shape deformations consistency constraint item EbIn,WithIt respectively represents by target rigidity
Model vertices coordinate after movement driving estimates the model vertices coordinate after obtained movement driving as possible with by three-dimensional framework
Unanimously, the rigid backbone and the skeleton identified which is used to ensure to resolve are consistent as far as possible, pass through single frames bone
The case where frame identification, anti-dynamic tracks the deviation accumulation occurred in the process and can not restore, it thereby may be ensured that final resolving
Non-rigid motion out meets object skeleton kinetic model, but adequately with the three-dimensional point cloud pair that is obtained from depth map
Together.
(4) in local stiffness kinematic constraint item EgIn, i indicates i-th of vertex on model,It indicates on model i-th
The set of neighbouring vertices around vertex,WithKnown non-rigid motion is respectively represented to model surface vertex viAnd vjDrive
Action is used,WithRole of delegate is in viAnd vjOn non-rigid motion simultaneously act on vjOn evolution effect,
To ensure that the non-rigid driving effect of neighbouring vertices on model is consistent as far as possible.It is that a robust is punished
Penalty function,WithRigid backbone movement is respectively represented to model surface vertex viAnd vjDriving effect, work as model surface
Two adjacent vertexs by rigid backbone move driving effect difference it is larger when, the robust penalty value is smaller, when two it is adjacent
When vertex is smaller by skeleton motion driving effect difference, robust penalty value is larger, can be with by the robust penalty
Ensure that rational non-rigid motion by a relatively large margin also can be fine while making model integrally by local stiffness constrained motion
Resolve, to make model be more accurately aligned with three-dimensional point cloud.
In step S105, GPU Optimization Solutions are carried out to energy function, to obtain the non-rigid shape of each surface vertices
Become, and the reconstruction threedimensional model of former frame is carried out by deformation according to solving result so that deformation model and present frame three-dimensional point cloud
It is aligned.
It is understood that as shown in Fig. 2, to energy function progress GPU Optimization Solutions, each on reconstruction model is solved
The non-rigid motion evolution parameter on vertex, and optimization object Three-dimensional Rigidity movable information, according to solving result to former frame
Reconstruction model carries out deformation, is allowed to be aligned with present frame three-dimensional point cloud.
Specifically, being solved to energy function, reconstruction model is aligned with three-dimensional point cloud according to solving result.
With the non-rigid motion evolution parameter and object skeleton motion parameter for solving each vertex on reconstruction model.It is final to solve
The information of acquisition be each threedimensional model vertex transformation matrix and object skeleton motion parameter, i.e., each bone it is individual
Transformation matrix.In order to realize requirement that fast linear solves, the method for the embodiment of the present invention is to utilization index mapping method to becoming
Shape equation is done such as lower aprons：
Wherein,For by the model vertices v of previous frameiAccumulation transformation matrix, be known quantity,For each surface
The non-rigid shape deformations on vertex；I is four-dimensional unit matrix；
Wherein,It enablesModel vertices i.e. after previous frame transformation, then
Have by transformation：
For each vertex, it is desirable that the unknown parameter of solution is 6 DOF transformation parameter x=(v1,v2,v3,wx,wy,wz)T.Bone
The linearized fashion of bone movement is identical as non-rigid motion.
In step s 106, present frame three-dimensional point cloud and deformation model are merged, to obtain the updated model of present frame,
To enter the iteration of next frame.
It is understood that as shown in Fig. 2, to after alignment model and point cloud carry out graph cut, obtain a new frame compared with
Complete threedimensional model.
Specifically, merging point cloud and threedimensional model, obtain the more new model of present frame.After being aligned with depth point cloud
Threedimensional model be updated and completion, the depth information newly obtained is fused in threedimensional model, update threedimensional model surface
Vertex position increases new vertex for threedimensional model, it is made more to meet the expression of current depth image.
To sum up, Core Feature of the embodiment of the present invention is to receive depth image code stream in real time, is calculated in real time per frame three-dimensional mould
Type.Simultaneously dynamic object is calculated using the surface non-rigid shape deformations information of the movement of the large scale rigid backbone of object and small scale
Time-varying threedimensional model.It is accurate that the method for the embodiment of the present invention solves, and may be implemented to carry out high-precision reconstruction to dynamic object in real time,
It since this method is real-time reconstruction method, and only needs to provide single depth camera input, system has equipment simple, facilitates portion
Administration waits a little with expansible, and required input information is very easy to acquisition, and can obtain dynamic 3 D model in real time.It should
Method solves accurate robust, and simple and practicable, the speed of service is real-time, gathers around and has broad application prospects, can be in PC (personal
Computer, personal computer) it fast implements on the hardware systems such as machine or work station.
The three-dimensional rebuilding method of the joint rigid movement and non-rigid shape deformations that propose according to embodiments of the present invention, by real-time
The method of non-rigid alignment, frame by frame merge dynamic object surface three dimension information, are robustly tracked to realize, realize in no head
The real-time dynamic three-dimensional reconstruction of robustness under the conditions of frame key frame three-dimensional template, so as to effectively improve reconstruction real-time,
Robustness and accuracy, autgmentability is strong, simple easily to realize.
Referring next to the three-dimensional of joint rigid movement and non-rigid shape deformations that attached drawing description proposes according to embodiments of the present invention
Reconstructing device.
Fig. 3 is that the joint rigid movement of one embodiment of the invention and the structure of the three-dimensional reconstruction apparatus of non-rigid shape deformations are shown
It is intended to.
As shown in figure 3, the three-dimensional reconstruction apparatus 10 of joint rigid movement and non-rigid shape deformations includes：Taking module 100,
Extraction module 200, matching module 300 resolve module 400, solve module 500 and model modification module 600.
Wherein, taking module 100 to target object for carrying out the shooting based on depth camera, to obtain individual depth map
Picture.Extraction module 200 is used to carry out three-dimensional framework extraction to depth point cloud by three-dimensional framework extraction algorithm.Matching module 300
Individual depth image is transformed to three-dimensional point cloud, and obtains the matching double points between three-dimensional point cloud and reconstruction model vertex.It resolves
Module 400 is used to establish energy function according to matching double points and three-dimensional framework information, and solves each vertex on reconstruction model
Non-rigid motion evolution parameter and optimization object matrix parameter.It is excellent for carrying out GPU to energy function to solve module 500
Change and solve, to obtain the non-rigid shape deformations of each surface vertices, and according to solving result by the reconstruction threedimensional model of former frame into
Row deformation so that deformation model is aligned with present frame three-dimensional point cloud.Model modification module 600 is for merging present frame three-dimensional
Point cloud and deformation model, to obtain the updated model of present frame, to enter the iteration of next frame.The dress of the embodiment of the present invention
10 real-times, robustness and accuracy that can effectively improve reconstruction are set, autgmentability is strong, simple easily to realize.
Further, in one embodiment of the invention, matching module 300 is further used for by depth camera
Ginseng matrix projects to individual depth image in three dimensions, to generate three-dimensional point cloud, wherein depth map projection formula is：
Wherein, u, v are pixel coordinate, and d (u, v) is the depth value at the position pixel (u, v) on depth image,For depth
The internal reference matrix of camera.
Further, in one embodiment of the invention, energy function is：
Et=λnEn+λsEs+λjEj+λgEg+λbEb,
Wherein, EtFor total energy quantifier, EnFor non-rigid surface's deformation bound term, EsFor rigid backbone kinematic constraint item, EjFor
Rigid backbone identifies bound term, EgFor local stiffness kinematic constraint item, λn、λs、λjAnd λgRespectively correspond to the power of each bound term
Weight coefficient.
Further, in one embodiment of the invention, wherein
Wherein, uiIndicate the position coordinates of three-dimensional point cloud in same matching double points, ciIndicate i-th in matching double points set
A element, in non-rigid surface's deformation bound termWithThe model vertices coordinate after non-rigid shape deformations drive is indicated respectively
And its normal direction, in rigid backbone kinematic constraint itemWithIndicate that the model vertices after object skeleton motion drives are sat respectively
Mark and its normal direction,WithIt respectively represents the model vertices coordinate after being driven by target rigid motion and estimates institute with by three-dimensional framework
Model vertices coordinate after obtained movement driving, in local stiffness kinematic constraint item, i indicates i-th of vertex on model,
Indicate the set of the neighbouring vertices on model around i-th of vertex,WithKnown non-rigid motion is respectively represented to model table
Vertex of surface viAnd vjDriving effect,WithRole of delegate is in viAnd vjOn non-rigid motion simultaneously act on vjOn
Evolution effect.
Further, in one embodiment of the invention, it is moved according to surface non-rigid shape deformations and object rigid backbone
Driving model vertex, wherein calculation formula is：
Wherein,To act on vertex viDeformation matrix, including rotation and translation two parts；For the deformation
The rotating part of matrix；For opposite vertexes viThere is the set of the bone of driving effect；αi,jFor j-th of bone pair, i-th of model vertices
Driving effect weight, indicate power of the bone to the vertex driving effect；TbjFor the motion deformation of j-th of bone itself
Matrix, rot (Tbj) be the deformation matrix rotating part.
It should be noted that the explanation of the aforementioned three-dimensional rebuilding method embodiment to joint rigid motion and non-rigid shape deformations
Illustrate the three-dimensional reconstruction apparatus of the joint rigid movement and non-rigid shape deformations that are also applied for the embodiment, details are not described herein again.
The three-dimensional reconstruction apparatus of the joint rigid movement and non-rigid shape deformations that propose according to embodiments of the present invention, by real-time
The method of non-rigid alignment, frame by frame merge dynamic object surface three dimension information, are robustly tracked to realize, realize in no head
The real-time dynamic three-dimensional reconstruction of robustness under the conditions of frame key frame three-dimensional template, so as to effectively improve reconstruction real-time,
Robustness and accuracy, autgmentability is strong, simple easily to realize.
In the description of the present invention, it is to be understood that, term "center", " longitudinal direction ", " transverse direction ", " length ", " width ",
" thickness ", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom" "inner", "outside", " up time
The orientation or positional relationship of the instructions such as needle ", " counterclockwise ", " axial direction ", " radial direction ", " circumferential direction " be orientation based on ... shown in the drawings or
Position relationship is merely for convenience of description of the present invention and simplification of the description, and does not indicate or imply the indicated device or element must
There must be specific orientation, with specific azimuth configuration and operation, therefore be not considered as limiting the invention.
In addition, term " first ", " second " are used for description purposes only, it is not understood to indicate or imply relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, the meaning of " plurality " is at least two, such as two, three
It is a etc., unless otherwise specifically defined.
In the present invention unless specifically defined or limited otherwise, term " installation ", " connected ", " connection ", " fixation " etc.
Term shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or integral；Can be that machinery connects
It connects, can also be electrical connection；It can be directly connected, can also can be indirectly connected through an intermediary in two elements
The interaction relationship of the connection in portion or two elements, unless otherwise restricted clearly.For those of ordinary skill in the art
For, the specific meanings of the above terms in the present invention can be understood according to specific conditions.
In the present invention unless specifically defined or limited otherwise, fisrt feature can be with "above" or "below" second feature
It is that the first and second features are in direct contact or the first and second features pass through intermediary mediate contact.Moreover, fisrt feature exists
Second feature " on ", " top " and " above " but fisrt feature be directly above or diagonally above the second feature, or be merely representative of
Fisrt feature level height is higher than second feature.Fisrt feature second feature " under ", " lower section " and " below " can be
One feature is directly under or diagonally below the second feature, or is merely representative of fisrt feature level height and is less than second feature.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiments or example.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
Although the embodiments of the present invention has been shown and described above, it is to be understood that above-described embodiment is example
Property, it is not considered as limiting the invention, those skilled in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, changes, replacing and modification.
Claims (10)
1. a kind of three-dimensional rebuilding method of joint rigid movement and non-rigid shape deformations, which is characterized in that include the following steps：
Shooting based on depth camera is carried out to target object, to obtain individual depth image；
Three-dimensional framework extraction is carried out to depth point cloud by three-dimensional framework extraction algorithm；
Individual described depth image is transformed to three-dimensional point cloud, and obtains between the three-dimensional point cloud and reconstruction model vertex
With point pair；
Energy function is established according to the matching double points and three-dimensional framework information, and solves each vertex on the reconstruction model
Non-rigid motion evolution parameter and optimization object matrix parameter；
GPU Optimization Solutions are carried out to the energy function, to obtain the non-rigid shape deformations of each surface vertices, and are tied according to solving
The reconstruction threedimensional model of former frame is carried out deformation by fruit so that deformation model is aligned with present frame three-dimensional point cloud；And
Present frame three-dimensional point cloud and the deformation model are merged, to obtain the updated model of present frame, to enter next frame
Iteration.
2. the three-dimensional rebuilding method of joint rigid movement and non-rigid shape deformations according to claim 1, which is characterized in that institute
It states and individual described depth image is transformed to three-dimensional point cloud, further comprise：
Individual described depth image is projected in three dimensions by the internal reference matrix of depth camera, to generate the three-dimensional point
Cloud, wherein depth map projection formula is：
Wherein, u, v are pixel coordinate, and d (u, v) is the depth value at the position pixel (u, v) on depth image,For the depth
The internal reference matrix of camera.
3. the three-dimensional rebuilding method of joint rigid movement and non-rigid shape deformations according to claim 1, which is characterized in that institute
Stating energy function is：
Et=λnEn+λsEs+λjEj+λgEg+λbEb,
Wherein, EtFor total energy quantifier, EnFor non-rigid surface's deformation bound term, EsFor rigid backbone kinematic constraint item, EjFor rigidity
Skeleton identifies bound term, EgFor local stiffness kinematic constraint item, λn、λs、λjAnd λgRespectively correspond to the weight system of each bound term
Number.
4. the three-dimensional rebuilding method of joint rigid movement and non-rigid shape deformations according to claim 3, which is characterized in that its
In,
Wherein, uiIndicate the position coordinates of three-dimensional point cloud in same matching double points, ciIndicate i-th yuan in matching double points set
Element, in non-rigid surface's deformation bound termWithIndicate that the model vertices after non-rigid shape deformations drive are sat respectively
Mark and its normal direction, in the rigid backbone kinematic constraint itemWithThe model after object skeleton motion drives is indicated respectively
Apex coordinate and its normal direction,WithRespectively represent the model vertices coordinate after being driven by target rigid motion and by three-dimensional framework
Estimate the model vertices coordinate after obtained movement driving, in the local stiffness kinematic constraint item, i is indicated i-th on model
A vertex,Indicate the set of the neighbouring vertices on model around i-th of vertex,WithIt respectively represents known non-rigid
Movement is to model surface vertex viAnd vjDriving effect,WithRole of delegate is in viAnd vjOn non-rigid motion it is same
When act on vjOn evolution effect.
5. special according to the three-dimensional rebuilding method of claim 1-4 any one of them joint rigids movement and non-rigid shape deformations
Sign is, according to surface non-rigid shape deformations and object rigid backbone movement driving model vertex, wherein calculation formula is：
Wherein,To act on vertex viDeformation matrix, including rotation and translation two parts；For the deformation matrix
Rotating part；For opposite vertexes viThere is the set of the bone of driving effect；αi,jFor j-th of bone pair, i-th of model vertices
The weight of driving effect indicates power of the bone to the vertex driving effect；TbjFor the motion deformation square of j-th of bone itself
Battle array, rot (Tbj) be the deformation matrix rotating part.
6. a kind of three-dimensional reconstruction apparatus of joint rigid movement and non-rigid shape deformations, which is characterized in that including：
Taking module, for carrying out the shooting based on depth camera to target object, to obtain individual depth image；
Extraction module, for carrying out three-dimensional framework extraction to depth point cloud by three-dimensional framework extraction algorithm；
Individual described depth image is transformed to three-dimensional point cloud, and obtains the three-dimensional point cloud and reconstruction model top by matching module
Matching double points between point；
Module is resolved, for establishing energy function according to the matching double points and three-dimensional framework information, and solves the reconstruction mould
The non-rigid motion evolution parameter and optimization object matrix parameter on each vertex in type；
Module is solved, for carrying out GPU Optimization Solutions to the energy function, to obtain the non-rigid shape of each surface vertices
Become, and the reconstruction threedimensional model of former frame is carried out by deformation according to solving result so that deformation model and present frame three-dimensional point cloud
It is aligned；And
Model modification module, for merging present frame three-dimensional point cloud and the deformation model, to obtain the updated of present frame
Model, to enter the iteration of next frame.
7. the three-dimensional reconstruction apparatus of joint rigid movement and non-rigid shape deformations according to claim 6, which is characterized in that institute
Matching module is stated to be further used for projecting to individual described depth image in three dimensions by the internal reference matrix of depth camera,
To generate the three-dimensional point cloud, wherein depth map projection formula is：
Wherein, u, v are pixel coordinate, and d (u, v) is the depth value at the position pixel (u, v) on depth image,For the depth
The internal reference matrix of camera.
8. the three-dimensional reconstruction apparatus of joint rigid movement and non-rigid shape deformations according to claim 6, which is characterized in that institute
Stating energy function is：
Et=λnEn+λsEs+λjEj+λgEg+λbEb,
Wherein, EtFor total energy quantifier, EnFor non-rigid surface's deformation bound term, EsFor rigid backbone kinematic constraint item, EjFor rigidity
Skeleton identifies bound term, EgFor local stiffness kinematic constraint item, λn、λs、λjAnd λgRespectively correspond to the weight system of each bound term
Number.
9. the three-dimensional reconstruction apparatus of joint rigid movement and non-rigid shape deformations according to claim 8, which is characterized in that its
In,
Wherein, uiIndicate the position coordinates of three-dimensional point cloud in same matching double points, ciIndicate i-th yuan in matching double points set
Element, in non-rigid surface's deformation bound termWithIndicate that the model vertices after non-rigid shape deformations drive are sat respectively
Mark and its normal direction, in the rigid backbone kinematic constraint itemWithThe model after object skeleton motion drives is indicated respectively
Apex coordinate and its normal direction,WithRespectively represent the model vertices coordinate after being driven by target rigid motion and by three-dimensional framework
Estimate the model vertices coordinate after obtained movement driving, in the local stiffness kinematic constraint item, i is indicated i-th on model
A vertex,Indicate the set of the neighbouring vertices on model around i-th of vertex,WithIt respectively represents known non-rigid
Movement is to model surface vertex viAnd vjDriving effect,WithRole of delegate is in viAnd vjOn non-rigid motion it is same
When act on vjOn evolution effect.
10. special according to the three-dimensional reconstruction apparatus of claim 6-9 any one of them joint rigids movement and non-rigid shape deformations
Sign is, according to surface non-rigid shape deformations and object rigid backbone movement driving model vertex, wherein calculation formula is：
Wherein,To act on vertex viDeformation matrix, including rotation and translation two parts；For the deformation matrix
Rotating part；For opposite vertexes viThere is the set of the bone of driving effect；αi,jFor j-th of bone pair, i-th of model vertices
The weight of driving effect indicates power of the bone to the vertex driving effect；TbjFor the motion deformation square of j-th of bone itself
Battle array, rot (Tbj) be the deformation matrix rotating part.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810460091.5A CN108711185B (en) | 2018-05-15 | 2018-05-15 | Three-dimensional reconstruction method and device combining rigid motion and non-rigid deformation |
| PCT/CN2019/086889 WO2019219012A1 (en) | 2018-05-15 | 2019-05-14 | Three-dimensional reconstruction method and device uniting rigid motion and non-rigid deformation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810460091.5A CN108711185B (en) | 2018-05-15 | 2018-05-15 | Three-dimensional reconstruction method and device combining rigid motion and non-rigid deformation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108711185A true CN108711185A (en) | 2018-10-26 |
| CN108711185B CN108711185B (en) | 2021-05-28 |
Family
ID=63869046
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810460091.5A Active CN108711185B (en) | 2018-05-15 | 2018-05-15 | Three-dimensional reconstruction method and device combining rigid motion and non-rigid deformation |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108711185B (en) |
| WO (1) | WO2019219012A1 (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109785440A (en) * | 2018-12-18 | 2019-05-21 | 合肥阿巴赛信息科技有限公司 | A kind of curved surface distorted pattern |
| CN109829972A (en) * | 2019-01-19 | 2019-05-31 | 北京工业大学 | A kind of 3 D human body standard framework extraction method towards successive frame point cloud |
| CN109840940A (en) * | 2019-02-11 | 2019-06-04 | 清华-伯克利深圳学院筹备办公室 | Dynamic three-dimensional reconstruction method, device, equipment, medium and system |
| CN110006408A (en) * | 2019-04-17 | 2019-07-12 | 武汉大学 | LiDAR data " cloud control " aviation image photogrammetric survey method |
| CN110070595A (en) * | 2019-04-04 | 2019-07-30 | 东南大学 | A kind of single image 3D object reconstruction method based on deep learning |
| WO2019219012A1 (en) * | 2018-05-15 | 2019-11-21 | 清华大学 | Three-dimensional reconstruction method and device uniting rigid motion and non-rigid deformation |
| CN110689625A (en) * | 2019-09-06 | 2020-01-14 | 清华大学 | Automatic generation method and device for customized face mixed expression model |
| CN111768504A (en) * | 2019-03-30 | 2020-10-13 | 华为技术有限公司 | Model processing method, deformation control method and related equipment |
| CN111862139A (en) * | 2019-08-16 | 2020-10-30 | 中山大学 | Dynamic object parametric modeling method based on color-depth camera |
| CN111968169A (en) * | 2020-08-19 | 2020-11-20 | 北京拙河科技有限公司 | Dynamic human body three-dimensional reconstruction method, device, equipment and medium |
| CN112991524A (en) * | 2021-04-20 | 2021-06-18 | 北京的卢深视科技有限公司 | Three-dimensional reconstruction method, electronic device and storage medium |
| CN113096249A (en) * | 2021-03-30 | 2021-07-09 | Oppo广东移动通信有限公司 | Method for training vertex reconstruction model, image reconstruction method and electronic equipment |
| CN114373018A (en) * | 2021-12-06 | 2022-04-19 | 聚好看科技股份有限公司 | Real-time driving method, device and equipment |
| CN114648613A (en) * | 2022-05-18 | 2022-06-21 | 杭州像衍科技有限公司 | Three-dimensional head model reconstruction method and device based on deformable nerve radiation field |
| WO2024007523A1 (en) * | 2022-07-08 | 2024-01-11 | 北京大学深圳研究生院 | Point cloud motion estimation method and apparatus, electronic device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100235118A1 (en) * | 2009-03-16 | 2010-09-16 | Bradford Allen Moore | Event Recognition |
| CN102800103A (en) * | 2012-06-18 | 2012-11-28 | 清华大学 | Unmarked motion capturing method and device based on multi-visual angle depth camera |
| CN103198523A (en) * | 2013-04-26 | 2013-07-10 | 清华大学 | Three-dimensional non-rigid body reconstruction method and system based on multiple depth maps |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101801749B1 (en) * | 2016-08-24 | 2017-11-28 | 한국과학기술연구원 | Method of deblurring multi-view stereo for 3d shape reconstruction, recording medium and device for performing the method |
| CN106469465A (en) * | 2016-08-31 | 2017-03-01 | 深圳市唯特视科技有限公司 | A kind of three-dimensional facial reconstruction method based on gray scale and depth information |
| CN107358645B (en) * | 2017-06-08 | 2020-08-11 | 上海交通大学 | Product three-dimensional model reconstruction method and system |
| CN108711185B (en) * | 2018-05-15 | 2021-05-28 | 清华大学 | Three-dimensional reconstruction method and device combining rigid motion and non-rigid deformation |
-
2018
- 2018-05-15 CN CN201810460091.5A patent/CN108711185B/en active Active
-
2019
- 2019-05-14 WO PCT/CN2019/086889 patent/WO2019219012A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100235118A1 (en) * | 2009-03-16 | 2010-09-16 | Bradford Allen Moore | Event Recognition |
| CN102800103A (en) * | 2012-06-18 | 2012-11-28 | 清华大学 | Unmarked motion capturing method and device based on multi-visual angle depth camera |
| CN103198523A (en) * | 2013-04-26 | 2013-07-10 | 清华大学 | Three-dimensional non-rigid body reconstruction method and system based on multiple depth maps |
Non-Patent Citations (1)
| Title |
|---|
| 于涛等: "BodyFusion：Real-time Capture of Human Motion and Surface Geometry Using a Single Depth Camera", 《2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019219012A1 (en) * | 2018-05-15 | 2019-11-21 | 清华大学 | Three-dimensional reconstruction method and device uniting rigid motion and non-rigid deformation |
| CN109785440A (en) * | 2018-12-18 | 2019-05-21 | 合肥阿巴赛信息科技有限公司 | A kind of curved surface distorted pattern |
| CN109829972A (en) * | 2019-01-19 | 2019-05-31 | 北京工业大学 | A kind of 3 D human body standard framework extraction method towards successive frame point cloud |
| CN109840940A (en) * | 2019-02-11 | 2019-06-04 | 清华-伯克利深圳学院筹备办公室 | Dynamic three-dimensional reconstruction method, device, equipment, medium and system |
| CN111768504B (en) * | 2019-03-30 | 2023-07-14 | 华为技术有限公司 | Model processing method, deformation control method and related equipment |
| CN111768504A (en) * | 2019-03-30 | 2020-10-13 | 华为技术有限公司 | Model processing method, deformation control method and related equipment |
| CN110070595A (en) * | 2019-04-04 | 2019-07-30 | 东南大学 | A kind of single image 3D object reconstruction method based on deep learning |
| CN110006408A (en) * | 2019-04-17 | 2019-07-12 | 武汉大学 | LiDAR data " cloud control " aviation image photogrammetric survey method |
| CN110006408B (en) * | 2019-04-17 | 2020-04-24 | 武汉大学 | LiDAR data cloud control aerial image photogrammetry method |
| CN111862139A (en) * | 2019-08-16 | 2020-10-30 | 中山大学 | Dynamic object parametric modeling method based on color-depth camera |
| CN111862139B (en) * | 2019-08-16 | 2023-08-18 | 中山大学 | Dynamic object parametric modeling method based on color-depth camera |
| CN110689625B (en) * | 2019-09-06 | 2021-07-16 | 清华大学 | Automatic generation method and device for customized face mixed expression model |
| CN110689625A (en) * | 2019-09-06 | 2020-01-14 | 清华大学 | Automatic generation method and device for customized face mixed expression model |
| CN111968169A (en) * | 2020-08-19 | 2020-11-20 | 北京拙河科技有限公司 | Dynamic human body three-dimensional reconstruction method, device, equipment and medium |
| CN111968169B (en) * | 2020-08-19 | 2024-01-19 | 北京拙河科技有限公司 | Dynamic human body three-dimensional reconstruction method, device, equipment and medium |
| CN113096249B (en) * | 2021-03-30 | 2023-02-17 | Oppo广东移动通信有限公司 | Method for training vertex reconstruction model, image reconstruction method and electronic equipment |
| CN113096249A (en) * | 2021-03-30 | 2021-07-09 | Oppo广东移动通信有限公司 | Method for training vertex reconstruction model, image reconstruction method and electronic equipment |
| CN112991524A (en) * | 2021-04-20 | 2021-06-18 | 北京的卢深视科技有限公司 | Three-dimensional reconstruction method, electronic device and storage medium |
| CN114373018A (en) * | 2021-12-06 | 2022-04-19 | 聚好看科技股份有限公司 | Real-time driving method, device and equipment |
| CN114648613A (en) * | 2022-05-18 | 2022-06-21 | 杭州像衍科技有限公司 | Three-dimensional head model reconstruction method and device based on deformable nerve radiation field |
| WO2024007523A1 (en) * | 2022-07-08 | 2024-01-11 | 北京大学深圳研究生院 | Point cloud motion estimation method and apparatus, electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019219012A1 (en) | 2019-11-21 |
| CN108711185B (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108711185A (en) | Joint rigid moves and the three-dimensional rebuilding method and device of non-rigid shape deformations | |
| CN108665537A (en) | The three-dimensional rebuilding method and system of combined optimization human body figure and display model | |
| Bogo et al. | Dynamic FAUST: Registering human bodies in motion | |
| US20170132827A1 (en) | Data Driven Design and Animation of Animatronics | |
| CN108122275A (en) | Dynamic realtime 3 D human body method for reconstructing and system based on skeleton tracking | |
| CN104915978B (en) | Realistic animation generation method based on body-sensing camera Kinect | |
| CN108629831A (en) | 3 D human body method for reconstructing and system based on parametric human body template and inertia measurement | |
| WO2019219014A1 (en) | Three-dimensional geometry and eigencomponent reconstruction method and device based on light and shadow optimization | |
| CN102222363A (en) | Method for fast constructing high-accuracy personalized face model on basis of facial images | |
| CN102306386B (en) | Method for quickly constructing third dimension tree model from single tree image | |
| US20090102834A1 (en) | Image processing apparatus and image processing method | |
| CN107194991A (en) | A kind of three-dimensional global visualization monitoring system construction method updated based on skeletal point local dynamic | |
| CN104794728A (en) | Method for reconstructing real-time three-dimensional face data with multiple images | |
| Saito et al. | Appearance-based virtual view generation from multicamera videos captured in the 3-d room | |
| CN108122281B (en) | Large-range real-time human body three-dimensional reconstruction method | |
| CN106875437A (en) | A kind of extraction method of key frame towards RGBD three-dimensional reconstructions | |
| CN112530005B (en) | Three-dimensional model linear structure recognition and automatic restoration method | |
| JP2000268179A (en) | Three-dimensional shape information obtaining method and device, two-dimensional picture obtaining method and device and record medium | |
| CN107862733A (en) | Large scale scene real-time three-dimensional method for reconstructing and system based on sight more new algorithm | |
| CN108776990A (en) | Three-dimensional rebuilding method and system when dynamic scene high-precision real based on three rank spherical harmonics | |
| Wu et al. | Ganhead: Towards generative animatable neural head avatars | |
| Zhang et al. | Adaptive affine transformation: A simple and effective operation for spatial misaligned image generation | |
| CN108615221A (en) | Light field angle super-resolution rate method and device based on the two-dimentional epipolar plane figure of shearing | |
| CN102855652B (en) | Method for redirecting and cartooning face expression on basis of radial basis function for geodesic distance | |
| KR101212110B1 (en) | Method for 2d and 3d virtual face modeling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |