CN110599535A - High-resolution human body real-time dynamic reconstruction method and device based on hash table - Google Patents
High-resolution human body real-time dynamic reconstruction method and device based on hash table Download PDFInfo
- Publication number
- CN110599535A CN110599535A CN201910717605.5A CN201910717605A CN110599535A CN 110599535 A CN110599535 A CN 110599535A CN 201910717605 A CN201910717605 A CN 201910717605A CN 110599535 A CN110599535 A CN 110599535A
- Authority
- CN
- China
- Prior art keywords
- model
- human body
- vertex
- reconstruction
- hash table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Generation (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a high-resolution human body real-time dynamic reconstruction method and a device based on a hash table, wherein the method comprises the following steps: acquiring a plurality of depth images of a single human body through a depth camera; projecting each depth image to a three-dimensional space through camera internal parameters to obtain three-dimensional point cloud, and solving corresponding points between the three-dimensional point cloud and the top point of the reconstructed model and the embedded human body parametric model; establishing an energy function according to the corresponding points, and performing optimization solution to obtain the posture of the human body parameterized model so as to obtain non-rigid transformation parameters of each vertex of the reconstructed model; and carrying out non-rigid transformation on the reconstruction model, transforming the reconstruction model to a real-time frame to enable the reconstruction model to be a newly-added voxel and allocate a video memory space in the hash table, and carrying out voxel fusion on all voxels in the hash table according to non-rigid transformation parameters to obtain a human body reconstruction model so as to dynamically reconstruct the human body. The method has accurate solution, can realize real-time human body dynamic reconstruction, and is suitable for the condition of high resolution.
Description
Technical Field
The invention relates to the technical field of computer vision and computer graphics, in particular to a high-resolution human body real-time dynamic reconstruction method and device based on a hash table.
Background
Dynamic three-dimensional reconstruction of a human body is a key problem in the fields of computer graphics and computer vision. The high-quality human body three-dimensional model has wide application prospect and important application value in the fields of movie and television entertainment, demographic data analysis and the like. However, the acquisition of high-quality human body three-dimensional models is usually realized by means of expensive laser scanners or multi-camera array systems, and although the accuracy is high, some disadvantages are also obviously existed: firstly, the video memory occupies a large area, and the conventional method is generally based on uniformly divided voxel grids, so that the method is difficult to adapt to the high-resolution situation. Second, the slow speed often requires at least 10 minutes to hours to reconstruct a three-dimensional phantom.
Disclosure of Invention
The present invention is directed to solving, at least to some extent, one of the technical problems in the related art.
Therefore, one purpose of the invention is to provide a high-resolution human body real-time dynamic reconstruction method based on a hash table, which is accurate in solving, can realize real-time human body dynamic reconstruction, can adapt to the high-resolution situation, can provide good interactive three-dimensional reconstruction experience for users, and has wide application prospects.
The invention also aims to provide a high-resolution human body real-time dynamic reconstruction device based on the hash table.
In order to achieve the above object, an embodiment of the invention provides a high resolution human body real-time dynamic reconstruction method based on a hash table, which includes:
acquiring a plurality of depth images of a single human body by a single depth camera;
projecting each depth image to a three-dimensional space through camera internal parameters to obtain three-dimensional point cloud, and solving corresponding points between the three-dimensional point cloud and a reconstructed model vertex and between the three-dimensional point cloud and an embedded human body parameterized model;
establishing an energy function according to the corresponding points, carrying out optimization solution on the energy function to obtain the posture of the human body parameterized model, and obtaining non-rigid transformation parameters of each vertex of the reconstructed model based on the posture of the human body parameterized model;
and carrying out non-rigid transformation on the reconstruction model, transforming the reconstruction model to a real-time frame to enable the reconstruction model to allocate a video memory space in a hash table for newly-added voxels, and carrying out voxel fusion on all the voxels in the hash table according to the non-rigid transformation parameters to obtain a human body reconstruction model so as to dynamically reconstruct the human body.
According to the high-resolution human body real-time dynamic reconstruction method based on the Hash table, a depth camera is used for shooting a human body to acquire a depth image, and the real-time dynamic three-dimensional reconstruction function of the human body is completed based on the depth image. The input information required by the method is very easy to collect, and the dynamic three-dimensional model of the human body can be obtained in real time. The method is accurate and robust in solving, simple and easy to implement, has real-time performance, is suitable for high-resolution requirements, has wide application prospect, and can be quickly realized on hardware systems such as a PC (personal computer) or a workstation.
In addition, the high-resolution human body real-time dynamic reconstruction method based on the hash table according to the above embodiment of the present invention may further have the following additional technical features:
further, in one embodiment of the present invention, the energy function is:
E＝Edata+λbindEbind+λregEreg
wherein the content of the first and second substances,for data items, vcAndrespectively representing the vertex coordinates and normal direction of the reconstructed model after non-rigid motion, u is the corresponding point coordinate corresponding to the vertex coordinates, and P is the corresponding point pair set;
and T (x)i)xiRespectively representing the model vertex coordinates driven by the human body parametric model deformation and the reconstruction model driven by non-rigid motionType vertex coordinates;
is a regularization term whereinIs a neighbor of node i, λbind,λregRespectively, the weight coefficients.
Further, in an embodiment of the present invention, the voxel fusing all voxels in the hash table according to the non-rigid transformation parameter includes:
and compressing the voxels of the allocated space into a linear memory space and putting the linear memory space into a graphics processor for voxel fusion.
Further, in one embodiment of the present invention, the voxel fusion formula is:
wk+1＝maX(maxWeight，wk+1)
wherein TSDF is a truncated symbolic distance function, TSDFk,TSDFk+1Respectively TSDF values before and after fusion, PSDF is a symbol truncation distance function value under a real-time frame, PSDF is obtained according to the non-rigid transformation parameter, wk,wk+1Respectively, the weights before and after fusion, and maxWeight is the maximum upper limit of the weight.
Further, in an embodiment of the present invention, the vertex positions of the parameterized human body model are driven by a human skeleton, and the calculation formula is as follows:
wherein the content of the first and second substances,for parameterizing vertices v of the model of the human bodyiWith a driveA collection of motile bones; alpha is alphai,jWeighting the driving action of the jth bone on the ith model vertex to represent the strength of the driving action of the bone on the vertex; t isbjIs the motion deformation matrix, rot (T) of the jth bone itselfbj) Is the rotating part of the motion deformation matrix.
In order to achieve the above object, another embodiment of the present invention provides a high resolution human body real-time dynamic reconstruction apparatus based on a hash table, including:
the acquisition module is used for acquiring a plurality of depth images of a single human body through a single depth camera;
the first processing module is used for projecting each depth image to a three-dimensional space through camera internal parameters to obtain a three-dimensional point cloud, and solving corresponding points between the three-dimensional point cloud and a reconstructed model vertex and between the three-dimensional point cloud and an embedded human body parameterized model;
the second processing module is used for establishing an energy function according to the corresponding points, performing optimization solution on the energy function to obtain the posture of the human body parameterized model, and obtaining non-rigid transformation parameters of each vertex of the reconstructed model based on the posture of the human body parameterized model;
and the reconstruction module is used for carrying out non-rigid transformation on the reconstruction model, transforming the reconstruction model to a real-time frame, so that the reconstruction model allocates a video memory space in a hash table for newly-added voxels, and carrying out voxel fusion on all the voxels in the hash table according to the non-rigid transformation parameters to obtain a human body reconstruction model so as to dynamically reconstruct the human body.
The high-resolution human body real-time dynamic reconstruction device based on the Hash table provided by the embodiment of the invention acquires a depth image by shooting a human body by using a depth camera, and completes a real-time dynamic three-dimensional reconstruction function of the human body based on the depth image. The input information required by the method is very easy to collect, and the dynamic three-dimensional model of the human body can be obtained in real time. The method is accurate and robust in solving, simple and easy to implement, has real-time performance, is suitable for high-resolution requirements, has wide application prospect, and can be quickly realized on hardware systems such as a PC (personal computer) or a workstation.
In addition, the high-resolution human body real-time dynamic reconstruction device based on the hash table according to the above embodiment of the present invention may further have the following additional technical features:
further, in one embodiment of the present invention, the energy function is:
E＝Edata+λbindEbind+λregEreg
wherein the content of the first and second substances,for data items, vcAndrespectively representing the vertex coordinates and normal direction of the reconstructed model after non-rigid motion, u is the corresponding point coordinate corresponding to the vertex coordinates, and P is the corresponding point pair set;
and T (x)i)xiRespectively representing the vertex coordinates of the model driven by the deformation of the human parametric model and the vertex coordinates of the reconstructed model driven by non-rigid motion;
is a regularization term whereinIs a neighbor of node i, λbind,λregRespectively, the weight coefficients.
Further, in an embodiment of the present invention, the voxel fusing all voxels in the hash table according to the non-rigid transformation parameter includes:
and compressing the voxels of the allocated space into a linear memory space and putting the linear memory space into a graphics processor for voxel fusion.
Further, in one embodiment of the present invention, the voxel fusion formula is:
wk+1＝maX(maxWeight，wk+1)
wherein TSDF is a truncated symbolic distance function, TSDFk,TSDFk+1Respectively TSDF values before and after fusion, PSDF is a symbol truncation distance function value under a real-time frame, PSDF is obtained according to the non-rigid transformation parameter, wk,wk+1Respectively, the weights before and after fusion, and maxWeight is the maximum upper limit of the weight.
Further, in an embodiment of the present invention, the vertex positions of the parameterized human body model are driven by a human skeleton, and the calculation formula is as follows:
wherein the content of the first and second substances,for parameterizing vertices v of the model of the human bodyiA collection of bones with a driving action; alpha is alphai,jWeighting the driving action of the jth bone on the ith model vertex to represent the strength of the driving action of the bone on the vertex; t isbjIs the motion deformation matrix, rot (T) of the jth bone itselfbj) The rotating portion of the matrix is distorted for motion.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flowchart of a method for real-time dynamic reconstruction of a high resolution human body based on a hash table according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a high-resolution human body real-time dynamic reconstruction device based on a hash table according to an embodiment of the invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The high-resolution human body real-time dynamic reconstruction method and device based on the hash table according to the embodiment of the invention are described below with reference to the accompanying drawings.
Firstly, a high-resolution human body real-time dynamic reconstruction method based on a hash table according to an embodiment of the invention will be described with reference to the accompanying drawings.
Fig. 1 is a flowchart of a high-resolution human body real-time dynamic reconstruction method based on a hash table according to an embodiment of the invention.
As shown in fig. 1, the high resolution human body real-time dynamic reconstruction method based on the hash table includes the following steps:
in step S101, a plurality of depth images of a single human body are acquired by a single depth camera.
Specifically, a single dynamic human body is photographed using a depth camera, obtaining a continuous sequence of depth images.
In step S102, each depth image is projected to a three-dimensional space by camera internal parameters to obtain a three-dimensional point cloud, and corresponding points between the three-dimensional point cloud and the reconstructed model vertex and the embedded human body parameterized model are solved.
Specifically, a plurality of depth images are obtained through the previous step, a group of three-dimensional point clouds are generated by projecting the unfoamed depth images back to a three-dimensional space through camera internal parameters, and corresponding points between the three-dimensional point clouds, the top points of the reconstructed model and the embedded human body parametric model are found.
In step S103, an energy function is established according to the corresponding points, the energy function is optimized and solved to obtain the pose of the human parametric model, and a non-rigid transformation parameter of each vertex of the reconstructed model is obtained based on the pose of the human parametric model.
Further, the driving mode of the human body parameterized model, namely the vertex position of the model is driven by the human body skeleton, and the calculation formula is as follows:
wherein the content of the first and second substances,is to vertex viA collection of bones with a driving action; alpha is alphai,jWeighting the driving action of the jth bone on the ith model vertex to represent the strength of the driving action of the bone on the vertex; t isbjIs the motion deformation matrix, rot (T) of the jth bone itselfbj) As a rotating part of the deformation matrix
Specifically, an energy function is established according to the corresponding points, the posture of the human body parameterized model is obtained by carrying out optimization solution on the energy function, and then the non-rigid transformation parameters of each vertex of the reconstructed model are determined according to the posture of the human body parameterized model.
Further, the energy function is:
E＝Edata+λbindEbind+λregEreg
wherein the content of the first and second substances,for data items, it is ensured that the reconstructed model after non-rigid motion can be aligned as much as possible with the three-dimensional point cloud obtained from the depth map, vcAndrespectively representing the vertex coordinates of the reconstructed model after non-rigid motion and the normal direction thereof,u is the corresponding point coordinate corresponding to it, and P is the corresponding point pair set.
Wherein the content of the first and second substances,the consistency of the human body parameterized model posture and the solved non-rigid motion is ensured,and T (x)i)xiRespectively representing the vertex coordinates of the model driven by the deformation of the human parametric model and the vertex coordinates of the reconstructed model driven by the non-rigid motion.
Is a regularization term whereinThe adjacent nodes of the node i ensure the consistency of non-rigid deformation between the adjacent nodes and ensure that the non-rigid deformation is smooth and continuous as much as possible in space. Lambda [ alpha ]bind,λregRespectively, the weight coefficients.
In step S104, the reconstructed model is subjected to non-rigid transformation, so that the reconstructed model is transformed to a real-time frame, so that the reconstructed model allocates a video memory space in the hash table for the newly added voxels, and all the voxels in the hash table are subjected to voxel fusion according to non-rigid transformation parameters, so as to obtain a human body reconstructed model for dynamically reconstructing a human body.
Specifically, after the non-rigid transformation is performed on the reconstruction model, the committed construction model is transformed to a real-time frame, so that the reconstruction model can allocate a space in the hash table for the newly added voxels.
Further, firstly, allocating a video memory space for the newly added voxels, and then compressing the voxels with allocated space into a linear memory space; these valid voxels are put into a GPU (graphics processor) and voxel fusion is performed in fast parallel.
Wherein the voxel fusion formula is as follows:
wk+1＝maX(maxWeight，wk+1)
wherein TSDF is a truncated symbolic distance function, TSDFk,TSDFk+1TSDF values before and after fusion, PSDF is a symbol truncation distance function value under a real-time frame, PSDF is obtained according to a non-rigid transformation parameter, wk,wk+1Respectively, the weights before and after fusion, and maxWeight is the maximum upper limit of the weight.
In other voxel fusion methods, each voxel contains a Truncated Symbolic Distance Function (TSDF) and a weight (weight), and a cubic-style set of voxels needs to be created by uniformly dividing the space. In the high-resolution human body real-time dynamic reconstruction method based on the hash table, the hash table is used for storing the voxel set, so that only a space needs to be opened up for voxels adjacent to the surface of a human body.
First, a neighboring 8 × 8 × 8 voxel is defined as a voxel block, and the coordinates of each voxel block are:
X＝[x/81
Y＝[y/8]
Z＝[z/81
where (X, Y, Z) is the coordinates of the voxel block, (X, Y, Z) is the three-dimensional discrete coordinates of any voxel in the voxel block, [ · ] denotes rounding down.
Then, the hash value of each voxel block is calculated through a hash mapping function, and the index of the voxel block can be obtained through the hash value. The hash function is
Wherein p is1,p2,p3Is three large prime numbers which are the number of the prime numbers,indicating a bit exclusive or and n is the hash table length.
To resolve hash collisions, multiple memory spaces may be reserved for each hash value, in which linear lookups are performed if a hash collision occurs. The frequency of hash collisions is small due to the effectiveness of the hash mapping function, and therefore, the hash table based access mechanism is very fast.
According to the high-resolution human body real-time dynamic reconstruction method based on the Hash table, which is provided by the embodiment of the invention, the human body is shot by using the depth camera to acquire the depth image, and the real-time dynamic three-dimensional reconstruction function of the human body is completed based on the depth image. The input information required by the method is very easy to collect, and the dynamic three-dimensional model of the human body can be obtained in real time. The method is accurate and robust in solving, simple and easy to implement, has real-time performance, is suitable for high-resolution requirements, has wide application prospect, and can be quickly realized on hardware systems such as a PC (personal computer) or a workstation.
Next, a high-resolution human body real-time dynamic reconstruction apparatus based on a hash table according to an embodiment of the present invention will be described with reference to the accompanying drawings.
Fig. 2 is a schematic structural diagram of a high-resolution human body real-time dynamic reconstruction device based on a hash table according to an embodiment of the invention.
As shown in fig. 2, the high resolution human body real-time dynamic reconstruction device based on the hash table comprises: an acquisition module 100, a first processing module 200, a second processing module 300 and a reconstruction module 400.
The acquisition module 100 acquires a plurality of depth images of a single human body through a single depth camera.
The first processing module 200 projects each depth image to a three-dimensional space through camera internal parameters to obtain a three-dimensional point cloud, and solves corresponding points between the three-dimensional point cloud and the vertex of the reconstructed model and the embedded human body parameterized model.
The second processing module 300 builds an energy function according to the corresponding points, performs optimization solution on the energy function to obtain the pose of the human body parameterized model, and obtains the non-rigid transformation parameters of each vertex of the reconstructed model based on the pose of the human body parameterized model.
The reconstruction module 400 performs non-rigid transformation on the reconstruction model, transforms the reconstruction model to a real-time frame, allocates a memory space in the hash table for the newly added voxels, performs voxel fusion on all voxels in the hash table according to non-rigid transformation parameters, and obtains a human body reconstruction model to dynamically reconstruct the human body.
Further, in one embodiment of the present invention, the energy function is:
E＝Edata+λbindEbind+λregEreg
wherein the content of the first and second substances,for data items, vcAndrespectively representing the vertex coordinates and normal direction of the reconstructed model after non-rigid motion, u is the corresponding point coordinate corresponding to the vertex coordinates, and P is the corresponding point pair set;
and T (x)i)xiRespectively representing the vertex coordinates of the model driven by the deformation of the human parametric model and the vertex coordinates of the reconstructed model driven by non-rigid motion;
is a regularization term whereinIs a neighbor of node i, λbind,λregRespectively, the weight coefficients.
Further, in an embodiment of the present invention, voxel fusing is performed on all voxels in the hash table according to the non-rigid transformation parameter, including:
and compressing the voxels of the allocated space into a linear memory space and putting the linear memory space into a graphics processor for voxel fusion.
Further, in one embodiment of the present invention, the voxel fusion formula is:
wk+1＝maX(maxWeight，wk+1)
wherein TSDF is a truncated symbolic distance function, TSDFk，TSDFk+1TSDF values before and after fusion, PSDF is a symbol truncation distance function value under a real-time frame, PSDF is obtained according to a non-rigid transformation parameter, wk，wk+1Respectively, the weights before and after fusion, and maxWeight is the maximum upper limit of the weight.
Further, in one embodiment of the present invention, the vertex positions of the parameterized human model are driven by the human skeleton, and the calculation formula is as follows:
wherein the content of the first and second substances,for vertices v of a parameterized model of the human bodyiA collection of bones with a driving action; alpha is alphai，jWeighting the driving action of the jth bone on the ith model vertex to represent the strength of the driving action of the bone on the vertex; t isbjIs the motion deformation matrix, rot (T) of the jth bone itselfbj) Is the rotating part of the motion deformation matrix.
It should be noted that the above explanation of the embodiment of the high-resolution human body real-time dynamic reconstruction method based on the hash table is also applicable to the apparatus of the embodiment, and is not repeated here.
According to the high-resolution human body real-time dynamic reconstruction device based on the Hash table, which is provided by the embodiment of the invention, the human body is shot by using the depth camera to acquire the depth image, and the real-time dynamic three-dimensional reconstruction function of the human body is completed based on the depth image. The input information required by the method is very easy to collect, and the dynamic three-dimensional model of the human body can be obtained in real time. The method is accurate and robust in solving, simple and easy to implement, has real-time performance, is suitable for high-resolution requirements, has wide application prospect, and can be quickly realized on hardware systems such as a PC (personal computer) or a workstation.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A high-resolution human body real-time dynamic reconstruction method based on a hash table is characterized by comprising the following steps:
acquiring a plurality of depth images of a single human body by a single depth camera;
projecting each depth image to a three-dimensional space through camera internal parameters to obtain three-dimensional point cloud, and solving corresponding points between the three-dimensional point cloud and a reconstructed model vertex and between the three-dimensional point cloud and an embedded human body parameterized model;
establishing an energy function according to the corresponding points, carrying out optimization solution on the energy function to obtain the posture of the human body parameterized model, and obtaining non-rigid transformation parameters of each vertex of the reconstructed model based on the posture of the human body parameterized model;
and carrying out non-rigid transformation on the reconstruction model, transforming the reconstruction model to a real-time frame to enable the reconstruction model to allocate a video memory space in a hash table for newly-added voxels, and carrying out voxel fusion on all the voxels in the hash table according to the non-rigid transformation parameters to obtain a human body reconstruction model so as to dynamically reconstruct the human body.
2. The method of claim 1, wherein the energy function is:
E＝Edata+λbindEbind+λregEreg
wherein the content of the first and second substances,for data items, vcAndrespectively representing the vertex coordinates and normal direction of the reconstructed model after non-rigid motion, u is the corresponding point coordinate corresponding to the vertex coordinates, and P is the corresponding point pair set;
and T (x)i)xiRespectively representing the vertex coordinates of the model driven by the deformation of the human parametric model and the vertex coordinates of the reconstructed model driven by non-rigid motion;
is a regularization term whereinIs a neighbor of node i, λbind，λregRespectively, the weight coefficients.
3. The method according to claim 1, wherein the voxel fusing all voxels in the hash table according to the non-rigid transformation parameters comprises:
and compressing the voxels of the allocated space into a linear memory space and putting the linear memory space into a graphics processor for voxel fusion.
4. The method of claim 3, wherein the voxel fusion formula is:
wk+1＝max(maxWeight，wk+1)
wherein TSDF is a truncated symbolic distance function, TSDFk，TSDFk+1Respectively TSDF values before and after fusion, PSDF is a symbol truncation distance function value under a real-time frame, PSDF is obtained according to the non-rigid transformation parameter, wk，wk+1Respectively, the weights before and after fusion, and maxWeight is the maximum upper limit of the weight.
5. The method of claim 1, wherein the vertex positions of the parameterized human model are driven by a human skeleton, and the calculation formula is as follows:
wherein the content of the first and second substances,for parameterizing vertices v of the model of the human bodyiA collection of bones with a driving action; alpha is alphai，jWeighting the driving action of the jth bone on the ith model vertex to represent the strength of the driving action of the bone on the vertex; t isbjIs the motion deformation matrix, rot (T) of the jth bone itselfbj) Is the rotating part of the motion deformation matrix.
6. A high-resolution human body real-time dynamic reconstruction device based on a hash table is characterized by comprising:
the acquisition module is used for acquiring a plurality of depth images of a single human body through a single depth camera;
the first processing module is used for projecting each depth image to a three-dimensional space through camera internal parameters to obtain a three-dimensional point cloud, and solving corresponding points between the three-dimensional point cloud and a reconstructed model vertex and between the three-dimensional point cloud and an embedded human body parameterized model;
the second processing module is used for establishing an energy function according to the corresponding points, performing optimization solution on the energy function to obtain the posture of the human body parameterized model, and obtaining non-rigid transformation parameters of each vertex of the reconstructed model based on the posture of the human body parameterized model;
and the reconstruction module is used for carrying out non-rigid transformation on the reconstruction model, transforming the reconstruction model to a real-time frame, so that the reconstruction model allocates a video memory space in a hash table for newly-added voxels, and carrying out voxel fusion on all the voxels in the hash table according to the non-rigid transformation parameters to obtain a human body reconstruction model so as to dynamically reconstruct the human body.
7. The apparatus of claim 6, wherein the energy function is:
E＝Edata+λbindEbind+λregEreg
wherein the content of the first and second substances,for data items, vcAndrespectively representing the vertex coordinates and normal direction of the reconstructed model after non-rigid motion, u is the corresponding point coordinate corresponding to the vertex coordinates, and P is the corresponding point pair set;
and T (x)i)xiRespectively representing the vertex coordinates of the model driven by the deformation of the human parametric model and the vertex coordinates of the reconstructed model driven by non-rigid motion;
is a regularization term whereinIs a neighbor of node i, λbind，λregRespectively, the weight coefficients.
8. The apparatus of claim 6, wherein the voxel fusing all voxels in the hash table according to the non-rigid transformation parameters comprises:
and compressing the voxels of the allocated space into a linear memory space and putting the linear memory space into a graphics processor for voxel fusion.
9. The apparatus of claim 8, wherein the voxel fusion formula is:
wk+1＝max(maxWeight，wk+1)
wherein TSDF is a truncated symbolic distance function, TSDFk，TSDFk+1Respectively TSDF values before and after fusion, PSDF is a symbol truncation distance function value under a real-time frame, PSDF is obtained according to the non-rigid transformation parameter, wk，wk+1Respectively, the weights before and after fusion, and maxWeight is the maximum upper limit of the weight.
10. The apparatus of claim 6, wherein the vertex positions of the parameterized human body model are driven by a human skeleton, and the calculation formula is as follows:
wherein the content of the first and second substances,for parameterizing vertices v of the model of the human bodyiA collection of bones with a driving action; alpha is alphai，jWeighting the driving action of the jth bone on the ith model vertex to represent the strength of the driving action of the bone on the vertex; t isbjIs the motion deformation matrix, rot (T) of the jth bone itselfbj) Is the rotating part of the motion deformation matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910717605.5A CN110599535A (en) | 2019-08-05 | 2019-08-05 | High-resolution human body real-time dynamic reconstruction method and device based on hash table |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910717605.5A CN110599535A (en) | 2019-08-05 | 2019-08-05 | High-resolution human body real-time dynamic reconstruction method and device based on hash table |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110599535A true CN110599535A (en) | 2019-12-20 |
Family
ID=68853490
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910717605.5A Pending CN110599535A (en) | 2019-08-05 | 2019-08-05 | High-resolution human body real-time dynamic reconstruction method and device based on hash table |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110599535A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111968169A (en) * | 2020-08-19 | 2020-11-20 | 北京拙河科技有限公司 | Dynamic human body three-dimensional reconstruction method, device, equipment and medium |
| CN111968165A (en) * | 2020-08-19 | 2020-11-20 | 北京拙河科技有限公司 | Dynamic human body three-dimensional model completion method, device, equipment and medium |
| CN112802186A (en) * | 2021-01-27 | 2021-05-14 | 清华大学 | Dynamic scene real-time three-dimensional reconstruction method based on binarization characteristic coding matching |
| CN113256789A (en) * | 2021-05-13 | 2021-08-13 | 中国民航大学 | Three-dimensional real-time human body posture reconstruction method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018011370A1 (en) * | 2016-07-13 | 2018-01-18 | Naked Labs Austria Gmbh | Efficient volumetric reconstruction with depth sensors |
| CN108053437A (en) * | 2017-11-29 | 2018-05-18 | 深圳奥比中光科技有限公司 | Three-dimensional model acquiring method and device based on figure |
| CN108122275A (en) * | 2017-12-22 | 2018-06-05 | 清华大学 | Dynamic realtime 3 D human body method for reconstructing and system based on skeleton tracking |
| CN108564652A (en) * | 2018-03-12 | 2018-09-21 | 中国科学院自动化研究所 | Efficiently utilize the high-precision three-dimensional method for reconstructing of memory and system and equipment |
| CN108665537A (en) * | 2018-05-15 | 2018-10-16 | 清华大学 | The three-dimensional rebuilding method and system of combined optimization human body figure and display model |
| CN109285215A (en) * | 2018-08-28 | 2019-01-29 | 腾讯科技（深圳）有限公司 | A kind of human 3d model method for reconstructing, device and storage medium |
-
2019
- 2019-08-05 CN CN201910717605.5A patent/CN110599535A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018011370A1 (en) * | 2016-07-13 | 2018-01-18 | Naked Labs Austria Gmbh | Efficient volumetric reconstruction with depth sensors |
| CN108053437A (en) * | 2017-11-29 | 2018-05-18 | 深圳奥比中光科技有限公司 | Three-dimensional model acquiring method and device based on figure |
| CN108122275A (en) * | 2017-12-22 | 2018-06-05 | 清华大学 | Dynamic realtime 3 D human body method for reconstructing and system based on skeleton tracking |
| CN108564652A (en) * | 2018-03-12 | 2018-09-21 | 中国科学院自动化研究所 | Efficiently utilize the high-precision three-dimensional method for reconstructing of memory and system and equipment |
| CN108665537A (en) * | 2018-05-15 | 2018-10-16 | 清华大学 | The three-dimensional rebuilding method and system of combined optimization human body figure and display model |
| CN109285215A (en) * | 2018-08-28 | 2019-01-29 | 腾讯科技（深圳）有限公司 | A kind of human 3d model method for reconstructing, device and storage medium |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111968169A (en) * | 2020-08-19 | 2020-11-20 | 北京拙河科技有限公司 | Dynamic human body three-dimensional reconstruction method, device, equipment and medium |
| CN111968165A (en) * | 2020-08-19 | 2020-11-20 | 北京拙河科技有限公司 | Dynamic human body three-dimensional model completion method, device, equipment and medium |
| CN111968169B (en) * | 2020-08-19 | 2024-01-19 | 北京拙河科技有限公司 | Dynamic human body three-dimensional reconstruction method, device, equipment and medium |
| CN111968165B (en) * | 2020-08-19 | 2024-01-23 | 北京拙河科技有限公司 | Dynamic human body three-dimensional model complement method, device, equipment and medium |
| CN112802186A (en) * | 2021-01-27 | 2021-05-14 | 清华大学 | Dynamic scene real-time three-dimensional reconstruction method based on binarization characteristic coding matching |
| CN112802186B (en) * | 2021-01-27 | 2022-06-24 | 清华大学 | Dynamic scene real-time three-dimensional reconstruction method based on binarization characteristic coding matching |
| CN113256789A (en) * | 2021-05-13 | 2021-08-13 | 中国民航大学 | Three-dimensional real-time human body posture reconstruction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110599535A (en) | High-resolution human body real-time dynamic reconstruction method and device based on hash table | |
| CN107730503B (en) | Image object component level semantic segmentation method and device embedded with three-dimensional features | |
| CN113099208B (en) | Method and device for generating dynamic human body free viewpoint video based on nerve radiation field | |
| US6069634A (en) | System for rapidly deforming a graphical object | |
| CN104616345B (en) | Octree forest compression based three-dimensional voxel access method | |
| CN110675489B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN101149842B (en) | Depth detail level adjustment of multi-dimensional image data with a client/server-based image rendering system | |
| CN111340944B (en) | Single-image human body three-dimensional reconstruction method based on implicit function and human body template | |
| EP3358529A2 (en) | Lightfield rendering based on depths from physically-based volume rendering | |
| CN108805979A (en) | A kind of dynamic model three-dimensional rebuilding method, device, equipment and storage medium | |
| CN103559737A (en) | Object panorama modeling method | |
| Rushmeier et al. | Perceptual issues in substituting texture for geometry | |
| CN114863038A (en) | Real-time dynamic free visual angle synthesis method and device based on explicit geometric deformation | |
| CN111476884A (en) | Real-time three-dimensional human body reconstruction method and system based on single-frame RGBD image | |
| CN114972634A (en) | Multi-view three-dimensional deformable human face reconstruction method based on feature voxel fusion | |
| JP4229398B2 (en) | Three-dimensional modeling program, three-dimensional modeling control program, three-dimensional modeling data transmission program, recording medium, and three-dimensional modeling method | |
| WO2020208976A1 (en) | Image processing device, image processing method, and program | |
| CN116309990A (en) | Face video generation method, device, equipment and storage medium | |
| Beneš et al. | GI-COLLIDE: collision detection with geometry images | |
| CN112233018B (en) | Reference image guided face super-resolution method based on three-dimensional deformation model | |
| CN108921908B (en) | Surface light field acquisition method and device and electronic equipment | |
| JP6443574B1 (en) | Ray casting program, search control data, search control data generation method, and ray casting apparatus | |
| JP7131080B2 (en) | volume rendering device | |
| US20030234783A1 (en) | Compression of surface light fields | |
| CN111178501A (en) | Optimization method, system, electronic equipment and device of dual-cycle countermeasure network architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221221 Address after: Room 3346, Floor 3, International Innovation Expo Center, No. 267, Kejiyuan Road, Baiyang Street, Qiantang District, Hangzhou, Zhejiang 310020 Applicant after: Hangzhou Xinchangyuan Technology Co.,Ltd. Address before: 100084 Tsinghua Yuan, Beijing, Haidian District Applicant before: TSINGHUA University |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191220 |
|
| RJ01 | Rejection of invention patent application after publication |