CN111402422A - Three-dimensional surface reconstruction method and device and electronic equipment - Google Patents
Three-dimensional surface reconstruction method and device and electronic equipment Download PDFInfo
- Publication number
- CN111402422A CN111402422A CN202010180200.5A CN202010180200A CN111402422A CN 111402422 A CN111402422 A CN 111402422A CN 202010180200 A CN202010180200 A CN 202010180200A CN 111402422 A CN111402422 A CN 111402422A
- Authority
- CN
- China
- Prior art keywords
- node
- data structure
- nodes
- dimensional
- voxel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000004364 calculation method Methods 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 13
- 230000014509 gene expression Effects 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/30—Polynomial surface description
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Mathematical Analysis (AREA)
- Algebra (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Image Generation (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention provides a three-dimensional surface reconstruction method, a three-dimensional surface reconstruction device and electronic equipment, wherein the method comprises the following steps: acquiring data of a target object to obtain image data; determining three-dimensional point information corresponding to the image data according to the image data; updating a pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure, wherein the first data structure at least comprises a first node, the first node corresponds to a first voxel, and the first node comprises a three-dimensional surface of a target object corresponding to the first voxel; and determining a three-dimensional surface reconstruction result of the target object according to the second data structure. The image data is acquired by acquiring the data of the target object, the pre-acquired first data structure is updated according to the image data, and the three-dimensional surface reconstruction result of the target object is determined by adopting the updated second data structure, so that the calculation expense in the three-dimensional reconstruction can be saved, and the efficiency of performing the three-dimensional surface reconstruction on line can be improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for reconstructing a three-dimensional surface, and an electronic device.
Background
In order to facilitate data fusion, a large part of space expense of the existing online three-dimensional reconstruction algorithm is derived from traditional uniform space division, a flat area needs to be modeled by using higher resolution, and under the condition that the resource of a Graphics Processing Unit (GPU) is certain, the area reconstruction of fine textures is influenced.
That is, the conventional three-dimensional reconstruction algorithm requires high computational resources.
Disclosure of Invention
The invention aims to provide a three-dimensional surface reconstruction method, a three-dimensional surface reconstruction device and electronic equipment, and aims to solve the problem that in the existing method, the calculation resources required in the online three-dimensional reconstruction process are high.
In order to achieve the above object, a first aspect of the present invention provides a three-dimensional surface reconstruction method, including:
acquiring data of a target object to obtain image data;
determining three-dimensional point information corresponding to the image data according to the image data;
updating a pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure, wherein the first data structure at least comprises a first node, the first node corresponds to a first voxel, and the first node comprises a three-dimensional surface of the target object corresponding to the first voxel;
and determining a three-dimensional surface reconstruction result of the target object according to the second data structure.
Further, the three-dimensional point information includes coordinates of a three-dimensional point, a distance function value of the three-dimensional point, and a curvature of the three-dimensional point;
the updating the pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure includes:
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is smaller than a first preset threshold value, and the absolute value of the distance function value of the three-dimensional points falling into the range of the first voxel is smaller than a second preset threshold value, or,
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
Further, the subdividing the first node to obtain the second data structure includes:
carrying out eighth equal division on the first voxels corresponding to the first node to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the eight equal divisions;
and taking the eight first sub-nodes as first sub-nodes of the first node to update the first data structure to obtain the second data structure, wherein the resolution of the three-dimensional surface of the voxel corresponding to each first sub-node in the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
Further, the nodes in the first data structure further include a symbolic distance field and weights;
the updating the pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure includes:
updating the symbolic distance field and the weight of each node in the first data structure with the following expressions, respectively, to obtain a second data structure:
wi＝min(wi-1+σw,255.0f)
wherein σ is the level of the node in the first data structure;
sdfi-1a symbol distance field for the node at a previous time instant;
wi-1the weight of the node at the last moment is obtained;
sdf is a symbol distance field of the node at the current moment, which is obtained by calculation according to the three-dimensional point information;
w is the weight of the node at the current moment;
sdfian updated symbol distance field for the node;
wiupdated weights for the nodes.
Further, after the determining the three-dimensional surface reconstruction result of the target object according to the second data structure, the method further includes:
for each second node of the second data structure, interpolating the symbol distance field of the second node to obtain a plurality of equipotential points, where the second node is a node of the second data structure that includes six second child nodes; the six second sub-nodes respectively correspond to six faces of the voxel corresponding to the second node; each second sub-node comprises four forest nodes, and when the four forest nodes are respectively equally divided with the voxel corresponding to the second node, one surface of the voxel is equally divided to obtain a surface corresponding to the four forest nodes;
and aligning equipotential points on the surface corresponding to the forest nodes in the second sub-nodes to equipotential points on the surface corresponding to the second sub-nodes so as to adjust the three-dimensional surface reconstruction result and obtain a new three-dimensional surface reconstruction result.
The second aspect of the present invention also provides a three-dimensional surface reconstruction apparatus, comprising:
the first acquisition module is used for acquiring data of a target object to acquire image data;
the first determining module is used for determining three-dimensional point information corresponding to the image data according to the image data;
a second obtaining module, configured to update a pre-obtained first data structure according to the three-dimensional point information to obtain a second data structure, where the first data structure at least includes a first node, the first node corresponds to a first voxel, and the first node includes a three-dimensional surface of the target object corresponding to the first voxel;
a second determining module for determining a three-dimensional surface reconstruction result of the target object according to the second data structure.
Further, the three-dimensional point information includes coordinates of a three-dimensional point, a distance function value of the three-dimensional point, and a curvature of the three-dimensional point;
the second obtaining module is configured to:
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is smaller than a first preset threshold value, and the absolute value of the distance function value of the three-dimensional points falling into the range of the first voxel is smaller than a second preset threshold value, or,
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
Further, the second obtaining module is configured to:
carrying out eighth equal division on the first voxels corresponding to the first node to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the eight equal divisions;
and taking the eight first sub-nodes as first sub-nodes of the first node to update the first data structure to obtain the second data structure, wherein the resolution of the three-dimensional surface of the voxel corresponding to each first sub-node in the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
Further, the nodes in the first data structure further include a symbolic distance field and weights;
the second obtaining module is configured to:
updating the symbolic distance field and the weight of each node in the first data structure with the following expressions, respectively, to obtain a second data structure:
wi＝min(wi-1+σw,255.0f)
wherein σ is the level of the node in the first data structure;
sdfi-1a symbol distance field for the node at a previous time instant;
wi-1the weight of the node at the last moment is obtained;
sdf is a symbol distance field of the node at the current moment, which is obtained by calculation according to the three-dimensional point information;
w is the weight of the node at the current moment;
sdfian updated symbol distance field for the node;
wiupdated weights for the nodes.
Further, the apparatus further comprises:
a third obtaining module, configured to interpolate, for each second node of the second data structure, a symbol distance field of the second node to obtain a plurality of equipotential points, where the second node is a node in the second data structure that includes six second child nodes; the six second sub-nodes respectively correspond to six faces of the voxel corresponding to the second node; each second sub-node comprises four forest nodes, and when the four forest nodes are respectively equally divided with the voxel corresponding to the second node, one surface of the voxel is equally divided to obtain a surface corresponding to the four forest nodes;
and the adjusting module is used for aligning equipotential points on the surface corresponding to the forest nodes in the second sub-nodes to equipotential points on the surface corresponding to the second sub-nodes so as to adjust the three-dimensional surface reconstruction result and obtain a new three-dimensional surface reconstruction result.
A third aspect of the present invention provides an electronic device comprising: the three-dimensional surface reconstruction method comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the steps in the three-dimensional surface reconstruction method provided by the embodiment of the invention when being executed by the processor.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the steps in the three-dimensional surface reconstruction method provided in the embodiment of the present invention.
In the embodiment of the invention, data acquisition is carried out on a target object to obtain image data; determining three-dimensional point information corresponding to the image data according to the image data; updating a pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure, wherein the first data structure at least comprises a first node, the first node corresponds to a first voxel, and the first node comprises a three-dimensional surface of the target object corresponding to the first voxel; and determining a three-dimensional surface reconstruction result of the target object according to the second data structure. The image data is acquired by acquiring the data of the target object, the pre-acquired first data structure is updated according to the image data, and the three-dimensional surface reconstruction result of the target object is determined by adopting the updated second data structure, so that the calculation expense in the three-dimensional reconstruction can be saved, and the efficiency of performing the three-dimensional surface reconstruction on line can be improved.
Drawings
FIG. 1 is a flow chart of a three-dimensional surface reconstruction method provided by an embodiment of the invention;
FIG. 2 is a diagram of a data structure provided by an embodiment of the present invention;
FIG. 3 is a block diagram of a three-dimensional surface reconstruction apparatus provided by an embodiment of the present invention;
fig. 4 is a structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
Referring to fig. 1, fig. 1 is a flowchart of a three-dimensional surface reconstruction method according to an embodiment of the present invention, where the embodiment provides a three-dimensional surface reconstruction method applied to an electronic device, including the following steps:
The target object may be an object, a building, a terrain, and so on. The method includes acquiring data of a target object by an acquisition device such as a scanner or a camera to obtain image data of the target object, wherein the image data can be point cloud data or RGBD (R is Red (Red), G is Green (Green), B is Blue (Blue), and D is Depth (Depth)) data.
And 102, determining three-dimensional point information corresponding to the image data according to the image data.
For different image data, the coordinates, normal and curvature of their three-dimensional points in three-dimensional space can be obtained by different methods.
And if the image data is RGBD data, obtaining the three-dimensional point coordinates by multiplying pixel coordinate points on the image by the camera conversion matrix.
If the image data is Point Cloud data, the eigenvalue and the eigenvector of the covariance matrix formed by a preset number of points in each Point Cloud data neighborhood are calculated, wherein the eigenvector corresponding to the minimum eigenvalue is the normal direction of the Point Cloud data, and the curvature of the Point Cloud data can be obtained through the principal curvaturetiming of a Point Cloud L library (PC L for short), which is not described herein again.
The three-dimensional point information includes coordinates of at least one three-dimensional point, a distance function value of each three-dimensional point, and a curvature of each three-dimensional point.
The first data structure may include one or more nodes, and if the first data structure includes a node, the node is a first node; if the first data structure includes a plurality of nodes, the first node may be any one of the plurality of nodes. The first data structure may be understood as a multi-way tree structure, for example an octree. Initially, the first data structure includes a root node, which is a first node. The first data structure may comprise nodes distributed over a plurality of levels, the nodes of different levels corresponding to voxels having different resolutions of the three-dimensional surface, the larger the level the higher the resolution of the three-dimensional surface of the voxel, wherein the level of the root node is lowest.
The first node comprises the three-dimensional surface of the target object corresponding to the first voxel, that is, the three-dimensional surface of the target object corresponding to the first voxel is stored on the first node corresponding to the first voxel. For each node in the first data structure, the node may include a three-dimensional surface of the target object corresponding to the voxel corresponding to the node, that is, the three-dimensional surface of the target object corresponding to the voxel is stored on the node corresponding to the voxel. As shown in fig. 2, for each node in the first data structure, the node may further include a symbolic distance value, a weight, a curvature, a location where a voxel corresponding to the node is located (i.e., location information in fig. 2), and so on, and further includes an address of its first child node for a parent node in the first data structure, and further includes an address of its parent node for a child node in the first data structure.
When the three-dimensional point information is used for updating the first data structure, the three-dimensional point information can be used for updating the information of the nodes in the first data structure, and the number of the nodes included in the first data structure can also be increased or decreased.
And 104, determining a three-dimensional surface reconstruction result of the target object according to the second data structure.
The second data structure may be a multi-way tree structure, such as an octree structure. The second data structure may comprise nodes distributed over a plurality of levels, the resolution of the three-dimensional surface of the voxels corresponding to nodes of different levels (alternatively referred to as depths) being different, the greater the level (or depth), the higher the resolution of the three-dimensional surface of the voxel, wherein the level (or depth) of the root node is lowest and the closer to the leaf nodes, the higher the level (or depth).
And determining a three-dimensional surface reconstruction result of the target object according to the information included in each node of the second data structure. For example, a three-dimensional model of the mesh representation, i.e. a three-dimensional surface reconstruction result of the target object, may be obtained using a conventional Marching Cubes algorithm (Marching Cubes). Because the resolution ratios of the three-dimensional surfaces of the voxels corresponding to the nodes of different levels in the second data structure are different, the mesh of the object flat region can be simplified, a three-dimensional model with local scale difference can be generated, the calculation cost in the three-dimensional reconstruction can be saved, and the efficiency of the on-line three-dimensional surface reconstruction can be improved.
In the embodiment, data acquisition is performed on a target object to obtain image data; determining three-dimensional point information corresponding to the image data according to the image data; updating a pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure, wherein the first data structure at least comprises a first node, the first node corresponds to a first voxel, and the first node comprises a three-dimensional surface of the target object corresponding to the first voxel; and determining a three-dimensional surface reconstruction result of the target object according to the second data structure. The image data is acquired by acquiring the data of the target object, the pre-acquired first data structure is updated according to the image data, and the three-dimensional surface reconstruction result of the target object is determined by adopting the updated second data structure, so that the calculation expense in the three-dimensional reconstruction can be saved, and the efficiency of performing the three-dimensional surface reconstruction on line can be improved.
In one embodiment of the present application, the three-dimensional point information includes coordinates of a three-dimensional point, a distance function value of the three-dimensional point, and a curvature of the three-dimensional point; step 103, updating the pre-acquired first data structure according to the three-dimensional point information to obtain a second data structure, including:
if the number of three-dimensional points falling into the range of the first voxel in the three-dimensional point information is smaller than a first preset threshold value, and the absolute value of the distance function value of the three-dimensional points falling into the range of the first voxel is smaller than a second preset threshold value, subdividing the first node to obtain a second data structure;
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
As shown in fig. 2, fig. 2 is a schematic diagram of a first data structure, and fig. 2 is an octree. A plurality of linear tables (A is a linear table and B is a tail node table in figure 2, and the linear tables are used for storing tail node addresses of the linear tables) are maintained in the electronic equipment, each linear table represents one layer of an octree, each element of the linear table corresponds to one node of the octree, and relevant information of a voxel corresponding to the node is stored, wherein the relevant information comprises a symbolic distance value, a weight, a first sub-node address, a three-dimensional surface of a target object corresponding to the voxel and the like. A voxel is an abbreviation of Volume element (Volume Pixel), and a Volume containing a voxel can be represented by Volume rendering or by extracting a polygonal isosurface of a given threshold contour. A voxel is the smallest unit on a three-dimensional spatial segmentation.
After the three-dimensional point information is obtained, sequentially judging each node in the first data structure, judging whether a subdivision condition is met, if so, subdividing the node to obtain a second data structure. In this embodiment, after the three-dimensional point information is obtained, the number of three-dimensional points falling into the range of the first voxel in the three-dimensional point information is determined, and if the number is smaller than a first preset threshold and the absolute value of the distance function value of the three-dimensional point falling into the range of the first voxel is smaller than a second preset threshold, the first node is subdivided, and the second data structure is obtained. The first preset threshold may be flexibly set according to actual conditions, and is not limited herein.
If the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
The first node is subdivided, that is, the first voxel corresponding to the first node is subdivided, for example, the first voxel is divided into eight equal parts, and each equal part corresponds to one child node of the first node. And dividing the first voxel, namely dividing the three-dimensional surface of the target object corresponding to the first voxel, wherein the voxel corresponding to each sub-node of the first node corresponds to one of the sub-nodes. The resolution of the three-dimensional surface of the voxel corresponding to the child node of the first node is greater than the resolution of the three-dimensional surface of the first voxel.
In an embodiment of the present application, the subdividing the first node to obtain the second data structure includes:
carrying out eighth equal division on the first voxels corresponding to the first node to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the eight equal divisions;
and taking the eight first sub-nodes as first sub-nodes of the first node to update the first data structure to obtain the second data structure, wherein the resolution of the three-dimensional surface of the voxel corresponding to each first sub-node in the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
The method comprises the steps of carrying out octave division on a first voxel corresponding to a first node needing to be subdivided to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the octaves respectively, namely a three-dimensional surface area of the voxel corresponding to each first sub-node corresponds to one of the octaves. The first node may be any node in the first data structure. In the first data structure, eight child nodes, i.e., eight first child nodes, are generated for the first node. The resolution of the three-dimensional surface of the voxel corresponding to each of the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
Marking the nodes meeting the subdivision conditions in the first data structure, placing the nodes into a mark array, then performing Scanning (SCAN) operation under a unified computing Device Architecture (CUDA) in a Graphics Processing Unit (GPU) to the mark array to obtain the offset of each layer of nodes at the next moment, and updating the tree structure.
In the method, the data structure of the octree is adopted to manage the three-dimensional space, the grids of the object flat area can be simplified, the three-dimensional model with local scale difference is generated, the calculation expense of an online three-dimensional reconstruction algorithm can be saved, the simplified three-dimensional grids with multi-scale geometric details are generated, and subsequent industrial links can be performed more efficiently. The hierarchical relation of the three-dimensional space division of the target object is dynamically adjusted by dynamically constructing a data structure, and the surface details of the target object are modeled with higher resolution to the maximum extent on the premise of ensuring the real-time performance of three-dimensional reconstruction so as to generate a more fidelity three-dimensional model.
In one embodiment of the present application, the nodes in the first data structure further comprise a symbolic distance field and weights;
the updating the pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure includes:
updating the symbolic distance field and the weight of each node in the first data structure with the following expressions, respectively, to obtain a second data structure:
wi＝min(wi-1+σw,255.0f)
wherein σ is the level of the node in the first data structure;
sdfi-1a symbol distance field for the node at a previous time instant;
wi-1the weight of the node at the last moment is obtained;
sdf is a symbol distance field of the node at the current moment, which is obtained by calculation according to the three-dimensional point information;
w is the weight of the node at the current moment;
sdfian updated symbol distance field for the node;
wiupdated weights for the nodes.
In the above, the symbolic distance field refers to a spatial field modeled by a symbolic distance function, σ is a scale parameter, and represents a level of a node in the first data structure, and the larger σ indicates that the node is located deeper in the first data structure, for example, a leaf node of an octree is larger than a level of a parent node of the octree, and a resolution of a three-dimensional reconstruction surface of a voxel corresponding to the node is higher, which is important in the fusion process, and accordingly, a weight of the node is also larger. And for each node in the first data structure, updating the information of each node by adopting the three-dimensional point information to obtain a second data structure. The step of updating the information of each node according to the three-dimensional point information may be preceded by the step of subdividing the first node according to the three-dimensional point information.
Further, after determining the three-dimensional surface reconstruction result of the target object according to the second data structure in step 104, the method further includes:
for each second node of the second data structure, interpolating the symbol distance field of the second node to obtain a plurality of equipotential points, where the second node is a node of the second data structure that includes six second child nodes; the six second sub-nodes respectively correspond to six faces of the voxel corresponding to the second node; each second sub-node comprises four forest nodes, and when the four forest nodes are respectively equally divided with the voxel corresponding to the second node, one surface of the voxel is equally divided to obtain a surface corresponding to the four forest nodes;
and aligning equipotential points on the surface corresponding to the forest nodes in the second sub-nodes to equipotential points on the surface corresponding to the second sub-nodes so as to adjust the three-dimensional surface reconstruction result and obtain a new three-dimensional surface reconstruction result.
In this embodiment, for each second node of the second data structure, the symbolic distance field of the second node is interpolated to obtain a plurality of equipotential points. And storing equipotential points obtained by the interpolation of the symbol distance field of the second node on six surfaces of the voxel corresponding to the second node. These six planes correspond to the voxels of the six second sub-nodes of the second node, respectively.
When the voxels corresponding to the second nodes are equally divided, six surfaces of the voxels are equally divided into four parts, so that each second sub-node comprises four forest nodes, and each forest node corresponds to one four-part surface in the corresponding surfaces of the second sub-nodes.
When the voxels are divided into eight equal parts, the six surfaces of the voxels are also divided into four equal parts, a quadtree forest of the surfaces is established and updated, each quadtree corresponds to one surface, six quadtrees are arranged on the six surfaces to form the quadtree forest, and the equi-potential points are aligned among the voxels with different resolutions according to the equi-potential points, i.e., accessing each face corresponding to each node in the quadtree in parallel, aligning the equipotential points stored on the faces to their respective ancestors, the method is characterized in that the method is aligned to equipotential points on a surface before quartering, each surface of a voxel is used as a basic unit for storing the equipotential points, a table is also a linear table, each element represents one surface, and the address of an ancestor, the corresponding equipotential point coordinate and the address of the voxel to which the equipotential point coordinate belongs are stored inside, so that any access from the voxel to the surface and from the surface to the voxel can be realized. In this embodiment, a voxel surface is used as a basic unit for storing equipotential points, so that adjacent voxel patches are aligned conveniently, and the three-dimensional surface reconstruction effect of the target object is improved.
When the large-scale three-dimensional point cloud video stream input is obtained, the method in the application can furthest reserve the local geometric information of the object to be reconstructed by utilizing the characteristic of the tree structure layering of the second data structure, and generate a multi-scale three-dimensional surface model. The method is realized by utilizing the high parallelism of the GPU, and can run in real time when large objects such as industrial workpieces are used as objects for reconstruction.
Referring to fig. 3, fig. 3 is a structural diagram of a three-dimensional surface reconstruction apparatus according to an embodiment of the present invention, and as shown in fig. 3, the three-dimensional surface reconstruction apparatus 300 includes:
the first obtaining module 301 is configured to perform data acquisition on a target object to obtain image data;
a first determining module 302, configured to determine, according to the image data, three-dimensional point information corresponding to the image data;
a second obtaining module 303, configured to update a pre-obtained first data structure according to the three-dimensional point information to obtain a second data structure, where the first data structure at least includes a first node, the first node corresponds to a first voxel, and the first node includes a three-dimensional surface of the target object corresponding to the first voxel;
a second determining module 304, configured to determine a three-dimensional surface reconstruction result of the target object according to the second data structure.
In one embodiment of the present application, the three-dimensional point information includes coordinates of a three-dimensional point, a distance function value of the three-dimensional point, and a curvature of the three-dimensional point;
the second obtaining module 303 is configured to:
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is smaller than a first preset threshold value, and the absolute value of the distance function value of the three-dimensional points falling into the range of the first voxel is smaller than a second preset threshold value, or,
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
In an embodiment of the present application, the second obtaining module 303 is configured to:
carrying out eighth equal division on the first voxels corresponding to the first node to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the eight equal divisions;
and taking the eight first sub-nodes as first sub-nodes of the first node to update the first data structure to obtain the second data structure, wherein the resolution of the three-dimensional surface of the voxel corresponding to each first sub-node in the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
In one embodiment of the present application, the nodes in the first data structure further comprise a symbolic distance field and weights;
the second obtaining module 303 is configured to:
updating the symbolic distance field and the weight of each node in the first data structure with the following expressions, respectively, to obtain a second data structure:
wi＝min(wi-1+σw,255.0f)
wherein σ is the level of the node in the first data structure;
sdfi-1a symbol distance field for the node at a previous time instant;
wi-1the weight of the node at the last moment is obtained;
sdf is a symbol distance field of the node at the current moment, which is obtained by calculation according to the three-dimensional point information;
w is the weight of the node at the current moment;
sdfian updated symbol distance field for the node;
wiupdated rights for the nodeAnd (4) heavy.
In an embodiment of the present application, the three-dimensional surface reconstruction apparatus 300 further includes:
a third obtaining module, configured to interpolate, for each second node of the second data structure, a symbol distance field of the second node to obtain a plurality of equipotential points, where the second node is a node in the second data structure that includes six second child nodes; the six second sub-nodes respectively correspond to six faces of the voxel corresponding to the second node; each second sub-node comprises four forest nodes, and when the four forest nodes are respectively equally divided with the voxel corresponding to the second node, one surface of the voxel is equally divided to obtain a surface corresponding to the four forest nodes;
and the adjusting module is used for aligning equipotential points on the surface corresponding to the forest nodes in the second sub-nodes to equipotential points on the surface corresponding to the second sub-nodes so as to adjust the three-dimensional surface reconstruction result and obtain a new three-dimensional surface reconstruction result.
It should be noted that, in this embodiment, the three-dimensional surface reconstruction apparatus 300 may implement any implementation manner in the method embodiment shown in fig. 1, that is, any implementation manner in the method embodiment shown in fig. 1 may be implemented by the three-dimensional surface reconstruction apparatus 300 in this embodiment, and achieve the same beneficial effects, and details are not repeated here.
Referring to fig. 4, fig. 4 is a structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 4, the electronic device 400 includes: a memory 401, a processor 402, and a computer program stored on the memory 401 and executable on the processor 402, wherein,
the processor 402 is configured to read the computing program in the memory 401, and execute the following processes:
acquiring data of a target object to obtain image data;
determining three-dimensional point information corresponding to the image data according to the image data;
updating a pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure, wherein the first data structure at least comprises a first node, the first node corresponds to a first voxel, and the first node comprises a three-dimensional surface of the target object corresponding to the first voxel;
and determining a three-dimensional surface reconstruction result of the target object according to the second data structure.
Further, the three-dimensional point information includes coordinates of a three-dimensional point, a distance function value of the three-dimensional point, and a curvature of the three-dimensional point;
the processor 402 further performs: if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is smaller than a first preset threshold value, and the absolute value of the distance function value of the three-dimensional points falling into the range of the first voxel is smaller than a second preset threshold value, or,
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
Further, the processor 402 further performs: carrying out eighth equal division on the first voxels corresponding to the first node to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the eight equal divisions;
and taking the eight first sub-nodes as first sub-nodes of the first node to update the first data structure to obtain the second data structure, wherein the resolution of the three-dimensional surface of the voxel corresponding to each first sub-node in the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
Further, the nodes in the first data structure further include a symbolic distance field and weights;
the processor 402 further performs: updating the symbolic distance field and the weight of each node in the first data structure with the following expressions, respectively, to obtain a second data structure:
wi＝min(wi-1+σw,255.0f)
wherein σ is the level of the node in the first data structure;
sdfi-1a symbol distance field for the node at a previous time instant;
wi-1the weight of the node at the last moment is obtained;
sdf is a symbol distance field of the node at the current moment, which is obtained by calculation according to the three-dimensional point information;
w is the weight of the node at the current moment;
sdfian updated symbol distance field for the node;
wiupdated weights for the nodes.
Further, the processor 402 further performs: for each second node of the second data structure, interpolating the symbol distance field of the second node to obtain a plurality of equipotential points, where the second node is a node of the second data structure that includes six second child nodes; the six second sub-nodes respectively correspond to six faces of the voxel corresponding to the second node; each second sub-node comprises four forest nodes, and when the four forest nodes are respectively equally divided with the voxel corresponding to the second node, one surface of the voxel is equally divided to obtain a surface corresponding to the four forest nodes;
and aligning equipotential points on the surface corresponding to the forest nodes in the second sub-nodes to equipotential points on the surface corresponding to the second sub-nodes so as to adjust the three-dimensional surface reconstruction result and obtain a new three-dimensional surface reconstruction result.
It should be noted that, in this embodiment, the electronic device may implement any implementation manner in the method embodiment shown in fig. 1, that is, any implementation manner in the method embodiment shown in fig. 1 may be implemented by the electronic device in this embodiment, and achieve the same beneficial effects, and details are not described here again.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the steps in the three-dimensional surface reconstruction method provided by the embodiment of the present invention.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (12)
1. A method of three-dimensional surface reconstruction, comprising:
acquiring data of a target object to obtain image data;
determining three-dimensional point information corresponding to the image data according to the image data;
updating a pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure, wherein the first data structure at least comprises a first node, the first node corresponds to a first voxel, and the first node comprises a three-dimensional surface of the target object corresponding to the first voxel;
and determining a three-dimensional surface reconstruction result of the target object according to the second data structure.
2. The method according to claim 1, wherein the three-dimensional point information includes coordinates of a three-dimensional point, a distance function value of the three-dimensional point, and a curvature of the three-dimensional point;
the updating the pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure includes:
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is smaller than a first preset threshold value, and the absolute value of the distance function value of the three-dimensional points falling into the range of the first voxel is smaller than a second preset threshold value, or,
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
3. The method of claim 2, wherein said subdividing said first node to obtain said second data structure comprises:
carrying out eighth equal division on the first voxels corresponding to the first node to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the eight equal divisions;
and taking the eight first sub-nodes as first sub-nodes of the first node to update the first data structure to obtain the second data structure, wherein the resolution of the three-dimensional surface of the voxel corresponding to each first sub-node in the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
4. The method of claim 1 wherein the nodes in the first data structure further comprise symbolic distance fields and weights;
the updating the pre-acquired first data structure according to the three-dimensional point information to acquire a second data structure includes:
updating the symbolic distance field and the weight of each node in the first data structure with the following expressions, respectively, to obtain a second data structure:
wi＝min(wi-1+σw，255.0f)
wherein σ is the level of the node in the first data structure;
sdfi-1a symbol distance field for the node at a previous time instant;
wi-1the weight of the node at the last moment is obtained;
sdf is a symbol distance field of the node at the current moment, which is obtained by calculation according to the three-dimensional point information;
w is the weight of the node at the current moment;
sdfian updated symbol distance field for the node;
wiupdated weights for the nodes.
5. The method of claim 1, further comprising, after said determining a three-dimensional surface reconstruction of the target object from the second data structure:
for each second node of the second data structure, interpolating the symbol distance field of the second node to obtain a plurality of equipotential points, where the second node is a node of the second data structure that includes six second child nodes; the six second sub-nodes respectively correspond to six faces of the voxel corresponding to the second node; each second sub-node comprises four forest nodes, and when the four forest nodes are respectively equally divided with the voxel corresponding to the second node, one surface of the voxel is equally divided to obtain a surface corresponding to the four forest nodes;
and aligning equipotential points on the surface corresponding to the forest nodes in the second sub-nodes to equipotential points on the surface corresponding to the second sub-nodes so as to adjust the three-dimensional surface reconstruction result and obtain a new three-dimensional surface reconstruction result.
6. A three-dimensional surface reconstruction apparatus, comprising:
the first acquisition module is used for acquiring data of a target object to acquire image data;
the first determining module is used for determining three-dimensional point information corresponding to the image data according to the image data;
a second obtaining module, configured to update a pre-obtained first data structure according to the three-dimensional point information to obtain a second data structure, where the first data structure at least includes a first node, the first node corresponds to a first voxel, and the first node includes a three-dimensional surface of the target object corresponding to the first voxel;
a second determining module for determining a three-dimensional surface reconstruction result of the target object according to the second data structure.
7. The apparatus according to claim 6, wherein the three-dimensional point information includes coordinates of a three-dimensional point, a distance function value of the three-dimensional point, and a curvature of the three-dimensional point;
the second obtaining module is configured to:
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is smaller than a first preset threshold value, and the absolute value of the distance function value of the three-dimensional points falling into the range of the first voxel is smaller than a second preset threshold value, or,
if the number of the three-dimensional points falling into the range of the first voxel in the three-dimensional point information is not smaller than the first preset threshold value, and the curvature of the three-dimensional points falling into the range of the first voxel is larger than the side length of the first voxel, subdividing the first node to obtain the second data structure.
8. The apparatus of claim 7, wherein the second obtaining module is configured to:
carrying out eighth equal division on the first voxels corresponding to the first node to obtain eight first sub-nodes, wherein each first sub-node corresponds to one of the eight equal divisions;
and taking the eight first sub-nodes as first sub-nodes of the first node to update the first data structure to obtain the second data structure, wherein the resolution of the three-dimensional surface of the voxel corresponding to each first sub-node in the eight first sub-nodes is greater than the resolution of the three-dimensional surface of the first voxel.
9. The apparatus of claim 6 wherein the nodes in the first data structure further comprise symbolic distance fields and weights;
the second obtaining module is configured to:
updating the symbolic distance field and the weight of each node in the first data structure with the following expressions, respectively, to obtain a second data structure:
wi＝min(wi-1+σw，255.0f)
wherein σ is the level of the node in the first data structure;
sdfi-1a symbol distance field for the node at a previous time instant;
wi-1the weight of the node at the last moment is obtained;
sdf is a symbol distance field of the node at the current moment, which is obtained by calculation according to the three-dimensional point information;
w is the weight of the node at the current moment;
sdfian updated symbol distance field for the node;
wiupdated weights for the nodes.
10. The apparatus of claim 6, further comprising:
a third obtaining module, configured to interpolate, for each second node of the second data structure, a symbol distance field of the second node to obtain a plurality of equipotential points, where the second node is a node in the second data structure that includes six second child nodes; the six second sub-nodes respectively correspond to six faces of the voxel corresponding to the second node; each second sub-node comprises four forest nodes, and when the four forest nodes are respectively equally divided with the voxel corresponding to the second node, one surface of the voxel is equally divided to obtain a surface corresponding to the four forest nodes;
and the adjusting module is used for aligning equipotential points on the surface corresponding to the forest nodes in the second sub-nodes to equipotential points on the surface corresponding to the second sub-nodes so as to adjust the three-dimensional surface reconstruction result and obtain a new three-dimensional surface reconstruction result.
11. An electronic device, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps in the method of three-dimensional surface reconstruction according to any one of claims 1 to 5.
12. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, realizes the steps in the method for three-dimensional surface reconstruction according to one of the claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010180200.5A CN111402422B (en) | 2020-03-16 | 2020-03-16 | Three-dimensional surface reconstruction method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010180200.5A CN111402422B (en) | 2020-03-16 | 2020-03-16 | Three-dimensional surface reconstruction method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111402422A true CN111402422A (en) | 2020-07-10 |
| CN111402422B CN111402422B (en) | 2024-04-16 |
Family
ID=71413495
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010180200.5A Active CN111402422B (en) | 2020-03-16 | 2020-03-16 | Three-dimensional surface reconstruction method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111402422B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113487690A (en) * | 2021-09-08 | 2021-10-08 | 北京医百科技有限公司 | Depth map processing method, system and storage medium |
| WO2024067716A1 (en) * | 2022-09-28 | 2024-04-04 | 先临三维科技股份有限公司 | Mesh reconstruction method and apparatus, device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105453139A (en) * | 2013-07-17 | 2016-03-30 | 微软技术许可有限责任公司 | Sparse GPU voxelization for 3D surface reconstruction |
| CN105654492A (en) * | 2015-12-30 | 2016-06-08 | 哈尔滨工业大学 | Robust real-time three-dimensional (3D) reconstruction method based on consumer camera |
| US20190266783A1 (en) * | 2018-02-27 | 2019-08-29 | Verizon Patent And Licensing Inc. | Methods and Systems for Volumetric Reconstruction Based on a Confidence Field |
| CN110738726A (en) * | 2019-09-27 | 2020-01-31 | 华南理工大学 | robot vision-guided three-dimensional object reconstruction method based on octree |
| CN110874864A (en) * | 2019-10-25 | 2020-03-10 | 深圳奥比中光科技有限公司 | Method, device, electronic equipment and system for obtaining three-dimensional model of object |
-
2020
- 2020-03-16 CN CN202010180200.5A patent/CN111402422B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105453139A (en) * | 2013-07-17 | 2016-03-30 | 微软技术许可有限责任公司 | Sparse GPU voxelization for 3D surface reconstruction |
| CN105654492A (en) * | 2015-12-30 | 2016-06-08 | 哈尔滨工业大学 | Robust real-time three-dimensional (3D) reconstruction method based on consumer camera |
| US20190266783A1 (en) * | 2018-02-27 | 2019-08-29 | Verizon Patent And Licensing Inc. | Methods and Systems for Volumetric Reconstruction Based on a Confidence Field |
| CN110738726A (en) * | 2019-09-27 | 2020-01-31 | 华南理工大学 | robot vision-guided three-dimensional object reconstruction method based on octree |
| CN110874864A (en) * | 2019-10-25 | 2020-03-10 | 深圳奥比中光科技有限公司 | Method, device, electronic equipment and system for obtaining three-dimensional model of object |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113487690A (en) * | 2021-09-08 | 2021-10-08 | 北京医百科技有限公司 | Depth map processing method, system and storage medium |
| WO2024067716A1 (en) * | 2022-09-28 | 2024-04-04 | 先临三维科技股份有限公司 | Mesh reconstruction method and apparatus, device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111402422B (en) | 2024-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7110000B2 (en) | Synthesis of progressively-variant textures and application to arbitrary surfaces | |
| CN108038906B (en) | Three-dimensional quadrilateral mesh model reconstruction method based on image | |
| Boubekeur et al. | A flexible kernel for adaptive mesh refinement on GPU | |
| Feng et al. | A parallel algorithm for viewshed analysis in three-dimensional Digital Earth | |
| CN107193923B (en) | Method and system for quickly superposing vectors in two-dimensional geographic space | |
| CN111402422B (en) | Three-dimensional surface reconstruction method and device and electronic equipment | |
| CN111581776A9 (en) | Geometric reconstruction model-based isogeometric analysis method | |
| CN115423978A (en) | Image laser data fusion method based on deep learning and used for building reconstruction | |
| Feng et al. | PRIF: Primary ray-based implicit function | |
| CN109785443B (en) | Three-dimensional model simplification method for large ocean engineering equipment | |
| CN114359269A (en) | Virtual food box defect generation method and system based on neural network | |
| US20180264736A1 (en) | Device for processing data for additive manufacturing | |
| CN116310060B (en) | Method, device, equipment and storage medium for rendering data | |
| CN111028349A (en) | Hierarchical construction method suitable for rapid visualization of massive three-dimensional live-action data | |
| CN113570712B (en) | 3D modeling optimization method based on GCN | |
| CN112002019B (en) | Method for simulating character shadow based on MR mixed reality | |
| WO2022104681A1 (en) | Method for extracting isosurface in cylindrical coordinate system | |
| Dubois et al. | Highly Efficient Controlled Hierarchical Data Reduction techniques for Interactive Visualization of Massive Simulation Data. | |
| CN110738719A (en) | Web3D model rendering method based on visual range hierarchical optimization | |
| CN117541755B (en) | RGB-D three-dimensional reconstruction-based rigid object virtual-real shielding method | |
| CN117237503B (en) | Geographic element data accelerated rendering and device | |
| CN117078825B (en) | Rendering modification method, system, equipment and medium on point cloud data line | |
| Feng et al. | Superpixel-based graph cuts for accurate stereo matching | |
| CN117765010A (en) | Tetrahedron surface marking Mesh construction method and system combined with unmanned aerial vehicle segmented image | |
| Yang et al. | Construction of Efficient Thinning Model of Point Cloud Data Based on PCA and Deep Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |