CN114140601A - Three-dimensional grid reconstruction method and system based on single image under deep learning framework - Google Patents
Three-dimensional grid reconstruction method and system based on single image under deep learning framework Download PDFInfo
- Publication number
- CN114140601A CN114140601A CN202111522113.4A CN202111522113A CN114140601A CN 114140601 A CN114140601 A CN 114140601A CN 202111522113 A CN202111522113 A CN 202111522113A CN 114140601 A CN114140601 A CN 114140601A
- Authority
- CN
- China
- Prior art keywords
- module
- network
- vertex
- loss
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000013135 deep learning Methods 0.000 title claims abstract description 15
- 238000000605 extraction Methods 0.000 claims abstract description 23
- 238000005070 sampling Methods 0.000 claims description 27
- 238000012549 training Methods 0.000 claims description 20
- 238000010606 normalization Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 13
- 230000004927 fusion Effects 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 239000013589 supplement Substances 0.000 abstract 1
- 230000000694 effects Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011960 computer-aided design Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
- G06T17/205—Re-meshing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a three-dimensional grid reconstruction method and a three-dimensional grid reconstruction system based on a single image under a deep learning framework. The invention obtains three scale characteristic graphs by using a VGG16 network as an image characteristic extraction network, and completes a complete reconstruction task by using a grid deformation network based on the obtained characteristics and an initial elliptical grid model, wherein the grid deformation network comprises a graph volume module and local characteristic information fused, and a Transformer coding module supplements global characteristic information. The method effectively improves the accuracy of the three-dimensional grid reconstruction of the single image.
Description
Technical Field
The invention belongs to the field of deep learning and image three-dimensional mesh reconstruction, and particularly relates to a three-dimensional mesh reconstruction method and system based on a single image under a deep learning framework.
Background
Three-dimensional modeling is an important computer aided design technology and plays a fundamental role in the industries of building engineering, industrial design, movie animation and the like. Nowadays, with the acceleration of industrialization process in various fields, the requirement for three-dimensional modeling efficiency is continuously improved. Compared with the traditional manual modeling, the three-dimensional reconstruction based on the single image has the characteristics of high reconstruction efficiency and low cost. Therefore, accurate three-dimensional model reconstruction from a single image is of increasing interest to researchers. Meanwhile, in recent years, deep learning is continuously developed and has a great success in the field of computer vision, so that research work combining deep learning and single image three-dimensional reconstruction becomes a hotspot.
The structural representation forms of the three-dimensional model for image reconstruction comprise various types of voxels, point clouds, grids and the like, the three-dimensional reconstruction research based on the images is mainly carried out based on the representation of the voxels and the point clouds in the early stage, but the geometric representation capability of the reconstructed model based on the voxels is limited, and inherent quantization errors exist. Point cloud based reconstructed models lack a complete surface representation. The mesh is represented by an irregular model structure and comprises vertexes, edges and surfaces, the mesh has the characteristics of light weight and rich shape details, the local neighborhood relation of the model can be further provided based on the reconstruction represented by the mesh, the continuity representation of the surface of the object is realized, and meanwhile, more accurate error calculation can be carried out based on the vertex coordinates. Therefore, model reconstruction studies based on mesh representations are becoming hot.
However, due to the ordering of the vertices of the mesh structure, the number of adjacent vertices of each point in the topological surface is not consistent, and the convolutional neural network cannot directly process the data of the non-euclidean structure, so that the mesh-based reconstruction work is full of challenges.
Disclosure of Invention
The invention aims to provide a single-image-based three-dimensional grid reconstruction method and a single-image-based three-dimensional grid reconstruction system under a deep learning framework.
A three-dimensional image reconstruction method based on deep learning specifically comprises the following steps:
acquiring a two-dimensional image and corresponding three-dimensional model data;
secondly, preprocessing the two-dimensional image and the three-dimensional model data;
step three, building and training a neural network;
3-1, building a complete reconstruction network
The complete reconstruction network comprises an image feature extraction network and a grid deformation network; the image feature extraction network is used for inputting a single two-dimensional image to obtain three scale feature maps; the grid deformation network is used for inputting an initial elliptical grid model and three scale feature maps output by the image feature extraction network, and carrying out grid deformation on the initial elliptical grid model so as to complete reconstruction;
3-2 model training
Inputting the image data preprocessed in the step two into the complete reconstruction network in the step 3-1 for training; calculating loss values of a reconstruction model and a target model output by the complete reconstruction network through a fusion type loss function in the training process, and then continuously reducing the loss values through network iteration parameters through back propagation of the loss values to finish training;
and step four, inputting a single two-dimensional image to be reconstructed according to the network model built and trained in the step three, and finally outputting a reconstructed three-dimensional model.
Another object of the present invention is to provide a depth learning-based three-dimensional image reconstruction system, which includes:
the data acquisition module is used for acquiring a two-dimensional image and a corresponding three-dimensional model;
the data preprocessing module is used for adjusting the format and the size of the two-dimensional image and the corresponding three-dimensional model;
and the complete reconstruction network module is used for training the complete reconstruction network by using the data processed by the data preprocessing module and then reconstructing a corresponding three-dimensional model of the two-dimensional image to be reconstructed by using the complete reconstruction network.
The invention has the following beneficial effects:
1. the invention is oriented to the single image three-dimensional grid reconstruction task, introduces a graph volume module to effectively process non-Euclidean grid model data, and completes three-dimensional grid reconstruction based on local grid information.
2. The method is oriented to a single image three-dimensional grid reconstruction task, and based on a graph volume module cascade transform coding module, global information is supplemented, and reconstruction accuracy is improved.
Drawings
Fig. 1 is a model structure diagram of a complete reconstruction network in the present invention.
Fig. 2 is a model configuration diagram of an image feature extraction network in the present invention.
Fig. 3 is a model structure diagram of a mesh deformation network in the present invention.
Detailed Description
The invention is further analyzed with reference to the following specific examples.
A three-dimensional image reconstruction method based on deep learning comprises the following steps:
step one, acquiring a two-dimensional image and corresponding three-dimensional model data. The data source is the Shapen public dataset.
And step two, preprocessing the two-dimensional image and the three-dimensional model data. The preprocessing method comprises two-dimensional image scaling, clipping and normalization, and three-dimensional model Poisson disc algorithm pre-sampling.
And step three, building and training a neural network.
3-1, building a complete reconstruction network
The complete reconstruction network shown in fig. 1 comprises an image feature extraction network and a mesh deformation network. The image feature extraction network is used for inputting a single two-dimensional image to obtain three scale feature maps; the mesh deformation network is used for inputting an initial elliptical mesh model and three scale feature maps output by the image feature extraction network, performing mesh deformation on the initial elliptical mesh model to complete reconstruction, namely performing vertex sampling on the input initial elliptical mesh model in the three scale feature maps for many times by the mesh deformation network, deforming the initial elliptical mesh by using a plurality of graph convolution modules and a Transfomer coding module according to a sampling result, performing vertex expansion on the mesh by using a plurality of vertex anti-pooling modules, and finally outputting a reconstructed three-dimensional model; the invention provides a cascade graph convolution module and a Transfomer coding module which are used for processing a grid model together, and can effectively combine local information and global information to complete reconstruction.
3-2 model training
And inputting the image data preprocessed in the step two into the complete reconstruction network in the step 3-1 for training. In the training process, loss values of a reconstruction model and a target model output by the complete reconstruction network are calculated through a fusion type loss function, and then the loss values are propagated reversely through the loss values, and the network iteration parameters continuously reduce the loss values to complete the training.
And step four, inputting a single two-dimensional image to be reconstructed according to the network model built and trained in the step three, and finally outputting a reconstructed three-dimensional model.
In step 3-1, the image feature extraction network structure shown in fig. 2 is formed by 13 two-dimensional convolution units connected in series of the existing VGG16 network. The first two-dimensional convolution unit is used as the input end of the feature extraction network and is connected with the preprocessed two-dimensional image data; the output end of the seventh two-dimensional convolution unit outputs a shallow layer feature map; the output end of the tenth two-dimensional convolution unit outputs a middle layer characteristic diagram; the output end of the thirteenth two-dimensional convolution unit outputs a deep characteristic map;
the shallow feature map, the middle feature map and the deep feature output by the feature extraction network are superposed to form three scale feature maps which are used as the input of the mesh deformation network;
the mesh deformation network as shown in fig. 3 includes a first deformation unit, a second deformation unit, and a third deformation unit connected in series in sequence;
the first deformation unit comprises a first vertex feature sampling module, a first graph convolution module, a first vertex inverse pooling module and a first Transfomer coding module; the first graph convolution module consists of five first to fifth graph convolution units which are connected in series; the first input end of the first vertex characteristic sampling module is used as the first input end of the mesh deformation network and is connected with an input initial elliptical mesh model, the second input end of the first vertex characteristic sampling module is used as the second input end of the mesh deformation network and is connected with three scale characteristic graphs output by the characteristic extraction network, the first output end of the first vertex characteristic sampling module is connected with the input end of the first graph convolution module, and the second output end of the first vertex characteristic sampling module is connected with the input end of the first Transfomer coding module; the output end data of the first graph convolution module and the output end data of the first Transfomer coding module are added and then connected with the input end of the first vertex anti-pooling module; the output end of the first vertex inverse pooling module is used as the output end of the first deformation unit and is connected with the second input end of the second deformation unit;
the second deformation unit comprises a second vertex feature sampling module, a second graph convolution module, a second vertex anti-pooling module and a second Transfomer coding module; the second graph convolution module is composed of five sixth graph convolution units to ten graph convolution units which are connected in series; the first input end of the second vertex characteristic sampling module is used as the third input end of the mesh deformation network and is connected with three scale characteristic graphs output by the characteristic extraction network, the second input end is connected with the output end of the first deformation unit, the first output end is connected with the input end of the second graph convolution module, and the second output end is connected with the input end of the second Transfomer coding module; the output end data of the second graph convolution module and the output end data of the second Transfomer coding module are added and then connected with the input end of the second vertex inverse pooling module; the output end of the second vertex anti-pooling module is used as the output end of the second deformation unit and is connected with the second input end of the third deformation unit;
the third deformation unit comprises a third vertex feature sampling module, a third graph convolution module and a third Transfomer coding module; the third graph convolution module is composed of five eleventh to fifteenth graph convolution units which are connected in series; the first input end of the third vertex characteristic sampling module is used as the fourth input end of the mesh deformation network and is connected with three scale characteristic graphs output by the characteristic extraction network, the second input end of the third vertex characteristic sampling module is connected with the output end of the second deformation unit, the first output end of the third vertex characteristic sampling module is connected with the input end of the third graph convolution module, and the second output end of the third vertex characteristic sampling module is connected with the input end of the third Transfomer coding module; adding the output end data of the third graph convolution module and the output end data of the third Transfomer coding module to serve as a reconstructed three-dimensional model of the grid deformation network for output;
the first Transfomer coding module comprises a first linear layer, a position coding layer, a multi-head attention mechanism layer, a first normalization layer, a forward propagation layer, a second normalization layer and a second linear layer which are sequentially connected in series; the second Transfomer coding module comprises a third linear layer, a position coding layer, a multi-head attention mechanism layer, a third normalization layer, a forward propagation layer, a fourth normalization layer and a fourth linear layer which are sequentially connected in series; the third Transfomer coding module comprises a fifth linear layer, a position coding layer, a multi-head attention mechanism layer, a fifth normalization layer, a forward propagation layer, a sixth normalization layer and a sixth linear layer which are sequentially connected in series;
3-2, performing model training by using a fusion loss function in the formula (6), wherein the fusion loss is formed by adding chamfer distance loss, side length loss, normal vector loss and Laplace loss;
the chamfer distance loss is expressed by the formula (1):
wherein P, Q are the two compared point sets, which herein represent the set of vertices of the reconstructed three-dimensional model and the real three-dimensional model, respectively; p and q represent unit vertexes in the corresponding point set; | P | represents the number of elements of the point set P; the calculation method is that the squared Euclidean distances from all unit points to the nearest point in the other point set are calculated for the two point sets respectively, and then the nearest average distances among the point sets are obtained respectively and added.
Side length loss formula (2):
where p is any vertex coordinate and k is any vertex coordinate in the set of contiguous vertices n (p) for vertex p, this penalty limits the appearance of excess long edges of the mesh model.
See formula (3) for normal vector loss:
wherein p represents the peak of the reconstructed three-dimensional model, q represents the peak with the nearest distance of the peak p in the real model, namely the peak is regarded as the approximate point of p, and nqDenotes the normal phasor of the vertex q, since p and q are approximate points nqThen, k represents the adjacent vertex of p, and the vector difference between the two is the required vertex edge vector,<,>representing the inter-vector inner product calculation.
Laplace loss formula (4):
wherein lpDenotes the Laplacian coordinate of the current mesh vertex, l'pCalculating Laplace coordinates of the grid vertexes before deformation of each deformation unit in a Laplace coordinate calculation mode shown in a formula (5);
wherein p is the vertex coordinate, lpIs the Laplace coordinate of the vertex, N (p) represents the adjacent vertex set with the coordinate being p vertex, k represents the coordinate of some adjacent vertex in the set, and the coordinate reflects the relative position of each vertex in the local mesh plane.
The fusion loss comprises the above four losses, which are expressed as follows:
loss＝losscd+αlossedg+βlossn+γlosslap (6)
wherein α is lossedgCorresponding loss weight, β is lossnCorresponding loss weight, γ is losslapThe corresponding loss weight; the invention takes alpha as 0.1, beta as 0.3 and gamma as 0.0001.
In order to verify the effectiveness of the invention, a plurality of existing reconstruction methods such as 3D-R2N2, PSGN, Pixel2Mesh and the like are compared, 13 different types of three-dimensional models are selected for training and testing based on a Shapenet data set in an experiment, and a CD (charfer distance) value and an F (F-Score) value are used as evaluation indexes, wherein the lower the CD value is, the better the reconstruction effect is, and the higher the F value is, the better the reconstruction effect is.
The experimental results are shown in tables 1 and 2, and the network model of the invention achieves excellent reconstruction effect compared with the existing model under two evaluation indexes.
Table 1 different network reconstruction results CD value comparison
TABLE 2 comparison of F-values for different network reconstruction results
Claims (9)
1. A three-dimensional image reconstruction method based on deep learning is characterized by comprising the following steps:
acquiring a two-dimensional image and corresponding three-dimensional model data;
secondly, preprocessing the two-dimensional image and the three-dimensional model data;
step three, building and training a neural network;
3-1, building a complete reconstruction network
The complete reconstruction network comprises an image feature extraction network and a grid deformation network; the image feature extraction network is used for inputting a single two-dimensional image to obtain three scale feature maps; the grid deformation network is used for inputting an initial elliptical grid model and three scale feature maps output by the image feature extraction network, and carrying out grid deformation on the initial elliptical grid model so as to complete reconstruction;
3-2 model training
Inputting the image data preprocessed in the step two into the complete reconstruction network in the step 3-1 for training; calculating loss values of a reconstruction model and a target model output by the complete reconstruction network through a fusion type loss function in the training process, and then continuously reducing the loss values through network iteration parameters through back propagation of the loss values to finish training;
and step four, inputting a single two-dimensional image to be reconstructed according to the network model built and trained in the step three, and finally outputting a reconstructed three-dimensional model.
2. The method as claimed in claim 1, wherein the preprocessing method in step two includes scaling, clipping, and normalizing the two-dimensional image, and the three-dimensional model is pre-sampled by using Poisson disk algorithm.
3. The deep learning-based image three-dimensional reconstruction method as claimed in claim 1, wherein the image feature extraction network structure comprises 13 two-dimensional convolution units connected in series of a VGG16 network; the first two-dimensional convolution unit is used as the input end of the feature extraction network and is connected with the preprocessed two-dimensional image data; the output end of the seventh two-dimensional convolution unit outputs a shallow layer feature map; the output end of the tenth two-dimensional convolution unit outputs a middle layer characteristic diagram; and the output end of the thirteenth two-dimensional convolution unit outputs the deep characteristic map.
4. The method as claimed in claim 1 or 3, wherein the shallow feature map, the middle feature map and the deep feature outputted by the feature extraction network are superimposed to form three scale feature maps as the input of the mesh deformation network.
5. The method as claimed in claim 1, wherein the mesh deformation network includes a first deformation unit, a second deformation unit, and a third deformation unit connected in series in sequence;
the first deformation unit comprises a first vertex feature sampling module, a first graph convolution module, a first vertex inverse pooling module and a first Transfomer coding module; the first input end of the first vertex characteristic sampling module is used as the first input end of the mesh deformation network and is connected with an input initial elliptical mesh model, the second input end of the first vertex characteristic sampling module is used as the second input end of the mesh deformation network and is connected with three scale characteristic graphs output by the characteristic extraction network, the first output end of the first vertex characteristic sampling module is connected with the input end of the first graph convolution module, and the second output end of the first vertex characteristic sampling module is connected with the input end of the first Transfomer coding module; the output end data of the first graph convolution module and the output end data of the first Transfomer coding module are added and then connected with the input end of the first vertex anti-pooling module; the output end of the first vertex inverse pooling module is used as the output end of the first deformation unit and is connected with the second input end of the second deformation unit;
the second deformation unit comprises a second vertex feature sampling module, a second graph convolution module, a second vertex anti-pooling module and a second Transfomer coding module; the first input end of the second vertex characteristic sampling module is used as the third input end of the mesh deformation network and is connected with three scale characteristic graphs output by the characteristic extraction network, the second input end is connected with the output end of the first deformation unit, the first output end is connected with the input end of the second graph convolution module, and the second output end is connected with the input end of the second Transfomer coding module; the output end data of the second graph convolution module and the output end data of the second Transfomer coding module are added and then connected with the input end of the second vertex inverse pooling module; the output end of the second vertex anti-pooling module is used as the output end of the second deformation unit and is connected with the second input end of the third deformation unit;
the third deformation unit comprises a third vertex feature sampling module, a third graph convolution module and a third Transfomer coding module; the first input end of the third vertex characteristic sampling module is used as the fourth input end of the mesh deformation network and is connected with three scale characteristic graphs output by the characteristic extraction network, the second input end of the third vertex characteristic sampling module is connected with the output end of the second deformation unit, the first output end of the third vertex characteristic sampling module is connected with the input end of the third graph convolution module, and the second output end of the third vertex characteristic sampling module is connected with the input end of the third Transfomer coding module; and adding the output end data of the third graph convolution module and the output end data of the third Transfomer coding module to output as a reconstructed three-dimensional model of the grid deformation network.
6. The method as claimed in claim 5, wherein the first image convolution module comprises five first to fifth image convolution units connected in series; the second graph convolution module comprises five sixth graph convolution units to ten graph convolution units which are connected in series; the third graph convolution module includes five eleventh to fifteenth graph convolution units connected in series.
7. The method for three-dimensional image reconstruction based on deep learning of claim 5, wherein the first fransformer coding module comprises a first linear layer, a position coding layer, a multi-head attention mechanism layer, a first layer normalization layer, a forward propagation layer, a second layer normalization layer, and a second linear layer, which are sequentially connected in series; the second Transfomer coding module comprises a third linear layer, a position coding layer, a multi-head attention mechanism layer, a third normalization layer, a forward propagation layer, a fourth normalization layer and a fourth linear layer which are sequentially connected in series; the third Transfomer coding module comprises a fifth linear layer, a position coding layer, a multi-head attention mechanism layer, a fifth normalization layer, a forward propagation layer, a sixth normalization layer and a sixth linear layer which are sequentially connected in series.
8. The three-dimensional image reconstruction method based on deep learning as claimed in claim 1, wherein the fusion loss function in the model training of step 3-2 is shown in formula (6), and the fusion loss is formed by adding chamfer distance loss, side length loss, normal vector loss and laplace loss;
the chamfer distance loss is expressed by the formula (1):
p and Q are vertex sets of the reconstructed three-dimensional model and the real three-dimensional model; p and q represent unit vertexes in the corresponding point set; | P | represents the number of elements of the point set P;
side length loss formula (2):
where k is any vertex coordinate in the contiguous set of vertices N (p) for vertex p;
see formula (3) for normal vector loss:
wherein n isqThe normal phasor of the vertex q is represented,<,>representing the inner product calculation between vectors;
laplace loss formula (4):
wherein lpDenotes the Laplacian coordinate, l 'of the current mesh vertex p'pCalculating the Laplace coordinate of the grid vertex p before deformation of each deformation unit in a Laplace coordinate calculation mode shown in a formula (5);
fusion loss is as follows:
loss＝losscd+αlossedg+βlossn+γlosslap (6)
wherein α is lossedgCorresponding loss weight, β is lossnCorresponding loss weight, γ is losslapCorresponding loss weight.
9. An image three-dimensional reconstruction system based on deep learning is characterized by comprising:
the data acquisition module is used for acquiring a two-dimensional image and a corresponding three-dimensional model;
the data preprocessing module is used for adjusting the format and the size of the two-dimensional image and the corresponding three-dimensional model;
and the complete reconstruction network module is used for training the complete reconstruction network by using the data processed by the data preprocessing module and then reconstructing a corresponding three-dimensional model of the two-dimensional image to be reconstructed by using the complete reconstruction network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111522113.4A CN114140601A (en) | 2021-12-13 | 2021-12-13 | Three-dimensional grid reconstruction method and system based on single image under deep learning framework |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111522113.4A CN114140601A (en) | 2021-12-13 | 2021-12-13 | Three-dimensional grid reconstruction method and system based on single image under deep learning framework |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114140601A true CN114140601A (en) | 2022-03-04 |
Family
ID=80382542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111522113.4A Pending CN114140601A (en) | 2021-12-13 | 2021-12-13 | Three-dimensional grid reconstruction method and system based on single image under deep learning framework |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114140601A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115619950A (en) * | 2022-10-13 | 2023-01-17 | 中国地质大学（武汉） | Three-dimensional geological modeling method based on deep learning |
| WO2024001311A1 (en) * | 2022-06-27 | 2024-01-04 | 京东科技信息技术有限公司 | Method, apparatus and system for training feature extraction network of three-dimensional mesh model |
| CN117994470A (en) * | 2024-04-07 | 2024-05-07 | 之江实验室 | Multi-mode hierarchical self-adaptive digital grid reconstruction method and device |
| CN118470254A (en) * | 2024-07-15 | 2024-08-09 | 湖南大学 | Three-dimensional grid reconstruction method based on self-adaptive template |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570522A (en) * | 2019-08-22 | 2019-12-13 | 天津大学 | Multi-view three-dimensional reconstruction method |
| CN112767539A (en) * | 2021-01-12 | 2021-05-07 | 杭州师范大学 | Image three-dimensional reconstruction method and system based on deep learning |
| CN113077554A (en) * | 2021-04-08 | 2021-07-06 | 华南理工大学 | Three-dimensional structured model reconstruction method based on any visual angle picture |
-
2021
- 2021-12-13 CN CN202111522113.4A patent/CN114140601A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570522A (en) * | 2019-08-22 | 2019-12-13 | 天津大学 | Multi-view three-dimensional reconstruction method |
| CN112767539A (en) * | 2021-01-12 | 2021-05-07 | 杭州师范大学 | Image three-dimensional reconstruction method and system based on deep learning |
| CN113077554A (en) * | 2021-04-08 | 2021-07-06 | 华南理工大学 | Three-dimensional structured model reconstruction method based on any visual angle picture |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024001311A1 (en) * | 2022-06-27 | 2024-01-04 | 京东科技信息技术有限公司 | Method, apparatus and system for training feature extraction network of three-dimensional mesh model |
| CN115619950A (en) * | 2022-10-13 | 2023-01-17 | 中国地质大学（武汉） | Three-dimensional geological modeling method based on deep learning |
| CN115619950B (en) * | 2022-10-13 | 2024-01-19 | 中国地质大学（武汉） | Three-dimensional geological modeling method based on deep learning |
| CN117994470A (en) * | 2024-04-07 | 2024-05-07 | 之江实验室 | Multi-mode hierarchical self-adaptive digital grid reconstruction method and device |
| CN117994470B (en) * | 2024-04-07 | 2024-06-07 | 之江实验室 | Multi-mode hierarchical self-adaptive digital grid reconstruction method and device |
| CN118470254A (en) * | 2024-07-15 | 2024-08-09 | 湖南大学 | Three-dimensional grid reconstruction method based on self-adaptive template |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112396703B (en) | Reconstruction method of single-image three-dimensional point cloud model | |
| CN114140601A (en) | Three-dimensional grid reconstruction method and system based on single image under deep learning framework | |
| Qian et al. | PUGeo-Net: A geometry-centric network for 3D point cloud upsampling | |
| CN110288695B (en) | Single-frame image three-dimensional model surface reconstruction method based on deep learning | |
| CN103279980B (en) | Based on the Leaf-modeling method of cloud data | |
| CN108921926B (en) | End-to-end three-dimensional face reconstruction method based on single image | |
| CN110458939A (en) | The indoor scene modeling method generated based on visual angle | |
| Wang et al. | Cascaded refinement network for point cloud completion with self-supervision | |
| CN112232134B (en) | Human body posture estimation method based on hourglass network and attention mechanism | |
| Lu et al. | Attention-based dense point cloud reconstruction from a single image | |
| Demir et al. | Skelneton 2019: Dataset and challenge on deep learning for geometric shape understanding | |
| CN110570522A (en) | Multi-view three-dimensional reconstruction method | |
| CN112509102A (en) | Lightweight visualization method of BIM model data | |
| CN111797692B (en) | Depth image gesture estimation method based on semi-supervised learning | |
| Zhu et al. | Nonlocal low-rank point cloud denoising for 3-D measurement surfaces | |
| CN118229885A (en) | Incremental three-dimensional reconstruction method based on fusion voxel octree and grid rendering | |
| CN116758219A (en) | Region-aware multi-view stereo matching three-dimensional reconstruction method based on neural network | |
| Yan et al. | GEOP-Net: Shape reconstruction of buildings from LiDAR point clouds | |
| CN108805886B (en) | Persistent clustering segmentation method for multi-fusion physical signatures | |
| CN113593007B (en) | Single-view three-dimensional point cloud reconstruction method and system based on variation self-coding | |
| CN113658323A (en) | Method for reconstructing network model in three dimensions based on graph convolution and computer equipment | |
| Xing et al. | Dynamic Guided Network for Monocular Depth Estimation | |
| CN113436314B (en) | Computer three-dimensional grid model reconstruction method based on structure | |
| Chen et al. | Channel Attention based Network for LiDAR Super-resolution | |
| CN118470221B (en) | Three-dimensional target reconstruction method based on non-calibrated single view |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |