CN112150612A - Three-dimensional model construction method and device, computer equipment and storage medium - Google Patents
Three-dimensional model construction method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112150612A CN112150612A CN202011009375.6A CN202011009375A CN112150612A CN 112150612 A CN112150612 A CN 112150612A CN 202011009375 A CN202011009375 A CN 202011009375A CN 112150612 A CN112150612 A CN 112150612A
- Authority
- CN
- China
- Prior art keywords
- mapping
- dimensional
- points
- mapping points
- mapping point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010276 construction Methods 0.000 title claims abstract description 11
- 238000013507 mapping Methods 0.000 claims abstract description 448
- 238000000034 method Methods 0.000 claims abstract description 50
- 238000012216 screening Methods 0.000 claims abstract description 35
- 239000011159 matrix material Substances 0.000 claims description 41
- 238000012545 processing Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 25
- 238000010586 diagram Methods 0.000 description 6
- 238000003062 neural network model Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Abstract
The application relates to a three-dimensional model construction method, a three-dimensional model construction device, computer equipment and a storage medium. The method comprises the following steps: acquiring a two-dimensional image, wherein the two-dimensional image comprises a target object of a three-dimensional model to be constructed; acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point; screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point; a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the plurality of target mapping points. In the embodiment of the application, the mapping points are filtered based on the vertex similarity prediction value of each mapping point, so that some mapping points which do not belong to the vertex can be filtered, and the method is better suitable for the reconstruction of three-dimensional models of target objects with different vertex numbers.
Description
Technical Field
The present application relates to the field of computer vision technologies, and in particular, to a method and an apparatus for building a three-dimensional model, a computer device, and a storage medium.
Background
In recent years, with rapid development of intelligent robots, unmanned technologies, virtual reality technologies, and the like, three-dimensional model reconstruction based on two-dimensional images is a very important research direction in the field of computer vision.
In the prior art, the process of reconstructing a three-dimensional model based on a two-dimensional image generally includes: and inputting the two-dimensional image into the trained neural network model to obtain the three-dimensional coordinates of the mapping points output by the neural network model, wherein the number of the mapping points is preset in the process of training the neural network model.
However, in the above method, when the three-dimensional model is reconstructed for the target object having a small number of vertices, excessive mapping points appear on the vertices of the three-dimensional model of the reconstructed target object, and the model is blurred, and therefore, the reconstruction accuracy of the three-dimensional model is not good.
Disclosure of Invention
In view of the above, it is desirable to provide a three-dimensional model construction method, apparatus, computer device, and storage medium capable of improving the accuracy of a three-dimensional model of a target object.
A method of constructing a three-dimensional model, the method comprising:
acquiring a two-dimensional image, wherein the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point;
a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the plurality of target mapping points.
In one embodiment, the screening of the plurality of target mapping points from the plurality of mapping points based on the predicted vertex similarity value of each mapping point comprises:
performing descending order arrangement on the mapping points based on the vertex similarity prediction value of each mapping point, and sequentially determining each mapping point as a candidate mapping point according to the descending order arrangement;
and calculating the distances from other mapping points to the candidate mapping points, and eliminating the mapping points with the distances to the candidate mapping points smaller than a distance threshold value to obtain a plurality of target mapping points.
In one embodiment, after eliminating the mapping points with the distance to the candidate mapping point smaller than the distance threshold, the method further comprises:
and removing the mapping points with the vertex similarity prediction value smaller than the similarity threshold value from the plurality of mapping points.
In one embodiment, before the step of screening out the plurality of target mapping points from the plurality of mapping points based on the predicted vertex similarity value of each mapping point, the method further comprises:
performing decentralized processing on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralized processing;
normalizing each first mapping point to obtain a plurality of second mapping points;
screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point, and the method comprises the following steps:
and screening a plurality of target mapping points from the plurality of second mapping points based on the predicted vertex similarity value of each second mapping point.
In one embodiment, the decentralizing the three-dimensional coordinates of each mapping point to obtain a plurality of decentered first mapping points includes:
acquiring a coordinate average value of each coordinate axis in three-dimensional coordinates of a plurality of mapping points;
and for each mapping point, subtracting the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of the mapping point to obtain the first mapping point after the decentralized processing.
In one embodiment, the normalizing each first mapping point to obtain a plurality of second mapping points includes:
acquiring the maximum distance from each first mapping point to the original point;
for each first mapping point, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain a second mapping point.
In one embodiment, acquiring three-dimensional coordinates of a plurality of mapping points of a target object and a vertex similarity prediction value of each mapping point based on a two-dimensional image includes:
acquiring three-dimensional structural features of a two-dimensional image;
uniformly collecting initial mapping points of a target number in a preset reference plane, and constructing an initial coordinate matrix according to two-dimensional coordinates of the initial mapping points, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of one initial mapping point;
splicing the three-dimensional structural features to each row of the initial coordinate matrix to obtain a spliced matrix;
and inputting the spliced matrix into a trained decoder, and obtaining the three-dimensional coordinates of the mapping points of the target number output by the decoder and the vertex similarity predicted value of each mapping point.
A three-dimensional model building apparatus, the apparatus comprising:
the image acquisition module is used for acquiring a two-dimensional image, and the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
the mapping point acquisition module is used for acquiring three-dimensional coordinates of a plurality of mapping points of the target object and vertex similarity prediction values of the mapping points on the basis of the two-dimensional image, wherein the vertex similarity prediction values of the mapping points represent the similarity between the mapping points and the three-dimensional vertex of the target object closest to the mapping points;
the screening module is used for screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity predicted value of each mapping point;
and the model building module is used for building a three-dimensional model of the target object based on the three-dimensional coordinates of the plurality of target mapping points.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring a two-dimensional image, wherein the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point;
a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the plurality of target mapping points.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring a two-dimensional image, wherein the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point;
a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the plurality of target mapping points.
According to the three-dimensional model construction method, the three-dimensional model construction device, the computer equipment and the storage medium, the two-dimensional image is obtained, wherein the two-dimensional image comprises the target object of the three-dimensional model to be constructed; acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point; screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point; a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the plurality of target mapping points. In the embodiment of the application, the mapping points are filtered based on the vertex similarity prediction value of each mapping point, so that some mapping points which do not belong to the vertex can be filtered, and the method is better suitable for the reconstruction of three-dimensional models of target objects with different vertex numbers.
Drawings
FIG. 1 is a schematic flow chart of a three-dimensional model construction method according to an embodiment;
FIG. 2 is a flow diagram of the steps for obtaining map points in one embodiment;
FIG. 3 is a schematic diagram of the structure of a multilayer sensor in one embodiment;
FIG. 4 is a flow diagram of the steps of screening map points in one embodiment;
FIG. 5 is a flow chart of another step of screening map points in one embodiment;
FIG. 6 is a block diagram showing the structure of a three-dimensional model building apparatus according to an embodiment;
FIG. 7 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In recent years, with rapid development of intelligent robots, unmanned technologies, virtual reality technologies, and the like, three-dimensional model reconstruction based on two-dimensional images is a very important research direction in the field of computer vision. The three-dimensional model reconstruction based on the two-dimensional image specifically includes: and restoring the three-dimensional model structure of the target object according to the two-dimensional image.
In the prior art, the process of reconstructing a three-dimensional model based on a two-dimensional image generally includes: and inputting the two-dimensional image into the trained neural network model to obtain the three-dimensional coordinates of the mapping points output by the neural network model, wherein the number of the mapping points is preset in the process of training the neural network model.
However, in the above method, when the three-dimensional model is reconstructed for the target object having a small number of vertices, excessive mapping points appear on the vertices of the three-dimensional model of the reconstructed target object, and the model is blurred, and therefore, the reconstruction accuracy of the three-dimensional model is not good.
In the embodiment of the application, the three-dimensional coordinates and the vertex similarity of a large number of mapping points are obtained for prediction, then the mapping points are optimized and screened based on the vertex similarity, the mapping points with high similarity to the three-dimensional vertex of the target object are reserved, and the mapping points with low similarity to the three-dimensional vertex of the target object are deleted, so that the obtained target mapping points are closer to the three-dimensional vertex of the target object, and the accuracy of the three-dimensional model of the target object constructed according to the target mapping points is higher.
In one embodiment, as shown in fig. 1, a three-dimensional model building method is provided, and this embodiment is illustrated by applying the method to a computer device, which may be a server, a computer, or the like. In this embodiment, the method includes the steps of:
Wherein the two-dimensional image comprises a target object of the three-dimensional model to be constructed. Optionally, the two-dimensional image is a single two-dimensional image.
The vertex similarity prediction value of each mapping point represents the similarity between the mapping point and the three-dimensional vertex of the target object closest to the mapping point. The three-dimensional vertices of the target object refer to vertices on the three-dimensional model of the target object.
In the embodiment of the application, the computer device may obtain a pre-trained three-dimensional feature extraction model, and then the computer device may input the two-dimensional image into the three-dimensional feature extraction model, and obtain three-dimensional coordinates of a plurality of mapping points of the target object and an order similarity prediction value of each mapping point, which are output by the three-dimensional feature extraction model.
And 103, screening a plurality of target mapping points from the plurality of mapping points by the computer equipment based on the predicted vertex similarity value of each mapping point.
In the embodiment of the present application, the greater the predicted vertex similarity value of each mapping point, the greater the probability that the mapping point is a three-dimensional vertex of the target object, and the smaller the predicted vertex similarity value, the smaller the probability that the mapping point is a three-dimensional vertex of the target object.
The computer device may screen the plurality of mapping points according to the magnitude of the vertex similarity prediction value of each mapping point, for example, the computer device may remove the mapping points of which the vertex similarity prediction value is equal to or less than a similarity threshold value, the remaining mapping points are target mapping points, and the target mapping points are mapping points closer to the three-dimensional vertex of the target object, so that the accuracy of the three-dimensional model of the target object constructed based on the target mapping points is higher.
Optionally, in this embodiment of the application, the computer device may input the two-dimensional image and the three-dimensional coordinates of the plurality of target mapping points into the mesh model, so as to obtain a three-dimensional model of the target object output by the mesh model.
According to the three-dimensional model building method provided by the embodiment of the application, the three-dimensional coordinates and the vertex similarity prediction of a large number of mapping points are obtained, then the mapping points are optimized and screened based on the vertex similarity, the mapping points with high similarity to the three-dimensional vertex of the target object are reserved, and the mapping points with low similarity to the three-dimensional vertex of the target object are deleted, so that the obtained target mapping points are closer to the three-dimensional vertex of the target object, and the precision of the three-dimensional model of the target object built according to the target mapping points is higher.
In one embodiment, as shown in fig. 2, the process of acquiring, by a computer device, three-dimensional coordinates of a plurality of mapping points of a target object and a vertex similarity prediction value of each mapping point based on a two-dimensional image includes:
In this embodiment of the present application, the computer device may obtain a pre-trained encoder, and optionally, the encoder may adopt a backbone network of ResNet 50. And then the computer equipment can input the two-dimensional image into a trained encoder, and the trained encoder is used for carrying out feature extraction on the two-dimensional image to obtain the three-dimensional structural features of the target object in the target image.
Alternatively, the three-dimensional structural feature of the target object may be a feature vector, for example, the three-dimensional structural feature of the target object may be a 1024-dimensional feature vector.
Each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of one initial mapping point.
In this embodiment, the preset reference plane may be a unit plane with an interval of [0,1 ].
The computer device may perform uniform sampling on the reference plane, and optionally, may adopt a uniform grid sampling manner, that is, the unit plane is divided into N equal parts in the X-axis and Y-axis directions respectively and uniformly. Wherein N is a predetermined value. For example, N is 1000. This results in 1000 × 1000 to 1000000 initial mapping points on the reference plane.
It should be noted that, in the embodiment of the present application, the number of initial mapping points is preset, and the value of N is determined by performing back-stepping according to the number of initial mapping points.
Optionally, in this embodiment of the application, the number of initial mapping points (target number) is greater than a preset threshold. Therefore, when the number of the vertexes contained in the three-dimensional model of the target object is large, the vertexes of the three-dimensional model of the target object can be ensured to be covered comprehensively, and model distortion caused by the small number of the initial mapping points is avoided.
In this embodiment, the computer device may represent the sampled coordinates of the initial mapping point as an initial coordinate matrix, and the initial coordinate matrix may be a matrix of 1000000 rows and 2 columns. The first column represents the X-axis coordinate and the second column represents the Y-axis coordinate.
For example, in this embodiment of the application, after the computer device splices the three-dimensional structure feature to each row of the initial coordinate matrix, for example, the computer device splices a 1024-dimensional feature vector to each row of a 1000000 row-2 column matrix, a spliced matrix of 1000000 rows (2+1024) columns is obtained.
And step 204, inputting the spliced matrix into a trained decoder by the computer equipment, and obtaining the three-dimensional coordinates of the mapping points with the target quantity output by the decoder and the vertex similarity predicted value of each mapping point.
In the embodiment of the present application, the computer device may obtain a trained decoder, and an optional decoder may adopt an MLP (Multi-Layer Perceptron, chinese), where a structure of the MLP may be as shown in fig. 3. The computer device may input the spliced matrix into the decoder to obtain a decoded matrix output by the decoder, and the decoded matrix is a 1000000 row matrix with 4 columns, where the first three columns are coordinates in the directions of the X axis Y axis and the Z axis, respectively, and the fourth column is a vertex similarity prediction value of the mapping point.
According to the embodiment of the application, the mapping points with the target number are obtained by obtaining the initial mapping points with the target number, and the mapping points with the target number are obtained in a large number, so that when the three-dimensional model reconstruction is carried out on the target object with the complex structure and multiple vertexes, all vertexes of the target object can be covered, and the precision of the three-dimensional model of the target object is ensured.
Furthermore, according to the embodiment of the application, the vertex similarity prediction value of each mapping point is obtained, and the similarity degree of each mapping point and the vertex can be determined, so that the mapping points are optimized and filtered conveniently, the model gelatinization is avoided, and the precision of the three-dimensional model of the target object is improved.
In one embodiment, as shown in fig. 4, the process of the computer device for screening out a plurality of target mapping points from a plurality of mapping points based on the predicted vertex similarity value of each mapping point comprises:
step 401, the computer device performs descending order arrangement on the mapping points based on the vertex similarity prediction value of each mapping point, and determines each mapping point as a candidate mapping point in sequence according to the descending order arrangement.
In the embodiment of the present application, a non-maximum constant method may be adopted to remove adjacent repeated mapping points for a plurality of mapping points, specifically: the computer device may sort in descending order according to the magnitude of the predicted values of the vertex similarity of the mapping points. And sequentially determining the mapping points as candidate mapping points according to the descending order.
For example, if the mapping points a, B, and C are arranged in descending order according to their respective vertex similarity prediction values as BCA, the computer device may determine B as a candidate mapping point according to the arrangement order, and after processing mapping point B, may determine C as a candidate mapping point if mapping point C is not removed, and after processing mapping point B, may remove C, and then may determine a mapping point a as a candidate mapping point.
Step 402, the computer device calculates the distances from other mapping points to the candidate mapping points, and eliminates the mapping points with the distances from the candidate mapping points smaller than the distance threshold value to obtain a plurality of target mapping points.
Taking the example of processing the mapping point B, the computer device may calculate the distance from the candidate mapping point to any other mapping point according to the three-dimensional coordinates of the candidate mapping point and the three-dimensional coordinates of other mapping points, and then eliminate the mapping points with the distance to the candidate mapping point smaller than the distance threshold.
Alternatively, the distance threshold may be 0.01.
In the embodiment of the present application, each of the mapping points that are not removed is determined as a candidate mapping point, and then the removing process is performed based on the candidate mapping points through the content disclosed in step 402 until the distance between any two mapping points in the remaining mapping points is greater than or equal to the distance threshold, and the remaining mapping points are the target mapping points.
Optionally, in this embodiment of the present application, after removing the duplicate mapping points, the computer device may further perform the following operations: the computer device may remove from the plurality of mapping points that remain, mapping points for which the vertex similarity prediction value is less than the similarity threshold.
In the embodiment of the application, the mapping points with the distance to the candidate mapping point smaller than the distance threshold value correspond to the same three-dimensional vertex, and each three-dimensional vertex corresponds to one mapping point in a mode of eliminating the mapping points with the distance to the candidate mapping point smaller than the distance threshold value, so that model blurring caused by the fact that a large number of mapping points are gathered near one three-dimensional vertex is avoided.
Furthermore, the mapping points with the highest similarity with the three-dimensional vertex can be reserved by arranging the vertex similarity predicted values of the mapping points in a descending order, so that the precision of the three-dimensional model is improved.
In one embodiment, as shown in fig. 5, the process of the computer device for screening out a plurality of target mapping points from a plurality of mapping points based on the predicted vertex similarity value of each mapping point may further include the following steps:
step 501, the computer device performs decentralized processing on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralized processing.
Optionally, in this embodiment of the present application, the process of performing the decentralized processing on each mapping point by the computer device may include the following steps:
in step a1, the computer device obtains a coordinate average value of each of the three-dimensional coordinate axes of the plurality of mapping points.
The computer device may calculate the mean Xmean, Ymean, Zmean of the first three columns in the decoded matrix in step 204.
That is, the computer device may average 1000000X-axis coordinates to obtain an X-axis coordinate average Xmean, average 1000000Y-axis coordinates to obtain a Y-axis coordinate average Ymean, and average 1000000Z-axis coordinates to obtain a Z-axis coordinate average Zmean.
And step A2, the computer equipment subtracts the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of each mapping point to obtain the first mapping point after the decentralized processing.
The computer device can subtract the respective coordinate average value from the three-dimensional coordinate of each mapping point to obtain the three-dimensional coordinate of each mapping point after the decentralization. For convenience of description, in the embodiment of the present application, a mapping point obtained by subtracting the coordinate average value of the corresponding coordinate axis is referred to as a first mapping point.
In this embodiment, the computer device may perform normalization processing on the plurality of first mapping points after the decentralization, specifically:
in step B1, the computer device obtains the maximum distance among the distances from the origin to each of the first mapping points.
In the embodiment of the present application, the computer device may calculate the euclidean distance from the origin point of each first mapping point, for example, for the first mapping point i, the euclidean distance from the origin point is Di ═ sqrt (Xi × Xi + Yi × Yi + Zi × Zi), where sqrt is an open operation. The computer device may determine the maximum distance Dmax from the plurality of distances.
And step B2, for each first mapping point, the computer device divides the three-dimensional coordinates of the first mapping point by the maximum distance to obtain a second mapping point.
In this embodiment, the computer device may divide each coordinate value in the three-dimensional coordinates of each first mapping point by the maximum distance to obtain a new three-dimensional coordinate of the first mapping point.
For convenience of description, the normalized first mapping point is referred to as a second mapping point.
In step 503, the computer device screens out a plurality of target mapping points from the plurality of second mapping points based on the predicted vertex similarity value of each second mapping point.
In the embodiment of the present application, the computer device may select a plurality of target mapping points from the plurality of second mapping points according to the vertex similarity prediction values of the respective second mapping points with reference to the disclosure of steps 401 to 402.
In the embodiment of the application, the three-dimensional coordinates of the plurality of mapping points are subjected to decentralized processing and normalization processing, and then the plurality of target mapping points are screened out based on the plurality of second mapping points subjected to decentralized processing and normalization processing, so that the process of optimizing the mapping points is realized, and the precision of the three-dimensional model of the target object is improved.
It should be understood that although the various steps in the flowcharts of fig. 1-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-5 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps or stages.
In one embodiment, as shown in fig. 6, there is provided a three-dimensional model building apparatus including: an image acquisition module 601, a mapping point acquisition module 602, a screening module 603, and a model construction module 604, wherein:
an image obtaining module 601, configured to obtain a two-dimensional image, where the two-dimensional image includes a target object of a three-dimensional model to be constructed;
a mapping point obtaining module 602, configured to obtain three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity prediction value of each mapping point based on the two-dimensional image, where the vertex similarity prediction value of each mapping point represents a similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
the screening module 603 is configured to screen a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point;
a model construction module 604 for constructing a three-dimensional model of the target object based on the three-dimensional coordinates of the plurality of target mapping points.
In an embodiment of the present application, the screening module 603 is further configured to perform descending order arrangement on the mapping points based on the vertex similarity prediction values of the mapping points, and sequentially determine the mapping points as candidate mapping points according to the descending order arrangement; and calculating the distances from other mapping points to the candidate mapping points, and eliminating the mapping points with the distances to the candidate mapping points smaller than a distance threshold value to obtain a plurality of target mapping points.
In an embodiment of the present application, the screening module 603 is further configured to, after removing the mapping points whose distance to the candidate mapping point is smaller than the distance threshold, remove the mapping points whose vertex similarity prediction value is smaller than the similarity threshold from the plurality of mapping points.
In an embodiment of the present application, the screening module 603 is further configured to perform a de-centering process on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after the de-centering process; normalizing each first mapping point to obtain a plurality of second mapping points; and screening a plurality of target mapping points from the plurality of second mapping points based on the predicted vertex similarity value of each second mapping point.
In an embodiment of the present application, the screening module 603 is further configured to obtain a coordinate average value of each coordinate axis in three-dimensional coordinates of a plurality of mapping points; and for each mapping point, subtracting the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of the mapping point to obtain the first mapping point after the decentralized processing.
In an embodiment of the present application, the screening module 603 is further configured to obtain a maximum distance of the distances from the first mapping points to the origin; for each first mapping point, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain a second mapping point.
In one embodiment of the present application, the mapping point obtaining module 602 is further configured to obtain three-dimensional structural features of the two-dimensional image; uniformly collecting initial mapping points of a target number in a preset reference plane, and constructing an initial coordinate matrix according to two-dimensional coordinates of the initial mapping points, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of one initial mapping point; splicing the three-dimensional structural features to each row of the initial coordinate matrix to obtain a spliced matrix; and inputting the spliced matrix into a trained decoder, and obtaining the three-dimensional coordinates of the mapping points of the target number output by the decoder and the vertex similarity predicted value of each mapping point.
For specific limitations of the three-dimensional model building apparatus, reference may be made to the above limitations of the three-dimensional model building method, which are not described herein again. The modules in the three-dimensional model building device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store pre-trained encoders and decoders. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a three-dimensional model building method.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring a two-dimensional image, wherein the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point;
a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the plurality of target mapping points.
In one embodiment, the processor, when executing the computer program, further performs the steps of: performing descending order arrangement on the mapping points based on the vertex similarity prediction value of each mapping point, and sequentially determining each mapping point as a candidate mapping point according to the descending order arrangement; and calculating the distances from other mapping points to the candidate mapping points, and eliminating the mapping points with the distances to the candidate mapping points smaller than a distance threshold value to obtain a plurality of target mapping points.
In one embodiment, the processor, when executing the computer program, further performs the steps of: and removing the mapping points with the vertex similarity prediction value smaller than the similarity threshold value from the plurality of mapping points.
In one embodiment, the processor, when executing the computer program, further performs the steps of: performing decentralized processing on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralized processing; normalizing each first mapping point to obtain a plurality of second mapping points; and screening a plurality of target mapping points from the plurality of second mapping points based on the predicted vertex similarity value of each second mapping point.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring a coordinate average value of each coordinate axis in three-dimensional coordinates of a plurality of mapping points; and for each mapping point, subtracting the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of the mapping point to obtain the first mapping point after the decentralized processing.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring the maximum distance from each first mapping point to the original point; for each first mapping point, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain a second mapping point.
In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring three-dimensional structural features of a two-dimensional image; uniformly collecting initial mapping points of a target number in a preset reference plane, and constructing an initial coordinate matrix according to two-dimensional coordinates of the initial mapping points, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of one initial mapping point; splicing the three-dimensional structural features to each row of the initial coordinate matrix to obtain a spliced matrix; and inputting the spliced matrix into a trained decoder, and obtaining the three-dimensional coordinates of the mapping points of the target number output by the decoder and the vertex similarity predicted value of each mapping point.
The implementation principle and technical effect of the computer device provided by the embodiment of the present application are similar to those of the method embodiment described above, and are not described herein again.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring a two-dimensional image, wherein the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point;
a three-dimensional model of the target object is constructed based on the three-dimensional coordinates of the plurality of target mapping points.
In one embodiment, the computer program when executed by the processor further performs the steps of: performing descending order arrangement on the mapping points based on the vertex similarity prediction value of each mapping point, and sequentially determining each mapping point as a candidate mapping point according to the descending order arrangement; and calculating the distances from other mapping points to the candidate mapping points, and eliminating the mapping points with the distances to the candidate mapping points smaller than a distance threshold value to obtain a plurality of target mapping points.
In one embodiment, the computer program when executed by the processor further performs the steps of: and removing the mapping points with the vertex similarity prediction value smaller than the similarity threshold value from the plurality of mapping points.
In one embodiment, the computer program when executed by the processor further performs the steps of: performing decentralized processing on the three-dimensional coordinates of each mapping point to obtain a plurality of first mapping points after decentralized processing; normalizing each first mapping point to obtain a plurality of second mapping points; and screening a plurality of target mapping points from the plurality of second mapping points based on the predicted vertex similarity value of each second mapping point.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring a coordinate average value of each coordinate axis in three-dimensional coordinates of a plurality of mapping points; and for each mapping point, subtracting the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of the mapping point to obtain the first mapping point after the decentralized processing.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring the maximum distance from each first mapping point to the original point; for each first mapping point, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain a second mapping point.
In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring three-dimensional structural features of a two-dimensional image; uniformly collecting initial mapping points of a target number in a preset reference plane, and constructing an initial coordinate matrix according to two-dimensional coordinates of the initial mapping points, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of one initial mapping point; splicing the three-dimensional structural features to each row of the initial coordinate matrix to obtain a spliced matrix; and inputting the spliced matrix into a trained decoder, and obtaining the three-dimensional coordinates of the mapping points of the target number output by the decoder and the vertex similarity predicted value of each mapping point.
The implementation principle and technical effect of the computer-readable storage medium provided in the embodiment of the present application are similar to those of the method embodiment described above, and are not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of constructing a three-dimensional model, the method comprising:
acquiring a two-dimensional image, wherein the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
acquiring three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity predicted value of each mapping point based on the two-dimensional image, wherein the vertex similarity predicted value of each mapping point represents the similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity predicted value of each mapping point;
constructing a three-dimensional model of the target object based on the three-dimensional coordinates of the plurality of target mapping points.
2. The method of claim 1, wherein the screening of the plurality of mapping points for the target mapping point based on the predicted vertex similarity value of each mapping point comprises:
performing descending order arrangement on the mapping points based on the vertex similarity prediction value of each mapping point, and sequentially determining each mapping point as a candidate mapping point according to the descending order arrangement;
and calculating the distances from other mapping points to the candidate mapping points, and eliminating the mapping points with the distances to the candidate mapping points smaller than a distance threshold value to obtain the plurality of target mapping points.
3. The method of claim 2, wherein after the rejecting the mapping points whose distance to the candidate mapping point is less than a distance threshold, the method further comprises:
and eliminating the mapping points with vertex similarity prediction values smaller than a similarity threshold value from the plurality of mapping points.
4. The method of claim 1, wherein before the step of screening the plurality of mapping points for the target mapping point based on the predicted vertex similarity value of each mapping point, the method further comprises:
performing decentralized processing on the three-dimensional coordinates of the mapping points to obtain a plurality of first mapping points after decentralized processing;
normalizing each first mapping point to obtain a plurality of second mapping points;
the screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point comprises the following steps:
and screening a plurality of target mapping points from the plurality of second mapping points based on the predicted vertex similarity value of each second mapping point.
5. The method according to claim 4, wherein the step of performing a de-centering process on the three-dimensional coordinates of each of the mapping points to obtain a plurality of first mapping points after the de-centering process comprises:
acquiring a coordinate average value of each coordinate axis in the three-dimensional coordinates of the plurality of mapping points;
and for each mapping point, subtracting the coordinate average value of the corresponding coordinate axis from the three-dimensional coordinate of the mapping point to obtain the first mapping point after the decentralized processing.
6. The method as claimed in claim 4, wherein the normalizing each of the first mapping points to obtain a plurality of second mapping points comprises:
acquiring the maximum distance from each first mapping point to the origin;
for each of the first mapping points, the three-dimensional coordinates of the first mapping point are divided by the maximum distance to obtain the second mapping point.
7. The method according to claim 1, wherein the obtaining three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity prediction value of each mapping point based on the two-dimensional image comprises:
acquiring three-dimensional structural features of the two-dimensional image;
uniformly collecting initial mapping points of a target number in a preset reference plane, and constructing an initial coordinate matrix according to two-dimensional coordinates of the initial mapping points, wherein each row of the initial coordinate matrix corresponds to the two-dimensional coordinates of one initial mapping point;
splicing the three-dimensional structural features to each row of the initial coordinate matrix to obtain a spliced matrix;
and inputting the spliced matrix into a trained decoder, and obtaining the three-dimensional coordinates of the mapping points of the target number output by the decoder and the vertex similarity predicted value of each mapping point.
8. A three-dimensional model building apparatus, characterized in that the apparatus comprises:
the image acquisition module is used for acquiring a two-dimensional image, and the two-dimensional image comprises a target object of a three-dimensional model to be constructed;
a mapping point obtaining module, configured to obtain, based on the two-dimensional image, three-dimensional coordinates of a plurality of mapping points of the target object and a vertex similarity prediction value of each mapping point, where the vertex similarity prediction value of each mapping point represents a similarity between the mapping point and a three-dimensional vertex of the target object closest to the mapping point;
the screening module is used for screening a plurality of target mapping points from the plurality of mapping points based on the vertex similarity prediction value of each mapping point;
a model construction module for constructing a three-dimensional model of the target object based on the three-dimensional coordinates of the plurality of target mapping points.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011009375.6A CN112150612A (en) | 2020-09-23 | 2020-09-23 | Three-dimensional model construction method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011009375.6A CN112150612A (en) | 2020-09-23 | 2020-09-23 | Three-dimensional model construction method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112150612A true CN112150612A (en) | 2020-12-29 |
Family
ID=73897804
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011009375.6A Pending CN112150612A (en) | 2020-09-23 | 2020-09-23 | Three-dimensional model construction method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112150612A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113190150B (en) * | 2021-04-20 | 2023-09-01 | 北京百度网讯科技有限公司 | Method, apparatus and storage medium for displaying overlay |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012008657A (en) * | 2010-06-22 | 2012-01-12 | Toshiba Corp | Apparatus and method for generating object shape |
| WO2019230813A1 (en) * | 2018-05-30 | 2019-12-05 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | Three-dimensional reconstruction method and three-dimensional reconstruction device |
| CN110727817A (en) * | 2019-06-20 | 2020-01-24 | 广州大学华软软件学院 | Three-dimensional model retrieval method based on t-CNN, terminal equipment and storage medium |
| CN110766808A (en) * | 2019-11-05 | 2020-02-07 | 北京智拓视界科技有限责任公司 | Method and device for processing face data and computer-readable storage medium |
| CN111401234A (en) * | 2020-03-13 | 2020-07-10 | 深圳普罗米修斯视觉技术有限公司 | Three-dimensional character model construction method and device and storage medium |
-
2020
- 2020-09-23 CN CN202011009375.6A patent/CN112150612A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012008657A (en) * | 2010-06-22 | 2012-01-12 | Toshiba Corp | Apparatus and method for generating object shape |
| WO2019230813A1 (en) * | 2018-05-30 | 2019-12-05 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | Three-dimensional reconstruction method and three-dimensional reconstruction device |
| CN110727817A (en) * | 2019-06-20 | 2020-01-24 | 广州大学华软软件学院 | Three-dimensional model retrieval method based on t-CNN, terminal equipment and storage medium |
| CN110766808A (en) * | 2019-11-05 | 2020-02-07 | 北京智拓视界科技有限责任公司 | Method and device for processing face data and computer-readable storage medium |
| CN111401234A (en) * | 2020-03-13 | 2020-07-10 | 深圳普罗米修斯视觉技术有限公司 | Three-dimensional character model construction method and device and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113190150B (en) * | 2021-04-20 | 2023-09-01 | 北京百度网讯科技有限公司 | Method, apparatus and storage medium for displaying overlay |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020186703A1 (en) | Convolutional neural network-based image processing method and image processing apparatus | |
| CN108960230B (en) | Lightweight target identification method and device based on rotating rectangular frame | |
| CN109614874B (en) | Human behavior recognition method and system based on attention perception and tree skeleton point structure | |
| KR20210036715A (en) | Neural processing apparatus and method for processing pooling of neural network thereof | |
| CN113469074B (en) | Remote sensing image change detection method and system based on twin attention fusion network | |
| CN111274999B (en) | Data processing method, image processing device and electronic equipment | |
| CN112183295A (en) | Pedestrian re-identification method and device, computer equipment and storage medium | |
| CN109859314B (en) | Three-dimensional reconstruction method, three-dimensional reconstruction device, electronic equipment and storage medium | |
| CN112001983B (en) | Method and device for generating occlusion image, computer equipment and storage medium | |
| CN113420640A (en) | Mangrove hyperspectral image classification method and device, electronic equipment and storage medium | |
| CN115100383B (en) | Three-dimensional reconstruction method, device and equipment for mirror surface object based on common light source | |
| CN114266894A (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN111709415B (en) | Target detection method, device, computer equipment and storage medium | |
| CN115457492A (en) | Target detection method and device, computer equipment and storage medium | |
| CN111340025A (en) | Character recognition method, character recognition device, computer equipment and computer-readable storage medium | |
| CN112150612A (en) | Three-dimensional model construction method and device, computer equipment and storage medium | |
| CN114429208A (en) | Model compression method, device, equipment and medium based on residual structure pruning | |
| CN115631192B (en) | Control method, device, equipment and medium for valve pressure tester | |
| CN114820755B (en) | Depth map estimation method and system | |
| CN113554164A (en) | Neural network model optimization method, neural network model data processing method, neural network model optimization device, neural network model data processing device and storage medium | |
| CN111179408A (en) | Method and apparatus for three-dimensional modeling | |
| CN114494441A (en) | Grape and picking point synchronous identification and positioning method and device based on deep learning | |
| CN112184884A (en) | Three-dimensional model construction method and device, computer equipment and storage medium | |
| CN110545373B (en) | Spatial environment sensing method and device | |
| CN113674383A (en) | Method and device for generating text image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |