CN107610221B - Three-dimensional model generation method based on isomorphic model representation - Google Patents
Three-dimensional model generation method based on isomorphic model representation Download PDFInfo
- Publication number
- CN107610221B CN107610221B CN201710810698.7A CN201710810698A CN107610221B CN 107610221 B CN107610221 B CN 107610221B CN 201710810698 A CN201710810698 A CN 201710810698A CN 107610221 B CN107610221 B CN 107610221B
- Authority
- CN
- China
- Prior art keywords
- model
- dimensional
- vector
- representation
- component
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a three-dimensional model generation method based on isomorphic model representation, which comprises the following steps: aiming at the component corresponding relation of the model set, constructing a unified structural representation of the model set; establishing isomorphic structure representation of each model by adopting a sub-graph coding mode; establishing a part representation of each model in a mode of a bounding box and a generalized cylinder; constructing a unified representation of the model according to the structural representation and the component representation of the model; training a self-encoder based on a neural network, and establishing a mapping relation between an isomorphic representation space and a two-dimensional numerical space; sampling the two-dimensional value, and decoding by using a self-encoder to obtain a model isomorphic representation; reconstructing a three-dimensional model according to the isomorphic representation of the model obtained by decoding, and judging the effectiveness of the reconstructed model; estimating the distribution of the effective space according to the effectiveness of the sampling data, and visualizing; and decoding to obtain model representation according to the effective two-dimensional data selected by the user, and reconstructing a new three-dimensional model.
Description
Technical Field
The invention belongs to the technical field of computer graphics, and particularly relates to a three-dimensional model generation method based on isomorphic model representation.
Background
Three-dimensional modeling is an important task of computer graphics and is the basis for satisfying other follow-up studies and applications. Three-dimensional modeling usually requires interaction of users and consumes a lot of time, and many applications have certain requirements on the number of three-dimensional models, so how to quickly and effectively construct large-scale three-dimensional models is an important research direction of three-dimensional modeling.
In fact, three-dimensional modeling has resulted in a number of related techniques and methods. In the traditional cad modeling method, a modeler needs to perform a large amount of interaction from the vertex and the patch of the bottom layer, such as vertex coordinate input and patch movement, to complete the modeling process.
Since the traditional method is heavily burdened with interaction, a method of reconstructing a digital model from real objects has emerged. As in document 1: bao S Y, Savaree S.Semantic structure from motion [ C ]// Computer Vision and Pattern Recognition (CVPR),2011IEEE Conference on.IEEE,2011 2025 plus 2032. the video sequence is utilized to estimate the camera parameters through adjacent video frames, thereby reconstructing a three-dimensional point cloud frame, and then fusing the three-dimensional point clouds reconstructed by a plurality of frames to obtain a reconstructed three-dimensional model. The method can generate the three-dimensional model completely and automatically, does not need interaction, but only can construct real objects existing in reality, and is difficult to meet the requirement of constructing large-scale three-dimensional model data.
A new three-dimensional model can also be constructed by editing an existing three-dimensional model. As in document 2: SorkineO, Alexa M.As-edge-as-passive surface modeling [ C ]// Sympossium on geomorphism.2007, 4. for an existing three-dimensional model, surface deformation points and fixed points are defined, the positions of the fixed points are kept unchanged while the positions of operation points are changed, and then the positions of all vertexes of the model are calculated, so that a new model is constructed. This method requires relatively simple interaction, and constructs a new three-dimensional model by editing an existing model. Although the method can construct new models different from the original models, the number of the constructed new models is small, and the difference between the constructed models is small.
As the size of three-dimensional model sets grows, some researchers have proposed model set-driven model generation methods. Document 3: chaudhuri S, Kalogerakis E, Guibas L, et al, basic reasoning for assembly-based 3D modeling [ C ]// ACM Transformations On Graphics (TOG). ACM 2011,30(4):35. A method for three-dimensional modeling using component assembly is proposed. Firstly, the model set is divided and analyzed to obtain the relationship between the component set and the components. The user then combines the components by selecting different components to build a new three-dimensional model. The method can generate models with large differences, but in the modeling process, user interaction is very frequent, and a large amount of time is needed for constructing one model, so that the method is difficult to generate a large amount of three-dimensional models. In summary, the prior art has the main defects: firstly, the modeling process takes long time and is difficult to construct a certain number of model sets; secondly, the difference of modeling results is small, and the generated result model set is monotonous; thirdly, more interaction is required and the user is burdened.
Disclosure of Invention
The purpose of the invention is as follows: the technical problem to be solved by the invention is to provide a method for carrying out unified representation on a model set based on an isomorphic model and automatically generating a new three-dimensional model through simple interaction to carry out computer three-dimensional modeling aiming at the defects of the prior art.
In order to solve the technical problem, the invention discloses a three-dimensional model generation method based on an isomorphic model, which comprises the following steps:
step 1, isomorphic model representation generation: analyzing the structural relationship of three-dimensional models in a given three-dimensional model set of the same category, performing shape fitting on a single three-dimensional model, and performing isomorphic unified representation on all three-dimensional models in the model set to obtain a unified representation vector of the single three-dimensional model;
step 2, model representation coding: converting the unified expression vector of each three-dimensional model into low-dimensional codes through a fully connected neural network to form a coding space;
step 3, constructing a visual space: on the basis of calculating the low-dimensional code represented by the model, judging the effectiveness of the model corresponding to each data point in the coding space, and visualizing the effectiveness distribution condition of the coding space.
Step 4, interactively generating a three-dimensional model: and (3) clicking and interacting the visual coding space in the step (3) by a user, selecting a data instance in the coding space, and decoding according to the selected data instance to generate a three-dimensional model.
The step 1 comprises the following steps:
step 1-1, modelingAnd generating a set unified structure representation: the unified structure of the model set is represented as an undirected graph G ═ V, E }, where V is a set of all nodes, each node represents each component type, and assuming that all models of the model set have n types of components, V ═ E }, where V is a set of all nodes, V represents a set of all components, and V ═ V1,v2…vnIn which v isiThe component I represents the component I, and the value range of i is 1-n; e is the set of all edges, each edge represents the relationship between every two types of components, and if the model set has m adjacent components, E is { E ═ E1,e1…emIn which ejRepresenting the j adjacent condition, wherein the value range of j is 1-m;
step 1-2, generating a single three-dimensional model structure representation: using a vector l with the length of n + m as a structural representation of a single three-dimensional model, extracting the existence condition and the adjacency relation of a single three-dimensional model component, representing as an undirected graph G ═ V, E }, judging whether each member of V and each member of E belong to G or not according to the undirected graph G ═ V, E }, and enabling the ith component l of the vector l to belong to G or not, and enabling the ith component l of the vector l to belong to GiIs 1, otherwise liIs 0; the jth member of E belongs to g, let the j + n component l of the vector lj+nIs 1, otherwise lj+nIs 0;
step 1-3, fitting the shape of a single model: for a single three-dimensional model, considering each part A it contains, its shape fitting vector is GeoA：
GeoA＝[BoxA,GcA]，
Wherein, BoxABounding Box parameter of A, GcAGeneralized cylinder parameter of A, bounding Box parameter BoxAThe following were used:
BoxA＝[cA,l1,l2,l3,d1,d2,d3]，
wherein c isAAs bounding box center coordinates,/1,l2,l2The lengths of three main shafts respectively from large to small of the bounding box length, d1,d2,d3The orientation vectors of three main axes with the lengths from large to small are obtained; generalized cylinder, meaning a cylinder fitted by a variant having adaptively deformable axes and radiiAnd (3) point sampling is carried out on the surface of the generalized cylinder by using a three-dimensional body with an arbitrary shape, and a coordinate set { P } of all sampling points is used as a parameter of the generalized cylinder. To calculate the generalized cylinder of the part a, the skeleton line of the part is first extracted. The skeleton line refers to a thin curve consistent with the original part shape connectivity and topological structure. After the skeleton lines are extracted, the skeleton lines are uniformly sampled to obtain skeleton line sampling points, and then Ske are sampled at every two adjacent skeleton line sampling points1,Ske2In between, calculate sample points Ske2The normal direction of the normal plane and the intersection points of all the triangular surface patches
Comprises the following steps:
wherein, (Sx)1,Sy1,Sz1),(Sx2,Sy2,Sz2) Are respectively Ske1Ske2Three-dimensional coordinates of (a);
normal to normal from normal plane and sample point Ske2And (3) solving a normal plane equation as follows:
(Sx1-Sx2)*(x-Sx2)+(Sy1-Sy2)*(y-Sy2)+(Sz1-Sz2)*(z-Sy2)＝0，
for a triangular patch fiAnd calculating the linear equations of the three edges respectively:
wherein (x)1,y1,z1)，(x2,y2,z2)，(x3,y3,z3) Respectively a triangular patch fiThree-dimensional coordinates of the three vertex spaces;
solving the joint method plane equation and any one of the linear equations, and assuming that the solution is (X, Y, Z), assuming that the first linear equation is used for solving:
if the equation is not solved, the normal plane is parallel to the straight line, and the normal plane is represented as fiThere is no intersection point on the first edge of (1);
if the equation has a solution, judging:
(X-x1)*(X-x2)<0，
(Y-y1)*(Y-y2)<0，
(Z-z1)*(Z-z2)<0，
if all three equations are satisfied, the coordinates (X, Y, Z) of the intersection of the normal plane and the straight line are fiIs determined to have an intersection, otherwise (X, Y, Z) is at fiIs judged to have no intersection point outside the range of the first edge;
calculating all intersection points (X, Y, Z) within the range determined as a certain side of the triangular patch as generalized cylindrical sampling points of the part A, namely:
GcA＝{Xi,Yi,Zi}；
wherein (X)i,Yi,Zi) Denotes the coordinates of the intersection point, Gc, determined as being within the range of one edge of a certain triangular patchAIs the set of coordinates of all eligible intersection points.
Step 1-4, generating unified expression vectors: and (3) fitting the single three-dimensional model structural representation g in the step 1-2 and the single three-dimensional model in the step 1-3 with the shape of the single three-dimensional model to construct a unified representation vector L of the single model: adding a single three-dimensional model structure representation G into a vector L, and judging each member V of V according to an undirected graph G ═ V, E in the step 1-1iWhether it belongs to g, if so, viShape fitting vector Geo of corresponding partAAdding the vector L, otherwise adding the full 0 vector with the same length into the vector L, and calculating any of the single three-dimensional modelLocal coordinates Con of the connection point of two adjacent parts A and BA,B：
ConA,B＝[ConA,B A,ConA,B B]，
Wherein ConA,BIs the coordinate vector, Con, of the connection point of the A and B parts in the respective local coordinate systemA,B AIs the coordinate of the connection point in the local coordinate system of the component A, ConA,B BCoordinates of the connecting point in a local coordinate system of the part B;
each member E of E is judgedjWhether it belongs to g, if it belongs to ejCorresponding local coordinates Con of connection pointsA,BIf not, adding the full 0 vector with the same length into the uniform expression vector L;
step 1-5, analyzing the symmetry relation of model parts: judging symmetry of any two parts A and B in the model set according to the shape fitting vectors, and calculating the difference degree D (A and B) of the shape fitting vectors:
D(A,B)＝‖GA-GB‖2/max(‖GA‖2,‖GB‖2)，
wherein G isAAnd GBThe shape fitting vector for part A and the shape fitting vector for part B, if D (A, B)<And 20%, judging as symmetrical, and expressing by using a matrix S, wherein if the component type A is symmetrical to the component type B, the element S (A, B) corresponding to A and B in the matrix S is 1, and otherwise, the element S (A, B) is 0.
In steps 1 to 4, Con is calculated by the following formulaA,B AAnd ConA,B B：
Wherein, cA,l1 A,l2 A,l3 A,d1 A,d2 A,d3 AAs parameters of the bounding box of component A, cB,l1 B,l2 B,l3 B,d1 B,d2 B,d3 BIs the bounding box parameter of part B, and C is the coordinate of the connection point of parts A and B in the global coordinate system. Calculated by the method of steps 1-3.
The step 2 comprises the following steps:
step 2-1, constructing a neural network model: constructing a neural network model with the depth of 5, wherein the neural network model comprises an input layer, a first hidden layer, a second hidden layer, a third hidden layer and an output layer, and the adjacent two layers are connected in a full-connection mode; setting the length of the uniform expression vector L obtained in the step 1-4 as s, the number of input layer full-connection neurons as s, the number of hidden layer one full-connection neurons as 1.2 × s, the number of hidden layer two full-connection neurons as 0.5 × s, the number of hidden layer three full-connection neurons as 0.25 × s, the number of output layer full-connection neurons as 2, and the number of offset neurons of each layer in the neural network model as 1;
step 2-2, training a neural network model by adopting a back propagation and random gradient descent method: the parameter to be trained for each hidden layer is WiAnd biThe input is vector x, and the output is vector y ═ x × Wi+bi(ii) a When the reverse propagation is performed, the input is vector y and the output is vector
In the training process, each three layers are trained, and the loss function is defined as the reconstruction error
Wherein
Is a reconstructed vector calculated by inversion from the input vector x. For each hidden layer, respectively using random gradient descent method to minimize
Obtain the parameter WiAnd bi；
And 2-3, encoding the representation vector by using the neural network model trained in the step 2-2, wherein the encoding result is a two-dimensional vector, and if the representation vector is L, the two-dimensional vector is encoded into L ═ W ((L × W) through three hidden layers1+b1)*W2+b2)*W3+b3. Wherein W1,W2,W3Weight coefficients of the first, second and third hidden layers, respectively, b1,b2,b3Respectively, the offsets of the first, second and third hidden layers. And l is the final output two-dimensional code.
The step 3 comprises the following steps:
step 3-1: and (4) multiplying the output two-dimensional coding range in the step (2-3) by a scaling coefficient, wherein the coefficient is generally set to be 1.2 to 1.5, and obtaining a two-dimensional numerical space. Uniformly sampling the two-dimensional numerical space, wherein the sampling data is l';
step 3-2: decoding the sample data by using the neural network model trained in step 3, and the decoding result is X '═ ((l' -b) ═ b3)*W3 T-b2)*W2 T-b1)*W1 T；
Step 3-3: reconstructing a grid model according to the decoding result of the step 3-2: obtaining whether the shape fitting vector Geo of each part is all 0 and whether the coordinates Con of the connection point of every two parts are all 0 according to the decoding result X', obtaining the existence condition and the connection condition of the parts of the model, reconstructing the grids of each part according to the existing shape fitting vector of each part, and combining the grids of each part to obtain a reconstructed model;
step 3-4: judging the effectiveness of the reconstruction model in the step 3-3: the validity criterion is divided into two parts: if any two parts in the reconstructed model have connection conditions in the step 1-1 and are not connected in the reconstructed model, judging that the connectivity is invalid; if any two parts in the reconstruction model have symmetry in the steps 1-5 and are not symmetric in the reconstruction model, the symmetry is judged to be invalid, if any standard is invalid, the sampled data of the reconstruction model does not have validity, otherwise the sampled data of the reconstruction model has validity;
step 3-5: constructing a visual space: to visualize the validity of the sampled data, a visual space is constructed. Firstly, a two-dimensional visual plane is constructed, and all pixel points on the plane are set to be white. Then, judging the two-dimensional data points with effectiveness in the step 3-4, and setting the color of pixel points corresponding to the coordinates on a two-dimensional plane as light color; and the color of a pixel point corresponding to the coordinate on the two-dimensional plane is set to be dark color.
Step 4 comprises the following steps:
step 4-1, when a user clicks a visual space, acquiring two-dimensional data corresponding to a user click coordinate;
step 4-2, inputting the two-dimensional data obtained in the step 4-1 into a self-encoder for decoding to obtain a model expression vector;
and 4-3, representing the vector by the model obtained in the step 4-2, and reconstructing a grid model.
The method analyzes the component composition, the connection relation and the component shape of the model set based on the existing model set, utilizes the neural network to encode the representation after constructing the unified representation of all the models, adopts the encoding space and judges the effectiveness of the adopted data. And on the basis of effectiveness judgment, visualizing effectiveness distribution of the coding space, decoding corresponding two-dimensional data and reconstructing a model according to click interaction of a user on the visualization space, and generating a new three-dimensional model.
Has the advantages that: the invention has the following advantages: firstly, the invention can complete the three-dimensional modeling process only by simple mouse click operation of a user. Secondly, the method can automatically ensure the effectiveness of the modeling result. By the technology of the invention, the visualization of the model set can be realized, and a user is allowed to synthesize a new model through simple clicking operation.
Drawings
The foregoing and other advantages of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.
FIG. 1 is a schematic process flow diagram of the present invention.
FIG. 2 is a schematic diagram of an isomorphic model representation.
FIG. 3a is a rendering of a single model.
Figure 3b is a schematic diagram of shape fitting of a single model.
FIG. 4 is a schematic diagram of the mapping of an isomorphic representation to two dimensions.
Fig. 5a is a schematic view of the effective space after the effectiveness test.
FIG. 5b is a new model generated from user interaction.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
As shown in fig. 1, the invention discloses a method for generating a three-dimensional model based on isomorphic model representation, which specifically comprises the following steps:
the method comprises the following steps: constructing an isomorphic model representation: and carrying out unified isomorphic structure representation on the three-dimensional models with the structure difference and the shape difference by analyzing the structure relation of the models in the model set and shape fitting.
Step two, representing encoding: on the basis of unified model representation, high-dimensional model representation data are converted into low-dimensional codes through a fully-connected neural network.
Step three, constructing a visual space: on the basis of calculating the low-dimensional code represented by the model, judging the effectiveness of the model corresponding to each data point in the coding space, and visualizing the effectiveness distribution condition of the coding space.
Step four, generating a three-dimensional model in an interactive mode: and (4) clicking and interacting the visual coding space in the third step by the user, selecting the data instance in the coding space, and decoding according to the selected data instance to generate the three-dimensional model.
The main flow of each step is specifically described as follows:
wherein, step one includes the following steps: step 11, generating a unified structural representation of the model set: and extracting the unified structural representation of the model set according to the component construction information and the component corresponding information of all the models in the model set. The unified structure of the model set is expressed as an undirected graph G ═ V, E }, V is a set of all nodes, all models in the model set share n-type components, and V ═ V1,v2…vn}(viIndicating a class i component). E is the set of all edges, and if the model set has m kinds of two kinds of different parts adjacent, E is { E ═ E }1,e1…em}(ejIndicating the jth adjacency).
And step 12, generating a single three-dimensional model structure representation: the existence and adjacency relationships of the single model component are extracted and expressed as an undirected graph g ═ v, e. Then, for each undirected graph G extracted in step 11, { V, E }, it is determined whether each member of V and each member of E belong to G. Using a vector l of length n + m as a single three-dimensional model structure representation, the ith member of V belongs to g, then liIs 1, otherwise liIs 0. If the jth member of E belongs to g, then lj+nIs 1, otherwise lj+nIs 0.
Step 13, fitting the shape of the single three-dimensional model: for each individual three-dimensional model, the bounding box and generalized cylinder for each part are computed.
The bounding box for each component uses a bounding box center c, the bounding box three principal axis lengths l1,l2,L3And three main axes of the bounding box d1,d2,d3And (4) showing. The present application uses document 4: tagliacchi A, Alhashim I, Olson M, et al]The method of/Computer Graphics for blackwell Publishing Ltd,2012,31(5):1735-1744 extracts the skeleton line, samples the points on the skeleton line, and calculates the slice at each sampling point. The generalized cylinder is represented using a surface slice point set P of the generalized cylinder. The connection points of two adjacent parts use the coordinate x of the connection point in the respective local coordinate system1,y1,z1And x2,y2,z2And (4) showing.
Step 14, generating a unified representation vector: from the single three-dimensional model structural representation g and the single three-dimensional model shape fit in step 12 and step 13, a unified representation vector L of the single three-dimensional model can be constructed. A single three-dimensional model unified structure representation g is first added to the vector L. From the undirected graph G ═ V, E in step 11, each member V of V is determinediWhether it belongs to g, if so, viAnd (4) adding a vector L into the shape fitting of the corresponding part, otherwise adding a full 0 vector with the same length into the vector L. Each member E of E is judgedjWhether it belongs to g, if it belongs to ejThe corresponding connection point represents adding the vector L, otherwise adding the full 0 vector with the same length into the vector L.
Step 15, analyzing the symmetry relation of the model parts: and judging symmetry of any two types of components in the model set according to the shape fitting vectors, and if the difference degree of the shape fitting vectors is not more than 20%, judging the components to be symmetrical and expressing the components by using a matrix S. If component type a is symmetrical to component type B, S (a, B) is 1, otherwise S (a, B) is 1.
The second step comprises the following steps: and step 21, constructing a neural network model. And constructing a neural network model with the depth of 5. The two adjacent layers of the network are connected in a full connection mode. If the length of the vector L in step 14 is s, the number of input layer fully-connected neurons is s, the number of hidden layer one fully-connected neurons is 1.2 × s, the number of hidden layer two fully-connected neurons is 0.5 × s, the number of hidden layer three fully-connected neurons is 0.25 × s, and the number of output layer fully-connected neurons is 2. The number of offset neurons per layer is 1.
And step 22, training the neural network model by adopting a back propagation and random gradient descent method. The present application uses document 5: hinton G E, Salakhutdinov R.reducing the dimensional of data with neural networks [ J]504, 507. science,2006,313 (5786). The parameter to be trained for each hidden layer is WiAnd biThe input is vector x, and the output is vector y ═ x × Wi+bi. When the reverse propagation is performed, the input is vector y and the output is vector
In the training process, each three layers are trained, and the loss function is defined as the reconstruction error
For each hidden layer, respectively using random gradient descent method to minimize
Obtain the parameter WiAnd bi。
And step 23, encoding the representation vector by using the trained neural network, wherein the encoding result is a two-dimensional vector. Let the expression vector be X, the final code is l ═ W ((X × W)1+b1)*W2+b2)*W3+b3。
The third step comprises the following steps: step 31: the data is sampled. And uniformly sampling the two-dimensional numerical space.
Step 32: the sampled data is decoded. Decoding is performed by using the self-encoder trained in the third step, and if the sampling data is l ', the decoding result is X ' ═ ((l ' -b)3)*W3 T-b2)*W2 T-b1)*W1 T。
Step 33: and (5) model reconstruction. The result is decoded in step 32 and the resulting model representation vector is used to reconstruct the mesh model. And a reconstruction process, wherein the existence condition and the connection condition of the components of the model are obtained according to the representation composition. Then, a mesh for each part is reconstructed from the shape fit of the individual parts. And merging the grids of the components to obtain a reconstructed model.
Step 34: and (5) judging the effectiveness of the reconstruction model. A decision step 33 reconstructs the validity of the model. The present application uses document 6: averkiou M, Kim V G, Zheng Y, et al, Shapesynth, parameter model selection for coordinated shape application and synthesis [ C ]// Computer Graphics Forum.2014,33(2): 125-. The standard is divided into two parts: connectivity criteria and symmetry criteria. If the two types of components in the reconstructed model are connected in step 11 and are not connected in the reconstructed model, the connectivity is determined to be failed. If the two types of components in the reconstructed model have symmetry in step 15 but are not symmetric in the reconstructed model, the symmetry is determined to be invalid. If any standard fails, the reconstructed model has no effectiveness, otherwise the reconstructed model has effectiveness.
Step 35: and visualizing the effectiveness distribution condition. And visualizing the effectiveness distribution of the two-dimensional array space according to the effectiveness of all the sampling data obtained in the step 34. The sampling data with validity is set, the color of the pixel point on the corresponding visual space is set to be light color, the sampling data without validity is set to be dark color, and the color of the pixel on the corresponding visual space is set to be dark color.
The fourth step comprises the following steps: and step 41, acquiring corresponding two-dimensional data according to the click operation of the user on the visual space.
Step 42: and (4) inputting the two-dimensional data acquired in the step (41) into a self-encoder for decoding to obtain a model expression vector.
Step 43: the mesh model is reconstructed from the model representation vectors obtained in step 42.
Examples
In this embodiment, a three-dimensional model of 13 chairs is used as a system input model library, unified isomorphic representation is performed on the 13 chairs, respective representation vectors are calculated, two-dimensional coding is performed on the representation vectors through a three-layer fully-connected neural network, sampling is performed in a two-dimensional coding space, effectiveness is detected, an effective space is rendered after visualization, and a new model is generated according to a click operation of a user.
The specific implementation process is as follows:
in step one, a user imports a model set, and then the unified structure diagram generation in step 11 is executed. The structure diagram of each model is first extracted, as shown in the second column from left to right in fig. 2. Then, the unified structure diagram is extracted, and the extracted unified structure diagram is shown in the third column from left to right in fig. 2. After the unified structure diagram is extracted, the structure of each model is isomorphic, and the isomorphic structure diagram of each model is the fourth column from left to right in fig. 2. The execution step then performs a single three-dimensional model shape fit in step 13, fitting each model using bounding boxes and generalized cylinders. The individual models and shape fitting results are shown in fig. 3a and 3 b.
In the second step, the model representation vectors obtained in the first step are trained to be a fully connected neural network, and the representation vectors of each model are mapped through the neural network to obtain a two-dimensional coding result, and fig. 4 is a schematic diagram of the distribution of the two-dimensional coding vectors of the 13 chair models in the space.
In the third step, respectively obtaining the maximum value and the minimum value of the X coordinate and the Y coordinate according to all two-dimensional coding results { X, Y } in the second step, and setting the maximum value and the minimum value as Xmax,xmin,ymax,ymin. Then set the two-dimensional coding space range to { x, y |1.2 x ×)min-xmax<x<1.2*xmax-xmin，1.2*ymin-ymax<y<1.2*ymax-ymin}. And mapping the two-dimensional code space to a region with the resolution of 500 x 500 on a screen, and solving the corresponding two-dimensional code of each pixel point in the region respectively to serve as uniform sampling data. And decoding all the uniform sampling points through a trained neural network to obtain a model expression vector, reconstructing a model according to the model expression vector, and judging the effectiveness of the reconstructed model. The validity criterion is divided into connection validity and symmetry validity. If the validity standard is met, setting the corresponding pixel points to be dark, otherwise, setting the corresponding pixel points to be light, and visualizing all the pixel points. Fig. 5a is a rendering of the active space, with dark areas being active areas and light areas being inactive areas.
In step four, the user clicks any point in the visualization space, and the white point in fig. 5a is the user click point. And solving the corresponding two-dimensional code according to the click position of the user, and decoding a model expression vector through a neural network. And reconstructing a model according to the model representation vector and displaying the model. The reconstruction model results are shown in fig. 5 b.
The present invention provides a method for generating a three-dimensional model based on isomorphic model representation, and a plurality of methods and approaches for implementing the technical solution, and the above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, a plurality of improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. All the components not specified in the present embodiment can be realized by the prior art.
Claims (2)
1. A three-dimensional model generation method based on isomorphic model representation is characterized by comprising the following steps:
step 1, isomorphic model representation generation: giving a three-dimensional model set of the same category, and performing isomorphic unified representation on all three-dimensional models in the model set to obtain a unified representation vector of a single three-dimensional model by analyzing the structural relationship of the three-dimensional models in the set and performing shape fitting on the single three-dimensional model;
step 2, model representation coding: converting the unified expression vector of each three-dimensional model into low-dimensional codes through a fully connected neural network to form a coding space;
step 3, constructing a visual space: judging the effectiveness of each data point in the coding space corresponding to the three-dimensional model, and visualizing the effectiveness distribution condition of the coding space;
step 4, interactively generating a three-dimensional model: selecting a data example in the coding space, and decoding according to the selected data example to generate a three-dimensional model;
the step 1 comprises the following steps:
step 1-1, generating a unified structural representation of a model set: the unified structure of the model set is represented as an undirected graph G ═ V, E }, where V is a set of all nodes, each node represents each component type, and assuming that all models of the model set have n types of components, V ═ E }, where V is a set of all nodes, V represents a set of all components, and V ═ V1，v2...vnIn which v isiThe component I represents the component I, and the value range of i is 1-n; e is the set of all edges, each edge represents the relationship between every two types of components, and if the model set has m adjacent components, E is { E ═ E1，e1…emIn which ejRepresenting the j adjacent condition, wherein the value range of j is 1-m;
step 1-2, generating a single three-dimensional model structure representation: using a vector l with the length of n + m as a structural representation of a single three-dimensional model, extracting the existence condition and the adjacency relation of a single three-dimensional model component, representing as an undirected graph G ═ V, E }, judging whether each member of V and each member of E belong to G or not according to the undirected graph G ═ V, E }, and enabling the ith component l of the vector l to belong to G or not, and enabling the ith component l of the vector l to belong to GiIs 1, otherwise liIs 0; the jth member of E belongs to g, let the j + n component l of the vector lj+nIs 1, otherwise lj+nIs 0;
step 1-3, fitting the shape of a single three-dimensional model: for a single three-dimensional model, considering each part A it contains, its shape fitting vector is GeoA：
GeoA＝[BoxA，GcA]，
Wherein, BoxABounding Box parameter of A, GcAGeneralized cylinder parameter of A, bounding Box parameter BoxAThe following were used:
BoxA＝[cA，z1，z2，z3，d1，d2，d3]，
wherein c isAAs bounding box center coordinates, z1，z2，z3The lengths of three main shafts respectively from large to small of the bounding box length, d1，d2，d3The orientation vectors of three main axes with the lengths from large to small are obtained; in order to calculate the generalized cylinder of the component A, firstly, skeleton lines of the component are extracted, the skeleton lines refer to thin curves which are consistent with the original component in shape connectivity and topological structure, after the skeleton lines are extracted, the skeleton lines are uniformly sampled to obtain skeleton line sampling points, and then, Ske are obtained at every two adjacent skeleton line sampling points1，Ske2In between, calculate sample points Ske2The normal direction of the normal plane and the intersection points of all the triangular surface patches
Comprises the following steps:
wherein, (Sx)1，Sy1，Sz1)，(Sx2，Sy2，Sz2) Are respectively Ske1Ske2Three-dimensional coordinates of (a);
normal to normal from normal plane and sample point Ske2And (3) solving a normal plane equation as follows:
(Sx1-Sx2)*(x-Sx2)+(Sy1-Sy2)*(y-Sy2)+(Sz1-Sz2)*(z-Sy2)＝0，
for a triangular patch fiAnd calculating the linear equations of the three edges respectively:
wherein (x)1，y1，z1)，(x2，y2，z2)，(x3，y3，z3) Respectively a triangular patch fiThree-dimensional coordinates of the three vertex spaces;
solving the joint method plane equation and any one of the linear equations, and assuming that the solution is (X, Y, Z), assuming that the first linear equation is used for solving:
if the equation is not solved, the normal plane is parallel to the straight line, and the normal plane is represented as fiThere is no intersection point on the first edge of (1);
if the equation has a solution, judging:
(X-x1)*(X-x2)＜0，
(Y-y1)*(Y-y2)＜0，
(Z-z1)*(Z-z2)＜0，
if all three equations are satisfied, the coordinates (X, Y, Z) of the intersection of the normal plane and the straight line are fiIs determined to have an intersection, otherwise (X, Y, Z) is at fiIs judged to have no intersection point outside the range of the first edge;
calculating all intersection points (X, Y, Z) within the range determined as a certain side of the triangular patch as generalized cylindrical sampling points of the part A, namely:
GcA＝{Xi，Yi，Zi}；
wherein (X)i，Yi，Zi) Denotes the coordinates of the intersection point, Gc, determined as being within the range of one edge of one triangular patchAThe intersection point coordinates which meet the conditions are collected;
step 1-4, generating unified expression vectors: and (3) fitting the single three-dimensional model structural representation g in the step 1-2 and the single three-dimensional model in the step 1-3 with the shape of the single three-dimensional model to construct a unified representation vector L of the single model: adding a single three-dimensional model structure representation G into a vector L, and judging each member V of V according to an undirected graph G ═ V, E in the step 1-1iWhether it belongs to g, if so, viShape fitting vector Geo of corresponding partAAdding a vector L, otherwise, adding a full 0 vector with the same length into the vector L, and calculating the local coordinate Con of the connecting point of any two adjacent parts A and B of the single three-dimensional modelA，B：
ConA，B＝[ConA，B A，ConA，B B]，
Wherein ConA，BIs the coordinate vector, Con, of the connection point of the A and B parts in the respective local coordinate systemA，B AIs the coordinate of the connection point in the local coordinate system of the component A, ConA，B BCoordinates of the connecting point in a local coordinate system of the part B;
each member E of E is judgedjWhether or not to belong tog, if belong to ejCorresponding local coordinates Con of connection pointsA，BIf not, adding the full 0 vector with the same length into the uniform expression vector L;
step 1-5, analyzing the symmetry relation of model parts: judging symmetry of any two adjacent parts A and B in the model set according to the shape fitting vector, and calculating the difference degree D (A and B) of the shape fitting vector:
D(A，B)＝||GA-GB||2/max(||GA||2，||GB||2)，
wherein G isAAnd GBRespectively taking the shape fitting vector of the component A and the shape fitting vector of the component B, judging as symmetrical if D (A, B) < 20%, and representing by using a matrix S, wherein if the component type A is symmetrical to the component type B, an element S (A, B) corresponding to A and B in the matrix S is 1, otherwise, the element S (A, B) is 0;
in steps 1 to 4, Con is calculated by the following formulaA，B AAnd ConA，B B：
Wherein, cA，l1 A，l2 A，l3 A，d1 A，d2 A，d3 AAs parameters of the bounding box of component A, cB，l1 B，l2 B，l3 B，d1 B，d2 B，d3 BThe parameters of the bounding box of the component B are shown, and the C is the coordinate of the connection point of the components A and B in the global coordinate system;
the step 2 comprises the following steps:
step 2-1, constructing a neural network model: constructing a neural network model with the depth of 5, wherein the neural network model comprises an input layer, a first hidden layer, a second hidden layer, a third hidden layer and an output layer, and the adjacent two layers are connected in a full-connection mode; setting the length of the uniform expression vector L obtained in the step 1-4 as s, the number of input layer full-connection neurons as s, the number of hidden layer one full-connection neurons as 1.2 × s, the number of hidden layer two full-connection neurons as 0.5 × s, the number of hidden layer three full-connection neurons as 0.25 × s, the number of output layer full-connection neurons as 2, and the number of offset neurons of each layer in the neural network model as 1;
step 2-2, training a neural network model by adopting a back propagation and random gradient descent method: the parameter to be trained for each hidden layer is WiAnd biThe input is vector x, and the output is vector y ═ x × Wi+bi(ii) a When the reverse propagation is performed, the input is vector y and the output is vector
In the training process, each three layers are trained, and the loss function is defined as the reconstruction error
Wherein
Is a reconstructed vector calculated by the inverse direction according to the input vector x; for each hidden layer, respectively using random gradient descent method to minimize
Obtain the parameter WiAnd bi；
And 2-3, encoding the representation vector by using the neural network model trained in the step 2-2, wherein the encoding result is a two-dimensional vector, the representation vector is L, and the representation vector is encoded into L1 ═ L ((L × W) through a three-layer hidden layer1+b1)*W2+b2)*W3+b3Wherein W is1，W2，W3Weight coefficients of the first, second and third hidden layers respectively，b1，b2，b3Respectively the offsets of the first, second and third hidden layers, and l1 is the final output two-dimensional code;
the step 3 comprises the following steps:
step 3-1: taking the output two-dimensional coding range in the step 2-3, multiplying by a scaling coefficient to obtain a two-dimensional value sampling space, and uniformly sampling the two-dimensional value sampling space, wherein the sampling data is l';
step 3-2: decoding the sample data by using the neural network model trained in step 3, and the decoding result is X '═ ((l' -b) ═ b3)*W3 T-b2)*W2 T-b1)*W1 T；
Step 3-3: reconstructing a grid model according to the decoding result of the step 3-2: obtaining whether the shape fitting vector Geo of each part is all 0 and whether the coordinates Con of the connection point of every two parts are all 0 according to the decoding result X', obtaining the existence condition and the connection condition of the parts of the model, reconstructing the grids of each part according to the existing shape fitting vector of each part, and combining the grids of each part to obtain a reconstructed model;
step 3-4: judging the effectiveness of the reconstruction model in the step 3-3: the validity criterion is divided into two parts: if any two types of components in the reconstructed model have connection conditions in the step 1-1 and are not connected in the reconstructed model, judging that the connectivity is invalid; if any two types of components in the reconstruction model have symmetry in the steps 1-5 and are not symmetric in the reconstruction model, the symmetry is judged to be invalid, if any standard is invalid, the sampled data of the reconstruction model does not have validity, otherwise the sampled data of the reconstruction model has validity;
step 3-5: constructing a visual space: constructing a two-dimensional visual plane, setting all pixel points on the plane to be white, judging two-dimensional data points with effectiveness in the step 3-4, and setting the color of the pixel points corresponding to the coordinates on the two-dimensional plane to be red; and the color of a pixel point corresponding to the coordinate on the two-dimensional plane is set to be black.
2. The method of claim 1, wherein step 4 comprises the steps of:
step 4-1, when a user clicks a visual space, acquiring two-dimensional data corresponding to a user click coordinate;
step 4-2, inputting the two-dimensional data obtained in the step 4-1 into a self-encoder for decoding to obtain a model expression vector;
and 4-3, representing the vector by the model obtained in the step 4-2, and reconstructing a grid model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710810698.7A CN107610221B (en) | 2017-09-11 | 2017-09-11 | Three-dimensional model generation method based on isomorphic model representation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710810698.7A CN107610221B (en) | 2017-09-11 | 2017-09-11 | Three-dimensional model generation method based on isomorphic model representation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107610221A CN107610221A (en) | 2018-01-19 |
| CN107610221B true CN107610221B (en) | 2020-06-05 |
Family
ID=61062493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710810698.7A Active CN107610221B (en) | 2017-09-11 | 2017-09-11 | Three-dimensional model generation method based on isomorphic model representation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107610221B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109389671B (en) * | 2018-09-25 | 2020-09-22 | 南京大学 | Single-image three-dimensional reconstruction method based on multi-stage neural network |
| CN110059073B (en) * | 2019-03-18 | 2021-04-06 | 浙江工业大学 | Web data automatic visualization method based on subgraph isomorphism |
| CN110633628B (en) * | 2019-08-02 | 2022-05-06 | 杭州电子科技大学 | RGB image scene three-dimensional model reconstruction method based on artificial neural network |
| CN110889893B (en) * | 2019-10-25 | 2021-10-08 | 中国科学院计算技术研究所 | Three-dimensional model representation method and system for expressing geometric details and complex topology |
| CN114491922A (en) * | 2021-12-09 | 2022-05-13 | 上海望友信息科技有限公司 | Method and system for modeling and parameterizing component, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976359A (en) * | 2010-09-26 | 2011-02-16 | 浙江大学 | Method for automatically positioning characteristic points of three-dimensional face |
| CN103236058A (en) * | 2013-04-25 | 2013-08-07 | 内蒙古科技大学 | Method for obtaining volume of interest of four-dimensional heart image |
-
2017
- 2017-09-11 CN CN201710810698.7A patent/CN107610221B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976359A (en) * | 2010-09-26 | 2011-02-16 | 浙江大学 | Method for automatically positioning characteristic points of three-dimensional face |
| CN103236058A (en) * | 2013-04-25 | 2013-08-07 | 内蒙古科技大学 | Method for obtaining volume of interest of four-dimensional heart image |
Non-Patent Citations (2)
| Title |
|---|
| Perception-based shape retrieval for 3D building models;ManZhang 等;《ISPRS Journal of Photogrammetry and Remote Sensing》;20130131;第76-91页 * |
| 交互式数字几何建模及其关键技术研究;宋沫飞;《中国博士学位论文全文数据库信息科技辑》;20151115;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107610221A (en) | 2018-01-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107610221B (en) | Three-dimensional model generation method based on isomorphic model representation | |
| CN109147048B (en) | Three-dimensional mesh reconstruction method by utilizing single-sheet colorful image | |
| CN111598998B (en) | Three-dimensional virtual model reconstruction method, three-dimensional virtual model reconstruction device, computer equipment and storage medium | |
| Lim et al. | Surface reconstruction techniques: a review | |
| CN110390638B (en) | High-resolution three-dimensional voxel model reconstruction method | |
| Gyulassy et al. | Efficient computation of Morse-Smale complexes for three-dimensional scalar functions | |
| Gao et al. | Mps-nerf: Generalizable 3d human rendering from multiview images | |
| CN108986221A (en) | A kind of three-dimensional face grid texture method lack of standardization approached based on template face | |
| Li et al. | A survey on the local refinable splines | |
| CN115951784B (en) | Method for capturing and generating motion of wearing human body based on double nerve radiation fields | |
| Liu et al. | High-quality textured 3D shape reconstruction with cascaded fully convolutional networks | |
| CN111028335A (en) | Point cloud data block surface patch reconstruction method based on deep learning | |
| CN115659445A (en) | Method for rendering and displaying CAD model on webpage in lightweight mode based on Open Cascade | |
| Huang et al. | Meshode: A robust and scalable framework for mesh deformation | |
| Liu et al. | Deep neural networks with attention mechanism for monocular depth estimation on embedded devices | |
| CN116758219A (en) | Region-aware multi-view stereo matching three-dimensional reconstruction method based on neural network | |
| Hu et al. | Learning structural graph layouts and 3D shapes for long span bridges 3D reconstruction | |
| Shalma et al. | A review on 3D image reconstruction on specific and generic objects | |
| CN112785684B (en) | Three-dimensional model reconstruction method based on local information weighting mechanism | |
| Stoll et al. | A volumetric approach to interactive shape editing | |
| CN115294265A (en) | Method and system for reconstructing three-dimensional human body grid by utilizing two-dimensional human body posture based on attention of graph skeleton | |
| Li et al. | Holistic parameteric reconstruction of building models from point clouds | |
| Lee et al. | Simplification of 3D CAD Model in Voxel Form for Mechanical Parts Using Generative Adversarial Networks | |
| Wang et al. | SparseFormer: Sparse transformer network for point cloud classification | |
| Zhang et al. | Fast Mesh Reconstruction from Single View Based on GCN and Topology Modification. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |