CN114004948A - Creative product solver based on generation network and application thereof - Google Patents
Creative product solver based on generation network and application thereof Download PDFInfo
- Publication number
- CN114004948A CN114004948A CN202110873799.5A CN202110873799A CN114004948A CN 114004948 A CN114004948 A CN 114004948A CN 202110873799 A CN202110873799 A CN 202110873799A CN 114004948 A CN114004948 A CN 114004948A
- Authority
- CN
- China
- Prior art keywords
- training
- dimensional
- product
- model
- hidden
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 claims abstract description 104
- 238000013461 design Methods 0.000 claims abstract description 92
- 239000011664 nicotinic acid Substances 0.000 claims abstract description 81
- 239000013598 vector Substances 0.000 claims abstract description 75
- 238000003860 storage Methods 0.000 claims abstract description 6
- 238000007781 pre-processing Methods 0.000 claims abstract description 3
- 238000005070 sampling Methods 0.000 claims description 48
- 238000000034 method Methods 0.000 claims description 44
- 230000006870 function Effects 0.000 claims description 35
- 238000009826 distribution Methods 0.000 claims description 24
- 238000004422 calculation algorithm Methods 0.000 claims description 15
- 238000005457 optimization Methods 0.000 claims description 14
- 230000003592 biomimetic effect Effects 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 7
- 238000012805 post-processing Methods 0.000 claims description 6
- 230000000750 progressive effect Effects 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000003213 activating effect Effects 0.000 description 5
- 230000000877 morphologic effect Effects 0.000 description 5
- 235000001968 nicotinic acid Nutrition 0.000 description 5
- 238000002360 preparation method Methods 0.000 description 4
- 238000012938 design process Methods 0.000 description 3
- 230000002068 genetic effect Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 208000009119 Giant Axonal Neuropathy Diseases 0.000 description 1
- 238000013256 Gubra-Amylin NASH model Methods 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011833 dog model Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 201000003382 giant axonal neuropathy 1 Diseases 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 101150049349 setA gene Proteins 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003746 surface roughness Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides an creative product solver based on a generation network, which comprises the following steps: the data preprocessing module is used for inputting a three-dimensional shape data set represented by a bionic object and a design target, expanding the data set through data enhancement based on the design target and performance, and obtaining a symbol distance field of a three-dimensional model required by model training through a mesh2voxel and an SDF sampler; a depth generation model which comprises an implicit automatic encoder and an creative product solver; the implicit automatic encoder comprises an encoder 3DCNN and a Decoder IM-Decoder; and the sample vector storage is used for storing the hidden vector point set, and the hidden vectors in the storage are used for obtaining the three-dimensional bionic product through a hidden decoder. The invention also provides an application of the creative product solver based on the generation network in the generation of the three-dimensional bionic product.
Description
Technical Field
The invention belongs to the technical field of machine learning and intelligence, and relates to an creative product solver based on a generation network and application thereof.
Background
Bionics is the science of building a technical system that mimics the principles of a biological system, or that makes an artificial technical system have or resemble the characteristics of a biological system. The main research method of bionics mainly provides a theoretical model, and realizes the realization of bionics principle to function through mathematical simulation. Bionic Design (Bionics Design) is creative Design for simulating the structure, function, form, color and other characteristics of nature and biological systems. The bionic design starts from the psychological needs of people, takes the individual bionic idea of designers as the trusting, creates a design product full of biological interest and natural form, and in turn serves the human, thereby completing the cycle from people to people in a natural state. Creative bionic products are designed to mimic or be inspired by nature and biology. On the basis of observing and refining the characteristics of natural objects, designers can complete the design process of the product by designing methods through creative forms such as association, exaggeration, simplification and the like, combining factors such as design objects, product use scenes and the like to form a design scheme and further carrying out deepened design suitable for the requirements of production and manufacturing technologies. According to different bionic application modes, bionic design is mainly divided into morphological bionic, functional bionic, structural bionic color bionic, texture bionic and the like of products. The shape bionics is widely applied to the modeling design of modern products, such as daily necessities, small household appliances, furniture, vehicles and the like. Designers have turned the direction of product modeling innovation to the field of biomimetic design today.
The outstanding creative design of biomimetic products is related to the designer's own competence, with contingencies and uncertainties, such as the design of the biomimetic product of korani. In order to generate more excellent creative bionic designs, the academic world and the industry have tried to explore the rules of creative design of bionic forms by means of computer technology for many years. Before the depth generation model is applied to the bionic design of product form, the academic world establishes a modeling database and extraction rules through evolutionary design, defines fitness and a product characteristic model by using an evolutionary algorithm, and generates a diversified design scheme on the product outline through extracting the characteristic elements of the bionic object outline. Recent research attempts to automatically synthesize the figure outline of a bionic product by using a deep learning technique. Particularly, Simiao Yu and the like firstly apply the depth generation models such as a generation countermeasure network and the like to the bionic design of the product appearance, and the depth generation models mathematically define the bionic design problem from the aspect of depth learning, so that a set of depth generation models is designed to form the bionic design of the product appearance outline.
A combined depth-generated model of a self-encoder and generation countermeasure network (GAN) has shown unique advantages and great potential in the fields of planar image generation, restoration, and style migration. In terms of three-dimensional design, wu et al, 2016 university of massachusetts proposed a specific method for applying a generation countermeasure network to three-dimensional shape generation, and a trained model can restore and reconstruct a picture of a chair into a three-dimensional model of the chair. The concept of a hidden vector space in a three-dimensional shape representation was first proposed, based on a voxelized 3D data model. Panos et al applied a self-encoder and an intermediate hidden generation countermeasure network (LatentGAN) to accomplish the embedding and representation of three-dimensional point cloud data. A self-encoder is used for directly training a hidden vector space based on a three-dimensional data set, then a series of three-dimensional generation tasks are completed by utilizing bijection between an Euclidean space and a three-dimensional shape, and the diversity of a generated result is ensured by the existence of the intermediate LatentGAN. chen et al implicitly optimizes the effect of 3D model fitting of the depth-generating model LatentGAN so that the effect of the interpolation-based generated three-dimensional model is more continuously smooth [11], and reuses the three-dimensional data of voxels and converts the voxel data to more widely applied three-dimensional mesh data through marching cube algorithm. However, the depth generation model has no relevant application in the bionic product form design process.
Disclosure of Invention
The invention provides an creative product solver based on a generating network, which comprises the following steps:
the data preprocessing module is used for inputting a three-dimensional shape data set represented by a bionic object and a design target, expanding the data set through data enhancement based on the design target and performance, and obtaining a symbol distance field of a three-dimensional model required by model training through a mesh2voxel and an SDF sampler;
a depth generation model comprising an implicit autoencoder, a potential generation countermeasure network/interpolator/creative product solver; the implicit automatic encoder comprises an encoder 3DCNN and a Decoder IM-Decoder;
the sample vector storage library is used for storing a hidden vector point set, and hidden vectors in the storage library are used for obtaining a three-dimensional bionic product through a hidden decoder;
and the post-processing module is used for carrying out post-processing on the three-dimensional bionic product to obtain a three-dimensional bionic design.
In the creative product solver based on the generated network, the implicit automatic encoder is used for fitting a symbolic distance field of a three-dimensional model and performing training in a first stage; in order to obtain a high-fidelity product meeting design requirements, a proper hidden vector point set needs to be searched in a hidden space; the hidden vector point set is obtained by Gaussian sampling after a training potential generation countermeasure network, or is obtained by the hidden vector pair interpolation obtained by an encoder through a three-dimensional model of a given training set bionic object and a design target, or is obtained by an creative product solver; and after the coding and decoding network training of the first stage is finished, the model to be trained is trained in the second stage to obtain the callable hidden vector generation model.
3DCNN encoder
Implicit decoder
| Layer(s) | Jumping link sources | Inputting shape | Activating a function | Output shape | Label (R) |
| Inputting hidden vector and coordinate point of query | (128+3) | (131) | a | ||
| Full connection | (131) | LReLU | |||
| Full connection | a | (2048+131) | LReLU | ||
| Full connection | a | (1024+131) | LReLU | ||
| Full connection | a | (512+131) | LReLU | ||
| Full connection | a | (256+131) | LReLU | ||
| Full connection | (128) | Sigmoid | (1) |
Potential generation of countermeasure networks
Generator
| Layer(s) | Activating a function | Output shape |
| Input hidden vector | (128) | |
| Full connection | LReLU | (2048) |
| Full connection | LReLU | (2048) |
| Full connection | Sigmoid | (128) |
Distinguishing device
| Layer(s) | Activating a function | Output shape |
| Input hidden vector | (128) | |
| Full connection | LReLU | (2048) |
| Full connection | LReLU | (2048) |
| Full connection | (1) |
Based on the system, the invention also provides an application of the creative product solver based on the generation network in the generation of the three-dimensional bionic product, which comprises the following steps:
step 1: establishing a training set;
step 2: establishing a depth generation model, and training the depth generation model through the training set; the depth generation model comprises an implicit automatic encoder and an implicit vector generation model; the implicit automatic encoder comprises an encoder 3DCNN and a Decoder IM-Decoder; the hidden vector generation model adopts an creative product solver;
and step 3: generating a three-dimensional biomimetic resultant using the depth generative model.
And 4, step 4: and carrying out post-processing on the three-dimensional bionic product in cooperation with a designer to obtain the three-dimensional bionic design.
The step 1 of the invention comprises the following steps: the method for constructing the identification conversion path from the 3D file format to the voxel model format specifically comprises the following steps:
step 1.1: the obj file of the three-dimensional grid is voxelized by the binvox and converted into a binvox file;
step 1.2: filling the voxelized file with flooding to obtain a point value pair, which is equivalent to discrete sampling of an isosurface;
step 1.3: learning, with a coder in an implicit autocoder, a mapping of a three-dimensional voxel to a 128-dimensional hidden vector, and learning, with a decoder, a mapping of the 128-dimensional hidden vector plus a three-dimensional voxel position to a symbolic distance field of the voxel; obtaining three-dimensional grid data by the symbol distance field through a marching cube algorithm; the symbolic distance field is thresholded to obtain voxel data.
The method further comprises the following step 1: and a training set enhancement step, which comprises enhancing based on design target data and enhancing based on performance data, and increasing the total amount of the training set from 113+6000 to more than 20 k.
The enhancing based on design objective data comprises: design-oriented data enhancement focuses on transferring design knowledge into the preparation of data sets to meet design requirements. For example, directional factors are also factored into the designer's design process. After evaluating existing simulcasts and brainstorms, generative designs that take into account multiple directions enable the present invention to explore more possibilities. Considering the symmetry of the target product and the balance between diversity and efficiency, the invention selects to apply the bionic object data set with seven-direction rotation transformation to realize diversified mixed results.
The performance data based enhancement includes: performance oriented data enhancement aims at improving the performance of deep generative models. The generator is subject to unbalanced training data, especially with only hundreds of animal data sets. Non-rigid deformations, such as tensile transformations, are an effective way to alleviate this problem. The present invention stretches the training data set in three orthogonal axes to an extent of from 0.25 to 2.
The method further comprises the following step 1: the resolution of the generated voxel file is improved from 16^3 to 32^3 to 64^ 3.
It includes: the model generates a symbolic distance field (sdf), which allows the generation of voxels of arbitrary resolution by controlling the sdf sampling density. But the degree of detail of the network fit depends on the voxel file resolution size used.
The progressive training is adopted during training, namely 100 rounds of training are performed on the voxel file with the resolution of 16^3, then 200 rounds of training are performed on the voxel file with the resolution of 32^3, and finally 1000 rounds of training are performed on the voxel file with the resolution of 64^ 3. Training may be terminated during the training process to control the generation of the three-dimensional model of the multi-scale detail.
In the invention, the depth generation model comprises an implicit automatic encoder and an implicit vector generation model; the implicit automatic encoder comprises an encoder 3DCNN and a Decoder IM-Decoder; the hidden vector generation model adopts an creative product solver.
Compared with a traditional Decoder based on a convolutional neural network, the IM-Decoder can enable a hidden vector space and generated three-dimensional voxels to be more continuous, and therefore a more continuous interpolation effect is obtained.
The loss function of the IM-Decoder model depends on the sampling mode, where wpThe weighting of the weighted sampling mode depends on the sparseness of the sampling points. The loss function is the weighted average squared error between the true value label and the predicted label for each point. Let S be the set of points for the target shape, the loss function L (θ) is:
where θ is the network weight, fθIn order to be mapped to the network,
the true value of the actual symbol distance field sampled at p points.
In the invention, the creative product solver comprises: a heuristic target function, wherein the heuristic target function comprises a heuristic sampling algorithm and a hidden space optimization algorithm; the heuristic sampling algorithm samples the hidden space; and training the generation model by a weighted retraining method, reducing inference, completing black box optimization, and obtaining the hidden vector generator.
The invention establishes an creative product solver, which comprises the following steps:
step 2.1: training an auto-encoder to learn a low-dimensional representation;
step 2.2: deploying an creative product solver in a hidden vector space between an encoder and a decoder of an automatic encoder;
step 2.3: and after the training set is prepared, respectively carrying out two-stage training on the automatic encoder and the creative generation solver.
The two-stage training refers to: in the first stage, a progressive training self-encoder is adopted, so that the generated model has more details; and in the second stage, training an creative product solver, firstly sampling the hidden space by a heuristic sampling method, then calculating by a heuristic target function to obtain the scores of sampling points, and using a high-score sampling point set for training non-parametric estimation kernel density estimation. While the set of high-scoring sampling points serves as the set for subsequent evaluation.
Considering that the calculation cost of the method in the previous step is large, high-resolution sampling points are used as a part of a training set, and a generation model of Sample-Efficient Optimization in the content Space of Deep genetic Models viaWeighted recovery is trained. The hidden vector generator can automatically input 32-dimensional training output which obeys standard multivariate Gaussian distribution and output sample points with higher scores in a heuristic target function, and the sample points can generate creative products which are considered to be high in quality after passing through an implicit Decoder IM-Decoder.
The hybrid heuristic objective function f (x) consists of a connected branch count function and a binary classifier for distinguishing design objectives from biomimetic objects. The connected branch count function makes the objective function f (x) black box, i.e., unknown to the general closed form or differential information.
Where δ is a threshold for discretizing the continuous symbol distance field, NrIs the resolution of the voxel. f. ofθ(z, p) is the network predictor at point p in the z-decoded symbol distance field, representing the integer ensemble.
Given embedded representations of a bionic object and a design target, respectively
And
and N is the number of bionic objects and design target training set samples. Zd and zb are only named symbols, k is the number from 1 to M or N, and ijk is used in this way.
Training each perceptual classifier fc(z)＝argmaxiP(Y＝i|z)。
The loss function is:
wherein:
is a hidden vector, Y represents a random variable of two classes, ideally fc(z) is e {0,1}, and the actual differentiable network values are consecutive fc(z)∈[0,1]. It is an object of the present invention to encourage production as a trade-off between biomimetic objects and design target classes. When the classifier uncertainty reaches a maximum, the percent () should reach an optimal value, and thus is defined as follows:
Percept(z)＝H(Z)＝-fc(z)logfc(z)-(1-fc(z))log(1-fc(z))
the hybrid heuristic objective function is finally expressed as:
S(z)＝Percept(z)+α1Ncg(z)
wherein alpha is1Are coefficients.
The hidden space optimization algorithm (LSO) maps a set of hidden vector points to a high value region of the objective function S (z). Wherein the number of times that the objective function f is evaluated is as small as possible, and finally an evaluation point list is obtained
Currently employed (Austin Tripp, Erik Daxberger, and Jos Miguel Hern ndez-Lobato.2020. Sampl)An algorithm of the efficient optimization in the relative space of deep genetic models, advanced in Neural Information Processing Systems 33(2020). The method finally obtains an implicit vector generator, and outputs the implicit vector in a high-value area of an objective function S (z) through simple prior distribution sampling.
In order to evaluate the effect of the hidden space optimization algorithm, a truth point set is formed by adopting a heuristic sampling method and kernel density estimation on the basis of Bayesian optimization.
The heuristic sampling method is a prime interval sampler. Firstly, the range of a high-quality sample hidden vector point set is reduced to a convex hull of a hidden vector of a training data set. Selecting the middle points from the interpolation is an acceptable creative exploration method, and is suitable for the instant evaluation of the convex hull S. If the present invention is applied to each pair
And
z in1And z2Carrying out interpolation sampling N times, and selecting m points in the middle of each pair to obtain a sample set
As follows:
wherein N represents the number of interpolation samples; λ takes values from 1 to m.
The number of samples obtained in this way is very large, Ninterpolation＝m×|Db||DdL. To reduce the amount of sampling, the present invention has discovered that sampling at intervals of a prime number can preserve a sample set
Algebraic structure of (c). Sample set
Is a cyclic group of order m, obtained by sampling
The subset of (a) is still a cyclic group of order m, and methods like uniform sampling yield subsets that converge probabilistically on this structure.
The resulting sample set was estimated by kernel density with a bandwidth of 0.1. Then sampling can be carried out through nuclear density estimation sampling to obtain creative products.
In the invention, the hidden vector generation model can also adopt a potential countermeasure generation network.
In the framework of implicit generation countermeasure networks (late-GANs), a generator and a discriminator both comprise three fully connected layers, wherein the first two layers are activation functions, the last layer is a sigmoid function. The network is used for fitting a hidden vector point set obtained by a training set through confrontation training through an encoder, and a generator can output hidden vectors which obey the distribution of the training point set in a hidden space of the training set through inputting standard normal distribution.
The potential generation countermeasure network includes a generator and a discriminator, and is used for fitting the distribution of the embedding points of the training set in the hidden space.
The invention establishes a potential countermeasure generation network, which comprises the following steps:
step 2.1: training an auto-encoder to learn a low-dimensional representation;
step 2.2: training a generative countermeasure network comprising a generator and a discriminator in a hidden vector space between an encoder and a decoder of an auto-encoder;
step 2.3: after the training set preparation is completed, the automatic encoder and the countermeasure generating network are trained in two stages respectively.
The two-stage training refers to: in the first stage, a progressive training automatic encoder is adopted, so that the generated model has more details; in the second stage, the generator and the discriminator are used for training the confrontation generation network in the hidden vector space, so that the result generated by the generator is more realistic and diversified.
Obtaining an embedding mapping AE after training of an automatic encoder and a countermeasure generation networkS→ZIncluding an encoder of an auto-encoder, and generating a mapping GZ→SIt includes a generator that generates the countermeasure network and a decoder of the self-encoder.
In the invention, the hidden vector generation model can also adopt an interpolation operator. The structure of the interpolation arithmetic unit is as follows: given a bionic object hidden vector x1 and a design target hidden vector x2, the interpolated vector is denoted as x ═ γ x1+(1-γ)x2Wherein gamma is 0,1]And uniformly sampling gamma to obtain a corresponding interpolation hidden vector, and finally obtaining a corresponding product through a decoder. And (4) obtaining an implicit field obtained by mapping the interpolated implicit vector through a generator as an creative product, and obtaining a triangular mesh file through a marching cube algorithm.
The method for establishing the depth generation model comprises the following steps:
step 2.1: training an auto-encoder to learn a low-dimensional representation;
step 2.2: training an interpolator in a hidden vector space between an encoder and a decoder of an autoencoder;
step 2.3: and after the training set preparation is completed, training the automatic encoder. And the automatic encoder is trained by adopting progressive training, so that the generated model has more details.
The interpolation arithmetic unit can directly obtain a closed solution through the data of the training set without training.
According to the three-dimensional bionic product generation method based on the depth generation model, the depth generation model can adopt one of a potential generation countermeasure network, an creative product solver and an interpolation operator independently or can adopt a plurality of combinations simultaneously.
The invention also provides an application of the method in three-dimensional representation, which specifically comprises the following steps:
and 3, utilizing an implicit coder decoder to learn mapping, remapping elements in the low-dimensional space back to the three-dimensional shape, and obtaining the three-dimensional shape of the object.
In the invention, the design in the three-dimensional bionic design refers to a design obtained by optimizing a product. Both "product" and "composition" correspond to the english term synthetic, and refer specifically to the product produced by the generator.
And a generator. The generator is a neural network model used to generate the task. In the model training phase, a hidden representation space Z needs to be learned for a space X where multimedia contents are located in a given training set, and the Z space is a low-dimensional Euclidean space. In the model reasoning phase, the invention has x ═ fθ(z) given a hidden spatial sample point z, the generator can generate data for a corresponding modality.
The implicit autoencoder comprises a 3DCNN encoder and an implicit decoder for fitting a symbolic distance field of the three-dimensional model, reconstructing the symbolic distance field, and representing the reconstructed symbolic distance field as a three-dimensional shape data set after the training of the first stage is completed. The loss function is the reconstruction error of the implicit field sampling point. The specific network structure is shown in the table.
The middle hidden layer of the automatic encoder is a low-dimensional European space, and the semantics of the space are extracted by a deep learning method. The specific method comprises the following steps: assume that the low dimensional space is RnDefined as a hidden vector space Z, given the samples of the design target domain
With samples of bionic object domains
Assuming that the distribution of the design target domain D is p (D), biomimeticThe distribution of object domain B is p (B). First step training encoder AES→ZTo learn from the samples the subset Z that embeds the distributions p (d) and p (b) into the low-dimensional hidden vector space ZBAnd ZDIn (1), assume their distribution as p (z)b) And p (z)d). Second, distribute p (z) according to a specific ruleb) And p (z)d) Is searched to obtain the potential representation space Z of the creative generation domainIAssuming that its distribution is p (z)i) At p (z)i) Obtaining a series of samples by intermediate sampling
Thirdly, using the generator G of the trained generative modelZ→SMixing the sample
Mapping into a three-dimensional shape domain, thereby obtaining a series of three-dimensional shapes with bionic creative inspiring values.
The invention adopts an interpolation method in a hidden space to synthesize a novel bionic design form. The related work of representational learning has shown that three-dimensional shapes can be transformed by operations on hidden vectors. The model obtained by the training of the invention can also generate two hidden vectors z by two different objects through an encoder1And z2Then, the interpolated implicit vector z' is equal to gamma z1+(1-γ)z2(0<γ<1) A generator is input to produce a three-dimensional shape. These three-dimensional shapes exhibit smooth transitions between objects as γ increases.
The potential generation countermeasure network is used to map the normal distribution samples of N (0,1) to new sample points fitting the hidden vector distribution of the training set.
Gaussian sampling is used to sample points directly in hidden space N (0, 1). The test results are not as good as the potential generation of a countermeasure network.
And selecting a proper bionic object and a design target from the voxel model in the input data set. By encoder AES→ZObtaining hidden vectors, and interpolating the hidden vectors to obtain a large amount of intermediate productsAnd then the method is delivered to a designer for intervention and selection, and different angle combinations are tried in the subsequent generation process to obtain a richer creative heuristic model.
Gaussian distribution in the hidden space of the latex-GAN model is also a method for generating a novel bionic design form. After sampling is completed, a designer intervenes to select a bionic object and a design target with ideal model learning effect, and preparation is made for subsequently calling the model and generating a bionic product. In order to make the results generated and more acceptable to designers, the present invention employs an expanded data set and orientation adjustments to produce a more morphologically rich and varied three-dimensional shape and morphological details.
Drawings
FIG. 1 is a schematic representation of the process of the present invention.
FIG. 2 is a schematic diagram of a three-dimensional bionic product generation method based on a depth generation model.
FIG. 3 is a flow chart of the two-stage training method of the present invention.
FIG. 4 is a schematic diagram of the data representation method and the conversion method between the four three-dimensional shapes according to the present invention.
FIG. 5 is a general framework diagram of the model invocation of the present invention.
FIG. 6 is a hypothetical graph of a creative generation problem targeting a biomimetic design in an embodiment of the present invention.
FIG. 7 is a data format diagram of the results generated by an embodiment of the present invention.
FIG. 8 is a schematic diagram of the three-dimensional shape, design initiation point and design development direction of a typical product according to an embodiment of the present invention.
Fig. 9 is a diagram of a design implemented in part by the elicitation of products in accordance with an embodiment of the present invention.
FIG. 10 is a comparison of the configuration of some of the products of the present invention with the configuration of a superior bionic children's home chair according to the present invention.
FIG. 11 is a comparison of a design scenario inspired by some products of the example of the present invention and an international excellent bionic children's home chair design.
Fig. 12 is a schematic diagram of the scoring result generated by a part of the creative product solver according to the embodiment of the present invention.
Fig. 13 is a schematic structural diagram of the creative product solver of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following specific examples and the accompanying drawings. The procedures, conditions, experimental methods and the like for carrying out the present invention are general knowledge and common general knowledge in the art except for the contents specifically mentioned below, and the present invention is not particularly limited.
Example 1
As shown in fig. 12, from left to right, the score of the result generated by the creative product solver is from top to bottom, and it can be seen that the senior product can often fuse the morphological features of the bionic object and the design target. While the products in the intermediate stage can only fit the distribution of the training data and do not necessarily result in a fused structure between the products. Products at low staging levels tend to exhibit substantial fragmentation, affecting product functionality.
In this embodiment, a creative product solver is used to obtain a three-dimensional bionic design, and the specific process is as follows:
firstly, sampling the hidden space by a heuristic sampling method, then calculating the fraction of sampling points by a heuristic target function, and using a high-fraction sampling point set for training non-parametric estimation kernel density estimation. While the set of high-scoring sampling points serves as the set for subsequent evaluation.
Considering that the calculation cost of the method in the previous step is large, high-resolution sampling points are used as a part of a training set, and a generation model of Sample-Efficient Optimization in the content Space of Deep genetic Models viaWeighted recovery is trained. The hidden vector generator can automatically input 32-dimensional training output which obeys standard multivariate Gaussian distribution and output sample points with higher scores in a heuristic target function, and the sample points can generate creative products which are considered to be high in quality after passing through an implicit Decoder IM-Decoder.
Example 2
The embodiment of the invention selects a 3-6 year old child as a design object, a bionic home chair as a design target, a dog which is familiar in people's life as a bionic object, and an urban family indoor as an application scene. The designers participating in the design include 1 earner of the design prize of the red dot product, a product designer who had been at the discretion of the infant education institution, a product designer who had a 5-year-old infant at home, and a team of computer engineers, a researcher and a teacher at the computer school of university of east china.
A bionic design resource library is constructed based on a work designer and a computer engineer together, an open source model library of a chair is adopted in training, and a simple and non-decorative backrest chair is used for preparing for training of generating a model; and selecting the dog models with obvious characteristics and consistent postures to construct a bionic modeling library.
The project is trained for multiple times according to a set three-dimensional deep learning model calculated through interpolation, each product consists of a three-view and a model file, and after 9 times of training and calling, more than 1.5 ten thousand products are generated for calling, evaluating and screening. Designers and computer engineering make continuous corrections and advances from the following 5 aspects, so that subsequent products have higher bionic design form value.
The first stage is a three-dimensional deep learning product calling and optimizing measure stage, as shown in the following table:
TABLE 1 three-dimensional deep learning product invocation and optimization measures
The deep learning product comprises a plane contour and a three-dimensional shape file, the bionic shape value evaluation of the product is divided into two aspects of bionic shape creative value evaluation and model generation quality evaluation, a designer selects the product with higher bionic design value more quickly according to the standards of whether the product embodies the important characteristics of an original bionic object, the correspondence between the shape structure of the product and the structure of the bionic object, the product structure and the like; and (3) evaluating the product by an algorithm engineer from the aspect of the finish effect of deep learning, finding out abnormal phenomena and reasons such as distortion, fracture, default and the like in the product, and observing and evaluating the influence of interpolation, parameter adjustment and data set on the generated result each time. After the adjustment and screening of multiple generation processes, about 5 types of products with high bionic form value are obtained, and the products can be further optimized into a design scheme. Both the three-dimensional morphology and the planar profile of the resultant product will be a great inspirational designer.
And the second stage is to further optimize the 3D product with high bionic morphological value into a design scheme. The designer's work at this stage is divided into two parts: 1, selecting a product with higher bionic design form value, and performing form optimization treatment to form a design scheme; 2, inspiring with single or multiple products, directly creating secondary creativity by designers to form a new design scheme. The sensitivity of the designer to the bionic form, the form processing skill and the experience of the product design play a key role in the process, and the computer engineer needs to fully understand the selection standard of the designer and assist the designer in completing the screening work and converting the product format and the model.
And the designer combines the product structure, the functional design and the like to adjust and optimize the product model to form a plurality of design schemes of the child home chairs.
In order to evaluate the value of the depth generation model in the aspect of three-dimensional bionic form creative generation, the invention compares the form of a product screened by a designer in the work with the form of a known bionic product design, as shown in fig. 10.
The comparison finds that the problems of surface roughness, insufficient integrity of local forms and the like of the product are eliminated, the form of the product is very close to that of most of the existing bionic product designs, and the product conforms to the bionic modeling rules of abstract bionic, intention bionic, image bionic and the like; the resultant is also relatively close to the bionic object in aspects of attitude dynamics, animal typical characteristics and the like. The product has rich forms and basically covers the change types of the existing creative forms of the bionic design.
In the embodiment of the invention, three product designers are combined to form 10 design schemes in a short time after observing the characteristics of the product, and a plurality of design creative sketches are also provided, as shown in fig. 11. From the distribution of the scheme, a plurality of products inspire designers and can form a design scheme after form optimization; from the creative inspiring point of the scheme, a designer can obtain creative inspiring from a single product, and can also fuse creative points in more than 2 products to form a new creative form. This process demonstrates that the product is an effective creative inspiration for designers and assists them in creating design solutions, in contrast to international designers' designs, which are superior in morphological processing skills. The designer also obtains elicitations from the contours of the product, forming a design.
Example 3
The structure of the fusion between the products is not necessarily obtained by the potential self-encoder being able to fit only the distribution of the training data. There are sometimes some levels of design abstraction that exist simply by reconstructing the structure. In this embodiment, a three-dimensional bionic design is obtained by using a potentially generated countermeasure network, and the specific process is as follows:
a potential generation countermeasure network for fitting a distribution of embedding points of the training set in the hidden space includes a generator and a discriminator.
In the training phase, the generator inputs 128-dimensional Gaussian noise that follows a normal distribution of N (0,1), and outputs a generated 128-dimensional vector. And inputting the generated vector and an embedded point set of the training set in a hidden space by the discriminator, and classifying whether the vector class belongs to the training set or is generated by the generator. The two parties carry out antagonistic training, and the generator can input Gaussian noise and output hidden vectors close to the distribution of the embedded points of the training set in the hidden space after the final training is finished.
In the calling phase, a 128-dimensional vector sampled from a normal distribution of N (0,1) is mapped to a new sample point fitting the hidden vector distribution of the training set.
Generator
| Layer(s) | Activating a function | Output shape |
| Input hidden vector | (128) | |
| Full connection | LReLU | (2048) |
| Full connection | LReLU | (2048) |
| Full connection | Sigmoid | (128) |
Distinguishing device
| Layer(s) | Activating a function | Output shape |
| Input hidden vector | (128) | |
| Full connection | LReLU | (2048) |
| Full connection | LReLU | (2048) |
| Full connection | (1) |
As shown in fig. 1, the structure of fusion between products is not necessarily obtained by the fact that the potential self-encoder can only fit the distribution of the training data. But sometimes there is some degree of design abstraction simply to reconstruct the structure.
Example 4
According to the three-dimensional bionic product generation method based on the depth generation model, the depth generation model can adopt one of a potential generation countermeasure network, an creative product solver and an interpolation operator independently or can adopt a plurality of combinations simultaneously. When multiple models are used simultaneously, the accuracy of the three-dimensional biomimetic design is improved.
The protection of the present invention is not limited to the above embodiments. Variations and advantages that may occur to those skilled in the art may be incorporated into the invention without departing from the spirit and scope of the inventive concept, which is set forth in the following claims.
Claims (10)
1. An creative product solver based on a generating network, comprising:
the data preprocessing module is used for inputting a three-dimensional shape data set represented by a bionic object and a design target, expanding the data set through data enhancement based on the design target and performance, and obtaining a symbol distance field of a three-dimensional model required by model training through a mesh2voxel and an SDF sampler;
a depth generation model which comprises an implicit automatic encoder and an creative product solver; the implicit automatic encoder comprises an encoder 3DCNN and a Decoder IM-Decoder;
and the sample vector storage is used for storing the hidden vector point set, and the hidden vectors in the storage are used for obtaining the three-dimensional bionic product through a hidden decoder.
2. The creative product solver based on a generating network of claim 1, further comprising: and the post-processing module is used for carrying out post-processing on the three-dimensional bionic product to obtain a three-dimensional bionic design.
3. The creative product solver of claim 1, wherein the implicit autocoder is configured to fit a symbolic distance field of a three-dimensional model for a first stage of training; in order to obtain a high-fidelity product meeting design requirements, a proper hidden vector point set needs to be searched in a hidden space; the hidden vector point set is obtained through an creative product solver; and after the coding and decoding network training of the first stage of the model to be trained is finished, the second stage of training is carried out to obtain the callable hidden vector generation model.
4. Use of a generative network-based creative product solver for three-dimensional biomimetic generation using any of claims 1-3, comprising the steps of:
step 1: establishing a training set;
step 2: establishing a depth generation model, and training the depth generation model through the training set; the depth generation model comprises an implicit automatic encoder and an implicit vector generation model; the implicit automatic encoder comprises an encoder 3DCNN and a Decoder IM-Decoder; the hidden vector generation model adopts an creative product solver;
and step 3: generating a three-dimensional biomimetic resultant using the depth generative model.
5. The use of claim 4, further comprising the step of 4: and carrying out post-processing on the three-dimensional bionic product in cooperation with a designer to obtain the three-dimensional bionic design.
6. The use of claim 4, wherein step 1 comprises: the method for constructing the identification conversion path from the 3D file format to the voxel model format specifically comprises the following steps:
step 1.1: the obj file of the three-dimensional grid is voxelized by the binvox and converted into a binvox file;
step 1.2: filling the voxelized file with flooding to obtain a point value pair, which is equivalent to discrete sampling of an isosurface;
step 1.3: learning mapping from a three-dimensional voxel to a 128-dimensional hidden vector by using a 3DCNN encoder in an implicit automatic encoder, and learning a mapping symbol distance field from the 128-dimensional hidden vector, a three-dimensional voxel position and a symbol distance field of the voxel by using a decoder to obtain three-dimensional grid data by a marching cube algorithm; obtaining voxel data by thresholding the symbolic distance field;
and/or the presence of a gas in the gas,
the step 1 further comprises the following steps: a training set enhancement step including design objective data based enhancement and performance data based enhancement;
the enhancing based on design objective data comprises: applying rotation transformation in seven directions to the bionic object data set to realize diversified mixed results;
the performance data based enhancement includes: stretching the training data set in three orthogonal axes to an extent of from 0.25 to 2;
and/or the presence of a gas in the gas,
the step 1 further comprises the following steps: increasing a resolution of a generated voxel file, which includes model generating a symbolic distance field, allowing generation of voxels of arbitrary resolution by density control of sampling of the symbolic distance field;
during training, progressive training is adopted, firstly 100 rounds of 16^3 resolution voxel file training are carried out, then 200 rounds of 32^3 resolution voxel file training are carried out, and finally 1000 rounds of 64^3 resolution voxel file training are carried out;
generating a three-dimensional model of multi-scale details is controlled by terminating training during the training process.
7. The application of claim 4, wherein the creative product solver comprises: a heuristic target function, wherein the heuristic target function comprises a heuristic sampling algorithm and a hidden space optimization algorithm; the heuristic sampling algorithm samples the hidden space; and training the generation model by a weighted retraining method, reducing inference, completing black box optimization, and obtaining the hidden vector generator.
8. The application of claim 4, wherein building a depth generative model comprises the steps of:
step 2.1: training an auto-encoder to learn a low-dimensional representation;
step 2.2: deploying an creative product solver in a hidden vector space between an encoder and a decoder of an automatic encoder;
step 2.3: and after the training set is prepared, respectively carrying out two-stage training on the automatic encoder and the creative generation solver.
9. The use of claim 8, wherein the two-stage training is: in the first stage, a progressive training self-encoder is adopted, so that the generated model has more details; in the second stage, training an creative product solver, firstly, sampling a hidden space by a heuristic sampling method, then, calculating by a heuristic target function to obtain scores of sampling points, using a high-score sampling point set for training non-parametric estimation kernel density estimation, and simultaneously using the high-score sampling point set as a set for subsequent evaluation; and/or the presence of a gas in the gas,
taking the high-resolution sampling points as a part of a training set, training a generation model by a weighted retraining method, reducing inference, completing black box optimization, and obtaining a hidden vector generator; the hidden vector generator automatically inputs 32-dimensional training output which obeys standard multivariate Gaussian distribution and has higher scores in a heuristic target function, and the sample points generate a creative product which is considered to be high in quality after passing through an implicit Decoder IM-Decoder.
10. The application of claim 4, wherein the loss function of the IM-Decoder model depends on a sampling pattern; wherein the loss function is a weighted average squared error between the true value label and the predicted label for each point; let S be the set of points for the target shape, the loss function L (θ) is:
where θ is the network weight, fθIn order to be mapped to the network,
a true value sampled at p points for the actual symbol distance field; w is apThe weighting of the weighted sampling mode depends on the sparseness of the sampling points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110873799.5A CN114004948B (en) | 2021-07-30 | 2021-07-30 | Creative product solver based on generation network and application thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110873799.5A CN114004948B (en) | 2021-07-30 | 2021-07-30 | Creative product solver based on generation network and application thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114004948A true CN114004948A (en) | 2022-02-01 |
| CN114004948B CN114004948B (en) | 2022-05-06 |
Family
ID=79921056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110873799.5A Active CN114004948B (en) | 2021-07-30 | 2021-07-30 | Creative product solver based on generation network and application thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114004948B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109154991A (en) * | 2016-07-26 | 2019-01-04 | 惠普发展公司，有限责任合伙企业 | Three-dimension object minor structure |
| CN110046712A (en) * | 2019-04-04 | 2019-07-23 | 天津科技大学 | Decision search learning method is modeled based on the latent space for generating model |
| CN110310345A (en) * | 2019-06-11 | 2019-10-08 | 同济大学 | A kind of image generating method generating confrontation network based on hidden cluster of dividing the work automatically |
| US20200327418A1 (en) * | 2019-04-12 | 2020-10-15 | Ultrahaptics Ip Ltd | Using Iterative 3D-Model Fitting for Domain Adaptation of a Hand-Pose-Estimation Neural Network |
| CN112489198A (en) * | 2020-11-30 | 2021-03-12 | 江苏科技大学 | Three-dimensional reconstruction system and method based on counterstudy |
-
2021
- 2021-07-30 CN CN202110873799.5A patent/CN114004948B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109154991A (en) * | 2016-07-26 | 2019-01-04 | 惠普发展公司，有限责任合伙企业 | Three-dimension object minor structure |
| CN110046712A (en) * | 2019-04-04 | 2019-07-23 | 天津科技大学 | Decision search learning method is modeled based on the latent space for generating model |
| US20200327418A1 (en) * | 2019-04-12 | 2020-10-15 | Ultrahaptics Ip Ltd | Using Iterative 3D-Model Fitting for Domain Adaptation of a Hand-Pose-Estimation Neural Network |
| CN110310345A (en) * | 2019-06-11 | 2019-10-08 | 同济大学 | A kind of image generating method generating confrontation network based on hidden cluster of dividing the work automatically |
| CN112489198A (en) * | 2020-11-30 | 2021-03-12 | 江苏科技大学 | Three-dimensional reconstruction system and method based on counterstudy |
Non-Patent Citations (2)
| Title |
|---|
| SIMIAO YU 等: "Generative Creativity:Adversarial Learning for Bionic Design", 《ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING – ICANN 2019: IMAGE PROCESSING》 * |
| 高峰 等: "基于人工智能的辅助创意设计", 《装饰》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114004948B (en) | 2022-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105741355B (en) | A kind of block dividing method of triangle grid model | |
| CN109871604B (en) | Indoor function division method based on depth countermeasure network model | |
| CN109166100A (en) | Multi-task learning method for cell count based on convolutional neural networks | |
| Ulhaq et al. | Efficient diffusion models for vision: A survey | |
| CN110097609A (en) | A kind of fining embroidery texture moving method based on sample territory | |
| CN108710906A (en) | Real-time point cloud model sorting technique based on lightweight network LightPointNet | |
| CN113344777B (en) | Face changing and replaying method and device based on three-dimensional face decomposition | |
| CN111028335A (en) | Point cloud data block surface patch reconstruction method based on deep learning | |
| CN112634438A (en) | Single-frame depth image three-dimensional model reconstruction method and device based on countermeasure network | |
| Geng et al. | Embedding visual cognition in 3D reconstruction from multi-view engineering drawings | |
| CN113239961A (en) | Method for generating sequence images based on text for generating confrontation network | |
| CN114445586B (en) | Three-dimensional bionic design method based on generation countermeasure network | |
| CN115984339A (en) | Double-pipeline point cloud completion method based on geometric feature refining and confrontation generation network | |
| Wang et al. | Sketch2map: a game map design support system allowing quick hand sketch prototyping | |
| Xing et al. | Diffsketcher: Text guided vector sketch synthesis through latent diffusion models | |
| CN106484511A (en) | A kind of spectrum attitude moving method | |
| CN114004948B (en) | Creative product solver based on generation network and application thereof | |
| Cunningham et al. | A sparsity preserving genetic algorithm for extracting diverse functional 3D designs from deep generative neural networks | |
| CN114463510A (en) | Three-dimensional bionic product generation system and method based on interpolation operator | |
| Guénard et al. | Reconstructing plants in 3D from a single image using analysis-by-synthesis | |
| CN105809742A (en) | Data-driving indoor scene coloring method | |
| CN109308732B (en) | Component grid fusion method and system based on control distortion of the mesh | |
| Kostiuk et al. | Hypercolor: A hypernetwork approach for synthesizing auto-colored 3d models for game scenes population | |
| Xiyu et al. | Shape reconstruction by genetic algorithms and artificial neural networks | |
| Kang et al. | Neural style transfer for 3D meshes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information |
Inventor after: Liang Tianyi Inventor after: Mao Xi Inventor before: Mao Xi Inventor before: Liang Tianyi |
|
| CB03 | Change of inventor or designer information |