CN116401723A - Cloth static deformation prediction method based on triangular meshes - Google Patents
Cloth static deformation prediction method based on triangular meshes Download PDFInfo
- Publication number
- CN116401723A CN116401723A CN202310058389.4A CN202310058389A CN116401723A CN 116401723 A CN116401723 A CN 116401723A CN 202310058389 A CN202310058389 A CN 202310058389A CN 116401723 A CN116401723 A CN 116401723A
- Authority
- CN
- China
- Prior art keywords
- cloth
- grid
- encoder
- obstacle
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000004744 fabric Substances 0.000 title claims abstract description 160
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000003068 static effect Effects 0.000 title claims abstract description 20
- 239000013598 vector Substances 0.000 claims abstract description 55
- 230000014509 gene expression Effects 0.000 claims abstract description 29
- 238000012886 linear function Methods 0.000 claims abstract description 19
- 238000004364 calculation method Methods 0.000 claims abstract description 10
- 230000006870 function Effects 0.000 claims description 23
- 239000011159 matrix material Substances 0.000 claims description 23
- 238000012549 training Methods 0.000 claims description 21
- 230000004888 barrier function Effects 0.000 claims description 18
- 238000011176 pooling Methods 0.000 claims description 17
- 230000004927 fusion Effects 0.000 claims description 13
- 230000035515 penetration Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 claims description 6
- 238000000547 structure data Methods 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 4
- 239000000463 material Substances 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims 1
- 238000004088 simulation Methods 0.000 abstract description 20
- 238000013135 deep learning Methods 0.000 abstract description 6
- 239000004973 liquid crystal related substance Substances 0.000 description 10
- 238000013528 artificial neural network Methods 0.000 description 7
- 230000001788 irregular Effects 0.000 description 5
- 230000005484 gravity Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000013213 extrapolation Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000007493 shaping process Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 101100481876 Danio rerio pbk gene Proteins 0.000 description 1
- 101100481878 Mus musculus Pbk gene Proteins 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 238000000053 physical method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Business, Economics & Management (AREA)
- Evolutionary Computation (AREA)
- Human Resources & Organizations (AREA)
- Software Systems (AREA)
- Strategic Management (AREA)
- General Engineering & Computer Science (AREA)
- Economics (AREA)
- Computer Hardware Design (AREA)
- Development Economics (AREA)
- Pure & Applied Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Mathematical Analysis (AREA)
- Artificial Intelligence (AREA)
- Mathematical Optimization (AREA)
- Game Theory and Decision Science (AREA)
- Computational Mathematics (AREA)
- Medical Informatics (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a cloth static deformation prediction method based on triangular meshes, which aims to solve the problems of large calculation amount and long calculation time in the traditional cloth simulation based on physics by using a deep learning method, and can capture the authenticity of a simulation result based on physics. The specific method of the invention comprises the following steps: the template cloth grid in the initial state and the obstacle or human body posture grid in the target state are input and converted into vector expression in the hidden space through the Encoder module respectively; the cloth vector expression in the hidden space is input into a group of trainable linear functions, and the obstacle vector expression fuses the group of linear functions in a weight form to obtain the vector expression of the cloth in the hidden space under the target state; the vector representation of the target state fabric is converted into a deformed fabric grid by a Decoder module. The method can be widely applied to scenes with high real-time performance such as virtual fitting and games, and can solve the problem of poor sense of reality existing in the existing high real-time performance method.
Description
Technical Field
The invention relates to the technical field of cloth simulation in flexible body motion simulation, in particular to a cloth static deformation prediction method based on triangular meshes.
Background
Cloth simulation technology is widely used in the fields of video animation, virtual fitting, game engines, and the like. The traditional cloth simulation adopts a physical-based method to discretize the cloth grid and discretize the solving time at the same time, and a nonlinear equation set can be solved in each time step, so that a simulation result with high sense of reality can be obtained, but a large amount of calculation time is needed to be consumed. The fabric simulation in the scenes with high real-time requirements such as games often adopts a simplified and optimized model to obtain the deformation of the fabric, and under the real-time requirements, the simulation result is often low in sense of reality, and complex physical and mechanical models are difficult to calculate on line. In these scenarios, the cloth is represented as a grid consisting of a finite number of triangles, deformation of the cloth is achieved by changing the vertex positions of the cloth grid in an initial state or under a template condition, and the deformation process generally keeps the topology of the initial state or the template cloth grid unchanged.
The cloth deformation result with physical reality is difficult to obtain under the requirement of high real-time performance, and the traditional real-time performance methods such as a physical-based method and a virtual fitting method cannot directly obtain good results. Combining the physics-based simulation method with the deep learning method provides a concept for solving the problem. And obtaining a high-quality cloth deformation result as data through simulation based on a physical method, constructing a neural network architecture based on learning, and training the nonlinearity of the neural network in the fitting cloth deformation process. The trained network can learn the simulation characteristics similar to those based on physics, and can predict the deformation result with good physical characteristics in real time by combining with the reasoning instantaneity of the network.
Although the method based on the combination of physical simulation and deep learning can capture the nonlinearity of cloth deformation in real time, the method has the following problems:
(1) The method for predicting cloth deformation based on the deep learning of the SMPL is often too dependent on the SMPL human body model, has good prediction capability on the clothes of a comparison body-building type, but is difficult to be directly applied to other human body or non-human body obstacle objects, or has reduced expressive capability on the clothes of a loose type;
(2) Since the cloth grid is expressed as an irregular grid structure, the deep learning method is mostly based on the grid data with regular shapes, and is difficult to be directly applied to the grid structure. Other methods for predicting cloth deformation based on learning generally use network modules such as MLP (multi-level programming) to design a framework structure, often lead to excessive parameter quantity of a trained network model, reduce reasoning speed and lead to lower generalization capability of the model;
(3) Some cloth deformation prediction methods related to deep learning adopt a cyclic input mode, and the prediction deformation of the current state is directly used as the input of the next prediction, and the cyclic input mode is easy to introduce accumulated errors, so that the accuracy of the prediction deformation is reduced.
Disclosure of Invention
In order to overcome the problems in the prior art, the invention provides a cloth static deformation prediction method based on triangular meshes, which can directly predict the deformation of cloth in a target state according to the cloth meshes in an initial state and the obstacle meshes in the target state. The invention can process not only the human body of the SMPL model, but also the human body of other non-SMPL models, even the human body obj expression, and can process the obstacle rigid body of the non-human body. The invention can predict not only regular clothes deformation, such as body shaping or loose type clothes, but also any irregularly shaped cloth.
In order to achieve the above functions, according to an aspect of the present disclosure, there is provided a cloth static deformation prediction method based on triangular mesh, including:
the cloth grid in the initial state is subjected to feature extraction through a cloth Encoder module, and vector expression of the cloth grid in the hidden space is obtained through downsampling;
the barrier grid in the target state is subjected to feature extraction through a barrier Encoder module, and vector expression of the barrier grid in the hidden space is obtained through downsampling;
the hidden space vector of the initial state cloth grid is input into a group of linear fusion functions, the hidden space vector of the target state obstacle grid participates in the weighted calculation of the group of linear fusion functions in a weighted mode, and the hidden space vector expression of the target state cloth grid (namely, the deformed cloth grid) is obtained;
and carrying out up-sampling on vector expression of the target state distribution grid in the hidden space through a Decoder module to obtain a deformed grid of the target state distribution in the three-dimensional space.
The cloth types in the technical proposal comprise clothes such as long sleeves and trousers for slimming, also comprise a loose type skirt, and also can be irregular cloth such as curtains or tablecloths which are not clothes types; the obstacle types in the technical scheme comprise a SMPL human body model, a non-SMPL human body model and a human body obj representation, and also comprise rigid body obstacles of a non-human body and the like.
According to the technical scheme, when the distribution Encoder module and the obstacle Encoder module are used for extracting the characteristics, the vector dimension in the obtained hidden space is optional, the vector dimension is set in a super-parameter mode, and the optimized numerical value is obtained through experimental verification.
When the hidden space vector expression fusion is carried out, the coefficient parameters of a group of linear fusion functions input by the hidden space vector of the initial state cloth grid are trainable, and the hidden space vector of the obstacle grid serving as the fusion weight is trainable.
The cloth Encoder and the obstacle Encoder module in the technical scheme are composed of a plurality of submodules. The invention expresses the cloth grid and the barrier grid as a graph structure formed by connecting nodes, and the grid graph structure is expressed as the spatial position attribute of the vertexes and the connection topology information of edges between the vertexes. The method comprises the steps that a plurality of sub-modules continuously extract features on each level of sub-image structures, wherein two sub-modules are respectively used as input information and output information of each sub-module, the vertex attribute of each level of sub-image and the topology information of a connecting edge, and the vertex attribute of an input grid of a first sub-module is the spatial position information of the vertex.
Specifically, the network architecture of each sub-module is mainly composed of the following network layers:
a graph Transformer layer, which aims at extracting characteristics of each level of subgraphs;
the Normalization layer aims at carrying out Normalization operation on the output of the network layer so as to accelerate the convergence of the neural network;
the PReLU layer activates the function layer, and aims to increase nonlinearity of the neural network and enhance the expression capability of the neural network;
and a Pooling layer, pooling operation, and downsampling the extracted features.
In the above sub-module, when the Pooling operation is performed, the vertices are sampled according to the vertex attributes of the graph, which may result in a large number of isolated point sets after sampling. These isolated point sets, because of the lack of topological connection with surrounding vertices, can affect subsequent feature extraction operations. To improve this, a second order adjacency matrix of the computation graph is needed to update the previous adjacency matrix before the Pooling operation. It should be noted that this operation of updating the adjacency matrix only improves the situation of the isolated point set, and the existence of the isolated point set cannot be avoided.
The Decoder module in the technical scheme comprises two parts: an MLP layer and a set of trainable mesh matrices. Vector expressions of the target state cloth (deformed cloth) in the hidden space can be directly input into the MLP layer for subsequent learning. The number of the trainable grid matrixes is a super parameter and is determined in experimental optimization. The number of rows of the grid matrix is the number of cloth vertexes, and the number of columns of the grid matrix is 3.
In the above technical solution, in order to improve the accuracy of the prediction result, the MLP part in the Decoder module connects the output of each layer in the obstacle Encoder module to the input part of the MLP by a linear function, so as to increase the influence of the obstacle grid in the target state on the fabric deformation.
Compared with the existing cloth deformation prediction method based on learning, the method has the beneficial effects that:
(1) The invention is not dependent on any manikin like SMPL, and can be applied to SMPL manikins, and can also be applied to other manikins and even human obj expressions. Compared with the prior SMPL-based method, the method has stronger applicability, does not need to carry out complicated modification on the model, and can be widely applied to scenes such as games, virtual fitting and the like.
(2) The present invention can process various types of cloth, not only a slim type of clothing and a loose type of clothing such as a skirt type, but also an irregular arbitrary shape of cloth which cannot be processed by the SMPL based method.
(3) The invention uses the graph converter network to directly extract the characteristics on the grid data, introduces a multi-head attention mechanism on the graph structure data, and compared with the prior method, the invention uses the topological structure of the grid, can extract the local characteristics of the grid more advantageously, and generates cloth deformation with good detail characteristics. The graph Transformer network structure can dynamically weight the impact of surrounding node information on the extracted features compared to the graph convolution network.
(4) The invention uses the graph converter network layer and the trainable grid matrix structure, avoids the problem of excessive redundancy of network parameters caused by using the full-connection layer structure, reduces the parameter quantity, and ensures the reasoning speed and extrapolation accuracy of the network.
Compared with the traditional cloth simulation method based on physics, the method has the beneficial effects that:
(1) The target deformed cloth is directly obtained by a network reasoning method without calculating a complicated nonlinear equation set, and the real-time performance is high.
(2) The time for obtaining the deformation of each frame of cloth by network reasoning is basically consistent, and the fluctuation is small; the traditional physical-based simulation method calculates the time fluctuation of deformation of each frame to be larger.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention;
fig. 2 is a schematic flow chart of a cloth static deformation prediction method based on triangle meshes according to an embodiment of the invention.
Fig. 3 is a schematic diagram of updating the adjacency matrix of the graph structure according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made more fully hereinafter with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The cloth static deformation prediction method based on the triangular meshes breaks the limitation of the previous method based on the SMPL human body model, can treat various different types of human bodies and barriers, and can also treat various types of cloth. According to the method, vertex attributes and topology information of the triangle mesh are directly utilized, and more detail features can be kept for predicting the single result.
Specifically, as shown in fig. 1 and 2, the invention inputs a cloth grid in an initial state and an obstacle grid in a target state, performs feature fusion by downsampling to a hidden space, and then obtains a deformed cloth grid in the target state by upsampling.
As shown in fig. 1 and 2, downsampling of the initial state cloth grid and the target state obstacle grid is performed by cloth Encoder and respectivelyThe barrier Encoder module. The input of the cloth Encoder is the cloth grid of the initial state, and the output is the initial cloth vector expression I c The input of the obstacle Encoder is the object state obstacle grid, and the output is the obstacle vector expression I o 。
The fabric Encoder and the barrier Encoder module have two inputs of the graph Transformer layer, namely nodes of the graph represented by the grid and an adjacent matrix representing the graph topology. After the feature extraction on the graph, only one output of the graph converter layer is the feature of each node of the extracted graph. In the process of feature extraction by the graph Transformer, the topological structure of the graph is not changed, and the dimension of each node feature vector of the extracted graph exists in the form of super-parameters.
Specifically, as a further technical scheme, each node attribute of the first layer graph input into the graph transform layer is expressed as follows:
n represents the number of nodes on the first layer of graph, h represents the feature vector of each node, and the dimension of the vector is a super parameter in the network structure design process. In particular the number of the elements to be processed,
representing the eigenvector at node i on the layer i graph,/>
Representing feature vectors at surrounding nodes j at node i.
As a further technical solution, it is necessary to calculate the multi-head attention feature from node j to node i, specifically, the attention feature at the c-th head is calculated as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
and b c,e Represented as trainable network parameters, e ij Representing the characteristic value of the provided connecting edge.
As a further technical solution, a multi-head attention coefficient from node j to node i needs to be calculated, specifically, the attention coefficient at the c-th head is calculated as follows:
where d represents the hidden layer size at the c-th head.
As a further technical solution, the output characteristics of the node i on the layer 1 +1 graph
Calculated from the following formula:
wherein C represents the number of heads in the multi-head attention mechanism.
And->
Is a trainable network parameter, +.>
Representing the surrounding neighboring nodes of node i. The multi-head attention coefficients in the multi-head attention mechanism are weighted and spliced together to form the output of the nodes on the next layer of graph.
To avoid that the predicted output result is too smooth, the following coefficients need to be further calculated:
wherein W is g (l) ，W r (l) A kind of electronic device
Are trainable network parameters.
After the calculated coefficients are blended, the final calculation formula of the output characteristics of the node i on the layer 1 +1 graph is as follows:
the Normalization layer in the cloth Encoder and obstacle Encoder modules here aims to speed up convergence speed when training the neural network. The specific operation of the Normalization Layer is implemented in the form of Layer Normal. Since the features extracted by the previous Layer map transducer are at each node, in order to match it to the inputs of the Layer Normal, the features at all nodes are flattened into one-dimensional vector, which is then input into the Layer Normal.
The PReLU layer in the cloth Encoder and obstacle Encoder modules here is intended to add nonlinearity, enhancing the fitting ability of the neural network.
The Pooling layer in the cloth Encoder and obstacle Encoder modules aims at Pooling nodes on the graph, and the Pooling algorithm adopts a Topk algorithm. The number of nodes of the graph after the Pooling operation is reduced, and new topological connection relations exist among the pooled nodes. The Pooling operation does not change the characteristic attribute of the node, but simply selects the node to be reserved according to the node characteristic.
Since the Pooling operation can reduce nodes of the graph, many isolated nodes can exist in the new graph obtained after the operation. To improve this, the second order adjacency matrix of the graph needs to be recalculated and updated to a new adjacency matrix before the Pooling operation.
The second-order adjacency matrix of the calculation map is shown in fig. 3, and the surrounding adjacency points of each point are changed from one circle to two circles after the second-order adjacency matrix is calculated. It should be noted that, the adjacency matrix of the update map is a second-order adjacency matrix, which can only improve the situation that isolated nodes exist after the Pooling operation, and cannot avoid the existence of the isolated nodes. The updating of the adjacency matrix operation is performed before the subsequent Pooling operation.
The sub-modules in the cloth Encoder and the obstacle Encoder are formed by a graph Transformer layer, a Normalization layer, a PReLU layer and a Pooling layer.
The cloth Encoder and the obstacle Encoder consist of a plurality of similar sub-modules. After each sub-module, the graph structure data can obtain a new graph structure with fewer nodes, and after the downsampling of a plurality of sub-modules, the distribution grid in the initial state and the barrier grid in the target state are respectively compressed into vector expressions in the hidden space.
After downsampling by a plurality of submodules in the cloth Encoder and the obstacle Encoder, a graph structure with fewer nodes is directly obtained.
Initial cloth vector I in hidden space c The expression is expressed as follows:
wherein E is c () The cloth Encoder in the present invention is shown,
representing the input initial state cloth grid.
The initial state of the cloth described in the invention refers to the undeformed state of the cloth, and the initial state of the cloth is the deformed state of the cloth worn on a human body under the corresponding T-shaped posture aiming at different types of clothes, and the cloth naturally hangs in a balanced state under the action of gravity; for irregular non-clothes type cloth, the initial state is an undeformed state under specific working conditions, such as a curtain hung at two points, and the initial state is an equilibrium state in which the cloth naturally hangs under the action of gravity. It should be noted that the initial state of the cloth is the hanging state of the cloth under the gravity balance.
The target state described in the invention refers to an obstacle state under a specific working condition, and is a human body under a specific posture aiming at different types of human body models; for a non-human body rigid body obstacle, the target state is a state when the target is in different spatial positions (such as after translational rotation and other operations). The cloth in the target state refers to a deformed cloth on an obstacle in the target state.
Object state obstacle vector I in hidden space o The expression is expressed as follows:
wherein E is o () Represents an obstacle Encoder in the present invention,
representing an input target state obstacle grid.
As a further technical proposal, the invention fits a group of linear functions { f in the hidden space 1 ,f 2 ,f 3 ,…,f n Specifically, wherein the i-th function is represented as follows:
the coefficients of the linear function are trainable network parameters. The linear function distributes the vector I in an initial state c For input, the vector of the initial state is intended to be deformed in hidden space.
As a further technical proposal, vector expression I of the object state obstacle in the hidden space o Participate in the form of weights in the aboveThe linear functions are fused in the following way:
after fusion, the vector expression I of the target state deformation cloth in the hidden space can be obtained t It should be noted that the dimensions of the cloth vector representation and the obstacle vector representation in the hidden space are not identical.
As shown in fig. 2, the cloth deformation vector I in the hidden space t And obtaining the deformed cloth in the three-dimensional space after upsampling by the Decoder module. Specifically, the Decoder module is composed of two parts: the MLP sub-module and the trainable matrix sub-module. The MLP submodule aims to provide more nonlinearity in the process of upsampling so that the resulting cloth deformation has more details. The trainable matrix sub-module is composed of a set of trainable grid matrices, initialized to zero during network training, and the values therein are engaged in optimization through network training. Compared with a Decoder module completely composed of MLP, the Decoder module composed of the MLP sub-module and the trainable matrix sub-module is greatly reduced in network training parameters, so that the model of the whole network becomes smaller, and the model has stronger extrapolation capability.
As shown in fig. 2, there is a data-intensive operation between the barrier Encoder module and the MLP sub-module in the Decoder module. Specifically, by inputting the output of each layer in the obstacle Encoder module simultaneously into a linear function composed of MLP modules, the formula is as follows:
f o (l) (o (l) )＝W o (l) o (l) +b (l)
wherein o is (l) Output vector f of the first layer of the barrier Encoder module o (l) () As a linear function of the first layer, W o (l) And b (l) Is f o (l) () Is provided for the training of the coefficient parameters.
The output of the linear function and the output of the corresponding layer of the MLP sub-module in the Decoder module are added and summed, and the summed data is input to the corresponding layer of the MLP sub-module, and the formula is as follows:
P o (l) ＝P (l) (f o (l) (o (l) )+d (l-1) )
wherein d (l-1) For the output of the first-1 layer of the MLP submodule in the Decoder module, P (l) () For the function expression of the first layer of the MLP submodule in the Decoder module, P o (l) The output of the first layer of the MLP submodule in the Decoder module.
The data reinforcement operation is used for increasing the influence of the barrier network branches on the final deformation result, so that a predicted result with higher precision is obtained.
The training data required by the network provided by the invention has diversity, and the data of the obstacle branches comprise the human body of the SMPL model, the human body without any model and the rigid body obstacle; the data of the cloth branches include body-shaping type clothes such as short sleeves, long sleeves, shorts, trousers and the like, loose type clothes such as skirts, gowns and the like, and non-clothes type cloth with irregular shapes.
In particular, for the human body of the SMPL model of training data in the invention, after setting specific body type parameters and posture parameters, the human body grid under the body type and posture is directly stored and used as the data of subsequent network training.
The training data in the invention is obtained by simulation calculation of a physical-based cloth simulation tool. According to the invention, the influence of the cloth material on deformation is not considered, so that all clothes and cloth which participate in simulation calculation are set to be the same material when training data are obtained.
The static state in the cloth static deformation prediction method based on the triangular meshes provided by the invention refers to the fact that the deformation predicted by the method is cloth deformation in a static stable state, and the influence of time dynamic factors on deformation results is not considered in the network architecture provided by the invention. Therefore, when training data are obtained, aiming at all the cloth, in simulation, before each frame of deformation data is obtained, the cloth is allowed to naturally droop on the obstacle only by gravity for a few frames to eliminate residual stress in the cloth, so that the dynamic influence of the deformation of the cloth is eliminated to ensure the uniqueness of the obtained deformation result.
In order to accelerate network convergence and reduce difficulty of network learning, the obstacle grid and cloth grid data in the invention are both translated to the vicinity of the origin to eliminate the absolute of the spatial position coordinates, and the characteristic to be fitted in the training of the network is prevented from being too scattered.
In order to reduce training time, the invention performs standardization processing on input and output grid space vertex position coordinates during training of a network, so that the grid space vertex position coordinates meet (0, 1) normal distribution.
As a further technical scheme, the deformed cloth grid obtained through simulation is used as a group trunk training network, and an Adam optimizer is adopted to optimize the loss function.
The loss function used in the network training process is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
representing the spatial position coordinates at node i on the predicted deformed cloth grid, +.>
Representing the spatial location coordinates at node i on the group route, N represents the number of nodes on the grid, | … || | 2 Representing calculation L 2 Distance.
In order to obtain a non-penetrating cloth grid, the invention provides a method for predicting a penetration loss function between the cloth grid and an obstacle grid, which comprises the following steps:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is the spatial position coordinate of the node i on the deformed cloth grid obtained by network prediction, +.>
Is the distance on the barrier grid +.>
Is the closest point of (c). />
Is +.>
Normal vector at d ε Is a set minimum penetration threshold, and N represents the number of nodes on the cloth grid.
The penetration loss function in the technical scheme can only solve the problem of penetration between the predicted cloth grid and the barrier grid, and as a further technical scheme, the invention provides the following loss function to reduce the self-penetration problem existing in the predicted cloth grid:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is to predict the spatial position coordinates of node i on the cloth grid,/->
Is the spatial position coordinates of the node j closest to the node i on the predicted cloth grid. />
Is the normal vector at the predicted cloth grid node j.
The comprehensive loss function used in the training of the network according to the invention is represented as follows:
L＝L p +λL e +μL s
where λ and μ are coefficients between the three loss functions of the mixture, the values of λ and μ can be set to 1 in general.
It should be noted that the penetration loss function between the cloth and the obstacle and the self-penetration loss function of the cloth can effectively avoid the occurrence of the penetration phenomenon in the predicted deformed cloth, but the predicted result of the network cannot completely guarantee no penetration state during reasoning.
Aiming at the network predicted deformed cloth grid, the invention provides the following measurement standard to quantitatively analyze the predicted cloth deformation so as to measure the quality of the predicted result:
wherein, the liquid crystal display device comprises a liquid crystal display device,
representing the spatial position coordinates of node i on the predicted deformed cloth,/->
And the spatial position coordinates of the node i on the group trunk cloth are represented. />
And->
And respectively representing the normal vector of the node i on the predicted deformed cloth and the ground trunk cloth. N represents a cloth netThe number of nodes on the grid.
The three measures of the predicted deformation cloth provided by the invention comprise three aspects epsilon dist Representing the prediction of the distance error epsilon between the vertices of the cloth grid and the ground trunk lap Representing the error between Laplace operator and predicted cloth grid and ground truth, and aiming at measuring the local detail characteristics epsilon of the predicted cloth deformation grid norm The method is characterized by predicting the error between normal vectors of vertexes between a cloth grid and a ground trunk cloth and aims at measuring the characteristics such as the surface curvature of the predicted cloth deformation grid.
It should be noted that the static fabric deformation prediction method based on the grid provided by the invention still has the following limitations:
(1) The problem studied by the invention is to predict the cloth deformation in a static state or in a balance state, and the influence of dynamic factors on the cloth deformation cannot be predicted.
(2) The predicted fabric deformation result cannot completely ensure no penetration, and the post-treatment operation is required for obtaining the complete no penetration result.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced with equivalents; these modifications or substitutions do not depart from the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.
Claims (7)
1. The cloth static deformation prediction method based on the triangular meshes is characterized by comprising the following steps of:
the cloth grid in the initial state is subjected to feature extraction through a cloth Encoder module, and vector expression of the cloth grid in the hidden space is obtained through downsampling;
the barrier grid in the target state is subjected to feature extraction through a barrier Encoder module, and vector expression of the barrier grid in the hidden space is obtained through downsampling;
the hidden space vector of the initial state cloth grid is input into a group of linear fusion functions, the hidden space vector of the target state obstacle grid participates in the weighted calculation of the group of linear fusion functions in a weight mode, and the hidden space vector expression of the target state cloth grid, namely the deformed cloth grid, is obtained;
and carrying out up-sampling on vector expression of the target state distribution grid in the hidden space through a Decoder module to obtain a deformed grid of the target state distribution in the three-dimensional space.
2. The cloth static deformation prediction method based on triangular meshes according to claim 1, wherein the cloth Encoder and the obstacle Encoder are composed of a plurality of submodules, and the input of the cloth Encoder and the obstacle Encoder are all graph structure data represented by meshes; the graph structure is represented as the spatial position attribute of the vertexes and the connection topology information of the edges between the vertexes, a plurality of sub-modules continuously extract the characteristics on each level of sub-graph structure, and the input information and the output information of each sub-module are two, namely the vertex attribute of each level of sub-graph and the topology information of the connection edges.
3. The cloth static deformation prediction method based on triangular meshes according to claim 2, wherein each sub-module composition structure in the cloth Encoder and the obstacle Encoder comprises: a graph transformation network layer, a Normalization layer, a PReLU layer and a Pooling layer which introduce a multi-head attention mechanism; the node topology structure of the graph is unchanged and the node characteristic attribute is changed before and after the graph structure data passes through the graph Transformer network layer; the number of node tables of the graph is small before and after the graph structure data passes through the Pooling layer, and the topology structure of the graph is changed.
4. A cloth static deformation prediction method based on triangular meshes according to claim 3, wherein the second order adjacency matrix of the graph structure data is calculated as a new adjacency matrix before the Pooling layer in the sub-modules in the cloth Encoder and the obstacle Encoder.
5. The cloth static deformation prediction method based on triangular meshes according to claim 1, wherein hidden space vectors obtained by the cloth Encoder are input into a group of trainable linear functions to fit and obtain vector expressions of deformed cloth in hidden space, and hidden space vectors obtained by the obstacle Encoder participate in fusion operation of the trainable linear functions in a form of weights, specifically:
initial cloth vector I in hidden space c The expression is as follows:
wherein E is c () The cloth material Encoder is shown as such,
representing an input initial state cloth grid;
object state obstacle vector I in hidden space o The expression is as follows:
wherein E is o () Representing the obstacle Encoder and the position of the obstacle,
a mesh of object state obstacles representing an input;
fitting a set of linear functions { f 1 ,f 2 ,f 3 ,…,f n -wherein the i-th function is represented as follows:
wherein coefficients of linear functions
Is a trainable network parameter, the set of linear functions distributes the vector I in an initial state c For input, the vector of the initial state is deformed in the hidden space; vector representation I of object state obstacle in hidden space o The fusion of the linear functions is participated in the form of weights, and the fusion mode is as follows:
after fusion, the vector expression I of the target state deformation cloth in the hidden space can be obtained t 。
6. The cloth static deformation prediction method based on triangular meshes according to claim 1, wherein the Decoder module consists of an MLP sub-module and a trainable mesh matrix group sub-module, wherein the mesh matrix group sub-module is initialized to zero during training, and the numerical value of each parameter in the mesh matrix group sub-module can be obtained during network convergence; the output of each network layer of the barrier Encoder is simultaneously input into a linear function consisting of MLPs, formulated as follows:
wherein o is (l) For the output vector of the first layer of the barrier Encoder module,
as a linear function of the first layer +.>
And b (l) Is->
Is provided;
the output of the linear function and the output of the corresponding layer of the MLP sub-module in the Decoder module are added and summed, and the summed data is input to the corresponding layer of the MLP sub-module, and the formula is as follows:
wherein d (l-1) For the output of the first-1 layer of the MLP submodule in the Decoder module, P (l) () For the functional representation of the first layer of the MLP submodule in the Decoder module,
the output of the first layer of the MLP submodule in the Decoder module.
7. The cloth static deformation prediction method based on triangular meshes according to claim 1, wherein the loss function in the network training process consists of three parts: the vertex position coordinate loss function of the training data and the prediction data, the penetration loss function between the obstacle and the prediction cloth and the self-penetration loss function of the prediction cloth existing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310058389.4A CN116401723A (en) | 2023-01-17 | 2023-01-17 | Cloth static deformation prediction method based on triangular meshes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310058389.4A CN116401723A (en) | 2023-01-17 | 2023-01-17 | Cloth static deformation prediction method based on triangular meshes |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116401723A true CN116401723A (en) | 2023-07-07 |
Family
ID=87009223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310058389.4A Pending CN116401723A (en) | 2023-01-17 | 2023-01-17 | Cloth static deformation prediction method based on triangular meshes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116401723A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118154282A (en) * | 2024-05-11 | 2024-06-07 | 武汉纺织大学 | Multi-mode personalized clothing recommendation method based on graphic neural network |
-
2023
- 2023-01-17 CN CN202310058389.4A patent/CN116401723A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118154282A (en) * | 2024-05-11 | 2024-06-07 | 武汉纺织大学 | Multi-mode personalized clothing recommendation method based on graphic neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bertiche et al. | Pbns: Physically based neural simulator for unsupervised garment pose space deformation | |
| Frishman et al. | Multi-level graph layout on the GPU | |
| CN110097609B (en) | Sample domain-based refined embroidery texture migration method | |
| CN109978165A (en) | A kind of generation confrontation network method merged from attention mechanism | |
| CN114662172B (en) | Neural network-based dynamic simulation method for clothing fabric | |
| CN110473284A (en) | A kind of moving object method for reconstructing three-dimensional model based on deep learning | |
| CN111724459B (en) | Method and system for redirecting movement of heterogeneous human bones | |
| CN109711401A (en) | A kind of Method for text detection in natural scene image based on Faster Rcnn | |
| JP7220083B2 (en) | Wind speed distribution estimation device and wind speed distribution estimation method | |
| CN116401723A (en) | Cloth static deformation prediction method based on triangular meshes | |
| CN112991503A (en) | Model training method, device, equipment and medium based on skin weight | |
| CN116362133A (en) | Framework-based two-phase flow network method for predicting static deformation of cloth in target posture | |
| Yandun et al. | Visual 3d reconstruction and dynamic simulation of fruit trees for robotic manipulation | |
| CN108520513A (en) | A kind of threedimensional model local deformation component extraction method and system | |
| CN107204040A (en) | Multiple-Point Geostatistics modeling method and device, computer-readable storage medium | |
| CN114742952A (en) | Three-dimensional garment simulation method and device, terminal equipment and storage medium | |
| CN112819951A (en) | Three-dimensional human body reconstruction method with shielding function based on depth map restoration | |
| CN116363308A (en) | Human body three-dimensional reconstruction model training method, human body three-dimensional reconstruction method and equipment | |
| Yang et al. | Algorithm for appearance simulation of plant diseases based on symptom classification | |
| CN112819930A (en) | Real-time role garment fabric animation simulation method based on feedforward neural network | |
| CN105321205B (en) | A kind of parameterized human body model method for reconstructing based on sparse key point | |
| CN106530384A (en) | Appearance texture synthesis method and device for three-dimensional model | |
| CN112926681B (en) | Target detection method and device based on deep convolutional neural network | |
| CN104517299A (en) | Method for restoring and resimulating physical video fluid driving model | |
| Diao et al. | Combating Spurious Correlations in Loose‐fitting Garment Animation Through Joint‐Specific Feature Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |