CN110084872A - A kind of the Animation of Smoke synthetic method and system of data-driven - Google Patents

Abstract

The invention relates to a data-driven smoke animation synthesis method, comprising: generating two-dimensional smoke profile data according to a smoke data set; generating a smoke generation model through the training of the two-dimensional smoke profile data; obtaining a smoke generation model through the smoke generation model , generating the two-dimensional smoke profile data into a three-dimensional smoke sequence; rendering the three-dimensional smoke sequence to generate smoke animation. The smoke animation synthesis method proposed by the present invention makes the smoke information and control mode simpler and more intuitive, maintains the authenticity of the smoke animation under simple input, and can generate realistic smoke animation with user-controllable shapes in real time.

Description

A data-driven smoke animation synthesis method and system

technical field

The invention relates to the field of computer graphics, in particular to a method and system for controllable synthesis of fluid animation.

Background technique

The simulation of natural phenomena has always been one of the hot issues in computer graphics research. Among them, fluid simulation, especially the real-time modeling and controllable animation technology of smoke animation, has more and more extensive application requirements and values in various fields such as film and television special effects, advertising, 3D game development, and virtual reality. However, natural smog is a complex physical phenomenon. Due to the influence of various factors such as buoyancy, obstacles, wind energy input and internal eddy, the internal concentration distribution of smog is irregular, and the shape is also various, which can be uniform. It can also have violent eddy current changes, its geometric shape is extremely irregular, and the boundary is difficult to distinguish and define. Over the years, in the research of fluid simulation, from the traditional fluid dynamics model, Navier-Stokes equations (Navier-Stokes equations, referred to as N-S equations) to solve the motion elements to the non-physical method, that is, data-driven fluid animation synthesis method, the realism and quality of the synthesized smoke are constantly improving, and the computing efficiency and real-time performance are also gradually improving. Therefore, focusing on the user's perspective, we hope to provide input in a simpler way and also obtain real-time and high-realistic simulation results. Demand also arises as the times require.

In the prior art, "Fluid animation generation method and device based on deep learning and SPH framework" (application publication number: CN108717722A), discloses a method and device for generating fluid animation based on deep learning and SPH framework, using neural network model training After importing into the SPH fluid simulation framework, it aims to achieve low-precision fluid and high-detail performance through offline rendering.

"Data-driven fluid animation acceleration generation method" (application publication number: CN106023286A), discloses a data-driven fluid animation acceleration generation method, using the artificial neural network after the training sample training is completed as a solver, in Euler The solution of the projection step in the fluid simulation process of the method is very fast, and the solution error can be kept small while the solution is fast, thus ensuring the accuracy of the solution result. The present invention uses the training data calculated before and after the projection step, and adjusts the weights of the transmission nodes of the artificial neural network through the training of the artificial neural network to directly obtain the final calculation model, completely avoiding the original time-consuming value calculation process of the projection step. The present invention is applicable to Euler's method for simulating fluid animation, and greatly accelerates the calculation of the solution projection step.

In the past, based on the N-S equation, it was necessary to rely on solving parametric equations to generate a velocity field and then obtain a synthetic smoke, while based on deep learning, it was necessary to rely on three-dimensional data input to obtain a three-dimensional velocity or density field output, and the use of parametric equations needs to be based on user Only a deep understanding of the fluid system can be effectively controlled, but in fact, the amount of calculation is large, it is difficult to meet the real-time requirements, and the control method is single, which still has a lot of restrictions and constraints on the use of users; and the input of 3D data is very difficult for users. It is not easy to obtain, process and use, so when using it, it still needs to invest a lot of troublesome data preprocessing work.

SUMMARY OF THE INVENTION

In order to solve the problems of large amount of calculation, poor real-time performance, and single control mode faced by the above-mentioned N-S equation method, and the problem that the input of three-dimensional data is not easy for users to acquire, process and use, the present invention proposes a data-driven The smoke animation synthesis method of the present invention uses a two-dimensional smoke profile to obtain a three-dimensional fluid field with a time series relationship through a smoke generation model, and generates a three-dimensional smoke animation.

Specifically, the smoke animation synthesis method of the present invention includes: generating two-dimensional smoke profile data according to the three-dimensional smoke data set; generating a network through the training of the two-dimensional smoke profile data to obtain a smoke generation model; through the smoke generation model, The two-dimensional smoke profile data is generated as a three-dimensional smoke sequence; the three-dimensional smoke sequence is rendered to generate a three-dimensional smoke animation.

The smoke animation synthesis method of the present invention, wherein the loss function of the smoke generation model for:

Among them, x is the two-dimensional smoke profile data, G(x) is the three-dimensional smoke sequence A single frame of 3D density field, y is the 3D smoke data set The single-frame three-dimensional density field of ～, D _S is the discriminator for judging the spatial consistency of G(x) and y, E _n is the discrimination expectation of G(x) passing D _S when the size of x is n, and E _m is Discriminator D _S pairs of y of scale m ~ is the discrimination expectation, D _t is the judgment and The discriminator of temporal consistency, F ^j is the output feature map of the jth layer of D _S extracted, For the loss function when extracting F ^j The influence weight coefficient, E _n,j is when the size of x is n to and The spatial consistency expectation compared when F ^j is extracted respectively, n and j are positive integers.

The smoke animation synthesis method of the present invention obtains the three-dimensional smoke data set by solving the Navier-Stokes equation.

The smoke animation synthesis method described in the present invention generates the three-dimensional smoke animation through GPU-based rendering under multiple perspectives.

The present invention also proposes a data-driven smoke animation synthesis system, including: a two-dimensional data generation module, used to generate two-dimensional smoke contour data according to a three-dimensional smoke data set; a smoke generation model training module, used to pass the two-dimensional smoke The training generation network of the outline data is used to obtain the smoke generation model; the three-dimensional sequence generation module is used to generate the two-dimensional smoke outline data into a three-dimensional smoke sequence through the smoke generation model; the smoke animation rendering module is used for the three-dimensional smoke The sequence is rendered to generate a 3D smoke animation.

The smoke animation synthesis system of the present invention, wherein the loss function of the smoke generation model for:

Among them, x is the two-dimensional smoke profile data, G(x) is the three-dimensional smoke sequence A single frame of 3D density field, y is the 3D smoke data set The single-frame three-dimensional density field of , D _S is the discriminator for judging the spatial consistency of G(x) and y, E _n is the discriminative expectation of G(x) passing D _S when the size of x is n, E _m is y The discriminator D _S pairs of scale m The discriminant expectation of D _t is the judgment and The discriminator of temporal consistency, F ^j is the output feature map of the jth layer of D _S extracted, For the loss function when extracting F ^j The influence weight coefficient, E _n,j is when the size of x is n to and The spatial consistency expectation compared when F ^j is extracted respectively, n and j are positive integers.

The smoke animation synthesis system of the present invention further includes: a smoke data set generation module, which is used to obtain the three-dimensional smoke data set by solving the Navier-Stokes equation.

In the smoke animation synthesis system of the present invention, the smoke animation rendering module specifically includes: generating the three-dimensional smoke animation through GPU-based multi-view rendering.

The present invention also provides a readable storage medium, which stores executable instructions, and the executable instructions are used to execute the aforementioned data-driven smoke animation synthesis method.

The present invention also proposes a data processing device, which includes the above-mentioned readable storage medium, and the data processing device calls and executes executable instructions in the readable storage medium to synthesize three-dimensional smoke animation.

The smoke animation synthesis method proposed by the present invention makes the smoke information and control mode simpler and more intuitive, maintains the authenticity of the smoke animation under simple input, and can generate realistic smoke animation with user-controllable shapes in real time.

Description of drawings

Fig. 1 is a flow chart of the data-driven smoke animation synthesis method of the present invention.

Fig. 2 is a schematic diagram of the smoke generation model network of the present invention.

Fig. 3 is a schematic diagram of the feature loss of the intermediate result of the present invention.

Fig. 4 is a schematic diagram of the data-driven smoke animation synthesis system of the present invention.

Detailed ways

In order to make the technical solution of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific examples described here are only used to explain the present invention and not limit the present invention.

The present invention aims at the problems of large amount of calculation, poor real-time performance, and single control mode faced by the N-S equation method, and the problem that the input of three-dimensional data is not easy for users to obtain, process and use, and proposes a data-driven smog The animation synthesis method uses a two-dimensional smoke profile to obtain a three-dimensional fluid field with a time series relationship through a smoke generation model, and generates a three-dimensional smoke animation.

Specifically, the smoke animation synthesis method of the present invention includes: generating two-dimensional smoke profile data according to the three-dimensional smoke data set; generating a network through the training of the two-dimensional smoke profile data to obtain a smoke generation model; through the smoke generation model, The two-dimensional smoke profile data is generated as a three-dimensional smoke sequence; the three-dimensional smoke sequence is rendered to generate a three-dimensional smoke animation.

Loss function for the smoke generation model for:

Among them, x is the two-dimensional smoke profile data, G(x) is the three-dimensional smoke sequence A single-frame 3D density field, y is a 3D smoke dataset The single-frame three-dimensional density field of , D _S is the discriminator for judging the spatial consistency of G(x) and y, E _n is the discriminative expectation of G(x) passing D _S when the size of x is n, E _m is y The discriminator D _S pairs of scale m The discriminant expectation of D _t is the judgment and The discriminator of temporal consistency, F ^j is the output feature map of the jth layer of D _S extracted, For the loss function when extracting F ^j The influence weight coefficient, E _n,j is when the size of x is n to and The spatial consistency expectation compared when F ^j is extracted respectively, n and j are positive integers.

In the embodiment of the present invention, the three-dimensional smoke data set is obtained by solving the Navier-Stokes equation.

In the embodiment of the present invention, the 3D smoke sequence is generated into a 3D smoke animation through GPU-based multi-view rendering.

The present invention also proposes a data-driven smoke animation synthesis system, including: a two-dimensional data generation module, used to generate two-dimensional smoke contour data according to a three-dimensional smoke data set; a smoke generation model training module, used to pass the two-dimensional smoke The training generation network of the outline data is used to obtain the smoke generation model; the three-dimensional sequence generation module is used to generate the two-dimensional smoke outline data into a three-dimensional smoke sequence through the smoke generation model; the smoke animation rendering module is used for the three-dimensional smoke The sequence is rendered to generate a 3D smoke animation.

The specific implementation process of the method of the present invention will be introduced below with reference to the accompanying drawings. Figure 1 is a flow chart of a data-driven smoke animation synthesis method. As shown in Figure 1, in order to carry out animation synthesis in realistic complex scenes for smoke, it is necessary to first obtain 3D smoke field data for comparison with real data, and at the same time obtain 2D projection through projection mapping and extract contour information according to the set threshold and Do data enhancement (such as adding noise), build a corresponding smoke generation model and input data to define a loss function suitable for the purpose to obtain a sequence of three-dimensional smoke attribute fields with a time series relationship, and then obtain smoke animation through rendering. Specifically:

1) Through the N-S equation solution to generate a 3D smoke data set in a complex scene with diversity

According to the current N-S equation method, the fluid simulation software mantaflow and parameterized settings are used to obtain the concentration field data sets of diverse and complex scenes.

2) A contour extraction and representation method based on projective transformation is proposed to represent the contour of smoke in a 2D image.

21) Obtain the two-dimensional projection matrix of the smoke field data

Take the orthographic projection of the concentration field as an example, take the left front and lower corner as the origin, and take the directions to the right, upward and into the paper as the positive directions of the X, Y and Z axes respectively, and take the same X and Y coordinates in the three-dimensional concentration field The concentration values of are added to the same plane of Z=0, that is, an orthographic projection is obtained.

22) Define and extract the smoke information contained in the projection as a two-dimensional outline by setting a threshold

The set of points defined to exceed a certain concentration value constitutes the contour of the two-dimensional projection of the smoke. By setting an appropriate concentration value as the threshold value, setting the value exceeding it as 1, and setting the value less than it as 0, the binarized contour information can be obtained.

3) Propose a 3D smoke generation model method based on generative network and autoencoder based on temporal correlation.

Fig. 2 is a schematic diagram of the smoke generation model network of the present invention. As shown in Figure 2, training and adjusting the smoke generation model can obtain a reasonable sequence of smoke density fields with a temporal relationship.

31) Generator loss during network training

is the partial loss representation of the smoke generation model training process, x is the two-dimensional smoke profile data, G(x) is the single-frame three-dimensional density field in the three-dimensional smoke sequence generated by the smoke generation model, D _s is the smoke generation model The representation of the discriminator for judging the spatial consistency between G(x) and the real control single-frame three-dimensional density field y, E _n is the input of the original two-dimensional smoke profile data scale is n when G(x) in the smoke generation model passes through D The representation of the discriminant expectation of _s , in the case of discrete data discussed in the context of this paper, can be understood as the discriminant mean.

By optimizing the score of the three-dimensional smoke samples generated by the generator in the discriminator, the effect of falsehood is improved to obtain a generator that can contain realistic three-dimensional smoke feature distribution.

32) Discriminator loss during network training

is the total representation of the loss during the training process of the smoke generation model, For the smoke generation model, the real data y is used to represent the temporally continuous flow field frame set obtained through fluid dynamics inference, For the smoke generation model, use the generated data G(x) to infer the representation of the time-series continuous convective frame set through fluid dynamics, E _n is the input sample size of n in the smoke generation model The expression of discriminative expectations after D _t can be understood as the discriminative mean in the case of discrete data discussed in the context of this article, and E _m is the discriminator pair when the size of the sample y is m In the case of discrete data discussed in the context of this article, it can be understood as the discriminative mean, and D _t is the judgment of the smoke generation model and Representation of the discriminator for timing consistency, t is a positive integer.

As the generator's confrontation goal, the discriminator's ability to identify real and fake data also needs to be continuously optimized, which indirectly promotes the generator's goal of generating more realistic results.

in By means of convection calculations:

in, To exploit the principles of fluid dynamics using the representation of convective operations, Respectively represent the three-dimensional velocity field of frame ^{t -} 1 and frame t+1 generated by the generator in the smoke generation model. Representation of frame sample data, G(x ^t-1 ), G(x ^t ), G(x ^t+1 ) are the three-dimensional density of frames t-1, t, and t+1 generated by the generator in the smoke generation model, respectively Field, t is a positive integer.

In order to make the generated sequence have temporal continuity, reduce noise and irregular small jitters caused by poor continuity between frames in visual effect, and because the velocity density between the previous and subsequent frames conforms to the law of fluid motion, it can be punished by punishing its neighbors. The inconsistency of flow results between consecutive frames is used to guide the optimization of the model.

33) Feature loss extracted by each layer Fig. 3 is a schematic diagram of the feature loss of the intermediate result of the present invention. As shown in Figure 3:

is the total representation of the multi-layer feature map loss in the smoke generation model during the training process of the smoke generation model, F ^j is the representation of the output feature map of the jth layer of the discriminator D _s that extracts the smoke generation model, It is the expression of the influence weight coefficient on the overall loss function when extracting F ^j , E _{n, j} is the representation of the spatial consistency expectation between the generated results and the real control when extracting F ^j when the input sample size is n, n , j is a positive integer.

The intermediate results often contain many implicit features, so the difference between the generated results and the real data can be calculated using the mirror network (G ^-1 ) of the generator and the difference between the results obtained by extracting some intermediate layers can be used to improve the generation of the model ability.

34) Total loss function for:

in, is the total loss representation of the smoke generation model training process, G(x) is the single-frame 3D density field generated in the smoke generation model, y is the single-frame 3D density field of the real control, D _s is the judgment of the smoke generation model G( x) The expression of the discriminator with the spatial consistency of the real control single-frame three-dimensional density field y, E _n is the expression of the discrimination expectation of G(x) in the smoke generation model after D _s when the input original two-dimensional data scale is n , which can be understood as the discriminant mean in the case of discrete data discussed in the context of this paper, For the smoke generation model, the real data y is used to represent the temporally continuous flow field frame set obtained through fluid dynamics inference, For the smoke generation model, the generated data G(x) is used to represent the temporally continuous pair of flow field frame sets obtained through fluid dynamics inference, and E _m is the discriminator pair when the size of the sample y is m In the case of discrete data discussed in the context of this article, it can be understood as the discriminative mean, and D _t is the judgment of the smoke generation model and The representation of the discriminator of temporal consistency, F ^j is the representation of the output feature map of the jth layer of the discriminator D _s that extracts the smoke generation model, It is the expression of the influence weight coefficient on the overall loss function when extracting F ^j , E _{n, j} is the representation of the spatial consistency expectation between the generated results and the real control when extracting F ^j when the input sample size is n, n , j is a positive integer.

The overall loss function is adjusted by controlling the weights of different losses to make it the most effective guidance for model training and the generator's generation ability is optimal.

4) Through GPU-based multi-view rendering of the generated 3D smoke sequence, a realistic smoke animation is generated.

Combined with the characteristics of GPU (Graphic Processing Unit), the 3D model of the smoke is obtained by using the ray-casting method to render the smoke from the 3D data under multiple perspectives, so as to realize the smoke simulation with good visual effect and high real-time performance.

Fig. 4 is a schematic diagram of the data-driven smoke animation synthesis system of the present invention. As shown in FIG. 4 , the embodiment of the present invention also provides a readable storage medium and a data processing device. The readable storage medium of the present invention stores executable instructions, and when the executable instructions are executed by the processor of the data processing device, the above-mentioned data-driven smoke animation synthesis method is realized. Those of ordinary skill in the art can understand that all or part of the steps in the above method can be completed by instructing related hardware (such as a processor) through a program, and the program can be stored in a readable storage medium, such as a read-only memory, magnetic disk or optical disk, etc. . All or part of the steps in the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module in the above-mentioned embodiment can be implemented in the form of hardware, such as implementing the corresponding functions through an integrated circuit, or can be implemented in the form of software function modules, such as executing programs/instructions stored in the memory by a processor to realize its corresponding functions. Embodiments of the invention are not limited to any specific combination of hardware and software.

Although the present invention has been disclosed above with the embodiments, it is not intended to limit the present invention. Any person of ordinary skill in the art can make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, this The scope of protection of the invention shall be defined by the scope of the appended patent application.

Claims (10)

1. A data-driven smoke animation synthesis method, characterized in that, comprising: Generate two-dimensional smoke profile data according to the three-dimensional smoke data set; generating a network through the training of the two-dimensional smoke profile data to obtain a smoke generation model; Generating the two-dimensional smoke profile data into a three-dimensional smoke sequence through the smoke generation model; Render the 3D smoke sequence to generate a 3D smoke animation. 2. smoke animation synthesis method as claimed in claim 1, is characterized in that, the loss function of this smoke generation model for: Among them, x is the two-dimensional smoke profile data, G(x) is the three-dimensional smoke sequence The single-frame 3D density field of , y is the 3D smoke data set The single-frame three-dimensional density field of , D _S is the discriminator for judging the spatial consistency of G(x) and y, E _n is the discrimination expectation of G(x) passing D _S when the size of x is n, and E _m is y The discriminator D _S pairs of scale m The discriminant expectation of D _t is the judgment and The time series consistent discriminator, F ^j is the output feature map of the jth layer of the extracted D _S , For the loss function when extracting F ^j The influence weight coefficient, E _n,j is when the scale of x is n to and The spatial consistency expectation compared when F ^j is extracted respectively, n and j are positive integers. 3. The smoke animation synthesis method according to claim 1, wherein the three-dimensional smoke data set is obtained by solving the Navier-Stokes equation. 4. The method for synthesizing smoke animation according to claim 1, wherein the three-dimensional smoke animation is generated by GPU-based multi-view rendering. 5. A data-driven smoke animation synthesis system, characterized in that it comprises: A two-dimensional data generation module, configured to generate two-dimensional smoke profile data according to the three-dimensional smoke data set; The smoke generation model training module is used to generate a network through the training of the two-dimensional smoke profile data to obtain a smoke generation model; A three-dimensional sequence generation module, configured to generate the two-dimensional smoke profile data into a three-dimensional smoke sequence through the smoke generation model; The smoke animation rendering module is used to render the 3D smoke sequence to generate a 3D smoke animation. 6. smoke animation synthesis system as claimed in claim 5, is characterized in that, the loss function of this smoke generation model for: Among them, x is the two-dimensional smoke profile data, G(x) is the three-dimensional smoke sequence A single frame of 3D density field, y is the 3D smoke data set The single-frame three-dimensional density field of , D _S is the discriminator for judging the spatial consistency of G(x) and y, E _n is the discriminative expectation of G(x) passing D _S when the size of x is n, E _m is y The discriminator D _S pairs of scale m The discriminant expectation of D _t is the judgment and The discriminator of temporal consistency, F ^j is the output feature map of the jth layer of D _S extracted, For the loss function when extracting F ^j The influence weight coefficient, E _n,j is when the size of x is n to and The spatial consistency expectation compared when F ^j is extracted respectively, n and j are positive integers. 7. The smoke animation synthesis system according to claim 5, further comprising: a smoke data set generation module, which is used to obtain the three-dimensional smoke data set by solving the Navier-Stokes equation. 8 . The smoke animation synthesis system according to claim 5 , wherein the smoke animation rendering module specifically comprises: generating the three-dimensional smoke animation through GPU-based multi-view rendering. 9. A readable storage medium storing executable instructions for executing the data-driven smoke animation synthesis method according to any one of claims 1-4. 10. A data processing device, comprising the readable storage medium according to claim 9, the data processing device retrieves and executes the executable instructions in the readable storage medium to synthesize three-dimensional smoke animation.