CN109598771B - Terrain synthesis method of multi-landform feature constraint - Google Patents

Abstract

The invention discloses a topography synthesis method of multi-relief feature constraint, which utilizes a topography data block of a Digital Elevation Model (DEM) and adopts a deep learning strategy combining a conditional variation self-encoder and a generation type countermeasure network, so that a realistic topography with the multi-relief feature constraint can be customized according to a sketch input by a user.

Description

Terrain synthesis method of multi-landform feature constraint

Technical Field

The invention relates to the technical field of virtual simulation, in particular to a terrain synthesis method with multiple landform feature constraints, which utilizes a terrain data block of a digital elevation model (Digital Elevation Mode, DEM for short), adopts a condition variation self-encoder to automatically encode a terrain image, and utilizes a generation type countermeasure network to learn the terrain feature; when the topography is synthesized, according to the sketch of the user by hand, and by combining with the input elevation data, a pre-trained deep learning network is utilized to automatically synthesize the customized topography with multiple topography features corresponding to the sketch of the user.

Background

The terrain synthesis technology has wide value in practical simulation application, and can better improve the use experience of users in natural disaster prevention or video game creation. Current terrain synthesis techniques can be broadly divided into the following categories: modeling methods based on procedural, modeling methods based on physical erosion, modeling methods based on user sketch, and synthetic strategies based on deep learning. In recent years, with the development of artificial intelligence technology, a topography synthesis method based on a convolutional neural network appears, depth information prediction is carried out on a mountain contour line input by a user, and the rationality of synthesized topography is improved. However, the method for synthesizing the terrain by using the deep learning method has the main problems that the structure of the network is complex, the training of network parameters is difficult to converge, and the problems are all existing in the current intelligent terrain synthesis research.

Disclosure of Invention

Aiming at the defects of the prior art and aiming at the practical problems in terrain synthesis, the invention provides a terrain synthesis method of multiple landform features. The method has the characteristics of simplicity and effectiveness, and can synthesize terrains with multiple landform features according to the sketch of the hand painting of the user.

The specific technical scheme for realizing the aim of the invention is as follows: a topography synthesis method of multi-relief feature constraint is characterized by comprising the following specific steps:

step 1: constructing a dataset

(1) Preparing elevation data blocks

The elevation data block J of the WGS84 coordinate system is downloaded from the SRTM website http:// srtm.csi.cgirar.org and stored in tiff format, the spatial resolution of which is between 90m x 90m and 200m x 200m, and the height of any point A of J is recorded as H _A And establishing a gray image G according to the height information of J: the highest point corresponds to white, the lowest point corresponds to black, and the interpolation is gray between the highest point and the lowest point; g resolution of N _t ×N _t ，N _t 256, 512 or 1024; the number of pixels in G is k=n _t ×N _t ；

(2) Constructing a multi-relief feature skeleton image B, wherein the multi-relief feature skeleton image B comprises river networks, ridge lines and geometric feature points: calculating three types of landform features of G by using a D8 algorithm and a 3X 3 window, specifically calculating a ridge line by using the D8 algorithm according to the gray value of G, subtracting each pixel gray value of G by using the maximum value of pixel gray in G to obtain a reverse image, and calculating by using the D8 algorithm to obtain a mark of a river network; the geometric feature point marking method comprises the following steps: accumulating absolute values of height differences between the height of A and the surrounding 8 neighborhood points, wherein the difference is larger than a threshold value theta (more than or equal to 400 and less than or equal to 1000), and marking the difference as a geometric feature point; different geomorphic features are marked by different colors: blue marked river network, red marked ridge line, greenThe point marks geometric feature points, the rest background part in B is represented by black, and the resolution of B is N _t ×N _t ，N _t 256, 512 or 1024;

(3) B and G form a data pair, and a data set S (Q is more than or equal to 1000 and less than or equal to 1500) formed by Q data pairs is established; step 2: network topology design

The network topology structure is composed of a condition variation automatic encoder and generator network, and the design method comprises the following steps:

(1) The automatic condition variation encoder is realized by adopting U-net coding, and the input is an image B and the shape is N _t ×N _t X 3, the output is a high-dimensional feature Z, the shape is 1×1×512, L is adopted _t (8≤L _t Less than or equal to 10) architecture of layer convolutional neural network, if N _t 256, L _t 8; if N _t 512, L _t 9; if N _t Is 1024L _t 10; the encoder has L _t Layer substructures, each of which is a 1-layer convolution: adopting a 4 multiplied by 4 convolution kernel, wherein the convolution step length is 2, carrying out neighborhood zero padding processing outside the boundary of an input image, carrying out batch normalization on the output of each convolution layer, and using a leakage Relu activation function; the number of the convolution kernels of the layer 1 is T _k When N _t 256, T _k Taking 64; when N is _t 512, T _k Taking 32; when N is _t Is 1024, T _k Taking 16; the number of convolution kernels of the layer 2 and the subsequent layers is doubled in turn until the layer L _t A layer;

(2) The generator network is L _t Each layer of substructure consists of a transposed convolution layer and a connecting layer; the convolution kernel of the transposed convolution layers is designed to be 4 multiplied by 4, the convolution step length is 2, each transposed convolution layer carries out batch normalization, and the activation function uses Relu; the number of convolution kernels of the 1 st layer convolution is T _k The number of convolution kernels of the layer 2 and the subsequent layers is reduced by half in turn until the layer L _t A layer; at L _t After the layer, the input Z is restored to N using an additional transposed convolution layer _t ×N _t X 3; the last layer of the generator is connected with a Tanh activation layer, and the network output is required to be a floating point number between-1 and 1;

step 3: training of neural networks

When the neural network is trained, a supervision method is adopted, a data set S is used, 100 rounds of training are performed, an NS GAN method is adopted to calculate a loss term of a discriminator during training, the definition of a loss function l of the network is composed of five items of a discrimination loss term c, a divergence loss term d, a consistency loss e of a generated terrain gray image, an antagonism loss term n and a skeleton structure loss term g, and the loss function l is defined as follows: l=c+λ ₁ d+λ ₂ e+λ ₃ f+λ ₄ g, wherein lambda _i (i=1, 2,3, 4) is the weight parameter of the corresponding loss, 1.0.ltoreq.lambda ₁ ≤5.0，1.0≤λ ₂ ≤5.0，0.0001≤λ ₃ ≤0.01，0.1≤λ ₄ ≤2.0；

c. d, e, n, g loss terms are defined as follows:

(1) c is defined as: c=e _x∈f(X) [log(D(x))]+E _G(z)∈Y [log(1-D(G(z)))]Wherein f (X) represents a gray scale image set of the network-output multi-relief feature topography; x represents the terrain supervision gray image of the sample, and the D (x) function represents the probability that the sample x is true; z is a high-dimensional vector of the output from the encoder by the conditional variation, Y represents the sample terrain supervisory gray image dataset, G (z) represents the terrain gray image generated by z, D (G (z)) represents the probability that the generated terrain gray image is false, E represents the energy loss function;

(2) d is defined as

Wherein μ and ε are the mean and covariance of Z, e is the base of the natural logarithm;

(3) e is defined as

Wherein x represents a sample topography supervision gray scale image, < >>

For generating a topographic gray image, f _D A feature map input for the full-connection convolution layer in the discriminator;

(4) The loss n is defined as

(5) Loss g is defined as

Wherein B is B, < >>

Is the skeleton of the network-generated topographic image;

the super parameters used by the neural network include: dropout rate takes 0.5, momentum beta using Adam optimizer ₁ =0.5, the number of samples per batch is 1, and the learning rates of the generator network and the arbiter network are all 0.002;

step 4: terrain synthesis

Drawing a sketch image V by drawing software to be used as a hand-drawn sketch, inputting the V into a network, inputting the V into a convolutional neural network N, predicting by using trained parameters in the N network, outputting a corresponding terrain gray image, and further obtaining a customized terrain synthesis result according to the maximum value and the minimum value of the height of the input terrain.

The invention has the characteristics of simplicity and practicability, and can synthesize the terrain customized by the user according to the sketch of the user by hand drawing and combining with the input elevation data.

Drawings

FIG. 1 is a graph of the results of the synthetic terrain of the present invention;

fig. 2 is a three-dimensional view of the synthetic terrain of the present invention.

Detailed Description

The invention will be further described with reference to the accompanying drawings and examples.

Examples

The embodiment is implemented under Windows 10-bit operating system on PC, and the hardware configuration is a processor

Core ^TM i5-7500 3.4GHz CPU,8GB memory, wherein the software environment is Matlab 2015b, and Python is adopted for programmingIn other words, the open source library OpenCV 2.4.4 and the open source grid space data conversion library GDAL are combined.

The embodiment specifically comprises the following steps:

step 1: constructing a dataset

(1) Preparing elevation data blocks

The elevation data block J of the WGS84 coordinate system is downloaded from the SRTM website http:// srtm.csi.cgirar.org and stored in tiff format, the spatial resolution of which is between 90m x 90m and 200m x 200m, and the height of any point A of J is recorded as H _A And establishing a gray image G according to the height information of J: the highest point corresponds to white, the lowest point corresponds to black, and the interpolation is gray between the highest point and the lowest point; g resolution of N _t ×N _t ，N _t 256; the number of pixels in G is k=n _t ×N _t ；

(2) Constructing a multi-relief feature skeleton image B, wherein the multi-relief feature skeleton image B comprises river networks, ridge lines and geometric feature points: calculating three types of landform features of G by using a D8 algorithm and a 3X 3 window, specifically calculating a ridge line by using the D8 algorithm according to the gray value of G, subtracting each pixel gray value of G by using the maximum value of pixel gray in G to obtain a reverse image, and calculating by using the D8 algorithm to obtain a mark of a river network; the geometric feature point marking method comprises the following steps: accumulating absolute values of height differences between the height of A and the surrounding 8 neighborhood points, wherein the difference is larger than a threshold value theta (600 is taken), and marking the difference as a geometric feature point; different geomorphic features are marked by different colors: blue mark river network, red mark ridge line, green mark geometric characteristic point, B the rest background part is represented by black, B resolution is N _t ×N；

(3) A data pair is formed of B and G, and a data set S is created of 1250 data pairs.

Step 2: network topology design

The network topology structure N is composed of a condition variation automatic encoder and generator network, and the design method comprises the following steps: (1) The automatic condition variation encoder is realized by adopting U-net coding, and the input is an image B and the shape is N _t ×N _t X 3, the output is a high-dimensional feature Z, the shape is 1×1×512, L is adopted _t Architecture of layer convolutional neural network, L _t 8; the encoder has L _t Layer substructures, each of which is a 1-layer convolution: adopting a 4 multiplied by 4 convolution kernel, wherein the convolution step length is 2, carrying out neighborhood zero padding processing outside the boundary of an input image, carrying out batch normalization on the output of each convolution layer, and using a leakage Relu activation function; the number of the convolution kernels of the layer 1 is T _k ，T _k Taking 64; the number of convolution kernels of the layer 2 and the subsequent layers is doubled in turn until the layer L _t A layer; (2) The generator network is L _t Each layer of substructure consists of a transposed convolution layer and a connecting layer; the convolution kernel of the transposed convolution layers is designed to be 4 multiplied by 4, the convolution step length is 2, each transposed convolution layer carries out batch normalization, and the activation function uses Relu; the number of convolution kernels of the 1 st layer convolution is T _k The number of convolution kernels of the layer 2 and the subsequent layers is reduced by half in turn until the layer L _t A layer; at L _t After the layer, the input Z is restored to N using an additional transposed convolution layer _t ×N _t X 3. The last layer of the generator is connected with a Tanh activation layer, and the network output is required to be a floating point number between-1 and 1.

Step 3: training of neural networks

When the network N is trained, a supervision method is adopted, a data set S is used, 100 rounds of training are performed, an NS GAN method is adopted to calculate a loss term of a discriminator during training, the definition of a loss function l of the network is composed of five items of a discrimination loss term c, a divergence loss term d, a consistency loss e of a generated terrain gray image, an antagonism loss term N and a skeleton structure loss term g, and the loss function l is defined as follows: l=c+λ ₁ d+λ ₂ e+λ ₃ f+λ ₄ g, wherein lambda _i (i=1, 2,3, 4) is the weight parameter of the corresponding loss, λ ₁ ＝2.0，λ ₂ ＝2.0，λ ₃ ＝0.001，λ ₄ ＝1.0；

c. d, e, n, g loss terms are defined as follows:

(1) c is defined as: c=e _x∈f(X) [log(D(x))]+E _G(z)∈Y [log(1-D(G(z)))]Wherein f (X) represents a gray scale image set of the network-output multi-relief feature topography; x represents the topography supervised gray scale image of the sample, and the D (x) function represents the sample x asProbability of true; z is a high-dimensional vector of the output from the encoder by the conditional variation, Y represents the sample terrain supervisory gray image dataset, G (z) represents the terrain gray image generated by z, D (G (z)) represents the probability that the generated terrain gray image is false, E represents the energy loss function;

(2) d is defined as

Wherein μ and ε are the mean and covariance of Z, e is the base of the natural logarithm;

(3) e is defined as

Wherein x represents a sample topography supervision gray scale image, < >>

For generating a topographic gray image, f _D A feature map input for the full-connection convolution layer in the discriminator;

(4) The loss n is defined as

(5) Loss g is defined as

Wherein B is B, < >>

Is the skeleton of the network-generated topographic image;

the super parameters used by the neural network include: dropout rate takes 0.5, momentum β1=0.5 is used by Adam optimizer, sample number of each batch takes 1, and learning rate of generator network and discriminator network takes 0.002;

step 4: terrain synthesis

Drawing a sketch image V by drawing software to be used as a hand-drawn sketch, inputting the V into a network, inputting the V into a convolutional neural network N, predicting by using trained parameters in the N network, outputting a corresponding terrain gray image, and further obtaining a customized terrain synthesis result according to the maximum value and the minimum value of the height of the input terrain.

Fig. 1 is a sketch of a user, and shows the result of synthesizing a terrain by using a DEM terrain sample, wherein the first column is a multi-landform feature sketch from left to right, and the second column is the result of synthesizing a multi-landform feature terrain by using the method of the invention.

Fig. 2 is a three-dimensional view of a synthetic topography, from left to right in the figure, the first column is a user sketch of a multi-topography feature, the second column is a multi-topography result synthesized by the present invention, and the third column is a three-dimensional view of a synthetic multi-topography feature topography, and from the figure, it can be seen that the method of the present invention can synthesize a result of a topography ridge, a river multi-topography feature with three-dimensional realism.

Claims (1)

1. A topography synthesis method of multi-relief feature constraint is characterized in that a network architecture combining a conditional variation self-encoder and a generation type countermeasure network is adopted to realize the realistic topography synthesis of multi-relief elements, and the method specifically comprises the following steps:

step 1: constructing a dataset

(1) Preparing elevation data blocks

The elevation data block J of the WGS84 coordinate system is downloaded from the SRTM website http:// srtm.csi.cgirar.org and stored in tiff format, the spatial resolution of which is between 90m x 90m and 200m x 200m, and the height of any point A of J is recorded as H _A And establishing a gray image G according to the height information of J: the highest point corresponds to white, the lowest point corresponds to black, and the interpolation is gray between the highest point and the lowest point; g resolution of N _t ×N _t ，N _t 256, 512 or 1024; the number of pixels in G is k=n _t ×N _t ；

(2) Constructing a multi-relief feature skeleton image B, wherein the multi-relief feature skeleton image B comprises river networks, ridge lines and geometric feature points: three kinds of landform features of G are calculated using a 3 x 3 window using a D8 algorithm, specifically, according to the gray value of G,calculating a ridge line by using a D8 algorithm, subtracting the gray value of each pixel of G by using the maximum value of the gray values of the pixels of G to obtain a reverse image, and calculating by using the D8 algorithm to obtain a mark of the river network; the geometric feature point marking method comprises the following steps: accumulating absolute values of height differences between the height of the A and the surrounding 8 neighborhood points, wherein the difference is larger than a threshold value theta, theta is more than or equal to 400 and less than or equal to 1000, and the difference is marked as a geometric feature point; different geomorphic features are marked by different colors: blue mark river network, red mark ridge line, green mark geometric characteristic point, B the rest background part is represented by black, B resolution is N _t ×N _t ，N _t 256, 512 or 1024;

(3) B and G form a data pair, and a data set S formed by Q data pairs is established, wherein Q is more than or equal to 1000 and less than or equal to 1500;

step 2: network topology design

The network topology structure is composed of a condition variation automatic encoder and generator network, and the design method comprises the following steps:

(1) The automatic condition variation encoder is realized by adopting U-net coding, and the input is an image B and the shape is N _t ×N _t X 3, the output is a high-dimensional feature Z, the shape is 1×1×512, L is adopted _t Architecture of layer convolutional neural network, wherein 8.ltoreq.L _t Less than or equal to 10; if N _t 256, L _t 8; if N _t 512, L _t 9; if N _t Is 1024L _t 10; the encoder has L _t Layer substructures, each of which is a 1-layer convolution: adopting a 4 multiplied by 4 convolution kernel, wherein the convolution step length is 2, carrying out neighborhood zero padding processing outside the boundary of an input image, carrying out batch normalization on the output of each convolution layer, and using a leakage Relu activation function; the number of the convolution kernels of the layer 1 is T _k When N _t 256, T _k Taking 64; when N is _t 512, T _k Taking 32; when N is _t Is 1024, T _k Taking 16; the number of convolution kernels of the layer 2 and the subsequent layers is doubled in turn until the layer L _t A layer;

(2) The generator network is L _t Each layer of substructure consists of a transposed convolution layer and a connecting layer; a convolution kernel of the transposed convolution layer, designed to4×4, the convolution step length is 2, each transposed convolution layer is subjected to batch normalization, and the activation function uses Relu; the number of convolution kernels of the 1 st layer convolution is T _k The number of convolution kernels of the layer 2 and the subsequent layers is reduced by half in turn until the layer L _t A layer; at L _t After the layer, the input Z is restored to N using an additional transposed convolution layer _t ×N _t X 3; the last layer of the generator is connected with a Tanh activation layer, and the network output is required to be a floating point number between-1 and 1;

step 3: training of neural networks

When the neural network is trained, a supervision method is adopted, a data set S is used, 100 rounds of training are performed, an NS GAN method is adopted to calculate a loss term of a discriminator during training, the definition of a loss function l of the network is composed of five items of a discrimination loss term c, a divergence loss term d, a consistency loss e of a generated terrain gray image, an antagonism loss term n and a skeleton structure loss term g, and the loss function l is defined as follows: l=c+λ ₁ d+λ ₂ e+λ ₃ f+λ ₄ g, wherein lambda _i I=1, 2,3,4 for the weight parameter corresponding to the penalty; lambda is more than or equal to 1.0 ₁ ≤5.0，1.0≤λ ₂ ≤5.0，0.0001≤λ ₃ ≤0.01，0.1≤λ ₄ ≤2.0；

c. d, e, n, g loss terms are defined as follows:

(1) c is defined as: c=e _x∈f(X) [log(D(x))]+E _G(z)∈Y [log(1-D(G(z)))]Wherein f (X) represents a gray scale image set of the network-output multi-relief feature topography; x represents the terrain supervision gray image of the sample, and the D (x) function represents the probability that the sample x is true; z is a high-dimensional vector of the output from the encoder by the conditional variation, Y represents the sample terrain supervisory gray image dataset, G (z) represents the terrain gray image generated by z, D (G (z)) represents the probability that the generated terrain gray image is false, E represents the energy loss function;

(2) d is defined as

Wherein μ and ε are the mean and covariance of Z, e is the base of the natural logarithm;

(3) e is defined as

Wherein x represents a sample topography supervision gray scale image, < >>

For generating a topographic gray image, f _D A feature map input for the full-connection convolution layer in the discriminator;

(4) The loss n is defined as

(5) Loss g is defined as

Wherein B is B, < >>

Is the skeleton of the network-generated topographic image;

the super parameters used by the neural network include: dropout rate takes 0.5, momentum beta using Adam optimizer ₁ =0.5, the number of samples per batch is 1, and the learning rates of the generator network and the arbiter network are all 0.002;

step 4: terrain synthesis

Drawing a sketch image V by drawing software to be used as a hand-drawn sketch, inputting the V into a network, inputting the V into a convolutional neural network N, predicting by using trained parameters in the N network, outputting a corresponding terrain gray image, and further obtaining a customized terrain synthesis result according to the maximum value and the minimum value of the height of the input terrain.

Images