CN108932742B - A real-time rendering method of large-scale infrared terrain scene based on remote sensing image classification - Google Patents

Abstract

The invention discloses a large-scale infrared terrain scene real-time rendering method based on remote sensing image classification, and belongs to the technical field of infrared physics and virtual reality. The method first establishes a terrain classification model; establishes a simplified terrain radiation model, calculates the terrain surface temperature values of the different terrains, and converts the calculated terrain surface temperature values of the different terrains into color data according to the fragment shader. Infrared noise to achieve infrared texture rendering; finally, based on the cfMMOC model, the infrared characteristics of large-scale terrain are rendered in real time. The rendering process of the present invention is based on the cfMMOC model and the shader program, has high rendering efficiency, and can realize real-time rendering of the infrared characteristics of the global terrain under the condition of low resource consumption. The frame rate of the whole rendering process can be stabilized at more than 55 frames, which meets the real-time rendering conditions.

Description

A real-time rendering method of large-scale infrared terrain scene based on remote sensing image classification

technical field

The invention belongs to the technical fields of infrared physics and virtual reality, and relates to a real-time rendering method for large-scale infrared terrain scenes based on remote sensing image classification, in particular to a large-scale method for classifying different features of large-scale terrain surfaces based on visible light terrain remote sensing images. A real-time rendering method for infrared terrain scenes.

Background technique

Infrared imaging simulation includes background imaging simulation and target imaging simulation. Target imaging simulation refers to objects of interest in the simulation area, such as vehicles, buildings, etc., and the background is a scene other than the target. No target can be isolated from the background. It exists alone, and the large-scale terrain is a very typical background. Terrain infrared characteristics are a very important part of the entire infrared terrain scene. Especially for infrared target recognition and infrared stealth applications that care about the infrared characteristics of terrain, accurate and real-time rendering of infrared terrain scenes is particularly important. Generally speaking, to meet the real-time rendering requirements, the rendering results need to be stable at 30 frames or more. Real-time rendering of large-scale infrared terrain scenes involves disciplines such as heat transfer, infrared physics, pattern recognition, computer graphics and images, and is a typical multidisciplinary problem.

SUMMARY OF THE INVENTION

Aiming at the problem that large-scale infrared terrain scenes cannot be rendered in real time in the prior art, the present invention proposes a real-time rendering method for large-scale infrared terrain scenes based on remote sensing image classification, focusing on realizing the infrared characteristic rendering function in large-scale terrain rendering, The rendering stability has been improved on the premise of ensuring the credibility of the simulation. The method of the invention first establishes a terrain classification model, and classifies the surface features of the images of different terrains in the visible light large-scale terrain remote sensing images; simplifies the surface radiation model, and calculates the terrain surface temperature of each different terrain from the simplified surface radiation model. According to the fragment shader (shader), the calculated terrain surface temperature values of different terrains are converted into color data, and infrared noise is superimposed on the color data to realize infrared texture rendering; framework of multiresolution management and occlusion culling for out-of-core hierarchical terrain rendering) model for real-time rendering of infrared properties of large-scale terrain.

The method for real-time rendering of large-scale infrared terrain scenes based on remote sensing image classification provided by the present invention specifically includes the following steps:

Step 1. Establish a terrain classification model to classify visible light large-scale terrain remote sensing images;

Step 101, obtaining a large-scale topographic remote sensing image in the visible light band;

Step 102: Select the surface material feature vector in the large-scale topographic remote sensing image.

According to the surface features of different terrains in the large-scale terrain remote sensing image, extract the surface material feature vector, the surface features include color features and texture features, select the hue, saturation and lightness in the color features as the color feature vectors, select the texture features Energy, entropy, moment of inertia, and standard deviation are used as texture feature vectors. The surface material feature vector includes a color feature vector and a texture feature vector.

Step 103: Generate corresponding training pictures and label data according to the surface material feature vector in the remote sensing image, and select a test picture for testing. The label data refers to terrain types, including five types: bare land, grassland, forest, dry forest and water body.

Step 104: Select the support vector machine as the terrain type determination basis, classify the surface material feature vector of the training image, and use the test image as training data to train the terrain classification model to establish a terrain classification model. The terrain classification model is applied to remote sensing images, and the terrain classification results of all 15 levels of remote sensing images are obtained.

Step 2: Simplify the surface radiation balance equation to obtain a simplified surface radiation model;

The process of simplifying the surface radiation model is as follows:

A: In solar radiation, the influence of reflection on radiation balance is ignored; B: In inward radiation, the law of temperature distribution is simplified: it is assumed that the temperature at 0.5m below the surface is constant and the temperature changes uniformly from the surface down; C. For The surface energy balance equation affects the empirical values of parameters that are not directly or have relatively stable values.

Step 3: Based on the cfMMOC model, infrared rendering of large-scale terrain.

The advantages of the present invention are:

(1) Based on visible light terrain remote sensing images and environmental data commonly used by weather stations, the data are relatively easy to obtain.

(2) Calculate the surface temperature of bare land, grassland, dry forest, green forest and water body in real time according to the environmental data while ensuring the calculation accuracy.

(3) The rendering process is based on the cfMMOC model and shader program, which has high rendering efficiency and can realize real-time rendering of the infrared characteristics of the global terrain with low resource consumption.

Description of drawings

Fig. 1 is that the present invention establishes the flow chart of terrain classification model;

Fig. 2 is the infrared texture real-time generation method of the present invention;

Fig. 3 is the structure comparison diagram of the surface radiation model before and after the simplification of the present invention;

Figure 4 is a large-scale terrain rendering framework diagram of cfMMOC;

Figure 5 is the result of large-scale terrain infrared rendering.

Detailed ways

The specific implementation method of the present invention will be described in detail below with reference to the accompanying drawings.

Aiming at the problem that the large-scale infrared terrain scene cannot be rendered in real time in the prior art, the present invention proposes a real-time rendering method for a large-scale infrared terrain scene based on a visible light large-scale terrain remote sensing image to classify different features of the large-scale terrain surface, The method first classifies different features of different terrain surfaces based on visible light large-scale terrain remote sensing images, and establishes a terrain classification model; establishes a surface radiation model according to the different terrains, and calculates the different terrains according to the surface energy balance equation respectively. The terrain surface temperature value of the terrain, according to the fragment shader (shader) will calculate the terrain surface temperature value of each different terrain for infrared rendering; finally, based on the unified rendering framework of out-of-core terrain, cfMMOC (A consolidated framework of multiresolution management and occlusionculling for out -of-core hierarchical terrain rendering) model for real-time rendering of large-scale infrared terrain scenes. The method specifically includes the steps:

Step 1: Based on the visible light large-scale terrain remote sensing image, classify each terrain surface, establish a terrain classification model, and provide support for the calculation of infrared characteristics; as shown in Figure 1, the specific steps are as follows:

Step 101: Obtain a large-scale topographic remote sensing image in the visible light band, and divide it into a data set of 15 LOD (Levels of Detail) from 0 to 14 according to the resolution fineness;

Step 102 , respectively selecting surface features of images of different terrains in the large-scale terrain remote sensing image as surface material feature vectors.

The described terrain types are set as five types of terrains: bare land, grassland, forest, dry forest and water body, so the supervised learning method is used to identify and classify these five terrains. The surface features include color features and texture features. The color features include hue, saturation, and lightness, each with 10 dimensions as the color feature vector, and the texture features include energy, entropy, moment of inertia, and standard deviation. A total of 8 dimensions are used as the texture feature vector. , which constitutes a total of 38-dimensional surface material feature vectors, including 30-dimensional color feature vectors and 8-dimensional texture feature vectors.

Step 103: Select the most refined level (14 level) remote sensing image in the remote sensing image data set as the training data in the supervised learning, and set the label data. The label data refers to terrain types, including five types: bare land, grassland, forest, dry forest and water body.

Step 104: Randomly select a certain amount of remote sensing images as a test data set for testing the training effect, adopt the support vector machine method (Support Vector Machine) method in the machine learning integration method to perform classification learning, and select a Gaussian kernel (RBF) for the kernel function, In the hyperparameters, the Gaussian kernel coefficient is selected as 0.43, and the penalty coefficient is selected as 1.0. Iterative training is performed on the test dataset to obtain terrain classification results. When the terrain classification results meet the classification requirements (the Kappa coefficient is greater than 0.8), the model training process is completed, and the terrain classification model is obtained. The terrain classification model is applied to remote sensing images, and the terrain classification results of all 15 levels of remote sensing images are obtained.

Step 2: Simplify the surface radiation balance equation to obtain a simplified surface radiation model;

The surface radiation model is established according to the surface radiation balance equation, which is mainly divided into six parts: inward radiation Q _Mg , sensible heat flux Q _H , latent heat flux Q _LE , solar radiation Q _sun , atmospheric radiation Q _sky and outward radiation Q _G , there is the following surface energy balance equation:

Among them, k _λ is the thermal conductivity of the surface material, S is the surface area, n is the normal direction of the surface, and T is the temperature.

In order to improve the operation efficiency, the present invention simplifies the surface radiation model, as shown in Figure 3, the process is as follows:

A: In solar radiation, the influence of reflection on radiation balance is ignored; B: In inward radiation, the law of temperature distribution is simplified: it is assumed that the temperature at 0.5m below the surface is constant and the temperature changes uniformly from the surface down; C. For Appropriate empirical values should be set for parameters that are not directly affected by the surface energy balance equation or whose values are relatively stable, as shown in Table 1 below. The simplified parts described in A and B are marked with the dotted box in Figure 3; the simplified surface radiation model, The local time, ambient temperature, wind speed and relative humidity are received as input data, and the surface temperature values corresponding to the five terrains are output. After measurement, a single calculation of the temperature value of the terrain surface takes about 1.5 milliseconds, which provides a good guarantee for the real-time rendering performance of the present invention.

Table 1 parameter empirical value

Step 3: Based on the cfMMOC large-scale terrain rendering model and the shader-based infrared texture generation in real time, the infrared characteristics of the large-scale terrain are rendered in real time, as shown in Figure 5;

Specific steps are as follows:

Step 301: Obtain the elevation data of the 15-level LOD remote sensing image, and convert the elevation data into three-dimensional mesh data describing the terrain, and perform structured division according to the requirements of the cfMMOC model, and store different levels according to the quad-tree structure. Remote sensing imagery with 3D mesh Mesh data.

Step 302, realizing real-time dynamic rendering of infrared textures based on shaders;

As shown in Figure 2, the shader control script is first written based on the Cg language, and the terrain classification results obtained in step 1 are input into the fragment shader material input interface; then the local environment data obtained from the weather station is input into the surface radiation model, Calculate the terrain surface temperature value in the remote sensing image, and input the terrain surface temperature value into the fragment shader value input interface.

The local environmental data includes local time, ambient temperature, wind speed and relative humidity.

Input the corresponding relationship between grayscale and temperature or the corresponding relationship between infrared pseudo-color and temperature into the fragment shader in the form of an image, write the corresponding code to realize the conversion of temperature to color data/grayscale data, and superimpose infrared noise on the conversion result, and finally Real-time dynamic rendering of terrain surface temperature values to infrared textures.

Step 303 , based on the cfMMOC model, read the infrared texture data obtained in step 302 to realize real-time rendering of a three-dimensional infrared scene of a large-scale terrain remote sensing image.

The rendering of the cfMMOC model is divided into two parts: the pre-process and the post-process. As shown in Figure 4, the renderer in the former process sends the viewpoint information in the original window to the renderer in the latter process, and the renderer in the latter process implements pixel calculation in the small window, and the pixel calculation result reflects the visibility information. The visibility information is sent to the terrain block management unit of the previous process, and the terrain block management unit of the previous process feeds back the terrain block status to the terrain block management unit of the later process, and the terrain block management unit of the latter process judges The terrain resource is in the loading state of the previous process, and combined with the terrain block state data, the terrain block loader of the forward process sends data loading/unloading request information. After the previous process obtains the terrain data loading/unloading request information, the terrain grid data is updated in the original window, and the real-time 3D infrared scene rendering is performed in combination with the infrared texture data generated in real time in step 302 . In this process, the terrain block data is loaded through the terrain block loaders of the front and rear processes respectively.

The rendering results realized based on the above steps are shown in Figure 5. In Figure 5, A to D are the infrared scene renderings at different levels in the previous process, and E to H in Figure 5 are schematic diagrams of the scheduling of different levels of terrain in the latter process. It can be seen that the present invention realizes real-time rendering of large-scale infrared terrain scenes. The frame rate of the whole rendering process can be stabilized at more than 55 frames, which meets the real-time rendering conditions.

Claims (1)

1. A large-scale infrared terrain scene real-time rendering method based on remote sensing image classification specifically comprises the following steps:

establishing a terrain classification model, and carrying out terrain classification on a visible light large-scale terrain remote sensing image;

step two, establishing a simplified topographic radiation model;

selecting five terrains, establishing equilibrium equations of solar radiation, atmospheric radiation, earth surface outward radiation, latent heat flux, sensible heat flux and earth surface downward radiation according to an energy equilibrium equation, and establishing an earth surface radiation model; simplifying the earth surface radiation model, and the process is as follows:

a: neglecting the effect of surface reflections on ambient radiation;

b: assuming that the temperature 0.5m below the surface is constant and varies uniformly from the surface to the bottom;

C. setting an empirical value for parameters which are not directly influenced by the earth surface energy balance equation or have stable numerical values;

step three: generating infrared textures of a shader in real time based on the cfMMOC large-scale terrain rendering model, and rendering infrared characteristics of large-scale terrain in real time;

the method is characterized in that:

the specific process of the step one is as follows:

step 101, acquiring a large-scale terrain remote sensing image under a visible light wave band, and dividing the large-scale terrain remote sensing image into data sets of 15 levels of LOD (level of detail) of 0-14 according to the resolution fineness;

102, selecting a surface material characteristic vector in the large-scale terrain remote sensing image;

the surface material characteristic vector comprises a color characteristic vector and a texture characteristic vector;

the color feature vector comprises 10 dimensions of hue, saturation and lightness;

the texture feature vector comprises 8 dimensions of energy, entropy, moment of inertia and standard deviation;

103, selecting a 14-level remote sensing image in a remote sensing image data set as training data in supervised learning, generating corresponding training pictures and label data, and selecting test pictures for testing;

the label data refers to terrain types including five types of bare land, grassland, forest, withered forest and water body;

104, classifying and learning the training pictures by adopting a support vector machine method in a machine learning integration method, wherein a kernel function adopts a Gaussian kernel, the Gaussian kernel coefficient in the hyperparameter is selected to be 0.43, and a penalty coefficient is selected to be 1.0;

performing iterative training on the test data set to obtain a terrain classification result;

when the Kappa coefficient of the terrain classification result is larger than 0.8, completing a model training process to obtain a terrain classification model;

applying the terrain classification model to the remote sensing images to obtain terrain classification results of all the remote sensing images with 15 levels;

the third specific process is as follows:

301, acquiring elevation data of a 15-level LOD remote sensing image, converting the elevation data into three-dimensional grid Mesh data describing a terrain, performing structured division according to the requirements of a cfMMOC model, and storing remote sensing images and three-dimensional grid Mesh data of different levels according to a quadtree structure;

step 302, realizing real-time dynamic rendering of the infrared texture based on a shader;

inputting the terrain classification result obtained in the first step into a fragment shader material input interface; selecting local time, environment temperature, wind speed and relative humidity from meteorological station data as input data of a simplified earth surface radiation model, calculating a terrain surface temperature value in a remote sensing image, and inputting the terrain surface temperature value into a numerical value input interface of a fragment shader;

the fragment shader converts the terrain surface temperature value into color data or gray data according to the corresponding relation between gray and temperature or infrared pseudo color, and infrared noise is superposed on the color data or gray data to realize real-time dynamic rendering from the terrain surface temperature value to infrared texture;

step 303, reading the infrared texture data obtained in the step 302 based on the cfMMOC model to realize real-time rendering of the three-dimensional infrared scene of the large-scale terrain remote sensing image;

the rendering of the cfMMOC model is divided into a front process and a rear process;

the renderer in the front process sends the viewpoint information in the original window to the renderer in the back process;

a renderer in the later process realizes pixel calculation in a small window, the pixel calculation result embodies visibility information, and the visibility information is sent to a terrain block management unit of the former process;

the terrain block management unit of the front process feeds the terrain block state back to the terrain block management unit of the back process;

the terrain block management unit of the back process judges the loading state of the terrain resource in the front process and sends data loading/unloading request information to a terrain block loader of the front process by combining terrain block state data;

after acquiring the topographic data loading/unloading request information, the front process updates topographic grid data in the original window, and performs real-time three-dimensional infrared scene rendering by combining the infrared texture data generated in step 302 in real time;

in the process, the terrain block data are loaded through terrain block loaders in front and back processes respectively.

Images