CN108764039A - Building extracting method, medium and the computing device of neural network, remote sensing image - Google Patents

Abstract

The invention discloses a neural network, a building extraction method of a remote sensing image, a medium and a computing device. The disclosed neural network is used for building extraction of remote sensing images, including: the input layer in the VGG network, the first to fifth convolutional layers, and the first to fourth pooling layers; the first single-scale fusion layer, the second The input end of a single-scale fusion layer is connected to the output end of the first convolutional layer; the second to fifth single-scale fusion layers, and the input ends of the second to fifth single-scale fusion layers are respectively connected to the second to fifth volumes The output end of the product layer; the first to fourth upsampling layers, the input ends of the first to fourth upsampling layers are respectively connected to the output ends of the second to fifth single-scale fusion layers; multi-scale splicing fusion layer, multi-scale The input end of the splicing and fusion layer is connected to the output ends of the first single-scale fusion layer and the first to fourth upsampling layers; an output layer. The disclosed neural network can effectively process buildings with dense distribution and various scales, and improve the precision of automatic building extraction.

Description

Neural network, building extraction method, medium and computing equipment of remote sensing image

technical field

The invention relates to the fields of neural network and image processing, in particular to a neural network, a method for extracting buildings from remote sensing images, a medium and a computing device.

Background technique

With the rapid development of sensor technology, the spatial resolution of remote sensing images has been continuously improved. Inspired by deep learning algorithms in the field of computer vision, scholars currently use convolutional neural networks to implement semantic segmentation tasks for remote sensing images. Although some cutting-edge methods have achieved good results in semantic segmentation tasks of remote sensing images, they have not considered some characteristics of remote sensing images themselves. First of all, in conventional computer vision semantic segmentation tasks, there are generally only a few to dozens of targets on the image to be detected, and the distribution of targets is relatively loose, as shown in Figure 1(a). However, in remote sensing images, the distribution of buildings is generally dense, especially in residential areas, as shown in Figure 1(b). Secondly, in conventional computer vision semantic segmentation tasks, the size of the target to be detected is generally large, and the length and width are generally tens to hundreds of pixels, while the size of buildings in remote sensing images is generally much smaller, and the scale (the image itself of different buildings) The corresponding number of pixels) also changes greatly, as shown in Figure 1(c).

In order to ensure the accuracy of semantic segmentation, the accuracy of building (feature) extraction must first be ensured. Although there are already some technical solutions in the prior art that combine convolutional neural networks to extract some specific targets in remote sensing images. For example, the patent application with the publication number CN107025440A discloses a method for extracting roads from remote sensing images based on fully convolutional neural networks. The disclosed technical solution uses fully convolutional neural networks to achieve structural output, which can fully tap the 2D geometry dependency of roads. However, there is no effective method in the prior art that can make full use of convolutional neural networks to extract feature information of buildings of different scales in remote sensing images.

Therefore, it is necessary to propose a new technical solution to combine the convolutional neural network to fuse image features at different scales, so as to effectively improve the accuracy of automatic extraction of buildings of different scales.

Contents of the invention

According to the neural network system of the present invention, it is used for building extraction of remote sensing images, including:

The input layer, the first to fifth convolutional layers, and the first to fourth pooling layers in the VGG network;

The first single-scale fusion layer, the input end of the first single-scale fusion layer is connected to the output end of the first convolutional layer, which is used to fuse the first-scale multi-channel feature map output by the first convolutional layer and output the fused First-scale fusion of single-channel feature maps;

The second to fifth single-scale fusion layers, the input ends of the second to fifth single-scale fusion layers are respectively connected to the output ends of the second to fifth convolutional layers, for respectively fusing the second to fifth convolutional layers Output the second to fifth scale multi-channel feature maps and output the fused second to fifth scale single-channel feature maps respectively;

The first to fourth upsampling layers, the input ends of the first to fourth upsampling layers are respectively connected to the output ends of the second to fifth single-scale fusion layers;

Multi-scale splicing fusion layer, the input end of the multi-scale splicing fusion layer is connected to the output end of the first single-scale fusion layer, the first to the fourth upsampling layer, and is used to fuse the first single-scale fusion layer, the first to the fourth The feature map output by the upsampling layer and output the fused multi-scale fusion single-channel feature map;

The output layer, the input end of the output layer is connected to the output end of the multi-scale splicing and fusion layer, which is used to output the building feature map based on the multi-scale fusion single-channel feature map,

Among them, the outputs of the first single-scale fusion layer, the output terminals of the first to fourth up-sampling layers, and the output terminals of the multi-scale splicing and fusion layer are two-dimensional single-channel feature maps with the same resolution as the remote sensing image.

According to the neural network system of the present invention, it also includes:

The first to fourth clipping layers, the first to fourth clipping layers are respectively set between the first to fourth upsampling layers and the multi-scale splicing and fusion layer, for respectively outputting the first to fourth upsampling layers The feature maps are cropped to the same resolution as the original input image.

According to the neural network system of the present invention, the following layers are also included after the first to fifth convolutional layers:

The first to fifth ReLU layers, the first to fifth Batch Normalization layers, and the first to fifth Dropout layers are used to avoid overfitting and improve the generalization ability of the neural network system.

The building extraction method for remote sensing images according to the present invention includes:

Construct a trained neural network system as described above;

The trained neural network system is used to obtain the building feature map corresponding to the remote sensing image.

According to the building extraction method for remote sensing images of the present invention, before the step of constructing the trained neural network system as described above, it also includes:

The neural network system is trained by using a data set including remote sensing training images of buildings and label images corresponding to the remote sensing training images to obtain a trained neural network system.

According to the building extraction method for remote sensing images of the present invention, after obtaining the building feature map corresponding to the remote sensing image, a threshold method is used to obtain the final building distribution map.

According to the building extraction method for remote sensing images of the present invention, the Sigmoid Cross Entropy Loss loss function and stochastic gradient descent algorithm are used when training the neural network system.

According to the computer-readable storage medium of the present invention, a computer program is stored on the storage medium, and when the program is executed by a processor, the steps of the method as described above are realized.

The computing device according to the present invention includes a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the program, the steps of the method described above are realized.

According to the above technical solution of the present invention, the multi-scale information in the deep convolutional neural network can be directly used to effectively process buildings with dense distribution and various scales, and improve the accuracy of automatic building extraction. In addition, according to the above technical solution of the present invention, the entire image is used as input, and the segmentation (ie, building extraction) result is directly output without overlapping slices, which greatly improves the efficiency of building extraction.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate the embodiments of the invention and together with the related description serve to explain the principles of the invention. In the drawings, like reference numerals are used to denote like elements. The drawings in the following description are some, but not all, embodiments of the present invention. Those skilled in the art can obtain other drawings based on these drawings without creative efforts.

Fig. 1 shows a schematic diagram of a conventional image to be detected and a remote sensing image to be detected by the present invention.

Fig. 2 exemplarily shows a schematic structural diagram of a neural network system according to the present invention.

Fig. 3 exemplarily shows a schematic flowchart of a method for extracting buildings from remote sensing images according to the present invention.

FIG. 4 exemplarily shows different image maps output by each layer of the neural network system shown in FIG. 2 .

Fig. 5 exemplarily shows an original satellite remote sensing image, its corresponding real label image map, and a building feature map actually output according to the technical solution of the present invention.

FIG. 6 exemplarily shows the precision-recall curves under different relaxation coefficients according to the technical solution of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

Fig. 1 shows a schematic diagram of a conventional image to be detected and a remote sensing image to be detected by the present invention.

As described in the background technology section in conjunction with Figure 1, due to the above-mentioned differences between remote sensing images and conventional images, it is necessary to propose a new technical solution to combine convolutional neural networks to fuse image features at different scales, so as to effectively improve the image quality of different images. Accuracy of automated extraction of scale buildings.

Fig. 2 exemplarily shows a schematic structural diagram of a neural network system according to the present invention.

As shown in Figure 2, according to the neural network system of the present invention, it is used for the building extraction of remote sensing images, including:

The input layer in the VGG network (corresponding to "input image" in Figure 2), the first to fifth convolutional layers (respectively corresponding to the layer set of "Conv1_2, Conv2_2, Conv3_3, Conv4_3, Conv5_3" in Figure 2 ), the first to fourth pooling layers (respectively corresponding to the "Pool1, Pool2, Pool3, Pool4" layers in Figure 2);

The first single-scale fusion layer (corresponding to the first "Conv" on the left side of the horizontal distribution in Figure 2), the input end of the first single-scale fusion layer is connected to the output end of the first convolutional layer for fusing the first A first-scale multi-channel feature map output by a convolutional layer and output a fused first-scale fused single-channel feature map;

The second to fifth single-scale fusion layers (respectively corresponding to the second to fifth "Conv" in the horizontal distribution in Figure 2), the input ends of the second to fifth single-scale fusion layers are respectively connected to the second to fifth The output end of the convolutional layer is used to respectively fuse the second to fifth scale multi-channel feature maps output by the second to fifth convolutional layers and respectively output the fused second to fifth scale single-channel feature maps;

The first to fourth upsampling layers (respectively corresponding to "2×Upsampling", "4×Upsampling", "8×Upsampling", "16×Upsampling" in Figure 2), the input of the first to fourth upsampling layers The terminals are respectively connected to the output terminals of the second to the fifth single-scale fusion layers;

Multi-scale splicing and fusion layer (corresponding to the "Concat" layer in Figure 2), the input of the multi-scale splicing and fusion layer is connected to the output of the first single-scale fusion layer and the first to fourth upsampling layers for fusion The feature map output by the first single-scale fusion layer, the first to the fourth upsampling layer, and output the fused multi-scale fusion single-channel feature map;

The output layer (corresponding to "Conv" above "P" in Figure 2), the input end of the output layer is connected to the output end of the multi-scale splicing and fusion layer, which is used to output the building feature map based on the multi-scale fusion single-channel feature map ( corresponds to “P” in Figure 2),

Among them, the outputs of the first single-scale fusion layer, the output terminals of the first to fourth up-sampling layers, and the output terminals of the multi-scale splicing and fusion layer are two-dimensional single-channel feature maps with the same resolution as the remote sensing image.

In the above technical solution, for the first to fifth single-scale fusion layers, although the number of channels (that is, the number) C of their corresponding input feature maps is different (as shown in Figure 2, they are 64, 128, 256, 512, 512), however, since they use different convolution kernels with dimensions of 1*(64, 128, 256, 512, 512)*1*1 to fuse all input feature maps at their respective scales, the final Both can output a single-channel feature map (that is, the first to fifth scale fusion single-channel feature maps, which are 5 different resolutions of single-channel feature maps with resolutions of 2562, 1282, 642, 322, and 162 ).

For the multi-scale splicing fusion layer, similar to the above-mentioned fusion method of the first to fifth single-scale fusion layers, the number of channels C (that is, the number) of the input feature map is 5 (including the output of the first single-scale fusion layer feature map, and the four new feature maps output by the first to fourth upsampling layers obtained by upsampling, and their resolutions are the same as those of the original remote sensing image). Therefore, these 5 feature maps can be spliced into a 5-channel probability map (i.e., feature map), and a single-channel prediction map (i.e., multi-scale fusion single channel feature map).

Although the technical solution shown in FIG. 2 does not need to clip the first to fourth upsampling layers, however, optionally, the above-mentioned neural network system may also include:

The first to fourth clipping layers (respectively corresponding to each "Crop" in the layer set of "P2" to "P5" in Figure 2), the first to fourth clipping layers are respectively set in the first to fourth upsampling layers Between the multi-scale splicing and fusion layer, it is used to crop the feature maps output by the first to fourth upsampling layers to the same resolution as the original input image. To automatically adapt to the situation where the resolution of the remote sensing image is inconsistent with the resolution of the output feature maps of the first to fourth upsampling layers.

Optionally, after the first to fifth convolutional layers, the above-mentioned neural network system also includes the following layers (not shown in Figure 2):

The first to fifth ReLU layers, the first to fifth Batch Normalization layers, and the first to fifth Dropout layers are used to avoid overfitting and improve the generalization ability of the neural network system.

The shallow layer of the network shown in Figure 2 generates feature maps with fine spatial resolution but low-level semantic information, the deep layer shown in Figure 2 generates rough feature maps with high-level semantic information, and the middle layer shown in Figure 2 Feature maps correspond to some intermediate-level features. The above technical solution is able to integrate these different feature maps, thus, buildings with different appearances or occlusions can be extracted efficiently.

Fig. 3 exemplarily shows a schematic flowchart of a method for extracting buildings from remote sensing images according to the present invention.

The building extraction method for remote sensing images according to the present invention includes:

Step S302: building a trained neural network system as described above;

Step S304: Use the trained neural network system to obtain the building probability map (ie, the building feature map as described above, that is, the building extraction prediction Figure "P").

Optionally, the method for extracting buildings from remote sensing images also includes before step S302:

Step S302': Use the data set containing the remote sensing training image of the building (ie, "input image" in Figure 2) and the label image corresponding to the remote sensing training image (ie, "input map" in Figure 2) to neural network The network system is trained to obtain a trained neural network system.

Optionally, in step S304 and step S302', the threshold method is used to obtain the above-mentioned building feature map (final building extraction result).

Optionally, in step S302', when training the neural network system, use the Sigmoid CrossEntropy Loss loss function (that is, the calculation function corresponding to "Loss" in Figure 2) and the stochastic gradient descent algorithm.

In order to enable those skilled in the art to better understand the beneficial technical effect of the present invention, the following will be described in conjunction with specific embodiments.

FIG. 4 exemplarily shows different image maps output by each layer of the neural network system shown in FIG. 2 .

As shown in Figure 4, Figure 4(a) is a (first scale) original satellite remote sensing image selected from the Massachusetts remote sensing dataset (that is, the “input image” in Figure 2), and Figure 4(b) is The interpolated feature map of the second-scale feature map with a small receptive field (ie, "P2" in Figure 2), based on this feature map, low-level features such as edges and corners of the original satellite remote sensing image can be extracted. Figure 4(c) is the interpolated feature map of the third-scale feature map with a larger receptive field (ie, "P3" in Fig. 2), which can outline the preliminary outline of the building. Figure 4(d) is the interpolated feature map of the fourth-scale feature map with a larger receptive field (ie, "P4" in Figure 2), based on which non-building areas such as lakes can be identified. Figure 4(e) is the interpolated feature map of the fifth-scale feature map with the largest receptive field (ie, "P5" in Figure 2), based on which non-building areas such as lakes and bare land can be identified. Finally, integrating multi-level semantic information and spatial information obtains a reliable prediction (the multi-scale fused single-channel feature map described above, i.e., “P” in Fig. 2), as shown in Fig. 4(f) .

Fig. 5 exemplarily shows the original satellite remote sensing image, its corresponding real label image map, and the actually output building feature map according to the technical solution of the present invention.

As shown in Figure 5, Figure 5(a) is an original satellite remote sensing image selected from the Massachusetts remote sensing dataset, Figure 5(b) is its real label image map, and Figure 5(c) is the predicted label map (ie, According to the actual output building feature map of the technical solution of the present invention). From the perspective of visual effects, the technical solution according to the invention can well predict the distribution of buildings, and the boundaries of buildings are accurate.

FIG. 6 exemplarily shows the precision-recall curves under different relaxation coefficients according to the technical solution of the present invention.

The accuracy rate is defined as the proportion of the detected pixels within the range of ρ adjacent pixels to the real pixel, and the recall rate is defined as the proportion of the detected pixels within the range of ρ adjacent pixels of the detected pixel. Fig. 6 (a) is the accuracy rate-recall rate curve when ρ=3 according to the technical solution of the present invention, and its corresponding model accuracy is about 0.9668 (corresponding to Fig. 6 (a) represented by symbol ×, accuracy rate and The point where the recall is equal breakeven). Fig. 6 (b) is the accuracy rate-recall rate curve when ρ=0 according to the technical solution of the present invention, and its corresponding model precision is about 0.8424 (corresponding to Fig. 6 (b) represented by symbol ×, accuracy rate and The point where the recall is equal breakeven).

Table 1 shows the Mnih-CNN scheme and the Mnih-CNN+CRF scheme disclosed by Mnih.V in his doctoral thesis "Machine learning for aerial imagelabeling, Doctoral (2013)", and Saito in "Multiple object extraction from aerial Imagery with convolutional neuralnetworks "Saito-multi-MA scheme and Saito-multi-MA&CIS scheme disclosed in "imagery with convolutional neural networks", and the architectural extraction performance comparison between different schemes including the technical scheme of the present invention.

Table 1 Performance comparison between different technical solutions

Model Breakeven (ρ=3) Breakeven (ρ=0) Prediction time (s) Mnih-CNN 0.9271 0.7661 8.7 Mnih-CNN+CRF 0.9282 0.7638 26.6 Saito-multi-MA 0.9503 0.7873 67.72 Saito-multi-MA&CIS 0.9509 0.7872 67.84 Technical scheme of the present invention 0.9668 0.8424 2.05

Note: The prediction time is the average time required for prediction of a single 1500*1500 test image, and the graphics card model used is NVIDIA TITAN X.

As can be seen from the results in Table 1, no matter in terms of model accuracy under different relaxation coefficients (ρ=3 and ρ=0), or in terms of prediction time, the above-mentioned technical scheme according to the present invention has better technical effects . Not only can the extraction accuracy be significantly improved, but also the operation time can be reduced.

In combination with the above-mentioned technical solution according to the present invention, a computer-readable storage medium is also proposed, on which a computer program is stored, and when the program is executed by a processor, the steps of the method shown in FIG. 3 are implemented.

In combination with the above-mentioned technical solution according to the present invention, a computing device is also proposed, including a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the program, the method shown in Figure 3 is implemented. A step of.

According to the above technical solution of the present invention, the VGG network is used as the basic structure, the last layer of each resolution feature map in the network is extracted, and the convolution operation is used to fuse into a single-channel feature map. Through resampling and feature map splicing, the final prediction result is obtained.

According to the above technical solution of the present invention, the multi-scale information in the deep convolutional neural network can be directly used to effectively process buildings with dense distribution and various scales, and improve the accuracy of automatic building extraction. In addition, according to the above technical solution of the present invention, the entire image is used as input, and the segmentation (ie, building extraction) result is directly output without overlapping slices, which greatly improves the efficiency of building extraction.

According to the above technical solution of the present invention, it also has the following advantages: 1) It is possible to fuse each resolution feature map, thereby extracting the multi-scale information of the input image, and realizing the precise extraction of buildings; Model integration is required, and post-processing operations are not required, thus greatly improving the efficiency of building extraction; 3) Due to the use of a fully convolutional network, it can accept input images of any size as long as the video memory allows.

In addition, according to the above technical solution of the present invention, the entire image is directly used as input, and the segmentation (ie, building extraction) result can be obtained through one network forward propagation, without the need for model integration by overlapping slices , and no post-processing operation is required, which greatly improves the efficiency of building extraction. The above comparison test results based on the Massachusetts remote sensing data set can show that the above technical solution according to the present invention is significantly better than other methods in terms of accuracy and efficiency.

The content described above can be implemented alone or combined in various ways, and these variants are all within the protection scope of the present invention.

Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and an appropriate combination thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those of ordinary skill in the art should understand that: it can still modify the technical solutions described in the aforementioned embodiments, or perform equivalent replacements for some of the technical features; and these The modification or replacement does not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (9)

1. a kind of nerve network system, which is characterized in that the building for remote sensing image extracts, including：

Input layer, the first to the 5th convolutional layer, first to fourth pond layer in VGG networks；

First single scale fused layer, the input terminal of the first single scale fused layer are connected to the output of first convolutional layer End, for merging the first scale multi-channel feature figure that first convolutional layer is exported and the first scale exported after fusion melts Close single channel characteristic pattern；

The input terminal of second to the 5th single scale fused layer, the described second to the 5th single scale fused layer is respectively connected to described The output end of two to the 5th convolutional layers, the second to the 5th scale exported for merging the described second to the 5th convolutional layer respectively The second to the 5th scale after multi-channel feature figure and respectively output fusion merges single channel characteristic pattern；

The input terminal of first to fourth up-sampling layer, the first to fourth up-sampling layer is respectively connected to described second to the 5th The output end of single scale fused layer；

Multiple dimensioned splicing fused layer, it is described it is multiple dimensioned splicing fused layer input terminal be connected to the first single scale fused layer, The output end of the first to fourth up-sampling layer, for merging the first single scale fused layer, described first to fourth Characteristic pattern that sample level is exported simultaneously exports the Multiscale Fusion single channel characteristic pattern after fusion；

Output layer, the input terminal of the output layer is connected to the output end of the multiple dimensioned splicing fused layer, for based on described Multiscale Fusion single channel characteristic pattern exports building feature figure,

Wherein, the first single scale fused layer, the output end of the first to fourth up-sampling layer and the multiple dimensioned splicing What the output end of fused layer was exported is two-dimentional single channel characteristic pattern identical with the resolution ratio of the remote sensing image.

2. nerve network system as described in claim 1, which is characterized in that further include：

First to fourth cuts layer, and the first to fourth cutting layer is separately positioned on the first to fourth up-sampling layer and institute Between stating multiple dimensioned splicing fused layer, for respectively by described first to fourth up-sample the characteristic pattern that is exported of layer be cut to It is originally inputted the identical resolution ratio of image.

3. nerve network system as claimed in claim 1 or 2, which is characterized in that after the described first to the 5th convolutional layer Further include following layers：

First to the 5th ReLU layers, the first to the 5th Normalization layers of Batch, first to the 5th Dropout layers, be used for Over-fitting is avoided, the generalization ability of the nerve network system is improved.

4. a kind of building extracting method for remote sensing image, which is characterized in that including：

Build the housebroken nerve network system as described in any one of claim 1-3；

The building feature figure corresponding to remote sensing image is obtained using housebroken nerve network system.

5. being used for the building extracting method of remote sensing image as claimed in claim 4, which is characterized in that in the structure through instruction Before the step of experienced nerve network system as described in any one of claim 1-3, further include：

Use the remote sensing training image comprising building and the data set pair of label image corresponding with remote sensing training image The nerve network system is trained, to obtain the housebroken nerve network system.

6. being used for the building extracting method of remote sensing image as described in claim 4 or 5, which is characterized in that described in acquisition After the building feature figure corresponding to remote sensing image, final building distribution map is obtained using threshold method.

7. being used for the building extracting method of remote sensing image as claimed in claim 5, which is characterized in that the nerve net When network system is trained, calculated using Sigmoid Cross Entropy Loss loss functions and using stochastic gradient descent Method.

8. a kind of computer readable storage medium, which is characterized in that be stored with computer program, the journey on the storage medium The step of any one of claim 4 to 7 the method is realized when sequence is executed by processor.

9. a kind of computing device, which is characterized in that including memory, processor and be stored on the memory and can be described The computer program run on processor, the processor realize any one of claim 4 to 7 institute when executing described program The step of stating method.