CN113435284A - Post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion - Google Patents
Post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion Download PDFInfo
- Publication number
- CN113435284A CN113435284A CN202110681049.8A CN202110681049A CN113435284A CN 113435284 A CN113435284 A CN 113435284A CN 202110681049 A CN202110681049 A CN 202110681049A CN 113435284 A CN113435284 A CN 113435284A
- Authority
- CN
- China
- Prior art keywords
- network model
- rcca
- cca
- attention
- remote sensing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to the technical field of remote sensing image road extraction, in particular to a post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion. The method comprises the steps of collecting remote sensing image data of a research area; inputting the remote sensing influence data into a DCCA network model obtained by pre-training; outputting an extraction result; the pre-training process of the DCCA network model comprises the following steps: collecting remote sensing image data of a research area; processing the remote sensing image data to obtain a data set containing an image and an annotated image; performing direction information attention on the image characteristics through a CCA network model and an RCCA network model, and respectively outputting direction attention information energy and a network extraction result output; respectively training a CCA network model and an RCCA network model by utilizing a data set; and fusing the CCA network model and the RCCA network model based on the direction attention information energy and the network extraction result output respectively output by the CCA network model and the RCCA network model to obtain the DCCA network model.
Description
Technical Field
The invention relates to the technical field of remote sensing image road extraction, in particular to a post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion.
Background
Roads are the main mode of urban transportation and occupy an important position in national economy and social life. When a serious disaster occurs, the normal operation of the road is an important guarantee for rescue and logistics supply. Therefore, the method has the advantages of quickly and accurately identifying and extracting the post-disaster roads and detecting the road range, and has important significance for activities such as post-disaster rescue and material supply. Object interpretation and identification of remote sensing images are always research hotspots in the field of remote sensing, so that extraction of a post-disaster road range based on high resolution and post-disaster rescue analysis research based on the extraction are important research contents.
With the development of artificial intelligence technology, the application of deep learning method in remote sensing image interpretation is becoming mature, and the use of convolutional neural network for high resolution remote sensing image identification becomes a research hotspot. The semantic segmentation network based on multiple attention mechanisms such as space attention and channel attention can accurately extract image road information, but road objects are less distributed in remote sensing images, the post-disaster scene content is complex, and the following problems still exist in post-disaster road extraction by the conventional method:
lack of directional information: the distribution of the road object in the remote sensing image has direction regularity, and in the process of remote sensing interpretation, the existing model mostly analyzes the whole image information and lacks the extraction of direction information.
Information redundancy: in the process of extracting image features by using a convolutional neural network, a large number of convolution operations are needed for feature extraction, but the obtained feature information is not all available for interpretation, and when the features are concerned, the calculation amount is large due to too many input features, so that a large amount of redundant information exists in the existing model.
Due to the above key problems, a remote sensing image interpretation method capable of utilizing the direction semantic information of the remote sensing image and realizing the targeted filtering of redundant information so as to effectively extract image features and realize efficient and accurate post-disaster road extraction is urgently needed.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides a post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion, which can realize targeted filtering of redundant information, can effectively extract image features and can realize efficient and accurate post-disaster road extraction.
The invention discloses a post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion, which adopts the technical scheme that the method comprises the following steps:
collecting remote sensing image data of a research area;
inputting the remote sensing influence data into a DCCA network model obtained by pre-training;
outputting an extraction result;
wherein the pre-training process of the DCCA network model comprises:
collecting remote sensing image data of a research area;
processing the remote sensing image data to obtain a data set comprising an image and an annotated image;
performing direction information attention on the image characteristics through a CCA network model and an RCCA network model, and respectively outputting direction attention information energy and a network extraction result output;
respectively training a CCA network model and an RCCA network model by utilizing a data set;
and fusing the CCA network model and the RCCA network model based on the direction attention information energy and the network extraction result output respectively output by the CCA network model and the RCCA network model to obtain the DCCA network model.
The method extracts road attention features from different directions, dynamically filters attention results through a median filtering activation function, removes redundant information in the attention results, and completes selective fusion of the attention results in all directions through a self-adaptive fusion algorithm so as to realize rapid identification and extraction of the post-disaster road range.
Preferably, the image features are extracted from the remote sensing image data through a Backbone network Backbone.
Preferably, the performing, by the CCA network model and the RCCA network model, the direction information attention on the image feature includes:
aggregating the horizontal and vertical direction information of each pixel position in the image characteristics through a CCA network model;
aggregating the information of the oblique direction of each pixel position in the image characteristics through an RCCA network model;
specific activation was performed by Mid ReLU.
Preferably, the merging the CCA network model and the RCCA network model based on the direction attention information energy and the network extraction result output respectively output by the CCA network model and the RCCA network model to obtain the DCCA network model includes:
respectively inputting direction attention information energy and a network extraction result output by the CCA network model and the RCCA network model into the SFA;
respectively taking the maximum values of the direction attention information energy of the CCA network model and the RCCA network model to obtain the attention weights of the CCA network model and the RCCA network model;
splicing, softmax activating and weight splitting attention weights of the CCA network model and the RCCA network model to obtain an interaction weight;
weighting the network extraction results output of the CCA network model and the RCCA network model by using the interaction weight;
and fusing the results of the weighting processing to obtain a DCCA network model.
Preferably, the calculation formula for fusing the results of the weighting processing is as follows:
wherein P represents the prediction score of each class of the final result, PCCAVarious types of predictive scores, P, representing CCA model resultsRCCAAnd (3) representing various types of prediction scores of the RCCA model result, f representing a foreground type, and b representing a background type.
Preferably, the calculation formula for specific activation by Mid ReLU is:
wherein x represents the input feature map, Pα(X) represents a threshold value for dynamic filtering.
Preferably, the calculation formula for performing direction information attention on the image features through the CCA network model and the RCCA network model is as follows:
wherein the content of the first and second substances,
shows the result after (i, j) prescription is focused,
denotes the Key value, V, at (i, j)kThe Value representing the corresponding direction at (i, j).
Preferably, the processing the remote sensing image data to obtain a data set includes:
preprocessing the remote sensing image data, wherein the preprocessing comprises splicing and cutting, geometric correction and atmospheric correction;
vectorizing and labeling the preprocessed remote sensing image data based on a visual interpretation result;
obtaining a thematic map of a research area through GIS processing, and rasterizing the thematic map to obtain a corresponding grid gray map;
selecting a cutting scale according to the image road distribution condition and the network model requirement, sampling the remote sensing image and the grid gray-scale image of the research area according to the cutting scale, and naming the cutting sampling results according to the sequence number in sequence to obtain a cutting sample;
the cut samples are divided into a training set, a validation set and a test set.
Preferably, the training set, the verification set and the test set are according to 7: 1: a ratio of 2.
Preferably, the method further comprises a hyper-parameter setting, wherein the hyper-parameter setting comprises:
setting the loss weight of the road and the background as 10: 1;
1/4 setting the ResNet101 output channel number as the original channel number;
setting the learning rate of network training to be 0.003;
batch _ size is 2;
the number of iterations of training is 100 rounds.
The invention has the beneficial effects that: the direction information of different roads in the high-resolution remote sensing image is fully utilized, road attention features are extracted from different directions, attention results are dynamically filtered, redundant information in the attention results is removed, the attention results in all directions are selectively fused, and rapid identification and extraction of the post-disaster road range can be achieved. According to the scheme, road characteristic aggregation can be performed in multiple directions such as horizontal, vertical and inclined directions, compared with global attention, the calculation complexity is reduced, meanwhile, redundant information is filtered, road information in different directions can be effectively aggregated through dynamic fusion, and the road range in post-disaster remote sensing images can be accurately extracted.
Drawings
FIG. 1 is a schematic flow chart of a post-disaster road extraction method based on dynamic filtering and multi-directional attention fusion according to the present invention;
FIG. 2 is a schematic diagram of a DCCA network model construction method;
FIG. 3 is a schematic diagram of the direction attention mechanism of the present invention;
FIG. 4 is a graph of results of comparative experiments with different activation functions;
FIG. 5 is a schematic diagram of the visual comparison result of before and after Mid ReLU-0.8 filtering feature maps;
FIG. 6 is a diagram showing comparison of predicted results.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly on the other element or be indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element.
It will be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like, refer to an orientation or positional relationship illustrated in the drawings for convenience in describing the present application and to simplify description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the present application.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
The invention designs a remote sensing image road extraction network with different direction attention mechanisms. The backbone part of the network extracts different direction information in the features based on a multi-direction Attention mechanism (DCCA), inhibits and extracts heterogeneous signals in the features by using a Median filter reconstructed Linear Unit (Mid ReLU), filters redundant information, and specifically fuses the Attention results by using a Self-adaptive Fusion algorithm (SFA). The method can aggregate road characteristics in multiple directions such as horizontal, vertical and inclined directions, reduces the calculation complexity compared with global attention, filters redundant information, can effectively aggregate road information in different directions through dynamic fusion, and realizes more accurate extraction of the road range in the post-disaster remote sensing image. The method comprises the following specific steps:
collecting remote sensing image data of a research area;
inputting the remote sensing influence data into a DCCA network model obtained by pre-training;
and outputting the extraction result.
Wherein the pre-training process of the DCCA network model comprises:
data acquisition: collecting remote sensing image data of a research area;
data preprocessing: preprocessing the acquired high-resolution remote sensing image data, including splicing and cutting, geometric correction, atmospheric correction and the like;
image interpretation: based on a visual interpretation result, carrying out vectorization labeling work on the remote sensing image by using GIS professional software such as ArcGIS and the like, obtaining a thematic map of a research area through GIS processing, and rasterizing the thematic map to obtain a corresponding grid gray map;
sample preparation: selecting a cutting scale according to the image road distribution condition and the network model requirement, sampling the remote sensing image and the grid gray-scale map of the research area by using a python script according to the cutting scale, and naming the cutting sampling result according to a sequence number rule to obtain a cutting sample;
and (3) data set generation: the cutting sample is divided into a training set, a verification set and a test set for network training test;
network construction: performing direction information attention on the image characteristics through a CCA network model and an RCCA network model, and respectively outputting direction attention information energy and a network extraction result output; among them, CCA (Cross Attention), RCCA (rotad Cross Attention). The extracted feature results can be selectively focused and different direction features can be aggregated from the horizontal, vertical and oblique directions respectively. The Mid ReLU as a specific activation can be embedded into a directional attention process, so that heterogeneous signals in an attention result are restrained, homogeneous signals are enhanced, and effective information in the attention result is enhanced. After different directions concern, different directions concern information (energy) and network extraction results (output) can be output respectively.
Setting the hyper-parameters: for the two-classification semantic segmentation task of road extraction, loss weight needs to be set according to road proportion so as to inhibit the background overfitting phenomenon. For the directional attention mechanism, the number of channels needs to be adjusted for ResNet101 (residual network) to adapt to the directional attention input. For network training, appropriate learning rate, batch _ size, iteration number and other superparameters need to be set to ensure smooth convergence of the network.
And (3) CCANet training: and (3) taking the image and the labeled image of the data set as input data to carry out iterative training on the CCANet network, so that the network learning carries out road information aggregation from the horizontal and vertical directions. And saving the network training process and the optimal model weight.
Model fusion: constructing DCCANet, embedding CCANet and RCCANet as 2 branches into DCCANet, specifically: the energy and output of the 2 branches are respectively input into the SFA.
In the SFA, firstly, taking the maximum value of energy output by a branch to obtain an attention weight, then splicing the attention weights of 2 branches, carrying out softmax operation, carrying out weight activation, splitting the activated weight, weighting output of CCANet and RCCANet respectively, selectively fusing the results after 2 weighting, adopting a minimum value-taking strategy for a background score and a maximum value-taking strategy for a foreground score to obtain a fused result, and realizing specific fusion of roads.
Example one
Fig. 1 and 2 show a flow chart of a post-disaster road extraction method based on dynamic filtering and multi-directional attention fusion provided in a preferred embodiment of the present application (fig. 1 and 2 show a first embodiment of the present application), and for convenience of explanation, only the parts related to the present embodiment are shown, and the following details are described below:
step 1: and acquiring a high-resolution remote sensing image of the research area. In this example, the Quickbird remote sensing image after earthquake disaster is obtained in the range of 30 degrees 28 '41 "-30 degrees 32' 29" in northern latitude of Wenchuan county in Sichuan province, 7.2008, and in the range of 114 degrees 22 '42 "-114 degrees 28' 11" in east longitude, and the spatial resolution of the image is 0.5 m.
Step 2: and inputting the remote sensing influence data into a DCCA network model obtained by pre-training.
And step 3: and outputting the extraction result.
Wherein, the pre-training process of the DCCA network model in the step 2 comprises the following steps:
step S201, acquiring data: and acquiring a high-resolution remote sensing image of the research area. In this example, the Quickbird remote sensing image after earthquake disaster is obtained in the range of 30 degrees 28 '41 "-30 degrees 32' 29" in northern latitude of Wenchuan county in Sichuan province, 7.2008, and in the range of 114 degrees 22 '42 "-114 degrees 28' 11" in east longitude, and the spatial resolution of the image is 0.5 m.
Step S202, data preprocessing: and inputting the acquired remote sensing image data into ENVI software, and carrying out preprocessing such as splicing and cutting, geometric correction, atmospheric correction and the like on the image.
Step S203, image interpretation: visual interpretation is carried out on the remote sensing image, a road range in the remote sensing image is identified, a vector layer is created by using GIS professional software such as ArcGIS and the like, a road interpretation result is labeled, a thematic map of a research area is obtained through GIS processing such as color rendering and the like, and a vector grid-to-grid tool is used for rasterizing the thematic map to obtain a corresponding grid gray scale labeling map.
Step S204, sample preparation: selecting a sample cutting scale of 256 × 256 according to the image road distribution condition and the network model requirement, sampling the remote sensing image and the grid gray-scale map of the research area by using a python script according to the cutting scale, and naming the cutting sampling result according to a sequence number rule to obtain 2000 sample data.
Step S205, data set generation: according to the following steps: 1: 2, randomly dividing 2000 samples into a training set, a verification set and a test set for network training test.
Step S206, network construction: and (3) building a network model based on directional attention and dynamic fusion weighting by using a pyrrch framework, wherein the network model process is as follows:
1. ResNet101 is selected as a backbone, and image features are extracted. Compared with other networks, the ResNet101 has moderate parameter quantity, has better feature extraction capability and is suitable for serving as a backhaul.
2. And the attention mechanism part of the network maps the input features into Key and Query, and then selectively pays attention to the extracted feature results and aggregates features in different directions from the horizontal, vertical and inclined directions by using CCA and RCCA respectively. The directional attention calculation of the CCA and the RCCA is shown as a formula (1):
ai，j＝∑kf(Ki，j，Vk)#(1)
in the formula, ai，jShows the result after the prescription of (i, j) is focused on, Ki，jDenotes the Key value, V, at (i, j)kThe Value representing the corresponding direction at (i, j), k representing the coordinates in the horizontal and vertical directions for CCA, and k representing the coordinates in the diagonal direction for RCCA. The information aggregation process of CCA and RCCA may be represented as fig. 3, with CCA implementing horizontal, vertical per pixel locationThe straight direction information is aggregated, the RCCA realizes the aggregation of the oblique direction information of each pixel position, and the multi-direction information aggregation can be realized through the subsequent Mid RELU activation. And performing specific activation on the concerned result by using Mid ReLU, inhibiting a heterogeneous signal in the concerned result, enhancing a homogeneous signal, and enhancing the characteristic expression capacity. The calculation process of Mid ReLU is shown in equation (2):
3. in the formula, x represents an input feature map, Pα(X) represents a threshold value for dynamic filtering. The Mid ReLU can realize the shielding of the inhibition signal and the enhancement of the homogeneous signal by filtering the result which is lower than the threshold value. For the selection of the filtering threshold, 80% of the eigenvalues are selected for filtering through a number of experiments.
The results of comparative experiments with different activation functions are shown in fig. 4.
Wherein, the ReLU represents the result of filtering activation by using the original ReLU function; the Mid ReLU-mean represents the result of filtering activation with the feature map mean as a threshold; mid ReLU-0.5 represents a 50% result of filtration; mid ReLU-0.5, softmax indicates the result of filtering 50% of the results before performing softmax activation; mid ReLU2-0.1, representing results that retained the highest, lowest, 10% each, and filtered the middle 80%. As can be seen, the effect of the Mid ReLU function varies according to the filtering ratio. The effect of the Mid ReLU-mean is better than that of the original ReLU function, and the effect of the Mid ReLU-0.8 is better, which is related to the fact that the road accounts for a lower proportion in the whole image. The activation mode of first Mid ReLU and then softmax is poor, which is related to that softmax performs integral scaling on Mid ReLU results and weakens the homogeneous signal strength. The Mid-value filtering approach of Mid-ReLU 2 retains some redundancy, and therefore results are inferior to Mid-ReLU-0.8.
In the operating process of the CCANet network, the feature map output before and after Mid ReLU-0.8 filtering is performed on the concerned result and the result is visualized as shown in fig. 5.
In the graph, (a) shows the visualization result of the feature map without using Mid ReLU for filtering, the feature value segmentation is messy, and a lot of redundant non-attention area information exists; (b) the results of visualization of the feature maps obtained by filtering with Mid ReLU are shown, and the regions of interest are more evident in the horizontal and vertical directions.
In conclusion, after the Mid ReLU-0.8 is used for filtering, the low-value region of the effective part can be shielded, the attention region signal is enhanced, and the direction attention effect is enhanced. Thus, Mid ReLU-0.8 is used in the present invention as a filter activation function.
Step S207, hyper-parameter setting: for the two-classification semantic segmentation task of road extraction, the loss weight 10 of the road proportion in the image as the road and background is required: 1 to suppress the background overfitting phenomenon. According to the requirement of a direction attention mechanism, the number of output channels of the ResNet101 is reduced to 1/4 of the original number of output channels so as to adapt to the direction attention input. For network training, the learning rate is set to be 0.003, the batch _ size is set to be 2, and the number of iterations of 100 rounds and other superparameters are set respectively to ensure the stable convergence of the network.
Step S208, CCANet training: and (3) taking the image and the labeled image of the data set as input data to carry out iterative training on the CCANet network, so that the network learning carries out road information aggregation from the horizontal and vertical directions. And (4) saving the network training process and the optimal model weight, wherein the optimal road segmentation precision reaches 0.4498.
Step S209, RCCANet training: and (3) taking the image and the labeled image of the data set as input data to carry out iterative training on the RCCANet network, so that the network learning carries out road information aggregation from the horizontal and vertical directions. And the network training process and the optimal model weight are saved, and the optimal road segmentation precision reaches 0.4908.
Step S210, model specific fusion: constructing DCCANet, embedding CCANet and RCCANet as 2 branches into DCCANet, and inputting energy and output by the 2 branches into SFA respectively.
In SFA fusion, firstly, the output energy is taken as the maximum value to obtain the attention weight, and then the attention weight of 2 branches is subjected to splicing, softmax activation and splitting operations, so that attention feature interaction is realized and the interaction weight is obtained. Respectively connecting the interaction weight with CWeighting output of CANet and RCCANet to obtain respective weighted prediction score PCCA、PRCCA。
And selectively fusing CCA and RCCA prediction scores, inhibiting redundant information again and enhancing effective information expression. The fusion algorithm in the SFA is represented by the formula:
wherein P represents the predicted score of each type of final result, PCCAVarious types of predictive scores, P, representing CCA model resultsRCCAAnd f represents a foreground class (particularly a road class in the foreground in the scheme), and b represents a background class. The fused result is obtained by adopting the minimum value-taking strategy for the background score and the maximum value-taking strategy for the foreground score, so that the road information in the concerned results in different directions can be aggregated, and more accurate road range extraction is realized.
Step S211, verification analysis: and analyzing the accuracy of the model and the identification effect of the remote sensing object.
The visualization results obtained by predicting part of the samples using CCANet, RCCANet and DCCNet are shown in fig. 6.
As can be seen from the figure, the prediction results of CCANet and RCCANet for roads in different directions are different, the prediction result of CCANet for horizontal and vertical roads is better, the prediction result of RCCANet for inclined roads is better, and DCCANet fuses 2 networks through SFA, so that 2 prediction results can be effectively combined, and the road prediction results in different directions are improved.
And (3) testing by using the test set sample to confirm the effect of the test set sample on road extraction in the remote sensing image and the generalization capability of the model, and obtaining a test set confusion matrix as shown in table 1.
TABLE 1 results of the experiment
As can be seen from Table 1, the extraction precision of the merged road after the disaster reaches 0.5140, the overall precision reaches 0.77, which shows that the method has better effect and higher practical level, and provides a scientific and effective method for the extraction of the road range after the disaster and the analysis of the influence thereof.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. . Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion is characterized by comprising the following steps:
collecting remote sensing image data of a research area;
inputting the remote sensing influence data into a DCCA network model obtained by pre-training;
outputting an extraction result;
wherein the pre-training process of the DCCA network model comprises:
collecting remote sensing image data of a research area;
processing the remote sensing image data to obtain a data set comprising an image and an annotated image;
performing direction information attention on the image characteristics through a CCA network model and an RCCA network model, and respectively outputting direction attention information energy and a network extraction result output;
respectively training a CCA network model and an RCCA network model by utilizing a data set;
and fusing the CCA network model and the RCCA network model based on the direction attention information energy and the network extraction result output respectively output by the CCA network model and the RCCA network model to obtain the DCCA network model.
2. The post-disaster road extraction method based on dynamic filtering and multi-directional attention fusion as claimed in claim 1, wherein: the image characteristics are extracted from the remote sensing image data through the Backbone network Backbone.
3. The post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion of claim 1, wherein the performing direction information attention on image features through a CCA network model and an RCCA network model comprises:
aggregating the horizontal and vertical direction information of each pixel position in the image characteristics through a CCA network model;
aggregating the information of the oblique direction of each pixel position in the image characteristics through an RCCA network model;
specific activation was performed by Mid ReLU.
4. The post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion as claimed in claim 1, wherein the method for fusing the CCA network model and the RCCA network model to obtain the DCCA network model comprises:
respectively inputting direction attention information energy and a network extraction result output by the CCA network model and the RCCA network model into the SFA;
respectively taking the maximum values of the direction attention information energy of the CCA network model and the RCCA network model to obtain the attention weights of the CCA network model and the RCCA network model;
splicing, softmax activating and weight splitting attention weights of the CCA network model and the RCCA network model to obtain an interaction weight;
weighting the network extraction results output of the CCA network model and the RCCA network model by using the interaction weight;
and fusing the results of the weighting processing to obtain a DCCA network model.
5. The post-disaster road extraction method based on dynamic filtering and multidirectional attention fusion as claimed in claim 4, wherein a calculation formula for fusing results of weighting processing is as follows:
wherein P represents the prediction score of each class of the final result, PCCAVarious types of predictive scores, P, representing CCA model resultsRCCAAnd (3) representing various types of prediction scores of the RCCA model result, f representing a foreground type, and b representing a background type.
6. The post-disaster road extraction method based on dynamic filtering and multi-directional attention fusion as claimed in claim 3, wherein the calculation formula for specific activation through Mid ReLU is as follows:
wherein x represents the input feature map, Pα(X) represents a threshold value for dynamic filtering.
7. The post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion as claimed in claim 1, wherein the calculation formula for performing direction information attention on image features through a CCA network model and a RCCA network model is as follows:
wherein, ai，jShows the result after the prescription of (i, j) is focused on, Ki，jDenotes the Key value, V, at (i, j)kThe Value representing the corresponding direction at (i, j).
8. The post-disaster road extraction method based on dynamic filtering and multi-directional attention fusion as claimed in claim 1, wherein the processing of the remote sensing image data to obtain a data set comprises:
preprocessing the remote sensing image data, wherein the preprocessing comprises splicing and cutting, geometric correction and atmospheric correction;
vectorizing and labeling the preprocessed remote sensing image data based on a visual interpretation result;
obtaining a thematic map of a research area through GIS processing, and rasterizing the thematic map to obtain a corresponding grid gray map;
selecting a cutting scale according to the image road distribution condition and the network model requirement, sampling the remote sensing image and the grid gray-scale image of the research area according to the cutting scale, and naming the cutting sampling results according to the sequence number in sequence to obtain a cutting sample;
the cut samples are divided into a training set, a validation set and a test set.
9. The post-disaster road extraction method based on dynamic filtering and multi-directional attention fusion as claimed in claim 1, wherein: the training set, the verification set and the test set are as follows: 1: a ratio of 2.
10. The post-disaster road extraction method based on dynamic filtering and multi-directional attention fusion as claimed in claim 1, further comprising hyper-parameter setting, wherein the hyper-parameter setting comprises:
setting the loss weight of the road and the background as 10: 1;
1/4 setting the ResNet101 output channel number as the original channel number;
setting the learning rate of network training to be 0.003;
batch _ size is 2;
the number of iterations of training is 100 rounds.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110681049.8A CN113435284B (en) | 2021-06-18 | 2021-06-18 | Post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110681049.8A CN113435284B (en) | 2021-06-18 | 2021-06-18 | Post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113435284A true CN113435284A (en) | 2021-09-24 |
| CN113435284B CN113435284B (en) | 2022-06-28 |
Family
ID=77756644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110681049.8A Active CN113435284B (en) | 2021-06-18 | 2021-06-18 | Post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113435284B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110299733A1 (en) * | 2008-12-18 | 2011-12-08 | Qinetiq Limited | System and method for processing radar imagery |
| CN104392451A (en) * | 2014-11-28 | 2015-03-04 | 河海大学 | Artificial scene image end point detection method |
| CN110490081A (en) * | 2019-07-22 | 2019-11-22 | 武汉理工大学 | A kind of remote sensing object decomposition method based on focusing weight matrix and mutative scale semantic segmentation neural network |
| US20200211415A1 (en) * | 2017-06-27 | 2020-07-02 | Mohammad Gheith | System For Performing Multiplication |
| CN112488115A (en) * | 2020-11-23 | 2021-03-12 | 石家庄铁路职业技术学院 | Semantic segmentation method based on two-stream architecture |
| CN112541503A (en) * | 2020-12-11 | 2021-03-23 | 南京邮电大学 | Real-time semantic segmentation method based on context attention mechanism and information fusion |
| CN112749706A (en) * | 2020-06-17 | 2021-05-04 | 腾讯科技（上海）有限公司 | Semantic segmentation method and related equipment |
-
2021
- 2021-06-18 CN CN202110681049.8A patent/CN113435284B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110299733A1 (en) * | 2008-12-18 | 2011-12-08 | Qinetiq Limited | System and method for processing radar imagery |
| CN104392451A (en) * | 2014-11-28 | 2015-03-04 | 河海大学 | Artificial scene image end point detection method |
| US20200211415A1 (en) * | 2017-06-27 | 2020-07-02 | Mohammad Gheith | System For Performing Multiplication |
| CN110490081A (en) * | 2019-07-22 | 2019-11-22 | 武汉理工大学 | A kind of remote sensing object decomposition method based on focusing weight matrix and mutative scale semantic segmentation neural network |
| CN112749706A (en) * | 2020-06-17 | 2021-05-04 | 腾讯科技（上海）有限公司 | Semantic segmentation method and related equipment |
| CN112488115A (en) * | 2020-11-23 | 2021-03-12 | 石家庄铁路职业技术学院 | Semantic segmentation method based on two-stream architecture |
| CN112541503A (en) * | 2020-12-11 | 2021-03-23 | 南京邮电大学 | Real-time semantic segmentation method based on context attention mechanism and information fusion |
Non-Patent Citations (4)
| Title |
|---|
| CHENFENG GUO ET AL.: "Canonical Correlation Analysis (CCA) Based", 《ARXIV.ORG》 * |
| MINGYU TAO ET AL.: "Attention U-Net for Road Extraction in Remote Sensing Images", 《INTERNATIONAL CONFERENCE ON SPATIAL DATA AND INTELLIGENCE》 * |
| ZILONG HUANG ET AL.: "Ccnet: Criss-cross attention for semantic segmentation", 《ARXIV.ORG》 * |
| 王哲伟: "基于深度学习的车道线检测算法研究", 《中国优秀博硕士学位论文全文数据库(硕士) 工程科技Ⅱ辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113435284B (en) | 2022-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111696094B (en) | Immunohistochemical PD-L1 membrane staining pathological section image processing method, device and equipment | |
| CN111612008B (en) | Image segmentation method based on convolution network | |
| Abdollahi et al. | Improving road semantic segmentation using generative adversarial network | |
| CN109493119B (en) | POI data-based urban business center identification method and system | |
| CN113780296A (en) | Remote sensing image semantic segmentation method and system based on multi-scale information fusion | |
| Su et al. | Deep convolutional neural network–based pixel-wise landslide inventory mapping | |
| CN110826429A (en) | Scenic spot video-based method and system for automatically monitoring travel emergency | |
| CN115239644B (en) | Concrete defect identification method, device, computer equipment and storage medium | |
| JP2023501820A (en) | Face parsing methods and related devices | |
| CN113012177A (en) | Three-dimensional point cloud segmentation method based on geometric feature extraction and edge perception coding | |
| CN113065551A (en) | Method for performing image segmentation using a deep neural network model | |
| CN111523439A (en) | Method, system, device and medium for target detection based on deep learning | |
| CN115565071A (en) | Hyperspectral image transform network training and classifying method | |
| CN114639102A (en) | Cell segmentation method and device based on key point and size regression | |
| CN112819753B (en) | Building change detection method and device, intelligent terminal and storage medium | |
| CN114332484A (en) | Key point detection method and device, computer equipment and storage medium | |
| CN116503677B (en) | Wetland classification information extraction method, system, electronic equipment and storage medium | |
| CN116342628B (en) | Pathological image segmentation method, pathological image segmentation device and computer equipment | |
| CN113435284B (en) | Post-disaster road extraction method based on dynamic filtering and multi-direction attention fusion | |
| CN115908363B (en) | Tumor cell statistics method, device, equipment and storage medium | |
| CN116994721A (en) | Quick processing system of digital pathological section graph | |
| CN111340139A (en) | Method and device for judging complexity of image content | |
| CN112232102A (en) | Building target identification method and system based on deep neural network and multitask learning | |
| Chen et al. | Built-up Area Extraction Combing Densely Connected Dual-Attention Network and Multi-Scale Context | |
| Cui et al. | CM-Unet: A novel remote sensing image segmentation method based on improved U-Net |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |