CN116310795A - A SAR aircraft detection method, system, device and storage medium - Google Patents

Abstract

The invention discloses a SAR aircraft detection method, system, device and storage medium. The method includes: acquiring an input SAR image; using an aircraft detection model to analyze the input SAR image to obtain an aircraft detection result; Generated by SAR image dataset training with marked aircraft category and target frame; aircraft detection model includes classification branch and regression branch, classification branch has deformable area association module, deformable area association module through deformable convolution branch and conventional convolution branch Perform feature weighted integration. The invention constructs classification branches through deformable association modules, significantly improves feature association capabilities, and performs model training through SAR image data sets marked with aircraft categories and target borders, and fully utilizes prior knowledge of SAR aircraft scattering feature information. The invention improves the performance of detecting aircraft in complicated SAR images, and realizes accurate detection and identification of aircraft.

Description

A SAR aircraft detection method, system, device and storage medium

technical field

The invention relates to the technical field of aircraft detection, in particular to a SAR aircraft detection method, system, device and storage medium.

Background technique

Synthetic Aperture Radar (SAR) is an active microwave imaging sensor, which plays an important role in the field of automatic target recognition because of its all-day and all-weather imaging observation capabilities. In the field of automatic object recognition, aircraft detection is a valuable application field. For example, in the civilian field, the dynamic monitoring of aircraft behavior is beneficial to the effective management of airports; in the military field, fast and accurate aircraft detection is of great significance for providing military reconnaissance information. Therefore, accurate detection of aircraft in high-resolution SAR images is a subject of significant value.

SAR transmits electromagnetic waves to the object through the antenna, receives the reflected electromagnetic waves, and finally forms an image through the recorded echo information. Due to its unique imaging mechanism, SAR images present complex appearances that are difficult to interpret. The original method of detecting objects in SAR images is constant false alarm detection (CFAR), which utilizes the property that object echoes are often stronger than background echoes, and extracts regions with strong echoes from SAR images as existing targets. However, in complex scenarios, such as airports with various other equipment or buildings, the detection performance of CFAR will be greatly affected, and the target cannot be accurately identified. In addition, the existing aircraft detection methods also require the ability to identify the category of the target, which cannot be achieved by CFAR.

There are various classic reflection structures such as dihedral angles, trihedral angles, and top hats on the aircraft. There are various scattering mechanisms under the irradiation of radar, such as direct scattering, multiple scattering, and diffraction scattering. Therefore, in the subsequent aircraft identification method, experts in the SAR field proposed to use the scattering feature information of the aircraft in the SAR image and use the template matching method to detect, and the feature extraction for matching is a key step. For example, there is a method to use the Harris-Laplace corner detector to extract salient points with stable scattering features on the plane, and describe them with salient point vectors. Another method uses the Gaussian mixture model to extract the scattering structure features including the strong scattering points of the aircraft and their corresponding distribution. However, due to the insufficient manual feature extraction and the inefficiency of matching the measured image with a variety of candidate templates, such methods still have shortcomings in accuracy and speed.

With the rapid development of SAR technology, more high-resolution and high-quality SAR images with expert annotations can be obtained, which provides favorable conditions for the application of deep learning methods in SAR target automatic recognition. In recent years, deep learning methods based on convolutional neural network (CNN) have made remarkable progress in target detection, and many CNN methods have shown better performance than traditional methods in SAR target detection, such as YOLOX, Cascade R-CNN, CenterNet, RepPoints, and more. However, most CNN methods are designed for optical object detection and cannot fully exploit their detection performance when directly introduced to SAR aircraft detection without considering aircraft scattering characteristics. The scattering characteristics of the aircraft are embodied in the following two points; 1) Discreteness. Due to the irregular distribution of various reflection structures on the aircraft, the radar cross-sectional area (RadarCross Section, RCS) of each part of the aircraft is different, so the aircraft in the SAR image appears as a collection of discrete scattering points. 2) Variability. Due to the complex structure of the aircraft, the existence of multiple scattering mechanisms makes the imaging results change with the incident angle and sensor parameters. Therefore, under different imaging conditions, even the scattering results of the same target may vary greatly. Under the SAR imaging mechanism, the SAR aircraft image presents completely different characteristics from the corresponding optical image. Therefore, existing CNN methods cannot sufficiently extract aircraft features through conventional convolutions.

In view of this, how to use the prior knowledge of SAR aircraft scattering feature information to achieve high-precision aircraft detection performance is an urgent problem to be solved.

Contents of the invention

In view of this, embodiments of the present invention provide a SAR aircraft detection method, system, device, and storage medium, which can effectively improve the detection and recognition performance of aircraft by network models.

On the one hand, the embodiment of the present invention provides a kind of SAR aircraft detection method, comprising:

Get the input SAR image;

Use the aircraft detection model to analyze the input SAR image and obtain the aircraft detection results;

Wherein, the aircraft detection result includes at least one target frame regression result and the category confidence score of the corresponding aircraft;

The aircraft detection model is generated by training the SAR image data set that has marked the aircraft category and the target frame; the aircraft detection model includes a classification branch and a regression branch. The classification branch has a deformable area association module, and the deformable area association module uses the deformable convolution branch Feature weighted integration with conventional convolution branches.

Optionally, also include:

Create a deformable scattering feature association network model based on a classification branch and a regression branch with a deformable region association module;

Among them, the deformable scattering feature association network model includes a Swin Transformer backbone, a path aggregation feature pyramid network, and a decoupling head; the decoupling head includes a classification branch and a regression branch with a deformable area association module.

Optionally, based on the classification branch and the regression branch with the deformable region association module, a deformable scattering feature association network model is created, including:

Construct the YOLOX model; among them, the YOLOX model includes the Darknet-53 backbone, the path aggregation feature pyramid network and the original decoupling head; the original decoupling head includes the original classification branch and regression branch;

By introducing the Swin Transformer backbone to replace the Darknet-53 backbone of the YOLOX model, and by introducing a classification branch with a deformable region association module to replace the original classification branch of the YOLOX model, a deformable scattering feature association network model is created;

Among them, the Swin Transformer backbone includes 24 layers of Swin Transformer layers.

Optionally, also include:

Determine the training samples according to the SAR image data set that has marked the aircraft category and the target frame; wherein, the target frame is marked with a preset number of strong scattering areas;

According to the training samples and based on the preset training parameters, the stochastic gradient descent SGD training is used to classify and regress the deformable scattering feature association network model, and combine the model loss function to optimize the network model parameters through the loss value backpropagation method to obtain aircraft detection Model.

Optionally, the aircraft detection model is used to analyze the input SAR image to obtain aircraft detection results, including:

Carry out segmentation splicing and linear mapping on the SAR image to obtain the first feature map;

Through the Swin Transformer backbone, use continuous Swin Transformer layers to alternately perform regular window division and transfer window division, and obtain the second feature map of the preset specification;

Through the path aggregation feature pyramid network, the semantic feature extraction is performed on the second feature map to obtain the third feature map with preset specifications; wherein, the semantic feature extraction includes upsampling, splicing and convolution;

Predict the position of the third feature map through the regression branch to obtain the regression result of the target border; wherein, the regression result of the target border includes the center point, width and height of the border;

Through the classification branch with the deformable area association module, use the deformable convolution branch and the conventional convolution branch to carry out feature weighted integration on the third feature map, and obtain the category confidence score of the aircraft of the preset category corresponding to the regression result of the target frame .

Optionally, the Swin Transformer layer includes a normalization function, a window-based multi-head self-attention mechanism and a multi-layer perceptron, and the steps of the Swin Transformer layer processing feature maps include:

The target feature map is normalized by a normalization function to obtain the feature map X ₂ ;

According to the feature map X ₂ , using the window-based multi-head self-attention mechanism, the feature map X ₃ is obtained through linear mapping and splicing based on the channel dimension;

According to the addition of the target feature map and the feature map X ₃ , the feature map X ₄ is obtained;

The feature map _X4 is normalized by a normalization function to obtain the feature map _X5 ;

Through the multi-layer perceptron, use the GELU nonlinear activator to activate the feature map _X5 , and obtain the feature map _X6 ; wherein, the multi-layer perceptron includes two fully connected layers;

According to the addition of the feature map X ₄ and the feature map X ₆ , the feature map X ₇ is obtained.

Optionally, through the classification branch with the deformable area association module, the deformable convolution branch and the conventional convolution branch are used to carry out feature weighted integration on the third feature map to obtain the aircraft of the preset category corresponding to the regression result of the target frame Category confidence scores, including:

Through the deformable convolution branch, perform the first convolution on the third feature map to obtain the feature map X ₉ ;

Perform the second convolution on the feature map _X9 to obtain the sampling point offset map; perform the third convolution on the feature map _X9 to obtain the score mask;

According to the feature map X ₉ , the sampling point offset map and the score mask, deformable convolution is performed to obtain the feature map Y;

Through the conventional convolution branch, perform two consecutive convolutions on the third feature map to obtain the feature map Z;

Based on the preset learnable hyperparameters, the feature map Y and the feature map Z are weighted and added and the feature channel information is integrated to obtain the category confidence score of the preset category of aircraft corresponding to the target frame regression result.

On the other hand, embodiments of the present invention provide a SAR aircraft detection system, including:

The first module is used to obtain the input SAR image;

The second module is used to use the aircraft detection model to analyze the input SAR image to obtain the aircraft detection result;

Wherein, the aircraft detection result includes at least one target frame regression result and the category confidence score of the corresponding aircraft;

The aircraft detection model is generated by training the SAR image data set that has marked the aircraft category and the target frame; the aircraft detection model includes a classification branch and a regression branch. The classification branch has a deformable area association module, and the deformable area association module uses the deformable convolution branch Feature weighted integration with conventional convolution branches.

On the other hand, an embodiment of the present invention provides a SAR aircraft detection device, including a processor and a memory;

The memory is used to store programs;

The processor executes the program to realize the method as above.

On the other hand, an embodiment of the present invention provides a computer-readable storage medium, where a program is stored in the storage medium, and the program is executed by a processor to implement the foregoing method.

The embodiment of the present invention also discloses a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

In the embodiment of the present invention, the input SAR image is first obtained; the aircraft detection model is used to analyze the input SAR image, and the aircraft detection result is obtained; wherein, the aircraft detection result includes at least one target frame regression result and the corresponding category confidence score of the aircraft ;The aircraft detection model is generated by training the SAR image data set that has marked the aircraft category and the target frame; the aircraft detection model includes a classification branch and a regression branch. The classification branch has a deformable area association module, and the deformable area association module is integrated through the deformable volume Feature-weighted integration of branch and conventional convolution branch. In the present invention, the classification branch is constructed by introducing a deformable association module, which significantly improves the feature association ability, further performs model training through the SAR image data set with marked aircraft category and target frame, and makes full use of the prior knowledge of SAR aircraft scattering feature information. The invention improves the performance of detecting aircraft in complicated SAR images, and realizes accurate detection and identification of aircraft.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

Fig. 1 is a schematic flow chart of a SAR aircraft detection method provided by an embodiment of the present invention;

Figure 2 is a schematic diagram of the structure of the YOLOX model;

Fig. 3 is a schematic structural diagram of the Swin Transformer provided by the embodiment of the present invention;

Fig. 4 is the structural representation of the Swin Transformer layer that the embodiment of the present invention provides;

FIG. 5 is a schematic structural diagram of a DRCM provided by an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a DSFCN provided by an embodiment of the present invention;

Fig. 7 is a schematic flowchart of constructing a DSFCN model provided by an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

First of all, it should be explained that the existing SAR aircraft detection technologies include:

1. Fu et al. proposed a paper "Aircraft Recognition in SAR Images Based on Scattering Structure Feature and Template Matching." (IEEE Journal of SelectedTopics in Applied Earth Observation and Remote Sensing, (2018) 4206-4217.) An aircraft recognition method based on template matching, which analyzes the scattering characteristics of the aircraft, and uses a Gaussian mixture model to extract the scattering structure features of the target. In the matching stage, the proposed sample decision optimization algorithm is used to improve the efficiency of candidate template selection, and a coordinate translation Kullback-Leibler divergence method is proposed in the detection stage to achieve the translation invariance of detection. However, this improved template matching method still has the problems of low matching accuracy and the need to manually craft features according to the attributes of specific data, making it difficult to apply to other resolution SAR images.

2. In their paper "Pyramid Attention Dilated Network for Aircraft Detection in SAR Images." (IEEE GEOSCI ENCE AND REMOTE SENSINGLETTERS, (2021) 662-666.), Zhao et al. proposed a pyramid attention expansion network to detect airplane. This method considers the discrete characteristics of SAR aircraft, uses the multi-branch dilated convolution module to enhance the relationship between the discrete backscattering features of the aircraft, and uses the convolutional fast attention module to refine redundant information and highlight the salient features of the aircraft . Although this method considers the discrete features of the aircraft, the sampling flexibility of the convolution module used to extract features is still limited, and it can only be sampled on a regular grid, which is not enough to fully model the relationship between important discrete areas of the aircraft , which limits the detection performance.

3. Guo et al. proposed a scattering enhanced attention pyramid network in their paper "Scattering Enhanced Attention Pyramid Network for Aircraft Detection in SAR Images." (IEEE Transactions on Geoscience and Remote Sensing, (2021) 7570-7587.) to inspect the aircraft. The method uses the Harris-Laplace corner detector to extract the strong scattering points of the aircraft, and then uses the density-based noise space clustering method and the Gaussian mixture model to model, and finally uses the Kullback-Leibler divergence to measure the distance between the known target and the template. Correlation, if it matches, the scattering information of the SAR picture will be enhanced and then sent to the network. Although this method considers the idea of using prior knowledge to model SAR aircraft and then assisting the network to extract features, it only performs γ-distributed CFAR enhancement on some images in the image preprocessing stage, which cannot give full play to the powerful modeling capabilities of the network.

However, the relevant prior art has the following disadvantages: 1. The existing detection method based on template matching has low matching accuracy, and needs to manually make features according to the attributes of specific data, which is not suitable for SAR data with different resolutions. 2. Most of the existing CNN-based detection methods were originally designed for optical image detection tasks. When introducing SAR aircraft detection, there is still the problem of insufficient consideration of aircraft scattering characteristics in SAR images, resulting in limited detection effects. 3. The existing method of considering the aircraft scattering characteristics of the SAR image to assist the network to extract features has two independent processes, the manual feature extraction and the network feature extraction, and there is no problem of organic integration, which is not conducive to the significant improvement of network performance.

In view of this, on the one hand, with reference to FIG. 1, an embodiment of the present invention provides a SAR aircraft detection method, including:

S100. Obtain an input SAR image;

Obtain the input SAR image for aircraft detection, usually using 1m resolution SAR image, and uniformly 640×640 pixels.

S200. Using the aircraft detection model, analyze the input SAR image to obtain an aircraft detection result;

It should be noted that the aircraft detection results include at least one target frame regression result and the category confidence score of the corresponding aircraft; the aircraft detection model is generated by training the SAR image dataset that has marked the aircraft category and target frame; the aircraft detection model includes classification Branch and regression branch, the classification branch has a deformable area association module, and the deformable area association module performs feature weighted integration through the deformable convolution branch and the conventional convolution branch;

In some embodiments, it also includes: based on the classification branch and the regression branch with the deformable area association module, creating a deformable scattering feature association network model; wherein, the deformable scattering feature association network model includes SwinTransformer backbone, path aggregation feature pyramid network and Decoupling head; the decoupling head includes a classification branch and a regression branch with deformable region association modules. Among them, based on the classification branch and the regression branch with the deformable area association module, the deformable scattering feature association network model is created, including: constructing the YOLOX model; wherein, the YOLOX model includes the Darknet-53 backbone, the path aggregation feature pyramid network and the original decoupling head; the original decoupling head includes the original classification branch and regression branch; by introducing the Swin Transformer backbone to replace the Darknet-53 backbone of the YOLOX model, and by introducing the classification branch with a deformable region association module to replace the original classification branch of the YOLOX model, the created Deformable scattering feature association network model; among them, the Swin Transformer backbone includes 24 layers of Swin Transformer layers.

Among them, in some embodiments, it also includes: according to the SAR image data set of the marked aircraft category and the target frame, determine the training sample; wherein, the target frame is marked with a preset number of strong scattering areas; according to the training sample, based on the preset For training parameters, the stochastic gradient descent SGD training is used to classify and regression train the deformable scattering feature association network model, and combine the model loss function to optimize the network model parameters through the loss value backpropagation method to obtain the aircraft detection model.

Wherein, in some specific embodiments, the specific steps of obtaining the SAR image data set of marked aircraft category and target frame are as follows:

Step (1): The 1m resolution SAR images collected from GF-3 satellites around the world are uniformly cropped to a size of 640×640 pixels, and a data set of 2204 SAR images with 5429 aircraft in 7 categories is obtained. Among them, the category of each aircraft and the target rectangular frame (i.e. target frame) are marked by experts. The aircraft types are divided into A220, A320-321, A330, ARJ21, Boeing737-800, Boeing787 and others according to the model. The target rectangular frame is the outer frame of the aircraft. Cut rectangles.

Step (2): Randomly divide the SAR images into a training set (ie training samples) and a test set at a ratio of 3:1.

Step (3): mark the strong scattering region (Strong Scattering Region, SSR), for each aircraft in the SAR picture, extract 25 SSRs within its target rectangular frame, further, the steps are:

Step (31): Perform mean filtering on the pixels in the border, where the radius of the filter is one-tenth of the shorter side of the border.

Step (32): Select the point with the largest value from the pixels in the filtered border as the center point of the first circular SSR with a radius.

Step (33): exclude the pixels falling in the first SSR, and select the point with the largest value from the remaining pixels as the center point of the second SSR.

Step (34): Repeat the above steps to finally get 25 SSRs.

Specifically, the aircraft detection model is used to analyze the input SAR image, and the aircraft detection result is obtained, including: segmenting and splicing the SAR image and linear mapping to obtain the first feature map; through the Swin Transformer backbone, using continuous Swin Transformer layers Alternately perform conventional window division and transfer window division to obtain the second feature map of the preset specification; through the path aggregation feature pyramid network, perform semantic feature extraction on the second feature map to obtain the third feature map of the preset specification; where, semantic Feature extraction includes upsampling, splicing and convolution; through the regression branch, the position prediction of the third feature map is performed to obtain the target frame regression result; wherein, the target frame regression result includes the center point, width and height of the frame; through a deformable The classification branch of the area association module uses the deformable convolution branch and the conventional convolution branch to carry out feature weighted integration on the third feature map to obtain the category confidence score of the preset category of aircraft corresponding to the target frame regression result. It should be noted that the target frame regression results include the regression results of several target frames detected and recognized in the SAR image, and the number of aircraft category confidence scores is the same as the number of target frames, and one-to-one corresponds to each target frame. airplane.

Wherein, in some embodiments, the Swin Transformer layer includes a normalization function, a window-based multi-head self-attention mechanism and a multi-layer perceptron, and the step of processing the feature map in the Swin Transformer layer includes: applying a normalization function to the target feature map Perform normalization to obtain the feature map X ₂ ; according to the feature map X ₂ , use the window-based multi-head self-attention mechanism to obtain the feature map X ₃ through linear mapping and splicing based on the channel dimension; according to the target feature map and feature map X ₃ are added together to obtain the feature map X ₄ ; the feature map X ₄ is normalized by a normalization function to obtain the feature map X ₅ ; the feature map X ₅ is activated by a GELU nonlinear activator through a multi-layer perceptron, The feature map X ₆ is obtained; wherein, the multi-layer perceptron includes two fully connected layers; the feature map X ₇ is obtained according to the addition of the feature map X ₄ and the feature map X ₆ .

Among them, in some embodiments, through the classification branch with the deformable region association module, the deformable convolution branch and the conventional convolution branch are used to carry out feature weighted integration on the third feature map to obtain the preset category corresponding to the regression result of the target frame The category confidence score of the aircraft, including: through the deformable convolution branch, the first convolution is performed on the third feature map to obtain the feature map X ₉ ; the second convolution is performed on the feature map X ₉ to obtain the sampling point offset map ; The third convolution is performed on the feature map X ₉ to obtain the score mask; according to the feature map X ₉ , the sampling point offset map and the score mask, deformable convolution is performed to obtain the feature map Y; through the conventional convolution branch, the The third feature map performs two consecutive convolutions to obtain the feature map Z; based on the preset learnable hyperparameters, the feature map Y and the feature map Z are weighted and added and the feature channel information is integrated to obtain the corresponding target border regression result Class confidence scores for aircraft in the preset class of .

In some specific embodiments, creating a deformable scattering feature association network (DSFCN) model includes the following steps:

(1) As shown in Figure 2, first construct the YOLOX model. Among them, the YOLOX model includes the Darknet-53 backbone, Path Aggregation Feature Pyramid Network (Path Aggregation Feature Pyramid Network, PAFPN) and the original decoupled head (Decoupled Head); the original decoupled head includes the original classification branch (Classification) and regression branch (Regression ). The steps of the YOLOX model for aircraft detection include:

1.1. For an input 640×640 SAR picture, the basic semantic features are extracted through the Darknet-53 backbone, and three feature maps of 80×80, 40×40 and 20×20 are output.

1.2. Send the three feature maps to the neck of the Path Aggregation Feature Pyramid Network (PAFPN), fully generate semantic features through operations such as upsampling, splicing, and convolution, and output 80×80, 40×40 and 20×20 feature maps of three different sizes.

1.3. Send the three feature maps to the decoupled head (the original decoupled head) to output the final prediction result. The decoupling head includes a 3×3 convolutional layer and two branches stacked by 3×3 convolutional layers: classification branch Classification and regression branch Regression. The regression branch outputs the regression result of the target frame predicted by each position on the feature map, including the center point, width and height of the frame, and whether the frame contains the confidence score of the target; the classification branch outputs each possible contained in the frame predicted by the regression branch. Confidence score for a category.

(2) Introduce the Swin Transformer backbone to replace the Darknet53 backbone (the structure of the Swin Transformer backbone is shown in Figure 3), which includes a total of 24 layers of Swin Transformer layers (STL) (the structure of the Swin Transformer layer is shown in Figure 4). For the Swin Transformer backbone, the steps for SAR image processing are:

2.1. The input SAR image is evenly divided into 16 parts and stitched together, and then the feature map X ₁ is obtained through linear mapping.

2.2. The feature map X ₁ is normalized by a layer normalization function (Layer Normalization, LN) to obtain the feature map X ₂ .

2.3. Use Window-based Multi-head Self-attention (W-MSA) to extract features. The feature map is first divided into non-overlapping local windows with 7×7 feature vectors, and all feature vectors in each window are regarded as tokens. Then use linear mapping to get query vector matrix (Q), keyword vector matrix (K) and content vector matrix (V) of all tokens. The corresponding expression is as follows:

Q＝X ₂ P _Q , K＝X ₂ P _K , V＝X ₂ P _V

Among them, P _Q , P _K and _PV represent three matrices of learnable linear maps.

Perform a separate self-attention mechanism on the obtained matrix, that is, divide the obtained Q, K, and V vectors into n groups along the channel dimension, and perform self-attention on each subgroup Q _s , K _s , and V _s in the n groups. attention mechanism. The corresponding expression is as follows:

Among them, B represents the learnable relative position encoding matrix, and d represents the channel dimension of Q, K, V vectors. Then the results obtained by each subgroup are spliced along the channel dimension to obtain the value of each feature vector on the output feature map X ₃ .

2.4. The feature map X ₃ is added to the original input feature map X ₁ to obtain the feature map X ₄ .

2.5. Feature map X ₄ obtains feature map X ₅ through an LN normalized feature.

2.6. The feature map X ₅ obtains the feature map X ₆ through a multilayer perceptron (Multilayer Perceptron, MLP) with two fully connected layers and a GELU nonlinear activator in the middle.

2.7. The feature map X ₆ is added to the feature map X ₄ obtained in step 2.4 to obtain the feature map X ₇ .

2.8. Steps (2.2) to (2.7) are a process of STL feature extraction, wherein, the above-mentioned X ₁ to X ₇ . It is only used to symbolize the feature map data of the corresponding stage, and cannot be regarded as a restriction on the feature map data. In different STLs in the Swin Transformer backbone, X ₂ to X ₇ corresponding to each stage are different. Among them, the target feature map Including feature map X ₁ and X ₇ output by the previous layer of STL. The STL division window is divided into two forms, one is to divide the local window directly from the feature map (0,0), which is called conventional window division; the other is to divide the window from (3,3) point, which is called transfer window division. The feature map repeats steps (2.2) to (2.7), and regular window division and transfer window division are alternately performed in consecutive STLs. At the 4th, 8th, 20th, and 24th STL, the feature map is evenly divided into 4 parts and stitched together to obtain a feature map with a size halved and a dimension doubled, and finally output 80×80, 40 at the 8th, 20, and 24th STL ×40 and 20×20 three feature maps of different sizes (that is, the second feature map of the preset specification).

(3) The deformable region association module DRCM is introduced to replace the classification branch of YOLOX (the structure of DRCM is shown in Figure 5). DRCM can be divided into deformable convolution branch (top) and conventional convolution branch (bottom). The steps of data processing by DRCM include:

3.1. The feature map X ₈ (that is, the third feature map) after the feature extraction by PAFPN is performed on the upper branch to perform 3×3 convolution to obtain the feature map X ₉ .

3.2 and X ₉ go through two convolutional layers respectively to obtain a sampling point offset map Δp with a dimension of H×W×2 and a fractional mask Δm with a dimension of H×W×1.

3.3. Embed Δp and Δm into a 5×5 deformable convolution, and X ₉ obtains an output feature map Y through this convolution. The expression is as follows:

Among them, p represents the position on the feature map, k represents the k-th sampling point, ω _k represents the weight of the k-th offset sampling point, p+p _k represents the k-th sampling position of the 5×5 regular convolution, Δp _k represents the offset of the k-th sampling position of the 5×5 regular convolution, and Δm _k represents the fractional value of the k-th offset sampling point.

3.4. In the following branch, perform two consecutive 3×3 convolutions on the feature map X ₈ to obtain the feature map Z. Set two learnable hyperparameters α and β, where α+β=1, add Y and Z weights through α and β, and integrate feature channel information through a 1×1 convolution, and output each position on the feature map Confidence score for class 7 aircraft.

In some specific embodiments, the specific steps of optimizing the network model parameters through the loss value backpropagation method based on the loss function are as follows:

4.1. In the classification branch, calculate the cross-entropy classification loss function with Sigmoid activation for the score vector of each position on the feature map.

Among them, N is the number of categories of the aircraft; y _i represents whether it is the i-th type of target, if so, it is 1, otherwise it is 0; p _i represents the confidence score of the network predicting the i-th type of target. Sigmoid computes activations for input x.

4.2. Take the average of all classification losses to obtain the total classification loss L _cls .

L _cls = mean(l _cls )

4.3. Calculate the chamfering distance loss for the offset sampling points that are responsible for predicting the position of the target on the feature map.

代表在第i个目标边框中，第n个预测出的偏移采样点位置；/>

代表在第i个目标边框中，第m个标记的强散射区域的中心点位置；rⁱ代表第i个目标边框中标记的强散射区域的半径。in,

Represents the position of the nth predicted offset sampling point in the i-th target frame; />

Represents the center point position of the m-th marked strong scattering area in the i-th target frame; r ⁱ represents the radius of the marked strong-scattering area in the i-th target frame.

4.4. Average all SSR prediction losses to obtain the total SSR prediction loss.

L _ssr = mean(l _ssr )

4.5. In the regression branch, calculate the Intersection over Union (IoU) regression loss function for the bounding box (boundingbox, bbox) that is responsible for predicting the position of the target on the feature map.

Among them, bbox _pre is the bounding box predicted by the network model, bbox _gt is the real bounding box label of the target, Intersection is a function to calculate the intersection, and Union is a function to calculate the union.

4.6. Take the average of all regression losses to obtain the total regression loss L _reg .

L _reg = mean(l _reg )

4.7. In the regression branch, the 1-norm loss is calculated for the score of whether there is a target predicted for each position on the feature map.

为网络模型的预测值。Among them, y is the true value, which is 1 when there is a target, and 0 otherwise,

is the predicted value of the network model.

4.8. Average the target existence loss of all positions to obtain the total target existence loss.

L _obj ＝mean(l _obj )

4.9. Calculate the total loss value of the training.

L＝αL _cls +βL _reg +γL _obj

Wherein, the weighting coefficients α=3, β=3, γ=1, λ=1.

Finally, the network model parameters are optimized through the loss value backpropagation method.

In some specific embodiments, based on the preset training parameters, the specific steps for classifying and regressing the deformable scattering feature association network model using stochastic gradient descent SGD training are as follows:

5.1. Use stochastic gradient descent SGD training, the number of cycles is 100, the batch size is 8, and the weight decay parameter and momentum of the optimizer are set to 0.0005 and 0.9, respectively.

5.2. The learning rate is initialized to 0.002, and the cosine annealing strategy is used to gradually decrease to 0.0001 with the number of cycles.

5.3. Given a batch of input SAR images, first pass through the Swin Transformer backbone, then pass through the PAFPN neck to fully generate semantic features, and then pass through DRCM to output classification results and output regression results through the regression branch. And adjust the model parameters based on the output results.

In some specific embodiments, it also includes the step of testing the deformable scattering feature association network (DSFCN) model, as shown in Figure 6, specifically including the following steps:

6.1. For the trained DSFCN model, calculate the precision rate (Precision), recall rate (Recall), and mean average precision (mAP) through the test data set, where the score of the predicted frame is whether the target is included in the regression branch The product of the confidence score of the classification branch and the highest category confidence score in the classification branch, when the value is greater than 0.5, it is regarded as the existence of the target, and the category is the category with the highest confidence in the classification branch; When it is greater than 0.5 and the categories are consistent, it is considered a successful match.

6.2. Calculation of model parameters.

6.3. Visualize the activation degree of each image area to the detection result through the feature visualization technology ScoreCAM.

6.4 Visualizing offset-sampled prediction results of deformable convolutions in DRCM.

Among them, as shown in Figure 7, the overall construction steps of the DSFCN model are:

(1) Obtain the input SAR image. It should be noted that the SAR image here represents the SAR image data set that has marked the aircraft category and the target frame;

(2) Construct DSFCN model; including: construct YOLOX model; introduce Swin Transformer; introduce DRCM; construct model loss function;

(3) Training DSFCN model;

(4) Test the DSFCN model.

Specifically, in some embodiments, the effects of the present invention are verified through ablation experiments, and the results of the ablation experiments in steps 6.1 and 6.2 are shown in Table 1.

Table 1

The performance comparison results with other existing algorithms are shown in Table 2.

Table 2

As can be seen from the results in Tables 1 and 2, the two improved modules proposed by the present invention enable the network model to achieve higher scores on various indicators while reducing the amount of parameters, significantly improving the detection performance, and better than other existing algorithms. The superior performance of the network model shows that the Swin Transformer module with a self-attention mechanism can finely extract the scattering features of the SAR image, and the DRCM with flexible sampling capabilities can correlate the salient features of the aircraft to adapt to the variability of the aircraft in the SAR image .

In summary, the purpose of the present invention is to utilize the scattering characteristics of aircraft discreteness and variability in SAR images, improve the modules in the neural network model to fully extract aircraft features, and improve the detection and recognition performance of the network model for aircraft. Convolution is based on the idea of template matching to model the relationship between adjacent sampling points, so it has a local inductive bias, that is, adjacent sampling points have a strong correlation. The self-attention mechanism used by Swin Transformer can be regarded as an adaptive filter, and its weight is determined by the correlation between query vector and keyword vector between points, which is more suitable for long-range dependent information of sampling points. The present invention considers that the aircraft appears as a collection of discrete and sparsely distributed scattering points in the SAR image, and the correlation between the pixels on the aircraft is relatively weak. Therefore, the backbone of the traditional convolution architecture is abandoned and the backbone of the Swin Transformer is adopted, so that the network The model is able to more fully extract the scattering characteristics of the aircraft. To solve the limitation that original convolutions can only be sampled on regular grids, the present invention embeds deformable convolutions into the classification branch. At the same time, considering that in the face of the variability of SAR aircraft, the strong scattering area on the aircraft is the key to identifying the aircraft, the present invention adds the supervision information of the strong scattering area in the form of a loss function to the network model training process to guide the offset sampling point forecast. Compared with the previous method of manually extracting features and then preprocessing and enhancing pictures, the present invention makes full use of the powerful modeling capabilities of the network, and proposes DRCM to adaptively associate strong scattering areas with prominent aircraft features, realizing the manual extraction of features and network The organic combination of extracted features has stronger adaptability. In the embodiment of the present invention, on the basis of the advanced YOLOX detector, the Swin Transformer with self-attention mechanism is used as the backbone of the network to extract features from the original high-resolution SAR image, and the detection head is added with the ability to automatically correlate significant areas of the aircraft The Deformable Region Correlation Module (DRCM) was used to build a new SAR aircraft detector named Deformable Scatter Feature Correlation Network (DSFCN). Compared with the prior art solutions, the algorithm of the present invention has excellent feature extraction and association capabilities, improves the performance of detecting aircraft in complex SAR images, and realizes more accurate detection and recognition capabilities of aircraft than other methods.

On the other hand, an embodiment of the present invention provides a SAR aircraft detection system, including: a first module for acquiring an input SAR image; a second module for analyzing the input SAR image by using an aircraft detection model , to obtain the aircraft detection result; wherein, the aircraft detection result includes at least one target frame regression result and the category confidence score of the corresponding aircraft; the aircraft detection model is generated by training the SAR image data set that has marked the aircraft category and the target frame; the aircraft detection The model includes a classification branch and a regression branch. The classification branch has a deformable area association module, and the deformable area association module performs feature weighted integration through the deformable convolution branch and the regular convolution branch.

The content of the method embodiment of the present invention is applicable to the system embodiment. The functions realized by the system embodiment are the same as those of the method embodiment above, and the beneficial effects achieved are also the same as those achieved by the above method.

Another aspect of the embodiments of the present invention also provides a SAR aircraft detection device, including a processor and a memory;

The memory is used to store programs;

The processor executes the program to realize the method as above.

The content of the method embodiment of the present invention is applicable to the device embodiment, and the specific functions realized by the device embodiment are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method.

Another aspect of the embodiments of the present invention also provides a computer-readable storage medium, where the storage medium stores a program, and the program is executed by a processor to implement the aforementioned method.

The content of the method embodiment of the present invention is applicable to the embodiment of the computer-readable storage medium. The functions realized by the embodiment of the computer-readable storage medium are the same as those of the above-mentioned method embodiment, and the beneficial effect achieved is the same as that achieved by the above-mentioned method. The effect is also the same.

The embodiment of the present invention also discloses a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

In some alternative implementations, the functions/operations noted in the block diagrams may occur out of the order noted in the operational diagrams. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/operations involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logical flow presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are performed independently.

Furthermore, although the invention has been described in the context of functional modules, it should be understood that one or more of the described functions and/or features may be integrated into a single physical device and/or unless stated to the contrary. or software modules, or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to understand the present invention. Rather, given the attributes, functions and internal relationships of the various functional blocks in the devices disclosed herein, the actual implementation of the blocks will be within the ordinary skill of the engineer. Accordingly, those skilled in the art can implement the present invention set forth in the claims without undue experimentation using ordinary techniques. It is also to be understood that the particular concepts disclosed are illustrative only and are not intended to limit the scope of the invention which is to be determined by the appended claims and their full scope of equivalents.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

The logic and/or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium, For use with an instruction execution device, device or device (such as a computer-based device, a device including a processor, or other devices that can fetch instructions from an instruction execution device, device or device and execute instructions), or in conjunction with these instruction execution devices, devices or equipment used. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use in or in conjunction with an instruction execution device, device or device.

More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection with one or more wires (electronic device), portable computer disk case (magnetic device), random access memory (RAM), Read Only Memory (ROM), Erasable and Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, as it may be possible, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or other suitable processing if necessary. The program is processed electronically and stored in computer memory.

It should be understood that various parts of the present invention can be realized by hardware, software, firmware or their combination. In the above-described embodiments, various steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques known in the art: Discrete logic circuits, ASICs with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions and modifications can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the invention is defined by the claims and their equivalents.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments, and those skilled in the art can also make various equivalent deformations or replacements without violating the spirit of the present invention. These equivalent modifications or replacements are all within the scope defined by the claims of the present invention.

Claims (10)

1. A SAR aircraft detection method, characterized in that, comprising: Get the input SAR image; Analyzing the input SAR image by using an aircraft detection model to obtain an aircraft detection result; Wherein, the aircraft detection result includes at least one target frame regression result and a corresponding aircraft category confidence score; The aircraft detection model is generated by training the SAR image data set that has marked the aircraft category and the target frame; the aircraft detection model includes a classification branch and a regression branch, and the classification branch has a deformable area association module, and the deformable area The association module performs feature weighted integration through deformable convolution branches and regular convolution branches. 2. A kind of SAR aircraft detection method according to claim 1, is characterized in that, also comprises: creating a deformable scattering feature association network model based on the classification branch and the regression branch having a deformable area association module; Wherein, the deformable scattering feature association network model includes a Swin Transformer backbone, a path aggregation feature pyramid network, and a decoupling head; the decoupling head includes the classification branch and the regression branch with a deformable area association module. 3. A kind of SAR aircraft detection method according to claim 2, is characterized in that, described based on the described classification branch and the regression branch with deformable area association module, create deformable scattering feature association network model, comprising: Build YOLOX model; Wherein, described YOLOX model comprises Darknet-53 backbone, path aggregation feature pyramid network and former decoupling head; Described former decoupling head comprises former classification branch and described regression branch; Deformable is created by introducing the Swin Transformer backbone to replace the Darknet-53 backbone of the YOLOX model, and by introducing the classification branch with a deformable region association module to replace the original classification branch of the YOLOX model Scattering feature association network model; Wherein, the Swin Transformer backbone includes 24 Swin Transformer layers. 4. A kind of SAR aircraft detection method according to claim 2, is characterized in that, also comprises: Determine the training sample according to the SAR image data set of the marked aircraft category and the target frame; wherein, the target frame is marked with a preset number of strong scattering areas; According to the training sample, based on the preset training parameters, the stochastic gradient descent SGD training is used to perform classification and regression training on the deformable scattering feature association network model, and optimize the network model parameters through the loss value backpropagation method in combination with the model loss function , to obtain the aircraft detection model. 5. a kind of SAR aircraft detection method according to claim 2, is characterized in that, described utilizing aircraft detection model, the SAR image of described input is analyzed, obtains aircraft detection result, comprises: Segmenting and splicing and linear mapping are performed on the SAR image to obtain a first feature map; Through the Swin Transformer backbone, using continuous Swin Transformer layers to alternately perform conventional window division and transfer window division to obtain a second feature map of a preset specification; Performing semantic feature extraction on the second feature map through the path aggregation feature pyramid network to obtain a third feature map of the preset specification; wherein the semantic feature extraction includes upsampling, splicing and convolution; Predicting the position of the third feature map through the regression branch to obtain a target frame regression result; wherein the target frame regression result includes the center point, width and height of the frame; Through the classification branch having a deformable area association module, the deformable convolution branch and the conventional convolution branch are used to carry out feature weighted integration on the third feature map to obtain a regression result corresponding to the target frame. Class confidence scores for aircraft in preset classes. 6. a kind of SAR aircraft detection method according to claim 5, is characterized in that, described Swin Transformer layer comprises normalization function, multi-head self-attention mechanism and multi-layer perceptron based on window, and described Swin Transformer layer The steps to process the feature map include: Normalize the target feature map by the normalization function to obtain the feature map X ₂ ; According to the feature map X ₂ , using the window-based multi-head self-attention mechanism, the feature map X ₃ is obtained through linear mapping and splicing based on channel dimensions; According to the addition of the target feature map and the feature map _X3 , the feature map _X4 is obtained; The feature map _X4 is normalized by the normalization function to obtain the feature map _X5 ; Through the multi-layer perceptron, use the GELU nonlinear activator to activate the feature map _X5 to obtain the feature map _X6 ; wherein, the multi-layer perceptron includes two fully connected layers; According to the addition of the feature map X ₄ and the feature map X ₆ , the feature map X ₇ is obtained. 7. A kind of SAR aircraft detection method according to claim 5, is characterized in that, described by having the described classification branch of deformable region association module, uses described deformable convolution branch and described conventional convolution branch Performing feature weighted integration on the third feature map to obtain the category confidence score of the aircraft of the preset category corresponding to the target frame regression result, including: Performing a first convolution on the third feature map through the deformable convolution branch to obtain a feature map X ₉ ; performing a second convolution on the feature map _X9 to obtain a sampling point offset map; performing a third convolution on the feature map _X9 to obtain a fractional mask; performing deformable convolution according to the feature map X ₉ , the sampling point offset map and the score mask to obtain a feature map Y; performing two consecutive convolutions on the third feature map through the conventional convolution branch to obtain a feature map z; Based on the preset learnable hyperparameters, the feature map Y and the feature map Z are weighted and added and feature channel information is integrated to obtain the category confidence of the preset category of aircraft corresponding to the target frame regression result. Fraction. 8. A SAR aircraft detection system, characterized in that, comprising: The first module is used to obtain the input SAR image; The second module is used to analyze the input SAR image by using the aircraft detection model to obtain the aircraft detection result; Wherein, the aircraft detection result includes at least one target frame regression result and a corresponding aircraft category confidence score; The aircraft detection model is generated by training the SAR image data set that has marked the aircraft category and the target frame; the aircraft detection model includes a classification branch and a regression branch, and the classification branch has a deformable area association module, and the deformable area The association module performs feature weighted integration through deformable convolution branches and regular convolution branches. 9. A SAR aircraft detection device, comprising a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1-7. 10. A computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images