CN115439706A - A multi-receptive field attention mechanism and system based on target detection - Google Patents

Abstract

The invention relates to a target detection-based multi-receptive-field attention mechanism and a system. The attention mechanism acquires feature maps of different receptive fields to obtain rich context information. Each feature map is divided into two groups, weight extraction of each feature map is realized through one-dimensional convolution of different receptive fields, then five groups of weights are fused and multiplied by input feature vectors, and channel attention learning of local cross-channel interaction without dimension reduction is realized. Experiments show that the module provided by the invention can improve the detection precision of different networks, and the visualization result shows that the networks provided by the invention have good detection effect in different scenes.

Description

A multi-receptive field attention mechanism and system based on target detection

technical field

The invention belongs to the field of target detection based on deep learning. Specifically, it refers to a multi-receptive field attention mechanism and system based on target detection.

Background technique

With the development of computer technology theory and the improvement of hardware equipment, object detection based on deep learning has been widely used. Due to the advantages of convolutional neural network (CNN) that can replace human manual feature extraction, CNN is widely used to extract image features.

The richness of the network for semantic feature extraction directly affects the accuracy of the target detection algorithm. The attention mechanism is mainly used to strengthen the network's attention to important channel features or to improve the network's attention to the region of interest in the picture. This method is often used in deep learning algorithms to improve semantic information.

The attention mechanism originated from the study of human vision and was first applied in the field of natural language to realize the efficient allocation of information processing resources. In recent years, the attention mechanism has been developed rapidly in the field of computer vision. In 2017, Dongcai Cheng and others proposed SENET. This attention mechanism realizes the channel attention mechanism through extrusion, incentives, and scale. SENET directly uses two fully connected layers to extract feature weights, but it is not very good. Good capture of feature dependencies between channels. Based on SENET, CABM adds a maximum pooling parallel branch in the extrusion stage, which makes the obtained information more comprehensive, but it still uses two fully connected layers to extract weights.

In order to improve the network's feature extraction of pictures, the attention mechanism usually adopts methods such as expanding the receptive field to enhance global information, extracting weights to strengthen or suppress channel features, etc. This paper proposes a multi-receptive field attention mechanism. It uses 4 parallel branches with different receptive fields, which not only expands the receptive field, but also prevents the loss of details caused by too large receptive field. Each branch is divided into two groups, and one-dimensional convolutions of different sizes are used to obtain channel weights, which can effectively prevent channel dependency loss caused by channel drop.

Contents of the invention

In order to overcome the above shortcomings, the present invention proposes a multi-receptive field attention mechanism and system based on target detection, which not only expands the receptive field, but also effectively prevents the loss of channel dependence caused by channel drop.

A kind of multi-receptive field attention mechanism based on target detection designed by the present invention is special in that it includes the following steps:

Step 1. Convolute the batch, channel number, spatial height and width of the input image with four different convolution kernels to obtain four tensors [x1,x2,x3,x4] with different receptive fields. and sum all tensors to get x5;

Step 2, divide the above 5 tensors into two groups in the channel dimension, use two C modules with different convolution kernel sizes to obtain the channel weight of each group, and then connect the two groups in the channel dimension, to get the weight of each tensor;

Among them, the C module is represented by the following formula:

F _{C module k*k} (X _CH )＝F _un F _sg W _1d F _s F _a X _CH

Among them, X _CH is the input feature map, F _a is an adaptive average pooling operator, F _sg is a sigmoid operator, W _1d is a k×k convolutional layer, F _s is a compression and exchange operator, F _un is a decompression and exchange operator, k*k is the convolution kernel used in local cross-channel interaction; the output formula of weight 5 is .

X _{5_1} , X _{5_2} = F _sp X ₅

weight5＝concat(F _{C module 3*3} (X _{5_1} ),F _{C module 5*5} (X _{5_2} ))

Among them, F _sp is a group operation operator, and concat is a splicing operator;

Step 3. Finally, fuse all channel weights, and then multiply the weights by the corresponding input vectors. And get the output after the channel is shuffled; the channel shuffling operator is to integrate the channel without increasing the amount of calculation;

The output formula 3 of the multi-receptive field attention mechanism is:

Among them, F _cs is the channel scrambling operator, and ⊙ is the multiplication operator.

Further, the four different convolution kernels in step 1 are randomly selected from numbers 1-9.

Further, the four different convolutions are 1*1, 3*3, 5*5, and 7*7 respectively.

Further, the convolution kernel sizes in step 2 are [3, 5].

Further, X _CH is the input feature map, whose size is [B, C, H, W], where B, C, H, W denote the batch, number of channels, spatial height and width, respectively. C module obtains X _a , X _a ∈ R ^(B,C,1,1) through the global average pooling operation. In order to avoid the model being too complicated, squeeze and replace X _a , and then get X _s , X _s ∈ R ^(B,1,C) ; After that, use the k*k convolution kernel to achieve local cross-channel interaction to get X _c , X _c ∈ R ^(B,1,C) ; X _sg , X _sg ∈ R ^{(B,1 ,C)} is obtained through the sigmoid activation function; finally, decompress and replace X _sg , and then get the weight X _weight , X _weight ∈ R ^(B,C,1,1) ;

The channel scrambling operator is to integrate channels without increasing the amount of calculation; X,X∈R ^(B ,C,H ^{,W) is} expanded into X _cs ,X _cs ,∈R ^{(B,G,C//G ,H,W)} , then change X _cs to get X _sc ,X _sc ∈R ^{(B,C//G,G,H,W)} ; finally restore to X,X∈R ^(B ,C,H ^,W) , to achieve global information exchange.

Based on the same inventive concept, the present invention also designs a system for realizing the multi-receptive field attention mechanism based on target detection, which is special in that:

Including backbone network, neck module and head module,

Described backbone network adopts retnet50 classification network, extracts feature to the picture of input, outputs four feature vectors with different semantic features.

The neck module uses four different convolution kernels to convolve the batch processing, channel number, spatial height and width of the input four feature vectors to obtain tensors [x1, x2, x3,x4], and add all tensors to get x5;

Divide the above 5 tensors into two groups in the channel dimension, use two C modules with different convolution kernel sizes to obtain the channel weights of each group, and then connect the two groups in the channel dimension to obtain each weights of tensors;

Among them, the C module is represented by the following formula:

F _{C module k*k} (X _CH )＝F _un F _sg W _1d F _s F _a X _CH

Among them, X _CH is the input feature map, F _a is an adaptive average pooling operator, F _sg is a sigmoid operator, W _1d is a k×k convolutional layer, F _s is a compression and exchange operator, F _un is a decompression and exchange operator, k*k is the convolution kernel used in local cross-channel interaction; the output formula of weight 5 is .

X _{5_1} , X _{5_2} = F _sp X ₅

weight5＝concat(F _{C module 3*3} (X _{5_1} ),F _{C module 5*5} (X _{5_2} ))

Among them, F _sp is a group operation operator, and concat is a splicing operator;

Finally, all channel weights are fused, and the weights are multiplied by the corresponding input vectors. And get the output after the channel is shuffled; the channel shuffling operator is to integrate the channel without increasing the amount of calculation;

The output formula 3 of the multi-receptive field attention mechanism is:

Where F _cs is the channel scrambling operator, ⊙ is the multiplication operator;

The head module receives the feature map input by the neck, and predicts all objects, calculates the loss by dividing positive and negative samples, and then optimizes through back propagation to realize the classification and regression of objects of different sizes.

The advantages of the present invention are:

Convolution processing using different convolution kernel sizes expands the receptive field of the feature vector, making the extracted features incorporate richer semantic features;

The C module realizes local cross-channel interaction, which reduces the information loss caused by the decrease of channel dimension, and also simplifies the model.

Grouping extraction weights can better measure the feature importance of different channels.

Weight fusion can balance weight distribution.

Channel scrambling can fully integrate channels without increasing the amount of calculation.

Description of drawings

Figure 1 is a schematic diagram of the multi-receptive field attention mechanism;

Figure 2 is a schematic diagram of the structure of the C module;

Figure 3 is a schematic diagram of the overall network structure

Figure 4 is the effect diagram of different network detection

detailed description

The technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

The structure of the multi-receptive field attention mechanism designed in the present invention is shown in FIG. 1 . X represents the input feature map, and its size is [B, C, H, W], where B, C, H, W represent the batch, number of channels, spatial height and width, respectively. First, it uses 1*1, 3*3, 5*5, 7*7 convolutions to convolve X to get four tensors [x1,x2,x3,x4] with different receptive fields. Their sizes are [B,C,H,W], then add to get x5.

It divides each tensor equally into 2 groups in the channel dimension. The size of each group is [B,C//2,H,W]. It uses two C modules with different kernel sizes to obtain channel weights for each group. The convolution kernel sizes are [3,5] respectively. The two groups in the channel dimension are then concatenated to obtain weights for each tensor. Different convolution kernels are used to capture cross-channel interaction information differently, so the effects can be synthesized by dividing them into two groups.

The structure of the C module is shown in Figure 2. X _CH is the input feature map, whose size is [B, C, H, W], where B, C, H, and W represent the batch, number of channels, spatial height, and width, respectively. It obtains X _a , X _a ∈ R ^(B,C,1,1) through global average pooling operation. In order to avoid the model being too complicated, X _a is squeezed and replaced to obtain X _s , X _s ∈ R ^(B,1,C) . Afterwards, the k*k convolution kernel is used to achieve local cross-channel interaction to obtain X _c , X _c ∈ R ^(B,1,C) . X _sg , X _sg ∈ R ^(B,1,C) is obtained through the sigmoid activation function. Finally, decompress and replace X _sg , and then get X _weight , X _weight ∈R ^(B,C,1,1) .

The C module is represented by Equation 1.

F _{C module k*k} (X _CH )＝F _un F _sg W _1d F _s F _a X _CH (1)

where F _a is an adaptive average pooling operator, F _sg is a sigmoid operator, W _1d is a k×k convolutional layer, F _s is a compression and exchange operator, F _un is a decompression and exchange operator. The output of weight5 is expressed by Equation 2.

Among them, F _sp is a group operation operator, and concat is a splicing operator. The fifth weight is the average of the first four weights, which can be introduced to a certain extent to balance the weight distribution.

Finally, all channel weights are fused, and the weights are multiplied by x. It gets the output after channel scrambling.

The output of the multi-receptive field attention mechanism is expressed by Equation 3.

Among them, F _cs is the channel scrambling operator, and ⊙ is the multiplication operator.

The channel scrambling operator integrates channels without increasing the amount of computation. X,X∈R ^(B,C,H,W) expands into X _cs ,X _cs ,∈R ^{(B,G,C//G,H,W)} , and then changes X _cs to get X _sc ,X _sc ∈ R ^{(B,C//G,G,H,W)} . Finally, it is restored to X,X∈R ^(B,C,H,W) to realize global information interaction.

Usually the weights are extracted using global average pooling alone, then using two fully connected layers with non-linearity, and then using a Sigmoid function to generate channel weights. Two connection layers are designed to capture non-linear cross-channel interactions, which includes dimensionality reduction to control model complexity. Although this strategy has been widely used in subsequent channel attention modules, dimensionality reduction has a side effect on channel attention prediction, and capturing dependencies between all channels is inefficient and unnecessary. This module avoids dimensionality reduction, effectively captures the information of cross-channel interaction, and also reduces the amount of parameters.

The present invention is based on the optimization neural network of MFPN network, and its steps comprise:

Step 1: Data entry and optimization strategy.

The PASCAL VOC dataset uses PASCAL VOC 2007 and PASCAL VOC 2012. They have a total of 21 categories, 16551 training images and 16492 testing images.

The Ms CoCo2017 dataset has a total of 80 categories and 118,287 images. It covers the most common objects in life and is a rich object detection dataset.

Crop all images to 512*512 for training, use the SGD optimizer, set the learning rate to 0.001, momentum to 0.9, and weight decay to 0.0001. The learning rate adopts a stepwise adjustment strategy, and the iteration period is 12 epochs.

Step 2: Construction of the model.

As shown in Figure 3, the network of the present invention consists of three parts: a backbone network, a neck module and a head module. The backbone network uses resnet50, which is used to extract the features of the picture. The network outputs 4 feature maps of different sizes [C2, C3, C4, C5], the step size is [4, 8, 16, 32], and the channel size is [256, 512, 1024, 2048]. The neck module structure is used to connect the backbone network and heads for feature fusion. This structure uses three feature maps [C3, C4, C5] of the backbone network. After 1*1 convolution, the channels are reduced to 256, and then all channels are processed by a multi-receptive field attention mechanism. FPN is used. Structural feature fusion, and then use the multi-receptive field attention mechanism to process the fused features, and finally use 3*3 convolution to ablate the feature map, and output 5 feature maps of different sizes [P3, P4, P5, P6 ,P7], the step size is [8,16,32,64,128], and the channel size is 256. The head module is used for object detection, classification and regression of targets.

Step 3: Train Test.

The evaluation standard of the experiment adopts the average precision (Average-Precision, AP), AP50, AP75, AP_S, AP_M, AP_L as the main evaluation standard.

Hardware: CPU: Intel Xeon E5-2683 V3@2.00GHz; RAM: 32GB; Graphics card: NvidiaGTX 1080Ti; Hard disk: 500GB.

Software: MMdetection2.6; PyTorch1.6.0; Torchvision＝0.7.0; CUDA10.0; CUDNN7.4.

The present invention has tested the impact of multi-receptive field attention mechanism on object precision, and has carried out comparative experiment on multiple networks, and experimental result is as shown in table 1.

Table 1 The influence of the multi-receptive field attention mechanism on the Ms CoCo2017 dataset on different networks, × means there is no attention mechanism, √ means there is an attention mechanism

The MFPN structure improves to varying degrees on the four networks. The AP of ATSS increases from 32.7% to 33.7%, and AP50 and AP_L even increase by 2%. The AP of FCOS has increased by 0.9% from 29.1%, and other indicators can also increase by about 1%. The AP of VFNet only increased by 0.4% from 34.1%, but the AP_L increased from 50.5% to 53.0%. The multi-receptive field attention mechanism has the most obvious improvement in Foveabox, its AP has increased by 2.6%, and its AP_L has increased from 43.8% to 47.3%.

Feature maps with different receptive fields have different effects on object detection with different sizes. The multi-receptive field attention mechanism structure integrates feature maps of four different receptive fields, which can effectively balance objects of different sizes. And its extraction of different channel weights can also enhance important features and reduce redundancy. Experimental results show that the multi-receptive field attention mechanism is effective in using feature maps of different receptive fields.

Figure 4 shows the detection effect of different networks. Faster-RCNN has obvious redundancy in detection. The first, second, and third images detected a lot of redundant objects, and the second image did not accurately return the position of the car. The missed detection of FCOS is obvious. The TV and the person on the right are not detected in the first image, and the laptop is also missing in the third image. The first and third images of FoveaBox have redundant detections, and the fourth image incorrectly detects a cat as a bear. ATSS also incorrectly detected the fourth image as a bear. The network proposed by the invention can not only accurately detect the target, but also has low false detection rate and redundancy rate. Compared with other networks, its detection accuracy is better than these networks.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention belongs can make various modifications or supplements to the described specific embodiments or adopt similar methods to replace them, but they will not deviate from the spirit of the present invention or go beyond the definition of the appended claims range.

Claims (6)

1. a multi-receptive field attention mechanism based on target detection, is characterized in that, comprises the steps: Step 1. Convolute the batch, channel number, spatial height and width of the input image with four different convolution kernels to obtain four tensors [x1,x2,x3,x4] with different receptive fields. and sum all tensors to get x5; Step 2, divide the above 5 tensors into two groups in the channel dimension, use two C modules with different convolution kernel sizes to obtain the channel weight of each group, and then connect the two groups in the channel dimension, to get the weight of each tensor; Among them, the C module is represented by the following formula: F _{C module k*k} (X _CH )＝F _un F _sg W _1d F _s F _a X _CH Among them, X _CH is the input feature map, F _a is an adaptive average pooling operator, F _sg is a sigmoid operator, W _1d is a k×k convolutional layer, F _s is a compression and exchange operator, F _un is a decompression and exchange operator, k*k is the convolution kernel used in local cross-channel interaction; the output formula of weight 5 is: X _{5_1} , X _{5_2} = F _sp X ₅ weight5＝concat(F _{C module 3*3} (X _{5_1} ),F _{C module 5*5} (X _{5_2} )) Among them, F _sp is a group operation operator, and concat is a splicing operator; Step 3, finally, fuse all channel weights, and then multiply the weights by the corresponding input vector; and get the output after the channels are shuffled; the channel shuffling operator integrates the channels without increasing the amount of calculation; The output formula 3 of the multi-receptive field attention mechanism is:

Among them, F _cs is the channel scrambling operator, and ⊙ is the multiplication operator. 2. The multi-receptive field attention mechanism based on target detection according to claim 1, characterized in that: in step 1, four different convolution kernels are arbitrarily selected from numbers 1-9. 3. The multi-receptive field attention mechanism based on target detection according to claim 2, wherein the four different convolutions are 1*1, 3*3, 5*5, and 7*7 respectively. 4. The multi-receptive field attention mechanism based on target detection according to claim 1, characterized in that: the sizes of the convolution kernels in step 2 are [3, 5]. 5. The multi-receptive field attention mechanism based on target detection according to claim 1, characterized in that: X _CH is an input feature map, and its size is [B, C, H, W], where B, C, H , W represent batch processing, channel number, space height and width respectively; C module obtains X _a , X _a ∈ R ^(B,C,1,1) through global average pooling operation, in order to avoid the model is too complicated, for X _a is squeezed and replaced, and then X _s , X _s ∈ R ^{(B,1,C) is} obtained; after that, the k*k convolution kernel is used to realize local cross-channel interaction to obtain X _c , X _c ∈ R ^{(B ,1 ,C)} ; X _sg , X _sg ∈ R ^(B,1,C) is obtained through the sigmoid activation function; finally, decompress and replace X _sg , and then get the weight X _weight , X _weight ∈ R ^{(B,C,1,1 )} ; The channel scrambling operator is to integrate channels without increasing the amount of calculation; X,X∈R ^{(B,C,H,W) is} expanded into X _cs ,X _cs ,∈R ^{(B,G,C//G ,H,W)} , then change X _cs to get X _sc ,X _sc ∈R ^{(B,C//G,G,H,W)} ; finally restore to X,X∈R ^(B,C,H,W) , to achieve global information exchange. 6. A network for realizing the arbitrary described multi-receptive field attention mechanism based on target detection of claim 1-5, characterized in that: Including backbone network, neck module and head module, The backbone network uses four different convolution kernels to convolve the batch processing, channel number, spatial height and width of the input image to obtain four tensors [x1,x2,x3,x4] with different receptive fields , and add all tensors to get x5; The neck module divides the above five tensors into two groups in the channel dimension, uses two C modules with different convolution kernel sizes to obtain the channel weights of each group, and then connects the two groups in the channel dimension up to get the weights of each tensor; Among them, the C module is represented by the following formula: F _C m _{odule k*k} (X _CH )＝F _un F _sg W _1d F _s F _a X _CH Among them, X _CH is the input feature map, F _a is an adaptive average pooling operator, F _sg is a sigmoid operator, W _1d is a k×k convolutional layer, F _s is a compression and exchange operator, F _un is a decompression and exchange operator, k*k is the convolution kernel used in local cross-channel interaction; the output formula of weight 5 is: X _{5_1} , X _{5_2} = F _sp X ₅ weight5＝concat(F _{C module 3*3} (X _{5_1} ),F _{C module 5*5} (X _{5_2} )) Among them, F _sp is a group operation operator, and concat is a splicing operator; The head module fuses all channel weights, and then multiplies the weights by the corresponding input vectors; and obtains the output after the channels are scrambled; the channel scramble operator integrates the channels without increasing the amount of calculation; The output formula 3 of the multi-receptive field attention mechanism is:

Among them, F _cs is the channel scrambling operator, and ⊙ is the multiplication operator.

Images