CN112560732B - Feature extraction method of multi-scale feature extraction network - Google Patents

Abstract

The invention discloses a feature extraction method of a multi-scale feature extraction network, which comprises a dimension reduction convolution layer, a scale feature extraction layer, a merging layer and a feature fusion layer which are sequentially connected, wherein the scale feature extraction layer comprises a large target detection branch, an original feature detection branch and a small target detection branch, features with different scales can be extracted, and feature fusion is carried out through the feature fusion layer, so that the multi-scale feature extraction network has the multi-scale feature extraction capability and low calculation complexity, the feature extraction network can be immediately applied when multi-scale feature extraction is needed, the target detection precision is improved, and the feature extraction method of the multi-scale feature extraction network can carry out feature dimension reduction, target detection and core multi-scale feature extraction on an input feature image to be extracted, can rapidly obtain the scale feature extraction image, and has the advantages of high target detection precision and less calculation quantity.

Description

A feature extraction method for multi-scale feature extraction network

technical field

The invention relates to a feature extraction neural network, in particular to a multi-scale feature extraction network and a feature extraction method of the network.

Background technique

Multi-scale target detection has always been a hotspot and difficulty in the field of computer vision research. In order to improve the accuracy of multi-scale target detection, network structures such as FPN, PA-Net, NAS-FPN, and BiFPN have been continuously proposed. , while improving the accuracy of target detection, it also carries too much calculation, which delays the reasoning time, making it difficult to apply and promote multi-scale target detection in the industry.

Contents of the invention

In order to overcome the deficiencies of the prior art, the present invention provides a feature extraction method of a multi-scale feature extraction network that improves target detection accuracy.

The technical solution adopted by the present invention to solve its technical problems is:

A feature extraction method of a multi-scale feature extraction network. The multi-scale feature extraction network includes a dimensionality reduction convolution layer, a scale feature extraction layer, a merge layer, and a feature fusion layer connected in sequence. The scale feature extraction layer includes a large target detection layer. branch, original feature detection branch and small object detection branch.

The large target detection branch includes a down-sampling feature layer, the first dilated convolution layer, and an up-sampling restoration layer connected in sequence; the original feature detection branch includes a second dilated convolution layer; the small target detection branch includes sequentially Connected upsampled feature layer, third dilated convolutional layer and downsampled recovery layer.

The first dilated convolutional layer includes three dilated convolutions whose convolution kernels are all 3*3, and the dilated rates of the three dilated convolutions are 1, 2 and 3 respectively; the second dilated convolutional layer The structures of the third dilated convolutional layer and the first dilated convolutional layer are the same.

The convolution kernels of the dimensionality reduction convolution layer and the feature fusion layer are both 1*1.

The feature extraction method of the multi-scale feature extraction network includes the following steps as follows:

(1) Input the feature map to be extracted into the dimensionality reduction convolution layer for 1x1 convolution, and perform feature fusion and dimensionality reduction on the feature map to be extracted to form the original feature map;

(2) The downsampled feature layer downsamples the original feature map to form a downsampled feature map; the upsampled feature layer upsamples the original feature map to form an upsampled feature map;

(3), the first hole convolution layer to the third hole convolution layer respectively perform three 3*3 hole convolutions on the downsampled feature map, original feature map and upsampled feature map to generate the first downsampled scale map , downsampled second scale map, downsampled third scale map, original first scale map, original second scale map, original third scale map, upsampled first scale map, upsampled second scale map, and upsampled third scale map Three-scale map;

(4) The upsampling restoration layer performs upsampling on the downsampled first scale map, the downsampled second scale map and the downsampled third scale map respectively, and the downsampling restoration layer respectively performs upsampling on the upsampled first scale map, upsampled second scale map The second-scale map and the up-sampled third-scale map are down-sampled;

(5) The merging layer performs upsampling of the downsampled first scale map, downsampled second scale map, and downsampled third scale map respectively to three scale maps obtained by upsampling, the original first scale map, the original second scale map, The original third scale image, the upsampled first scale image, the upsampled second scale image and the upsampled third scale image are respectively down-sampled to obtain three scale images that are merged and then 1*1 convolution fusion is performed by the feature fusion layer , forming a scale feature extraction map.

The beneficial effects of the present invention are: the present invention can extract features of different scales, and then perform feature fusion through the feature fusion layer, so that the multi-scale feature extraction network has multi-scale feature extraction capabilities and low computational complexity, requiring multi-scale feature extraction The feature extraction network can be applied immediately to improve the accuracy of target detection, and the feature extraction method of the multi-scale feature extraction network can perform feature dimensionality reduction, sub-target detection and core multi-scale feature extraction on the input feature map to be extracted. Quickly obtain the scale feature extraction map, and has the advantages of high target detection accuracy and less calculation.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

Fig. 1 is a schematic diagram of the network structure of the present invention;

Fig. 2 is a flowchart of the feature extraction method of the present invention.

Implementation

Referring to Fig. 1 , a feature extraction method of a multi-scale feature extraction network, a multi-scale feature extraction network, includes successively connected dimensionality reduction convolution layers 1, scale feature extraction layers, merging layers 2 and feature fusion layers 3, the scale The feature extraction layer includes the large target detection branch, the original feature detection branch and the small target detection branch, which can extract features of different scales, and then perform feature fusion through the feature fusion layer 3, so that the multi-scale feature extraction network has the ability to extract multi-scale features at the same time The computational complexity is low, and the feature extraction network can be applied immediately when the neural network needs to perform multi-scale feature extraction to improve the accuracy of target detection.

The large target detection branch includes a downsampling feature layer 4, a first dilated convolution layer 5, and an upsampling restoration layer 6 connected in sequence; the original feature detection branch includes a second dilated convolutional layer 7; the small target The detection branch includes an upsampling feature layer 8, a third atrous convolution layer 9, and a downsampling restoration layer 10 connected in sequence. In this embodiment, the upsampling method is UP-CONV, and the downsampling method is MAXPOOL.

The first dilated convolution layer 5 includes three dilated convolutions whose convolution kernels are 3*3, and the dilated rates of the three dilated convolutions are 1, 2 and 3 respectively; the second dilated convolution The structures of layer 7 and the third dilated convolutional layer 9 are the same as the first dilated convolutional layer 5, and they all have dilated convolutions with three convolution kernels of 3*3, and the dilated ratios are 1 and 2 respectively. And 3, each convolution kernel only acts on 1/3 of the channel number of the dilated convolutional layer, so that the large object detection branch, the original feature detection branch and the small object detection branch all have multi-scale feature extraction capabilities.

The convolution kernels of the dimensionality reduction convolution layer 1 and the feature fusion layer 3 are both 1*1, so that the dimensionality reduction convolution layer 1 and the feature fusion layer 3 have feature dimensionality reduction and feature fusion capabilities, and save computing time .

Referring to Figure 1 and Figure 2, the feature extraction method of the multi-scale feature extraction network includes the following steps as follows:

(1) Input the feature map to be extracted into the dimensionality reduction convolution layer 1 for 1x1 convolution, and perform feature fusion and dimensionality reduction on the feature map to be extracted to form the original feature map, which is the input of the dimensionality reduction convolution layer The feature map to be extracted realizes feature fusion and feature dimensionality reduction, which can save computing time, and the feature depth of the original feature map is reduced to 1/3 of the original depth of the feature map to be extracted.

(2) The downsampling feature layer 4 downsamples the original feature map to form a downsampling feature map. The depth of the downsampling feature map is the same as the original feature map, and the width and height of the image become 1/2 times of the original. The purpose of downsampling is to realize the detection of large objects and reduce the amount of computation.

The upsampling feature layer 8 upsamples the original feature map to form an upsampling feature map; the upsampling feature map is also the same as the original feature map, and the image width and height become twice the original. The purpose of upsampling is to achieve Detection of small objects.

(3), the first atrous convolution layer 5 to the third atrous convolution layer 9 respectively perform three 3*3 atrous convolutions on the downsampled feature map, original feature map and upsampled feature map, that is, the downsampled feature map , the original feature map and the upsampled feature map each perform three 3*3 hole convolutions, and the hole rates of the three 3*3 hole convolutions are 1 (ie standard 3x3 convolution), 2, 3, each A convolution kernel only acts on 1/3 of the number of channels in this layer. Therefore, the downsampled first scale map 11, the downsampled second scale map 12, the downsampled third scale map 13, the original first scale map 14, The original second scale map 15, the original third scale map 16, the upsampled first scale map 17, the upsampled second scale map 18 and the upsampled third scale map 19, so that the downsampled feature map, the original feature map and the upsampled feature map The feature map can accept the convolution of different receptive fields, so as to realize the extraction of multi-scale features.

(4) The upsampling restoration layer 6 respectively performs upsampling on the downsampled first scale map 11 , the downsampled second scale map 12 and the downsampled third scale map 13 , and the downsampling restoration layer 10 respectively performs upsampling on the upsampled first scale map The scale map 17, the upsampled second scale map 18 and the upsampled third scale map 19 are down-sampled, so that the upsampling restoration layer 6 and the downsampling restoration layer 10 will sample the first scale map 11 and the downsampling second scale map Figure 12. The width and height of the downsampled third-scale image 13, the up-sampled first-scale image 17, the up-sampled second-scale image 18, and the up-sampled third-scale image 19 are the same as those of the original first-scale image 14 and the original second-scale image. Figure 15 is consistent with the original third scale figure 16, which facilitates the merging and feature fusion of step five.

(5) Merging layer 2 performs upsampling of the first down-sampled scale map 11, down-sampled second scale map 12, and down-sampled third scale map 13 respectively to three scale maps obtained by upsampling, the original first scale map 14, the original The second scale map 15, the original third scale map 16, the upsampled first scale map 17, the upsampled second scale map 18, and the upsampled third scale map 19 are respectively downsampled to obtain three scale maps that are combined and then characterized The fusion layer 3 performs 1*1 convolution fusion to form a scale feature extraction map, completes multi-scale feature extraction, and maintains the same feature depth as the feature map to be extracted.

The above embodiments cannot limit the scope of protection of the present invention, and equivalent modifications and changes made by those in the technical field without departing from the overall concept of the present invention still fall within the scope of the present invention.

Claims (1)

1. The characteristic extraction method of the multi-scale characteristic extraction network is characterized in that the multi-scale characteristic extraction network comprises a dimension reduction convolution layer (1), a scale characteristic extraction layer, a merging layer (2) and a characteristic fusion layer (3) which are connected in sequence, wherein the scale characteristic extraction layer comprises a large target detection branch, an original characteristic detection branch and a small target detection branch;

the large target detection branch comprises a downsampling characteristic layer (4), a first cavity convolution layer (5) and an upsampling recovery layer (6) which are connected in sequence; the original feature detection branch comprises a second hole convolution layer (7); the small target detection branch comprises an up-sampling characteristic layer (8), a third cavity convolution layer (9) and a down-sampling recovery layer (10) which are connected in sequence; the first cavity convolution layer (5) comprises three cavity convolutions with convolution kernels of 3*3, and the cavity rates of the three cavity convolutions are 1, 2 and 3 respectively; the structures of the second hole convolution layer (7) and the third hole convolution layer (9) are the same as those of the first hole convolution layer (5);

the convolution kernels of the dimension reduction convolution layer (1) and the feature fusion layer (3) are 1*1;

the feature extraction method of the multi-scale feature extraction network comprises the following steps of:

firstly, inputting a feature image to be extracted into a dimension reduction convolution layer to carry out 1x1 convolution, and carrying out feature fusion and dimension reduction on the feature image to be extracted to form an original feature image;

secondly, the downsampling feature map layer downsamples the original feature map to form a downsampled feature map; the up-sampling feature map layer up-samples the original feature map to form an up-sampling feature map;

thirdly, the first hole convolution layer to the third hole convolution layer respectively conduct three 3*3 hole convolutions on the downsampled feature map, the original feature map and the upsampled feature map to generate a downsampled first scale map, a downsampled second scale map, a downsampled third scale map, an original first scale map, an original second scale map, an original third scale map, an upsampled first scale map, an upsampled second scale map and an upsampled third scale map;

fourthly, the up-sampling recovery layer up-samples the down-sampling first scale map, the down-sampling second scale map and the down-sampling third scale map respectively, and the down-sampling recovery layer down-samples the up-sampling first scale map, the up-sampling second scale map and the up-sampling third scale map respectively;

and fifthly, merging three scale maps obtained by respectively carrying out downsampling on the downsampled first scale map, the downsampled second scale map and the downsampled third scale map by a merging layer, and carrying out 1*1 convolution fusion on the three scale maps obtained by respectively carrying out downsampling on the upsampled first scale map, the upsampled second scale map and the upsampled third scale map by a feature fusion layer to form a scale feature extraction map.

Images