CN115205547A - A target image detection method, device, electronic device and storage medium - Google Patents

Abstract

The application provides a target image detection method, a target image detection device, electronic equipment and a storage medium, which are applied to the technical field of image detection, wherein the detection method comprises the following steps: acquiring a target image to be detected; inputting a target image to be detected into a backbone network of a pre-trained target detection model, performing feature extraction on the target image to be detected, and determining a shallow feature map and a deep feature map of the target image to be detected; inputting the shallow feature map and the deep feature map into a feature pyramid network of a target detection model, and weighting the shallow feature map and the deep feature map with different weights respectively to determine a fusion feature map; and inputting the fusion characteristic graph into a head network of the target detection model, and determining the target object in the target image to be detected. The shallow feature map and the deep feature map are weighted according to different weights in the feature pyramid network to obtain a fusion feature map, and the detection efficiency and accuracy of the target image are improved.

Description

A target image detection method, device, electronic device and storage medium

technical field

The present application relates to the technical field of image detection, and in particular, to a method, device, electronic device and storage medium for detecting a target image.

Background technique

In the process of image review, the reviewers usually rely on manpower to review the huge amount of data every day, which is time-consuming and labor-intensive in the review process, or use the deep learning neural network to audit the image. Including the backbone network, neck network and head network, the neck network plays a role in connecting the above, so a good neck network can better transfer features.

At this stage, in the current image detection methods, the neck network generally adopts Feature Pyramid Networks (FPN), and its main function is to fuse shallow local features and deep semantic features. However, in the process of feature fusion In the process, the importance of different features is often different, which will lead to the non-optimal features after fusion, thus affecting the detection results of the image by the head network. Therefore, how to improve the accuracy of image detection has become a problem that cannot be underestimated. technical issues.

SUMMARY OF THE INVENTION

In view of this, the purpose of this application is to provide a detection method, device, electronic device and storage medium of a target image, by inputting the shallow feature map and the deep feature map of the target image into the feature pyramid network, in the feature pyramid In the network, the shallow feature map and the deep feature map are weighted with different weights to obtain the fusion feature map, which improves the detection efficiency and accuracy of the target image.

An embodiment of the present application provides a method for detecting a target image, and the detecting method includes:

Obtain the target image to be detected;

Input the target image to be detected into the backbone network of the pre-trained target detection model, perform feature extraction on the target image to be detected, and determine the shallow feature map and deep feature of the target image to be detected. picture;

Inputting the shallow feature map and the deep feature map to the feature pyramid network of the target detection model, respectively weighting the shallow feature map and the deep feature map with different weights to determine fusion features picture;

The fusion feature map is input to the head network of the target detection model, and the target item in the target image to be detected is determined.

In a possible implementation manner, inputting the shallow feature map and the deep feature map to a feature pyramid network of the target detection model, respectively, for the shallow feature map and the deep feature map Perform weighting processing with different weights to determine the fusion feature map, including:

Inputting the shallow feature map and the deep feature map to the feature pyramid network for feature splicing, and determining the splicing feature map;

Inputting the splicing feature map to the convolution layer of the feature pyramid network, performing convolution processing on the splicing feature map, and determining the first target feature map;

Inputting the first target feature map to the pooling layer of the feature pyramid network, performing a global average pooling process on the first target feature map, and determining a second target feature map;

Inputting the second target feature map to the activation layer of the feature pyramid network, performing activation processing on the second target feature map, and determining the first target weight and the second target weight;

Inputting the first target weight and the second target weight to the fusion layer of the feature pyramid network, using the first target weight to weight the shallow feature map and using the second target weight Perform weighting processing on the deep feature map to determine the fusion feature map.

In a possible implementation manner, the splicing feature map is input into the convolution layer of the feature pyramid network, the convolution processing is performed on the splicing feature map, and the first target feature map is determined, including:

Obtain an initialized two-dimensional matrix, wherein the dimension of the initialized two-dimensional matrix is determined according to the number of convolution kernels and the number of feature vectors of the splicing feature map in different dimensions;

Based on the feature information corresponding to the feature vectors of the spliced feature maps in different dimensions, determine the target value of the feature vectors of the spliced feature maps in different dimensions;

The target value of the feature vector of the splicing feature map under the different dimensions is added to the two-dimensional matrix of the initialization, and the target two-dimensional matrix is generated;

Each of the convolution kernels filters out the corresponding feature vectors of the spliced feature maps in different dimensions according to the target values of the feature vectors of the spliced feature maps in different dimensions in the target two-dimensional matrix, and performs Convolution processing is performed to determine the first target feature map.

In a possible implementation manner, the second target feature map is input to the activation layer of the feature pyramid network, the second target feature map is activated, and the first target weight and the first target weight are determined. Two objective weights, including:

Use the sigmoid activation function to normalize the second target feature map to determine the weight vector;

The first target weight and the second target weight are determined by dividing based on the dimension of the weight vector.

In a possible implementation manner, the first target weight and the second target weight are input to a fusion layer of the feature pyramid network, and the shallow feature map is processed by using the first target weight. and performing weighting processing on the deep feature map by using the second target weight to determine the fusion feature map, including:

Perform weighting processing on the shallow feature map by using the first target weight to determine the weighted shallow feature map;

Perform weighting processing on the deep feature map using the second target weight to determine a weighted deep feature map;

The weighted shallow feature map and the weighted deep feature map are feature added to determine the fusion feature map.

In a possible implementation, the target detection model is trained through the following steps:

Obtain a plurality of sample pictures and label information corresponding to each sample picture, and divide the plurality of sample pictures into a training set and a verification set;

Use the training set and the label information corresponding to the sample pictures corresponding to the training set to perform multiple training on the initial training model to determine the target detection model;

The target detection model is tested by using the verification set to verify the detection accuracy of the target detection model.

The embodiment of the present application also provides a detection device for a target image, the detection device includes:

an acquisition module for acquiring the target image to be detected;

The feature extraction module is used to input the target image to be detected into the backbone network of the pre-trained target detection model, perform feature extraction on the target image to be detected, and determine the shallowness of the target image to be detected. Layer feature maps and deep feature maps;

A feature fusion module is used to input the shallow feature map and the deep feature map to the feature pyramid network of the target detection model, and weight the shallow feature map and the deep feature map with different weights respectively processing to determine the fusion feature map;

A determination module, configured to input the fusion feature map to the head network of the target detection model, and determine the target item in the target image to be detected.

In a possible implementation manner, in the feature pyramid network used for inputting the shallow feature map and the deep feature map to the target detection model, the feature fusion module performs the corresponding analysis on the shallow feature map respectively. The feature map and the deep feature map are weighted with different weights, and when the fusion feature map is determined, the feature fusion module is specifically used for:

Inputting the shallow feature map and the deep feature map to the feature pyramid network for feature splicing, and determining the splicing feature map;

Inputting the splicing feature map to the convolution layer of the feature pyramid network, performing convolution processing on the splicing feature map, and determining the first target feature map;

Inputting the first target feature map to the pooling layer of the feature pyramid network, performing a global average pooling process on the first target feature map, and determining a second target feature map;

Inputting the second target feature map to the activation layer of the feature pyramid network, performing activation processing on the second target feature map, and determining the first target weight and the second target weight;

The first target weight and the second target weight are input to the fusion layer of the feature pyramid network, and the shallow feature map is paired with the first target weight and the second target weight is used for the The deep feature map is weighted to determine the fusion feature map.

Embodiments of the present application further provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the The memories communicate with each other through a bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned method for detecting a target image are performed.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the above-mentioned method for detecting a target image are executed.

A target image detection method, device, electronic device, and storage medium provided by the embodiments of the present application include: acquiring a target image to be detected; inputting the target image to be detected into a pre-trained target The backbone network of the detection model performs feature extraction on the target image to be detected, and determines the shallow feature map and the deep feature map of the target image to be detected; the shallow feature map and the deep feature map are Input to the feature pyramid network of the target detection model, respectively weight the shallow feature map and the deep feature map with different weights to determine a fusion feature map; input the fusion feature map to the target The head network of the detection model determines the target item in the target image to be detected. By inputting the shallow feature map and deep feature map of the target image into the feature pyramid network, the shallow feature map and the deep feature map are weighted with different weights in the feature pyramid network to obtain the fusion feature map, which improves the target performance. Image detection efficiency and accuracy.

In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following drawings will briefly introduce the drawings that need to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore do not It should be regarded as a limitation of the scope, and for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

1 is a flowchart of a method for detecting a target image provided by an embodiment of the present application;

2 is a schematic diagram of convolution performed by a convolution kernel in a method for detecting a target image provided by an embodiment of the present application;

3 is a schematic diagram of a process of determining a fusion feature map in a method for detecting a target image provided by an embodiment of the present application;

4 is one of the schematic structural diagrams of a device for detecting a target image provided by an embodiment of the present application;

FIG. 5 is a second schematic structural diagram of a device for detecting a target image provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the The accompanying drawings are only for the purpose of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented in accordance with some embodiments of the application. It should be understood that the operations of the flowcharts may be performed out of order and that steps without logical context may be performed in reverse order or concurrently. In addition, those skilled in the art can add one or more other operations to the flowchart, and can also remove one or more operations from the flowchart under the guidance of the content of the present application.

In addition, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments. The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

In order to enable those skilled in the art to use the content of the present application, in combination with a specific application scenario "detecting images", the following embodiments are given. For those skilled in the art, without departing from the spirit and scope of the present application, The general principles defined herein may be applied to other embodiments and application scenarios.

The following methods, apparatuses, electronic devices, or computer-readable storage media in the embodiments of the present application can be applied to any scene where images need to be detected. The embodiments of the present application do not limit the specific application scenarios. The solution of a method, device, electronic device and storage medium for detecting a target image are all within the protection scope of the present application.

First, the applicable application scenarios of this application are introduced. The present application can be applied to the field of image detection technology.

The research found that at this stage, in the current image detection methods, the neck network generally adopts the feature pyramid (Feature Pyramid Networks, FPN for short), and its main function is to fuse the local features of the shallow layer and the semantic features of the deep layer, but, In the process of feature fusion, the importance of different features is often different, which will lead to the non-optimal features after fusion, which will affect the detection results of the image by the head network. Therefore, how to improve the accuracy of image detection It has become a technical problem that cannot be underestimated.

Based on this, an embodiment of the present application provides a method for detecting a target image. By inputting the shallow feature map and the deep feature map of the target image into the feature pyramid network, the shallow feature map and the deep feature map are detected in the feature pyramid network. The feature map is weighted with different weights to obtain a fusion feature map, which improves the detection efficiency and accuracy of the target image.

Please refer to FIG. 1 , which is a flowchart of a method for detecting a target image provided by an embodiment of the present application. As shown in Figure 1, the detection method provided by the embodiment of the present application includes:

S101: Acquire a target image to be detected.

In this step, the target image to be detected is obtained. Here, the target image to be detected can be an image of contraband to be reviewed by reviewers, or other images that need to be reviewed and detected. Here, the type of target image is not limited. Any image that needs to be reviewed and tested can be used.

S102: Input the target image to be detected into the backbone network of the pre-trained target detection model, perform feature extraction on the target image to be detected, and determine the shallow feature map of the target image to be detected and Deep feature maps.

In this step, the target image to be detected is input into the backbone network of the pre-trained target detection model, the backbone network performs feature extraction on the target image to be detected, and determines the shallow feature map and the deep layer of the target image to be detected. feature map.

Here, the shallow feature map is closer to the input target image to be detected, and contains more pixel information, some fine-grained information, such as some color, texture, edge and edge information of the target image to be detected, The receptive field of the shallow feature map is smaller, and the overlapping area of the receptive field is also smaller, so it is guaranteed that the network captures more details.

Here, the deep feature map contains some coarse-grained information, including more abstract information, that is, semantic information. The receptive field of the deep feature map increases, and the overlapping area between the receptive fields increases the image information for compression. Some information about the integrity of the detected target image.

In a possible implementation, the target detection model is trained through the following steps:

A: Obtain multiple sample images and label information corresponding to each sample image, and divide the multiple sample images into a training set and a validation set.

Here, multiple sample images are collected in the network, as well as the label information corresponding to each sample image, and the multiple sample images are divided into a training set and a validation set in a ratio of 9:1.

B: Perform multiple training on the initial training model by using the training set and the label information corresponding to the sample pictures corresponding to the training set to determine the target detection model.

Here, an initial training model is constructed first. The initial training model is a neural network model, and the initial training model is trained multiple times by using the training set and the label information corresponding to multiple sample pictures in the training set to obtain a target detection model.

C: Use the verification set to test the target detection model to verify the detection accuracy of the target detection model.

Here, after the training is completed, use the validation set to test the target detection model to verify the performance of the target detection model. If the performance of the target detection model is high, it means that the target detection model can detect the target image. If the performance of the target detection model is high If it is lower, you need to continue to train the target detection model.

In a specific embodiment, pictures of a plurality of sample firearms are obtained, a plurality of pictures of the sample firearms are made into a detection picture data set, and the detected picture data set is divided into a training set and a verification set; an initial training model is constructed, Use the label information corresponding to the training set and multiple sample gun pictures in the training set to train the initial training model for multiple times to obtain the target detection model. After the training is completed, use the validation set to test the target detection model to verify the performance of the target detection model. So that the target detection model can detect contraband firearms according to the target image.

S103: Input the shallow feature map and the deep feature map to the feature pyramid network of the target detection model, and perform weighting processing on the shallow feature map and the deep feature map with different weights, respectively, to determine Fusion feature maps.

In this step, the shallow feature map and the deep feature map obtained in the backbone network are input into the feature pyramid network, and the shallow feature map and the deep feature map are weighted with different weights in the feature pyramid network and feature fusion is performed. The fusion feature map is determined.

Here, the weights for weighting the shallow feature map and the deep feature map are different.

In a possible implementation manner, inputting the shallow feature map and the deep feature map to a feature pyramid network of the target detection model, respectively, for the shallow feature map and the deep feature map Perform weighting processing with different weights to determine the fusion feature map, including:

a: Input the shallow feature map and the deep feature map to the feature pyramid network for feature splicing, and determine the splicing feature map.

Here, the shallow feature map and the deep feature map are input to the feature pyramid network, and the shallow feature map and the deep feature map are spliced to obtain the spliced feature map.

Among them, the dimensions of the shallow feature map and the deep feature map are consistent. If the dimensions of the shallow feature map and the deep feature map are W*H*C, the dimension of the spliced feature map is W*H*2C.

b: Input the spliced feature map into the convolution layer of the feature pyramid network, perform convolution processing on the spliced feature map, and determine the first target feature map.

Here, the spliced feature map is input into the convolution layer of the feature pyramid network, and the spliced feature map is convolved to determine the first target feature map.

The first target feature map is a feature map that is consistent with the dimension of the spliced feature map after convolving the spliced feature map.

In a possible implementation manner, the splicing feature map is input into the convolution layer of the feature pyramid network, the convolution processing is performed on the splicing feature map, and the first target feature map is determined, including:

(1): Obtain an initialized two-dimensional matrix, wherein the dimension of the initialized two-dimensional matrix is determined according to the number of convolution kernels and the feature vectors of the splicing feature map in different dimensions.

Among them, if the dimension of the spliced feature map is W*H*2C, the feature vectors of the spliced feature maps in different dimensions are corresponding to the spliced feature maps in the dimensions of W*H*1, W*H*2...W*H*2C eigenvectors of .

Among them, the number of convolution kernels in the initialized two-dimensional matrix is consistent with the number of channels of the spliced feature map. If the dimension of the spliced feature map is W*H*2C, the dimension of the initialized two-dimensional matrix is 2C*2C.

Among them, the columns of the initialized two-dimensional matrix represent 2C convolution kernels, and the rows of the initialized two-dimensional matrix represent the feature vectors of the spliced feature maps in 2C different dimensions.

(2): Based on the feature information corresponding to the feature vectors of the spliced feature maps in different dimensions, determine the target values of the feature vectors of the spliced feature maps in different dimensions.

Here, the target value of the feature vector of the spliced feature maps in different dimensions is determined by using the importance of the feature information corresponding to the feature vectors of the spliced feature maps in different dimensions.

Here, in the process of training and learning of the initialized two-dimensional matrix, according to the importance of the feature information corresponding to the feature vectors of the spliced feature maps in different dimensions, the initialized two-dimensional matrix is selected for the splicing features in the dimension that needs to be convolved. The feature vector of the graph, and the target value of the feature vector of the splicing feature map in the dimension that needs to be convolutional is assigned greater than 0, and the target value of the feature vector of the feature splicing map in the dimension that does not require convolution is assigned less than 0.

Among them, the importance of the feature vectors corresponding to the feature information of the spliced feature maps in different dimensions is determined according to expert experience or network learning.

Among them, if the feature vectors of the spliced feature maps in different dimensions have a higher degree of importance corresponding to the feature information, the target value of the feature vector is given greater than 0, and the target value of the feature vector is given less than 0 if it is lower.

For example, the stitched feature map is the image of the contraband of the tool. The feature image of the contraband image of the tool in different dimensions is different. For example, the feature image of the contraband image of the tool in a certain dimension includes color and texture information. The importance of the feature information in this dimension is lower. For example, the feature image of the contraband image of the tool in a certain dimension includes the contour of the tool, and the importance of the feature information in this dimension is higher.

(3): Add the target value of the feature vector of the splicing feature map in the different dimensions to the initialized two-dimensional matrix to generate a target two-dimensional matrix.

Here, the target values of the feature vectors of the spliced feature maps in different dimensions are added to the initialized two-dimensional matrix to generate the target two-dimensional matrix.

Among them, each row in the target two-dimensional matrix is the eigenvector of the spliced feature map in 2C different dimensions and the target value corresponding to the eigenvector of the spliced feature map in this dimension, and each column in the target two-dimensional matrix has 2C volumes accumulated nucleus.

(4): According to the target value of the feature vector of the splicing feature map under different dimensions in the target two-dimensional matrix, each of the convolution kernels filters out the features of the splicing feature map in the corresponding different dimensions. vector, and perform convolution processing to determine the first target feature map.

Here, each convolution kernel filters out the corresponding eigenvectors of the spliced feature maps in different dimensions according to the target value of the eigenvectors of the spliced feature maps in different dimensions in the target two-dimensional matrix, and performs convolution processing to determine the The first target feature map.

Here, each convolution kernel needs to convolve the feature vector of the feature map under the corresponding dimension that needs to be convolved, and it is not necessary to convolve the feature vector of the feature map under the convolution under each dimension.

Among them, the value in the target two-dimensional matrix (M) is continuously updated during the training process. If M(i,j)>=0, it means that the i-th convolution kernel needs to stitch the features of the feature map in the j-th dimension. The vector is convolved. If M(i,j)<0, it means that the i-th convolution kernel does not convolve the feature vector of the spliced feature map in the j-th dimension.

Among them, after each convolution kernel convolves the feature vectors of the corresponding spliced feature maps in different dimensions, multiple convolution feature maps are obtained, and the multiple convolution feature maps are spliced to obtain the first target feature map.

Here, please refer to FIG. 2. FIG. 2 is a schematic diagram of convolution performed by a convolution kernel in a method for detecting a target image provided by an embodiment of the present application. As shown in FIG. 2, the black dots represent the target two-dimensional matrix in the The target value of the feature vector of the spliced feature map under the dimension is greater than 0, and the white dots represent that the target value of the feature vector of the spliced feature map under the dimension is less than 0. The first square is the feature vector of the spliced feature maps in different dimensions corresponding to the target two-dimensional matrix, and the second square is the convolution kernel corresponding to the target matrix. For example, the first black dot, the third black dot, the fourth black dot, and the seventh black dot in the first row of the target two-dimensional matrix represent the needs of the first convolution kernel Roll the feature vector of the mosaic feature map in the first dimension, the feature vector of the mosaic feature map under the third dimension, the feature vector of the mosaic feature map under the fourth dimension, and the feature vector of the mosaic feature map under the seventh dimension. Product; the first black dot, the fourth black dot, the fifth black dot, and the eighth black dot in the second row of the target two-dimensional matrix represent that the second convolution kernel needs to perform a The feature vector of the splicing feature map, the feature vector of the splicing feature map under the fourth dimension, the feature vector of the splicing feature map under the fifth dimension, and the feature vector of the splicing feature map under the eighth dimension are convolved, and the order is down. analogy. In this way, it avoids the technical problem that the fused features are not optimal because different feature vectors are fused with the same weight during the feature fusion process of FPN. In this solution, each convolution kernel is selected corresponding to It is not necessary to convolve the feature vectors of the stitched feature maps in all dimensions, so that the feature semantics of the first target feature map is higher.

c: Input the first target feature map to the pooling layer of the feature pyramid network, perform global average pooling on the first target feature map, and determine a second target feature map.

Here, the first target feature map is input to the pooling layer, the global average pooling process is performed on the first target feature map, and the second target feature map is determined.

d: Input the second target feature map to the activation layer of the feature pyramid network, perform activation processing on the second target feature map, and determine the first target weight and the second target weight.

Here, the second target feature map is input to the activation layer of the feature pyramid network, the second target feature map is activated, and the first target weight and the second target weight are determined.

In a possible implementation manner, the second target feature map is input to the activation layer of the feature pyramid network, the second target feature map is activated, and the first target weight and the first target weight are determined. Two objective weights, including:

1: Use the sigmoid activation function to normalize the second target feature map to determine the weight vector.

Here, a sigmoid activation function is used to normalize the second target feature map to determine a weight vector.

Step 2: Divide based on the dimension of the weight vector, and determine the first target weight and the second target weight.

Here, the dimension of the weight vector is used for division, and the first target weight and the second target weight are determined. For example, if the dimension of the first weight is 1*1*2C, the weight vector is restored to two first and second target weights with dimensions of 1*1*C, wherein the first target weight and the second target weight are Although the dimensions of the two target weights are the same, the content of the weights contained is different.

e: Input the first target weight and the second target weight to the fusion layer of the feature pyramid network, use the first target weight to combine the shallow feature map and the second target weight The deep feature map is weighted to determine the fusion feature map.

Here, the first target weight and the second target weight are input to the fusion layer of the feature pyramid network, and the shallow layer feature map and the deep layer feature map are weighted to determine the fusion feature map.

In a possible implementation manner, the first target weight and the second target weight are input to a fusion layer of the feature pyramid network, and the shallow feature map is processed by using the first target weight. and performing weighting processing on the deep feature map by using the second target weight to determine the fusion feature map, including:

1): Perform weighting processing on the shallow feature map by using the first target weight to determine a weighted shallow feature map.

Here, the first target weight is used to weight the shallow layer feature map to determine the weighted shallow layer feature map.

2): Perform weighting processing on the deep feature map by using the second target weight to determine a weighted deep feature map.

Here, the deep feature map is weighted by using the second target weight to determine the weighted deep feature map.

3): Perform feature addition on the weighted shallow feature map and the weighted deep feature map to determine the fusion feature map.

Here, the features of the weighted shallow feature map and the weighted deep feature map are added to determine the fusion feature map.

Here, the first target weight may be used to weight the deep feature map to determine the weighted deep feature map, and the second target weight may be used to weight the shallow feature map to determine the weighted shallow feature map.

Further, please refer to FIG. 3 , which is a schematic diagram of a process of determining a fusion feature map in a method for detecting a target image provided by an embodiment of the present application. As shown in Figure 3, the dimension of the shallow feature map is W*H*C, and the dimension of the deep feature map is W*H*C. The shallow feature map and the deep feature map are spliced to obtain a spliced feature map. The dimension of the feature map is W*H*2C, and each convolution kernel filters out the feature vectors of the corresponding feature maps in different dimensions from the target two-dimensional matrix, and performs convolution processing to determine the first target feature map. , the dimension of the first target feature map is W*H*2C, the first target feature map is pooled to determine the second target feature map, and the dimension of the second target feature map is 1*1*2C. The two target feature maps are activated, and the first target weight 1*1*C and the second target weight 1*1*C are determined, and the first target weight is used to weight the shallow feature map, and the weighted shallow feature map is determined. layer feature map, using the second target weight to weight the deep feature map, determine the weighted deep feature map, add the weighted shallow feature map and the weighted deep feature map, and determine the Fusion feature maps. Because the feature semantics of the shallow feature map and the deep feature map are different, if equal weights are used for weighting processing, the technical problem of inaccurate semantic information of the fusion feature map will result. The shallow feature map is weighted, and the second target weight is used to weight the deep feature map, which avoids that different feature vectors are fused with the same weight during the feature fusion process of FPN, and the fused features are not. The optimal technical problem improves the accuracy of the semantic information of the fusion feature map.

S104: Input the fusion feature map to the head network of the target detection model, and determine the target item in the target image to be detected.

In this step, the fusion feature map is input to the head network of the target detection model, and the target item in the target image to be detected is determined.

Here, if the target image is an image of a contraband to be reviewed by the reviewer, the identified target item may be other contraband such as knives and guns.

In a specific embodiment, the target image to be detected is acquired, the target image to be detected is input into the backbone network of the target detection model, and the shallow feature map and the deep feature map are output; the shallow feature map and the deep feature map are input to the target The feature pyramid network of the detection model, the shallow feature map and the deep feature map are weighted with different weights respectively, and the fusion feature map is output; among them, the feature pyramid network layer adds a two-dimensional matrix to make the shallow feature map and the deep feature map In the process of feature fusion, the method of unequal weight is used for fusion; the fusion feature map is input to the head network of the target detection model, and the fusion feature map is predicted, and the contraband included in the target image to be detected.

A method for detecting a target image provided by an embodiment of the present application includes: acquiring a target image to be detected; inputting the target image to be detected into a backbone network of a pre-trained target detection model, and performing the detection on the target image to be detected. Feature extraction, determine the shallow feature map and deep feature map of the target image to be detected; input the shallow feature map and the deep feature map to the feature pyramid network of the target detection model, respectively Different weights are weighted to determine the fusion feature map; the fusion feature map is input to the head network of the target detection model to determine the target object in the target image to be detected. By inputting the shallow feature map and the deep feature map of the target image into the feature pyramid network, the shallow feature map and the deep feature map are weighted with different weights in the feature pyramid network to obtain the fusion feature map, which improves the target performance. Image detection efficiency and accuracy.

Please refer to FIG. 4 and FIG. 5 , FIG. 4 is one of the schematic structural diagrams of a target image detection apparatus provided by an embodiment of the present application, and FIG. 5 is a structure of a target image detection apparatus provided by an embodiment of the present application Diagram two. As shown in FIG. 4 , the detection device 400 of the target image includes:

an acquisition module 410, configured to acquire a target image to be detected;

The feature extraction module 420 is used to input the target image to be detected into the backbone network of the pre-trained target detection model, perform feature extraction on the target image to be detected, and determine the characteristics of the target image to be detected. Shallow feature maps and deep feature maps;

The feature fusion module 430 is used for inputting the shallow feature map and the deep feature map to the feature pyramid network of the target detection model, and performing different weights on the shallow feature map and the deep feature map respectively. Weighted processing to determine the fusion feature map;

The determining module 440 is configured to input the fusion feature map into the head network of the target detection model, and determine the target item in the target image to be detected.

Further, the feature fusion module 430 is used to input the shallow feature map and the deep feature map into the feature pyramid network of the target detection model, respectively. The deep feature maps are weighted with different weights, and when a fusion feature map is determined, the feature fusion module 430 is specifically used for:

Inputting the shallow feature map and the deep feature map to the feature pyramid network for feature splicing, and determining the splicing feature map;

Inputting the splicing feature map to the convolution layer of the feature pyramid network, performing convolution processing on the splicing feature map, and determining the first target feature map;

Inputting the first target feature map to the pooling layer of the feature pyramid network, performing a global average pooling process on the first target feature map, and determining a second target feature map;

Inputting the second target feature map to the activation layer of the feature pyramid network, performing activation processing on the second target feature map, and determining the first target weight and the second target weight;

Inputting the first target weight and the second target weight to the fusion layer of the feature pyramid network, using the first target weight to weight the shallow feature map and using the second target weight Perform weighting processing on the deep feature map to determine the fusion feature map.

Further, the feature fusion module 430 performs convolution processing on the spliced feature map in the convolution layer used to input the spliced feature map into the feature pyramid network to determine the first target feature map. , the feature fusion module 430 is specifically used for:

Obtain an initialized two-dimensional matrix, wherein the dimension of the initialized two-dimensional matrix is determined according to the number of convolution kernels and the number of feature vectors of the splicing feature map in different dimensions;

Based on the feature information corresponding to the feature vectors of the spliced feature maps in different dimensions, determine the target value of the feature vectors of the spliced feature maps in different dimensions;

The target value of the feature vector of the splicing feature map under the different dimensions is added to the two-dimensional matrix of the initialization, and the target two-dimensional matrix is generated;

Each of the convolution kernels filters out the corresponding feature vectors of the spliced feature maps in different dimensions according to the target values of the feature vectors of the spliced feature maps in different dimensions in the target two-dimensional matrix, and performs Convolution processing is performed to determine the first target feature map.

Further, the feature fusion module 430 performs activation processing on the second target feature map in the activation layer for inputting the second target feature map into the feature pyramid network, and determines the first target. When the weight and the second target weight, the feature fusion module 430 is specifically used for:

Use the sigmoid activation function to normalize the second target feature map to determine the weight vector;

The first target weight and the second target weight are determined by dividing based on the dimension of the weight vector.

Further, in the fusion layer for inputting the first target weight and the second target weight into the feature pyramid network, the feature fusion module 430 uses the first target weight to perform a When the layer feature map is weighted and the second target weight is used to weight the deep feature map, and the fusion feature map is determined, the feature fusion module 430 is specifically used for:

Perform weighting processing on the shallow feature map by using the first target weight to determine the weighted shallow feature map;

Perform weighting processing on the deep feature map using the second target weight to determine a weighted deep feature map;

The weighted shallow feature map and the weighted deep feature map are feature added to determine the fusion feature map.

Further, as shown in FIG. 5 , the target image detection device 400 further includes a model training module 450, and the model training module 450 trains the target detection model through the following steps:

Obtain a plurality of sample pictures and label information corresponding to each sample picture, and divide the plurality of sample pictures into a training set and a verification set;

Use the training set and the label information corresponding to the sample pictures corresponding to the training set to perform multiple training on the initial training model to determine the target detection model;

The target detection model is tested by using the verification set to verify the detection accuracy of the target detection model.

An apparatus for detecting a target image provided by an embodiment of the present application includes: an acquisition module for acquiring a target image to be detected; a feature extraction module for inputting the target image to be detected into a pre-training The backbone network of a good target detection model performs feature extraction on the target image to be detected, and determines the shallow feature map and deep feature map of the target image to be detected; The layer feature map and the deep feature map are input to the feature pyramid network of the target detection model, and the shallow feature map and the deep feature map are weighted with different weights to determine the fusion feature map; , which is used to input the fusion feature map to the head network of the target detection model to determine the target item in the target image to be detected. By inputting the shallow feature map and the deep feature map of the target image into the feature pyramid network, the shallow feature map and the deep feature map are weighted with different weights in the feature pyramid network to obtain the fusion feature map, which improves the target performance. Image detection efficiency and accuracy.

Please refer to FIG. 6 , which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 6 , the electronic device 600 includes a processor 610 , a memory 620 and a bus 630 .

The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 through the bus 630, and the machine-readable instructions are executed. When executed, the processor 610 may execute the steps of the method for detecting a target image in the method embodiment shown in FIG. 1 above. For the specific implementation, please refer to the method embodiment, which will not be repeated here.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the computer program can execute the target image in the method embodiment shown in FIG. 1 above. For the steps of the detection method, reference may be made to the method embodiment for a specific implementation manner, which will not be repeated here.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present application, and are used to illustrate the technical solutions of the present application, rather than restricting them. The protection scope of the present application is not limited thereto, although referring to the foregoing The embodiments describe the application in detail, and those of ordinary skill in the art should understand that any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the application. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be covered in the application. within the scope of protection. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. a detection method of target image, is characterized in that, described detection method comprises: Obtain the target image to be detected; Input the target image to be detected into the backbone network of the pre-trained target detection model, perform feature extraction on the target image to be detected, and determine the shallow feature map and deep feature of the target image to be detected. picture; Inputting the shallow feature map and the deep feature map to the feature pyramid network of the target detection model, respectively weighting the shallow feature map and the deep feature map with different weights to determine fusion features picture; The fusion feature map is input to the head network of the target detection model, and the target item in the target image to be detected is determined. 2 . The detection method according to claim 1 , wherein, by inputting the shallow feature map and the deep feature map into the feature pyramid network of the target detection model, the shallow feature maps are respectively applied to the feature pyramid network of the target detection model. 3 . The map and the deep feature map are weighted with different weights to determine the fusion feature map, including: Inputting the shallow feature map and the deep feature map to the feature pyramid network for feature splicing, and determining the splicing feature map; Inputting the splicing feature map to the convolution layer of the feature pyramid network, performing convolution processing on the splicing feature map, and determining the first target feature map; Inputting the first target feature map to the pooling layer of the feature pyramid network, performing a global average pooling process on the first target feature map, and determining a second target feature map; Inputting the second target feature map to the activation layer of the feature pyramid network, performing activation processing on the second target feature map, and determining the first target weight and the second target weight; Inputting the first target weight and the second target weight to the fusion layer of the feature pyramid network, using the first target weight to weight the shallow feature map and using the second target weight Perform weighting processing on the deep feature map to determine the fusion feature map. 3. The detection method according to claim 2, wherein the splicing feature map is input into the convolution layer of the feature pyramid network, the splicing feature map is subjected to convolution processing, and the first splicing feature map is determined. A target feature map, including: Obtain an initialized two-dimensional matrix, wherein the dimension of the initialized two-dimensional matrix is determined according to the number of convolution kernels and the number of feature vectors of the splicing feature map in different dimensions; Based on the feature information corresponding to the feature vectors of the spliced feature maps in different dimensions, determine the target value of the feature vectors of the spliced feature maps in different dimensions; The target value of the feature vector of the splicing feature map under the different dimensions is added to the two-dimensional matrix of the initialization, and the target two-dimensional matrix is generated; Each of the convolution kernels filters out the corresponding feature vectors of the spliced feature maps in different dimensions according to the target values of the feature vectors of the spliced feature maps in different dimensions in the target two-dimensional matrix, and performs Convolution processing is performed to determine the first target feature map. 4 . The detection method according to claim 2 , wherein the second target feature map is input into the activation layer of the feature pyramid network, and the second target feature map is activated to determine the Get the first target weight and the second target weight, including: Use the sigmoid activation function to normalize the second target feature map to determine the weight vector; The first target weight and the second target weight are determined by dividing based on the dimension of the weight vector. 5 . The detection method according to claim 2 , wherein the first target weight and the second target weight are input into the fusion layer of the feature pyramid network, and the first target weight is used. 6 . Perform weighting processing on the shallow feature map and the deep feature map using the second target weight to determine the fusion feature map, including: Perform weighting processing on the shallow feature map by using the first target weight to determine the weighted shallow feature map; Perform weighting processing on the deep feature map using the second target weight to determine a weighted deep feature map; The weighted shallow feature map and the weighted deep feature map are feature added to determine the fusion feature map. 6. The detection method according to claim 1, wherein the target detection model is trained through the following steps: Obtain a plurality of sample pictures and label information corresponding to each sample picture, and divide the plurality of sample pictures into a training set and a verification set; Use the training set and the label information corresponding to the sample pictures corresponding to the training set to perform multiple training on the initial training model to determine the target detection model; The target detection model is tested by using the verification set to verify the detection accuracy of the target detection model. 7. A detection device for a target image, wherein the detection device comprises: an acquisition module for acquiring the target image to be detected; The feature extraction module is used to input the target image to be detected into the backbone network of the pre-trained target detection model, perform feature extraction on the target image to be detected, and determine the shallowness of the target image to be detected. Layer feature maps and deep feature maps; A feature fusion module is used to input the shallow feature map and the deep feature map to the feature pyramid network of the target detection model, and weight the shallow feature map and the deep feature map with different weights respectively processing to determine the fusion feature map; A determination module, configured to input the fusion feature map to the head network of the target detection model, and determine the target item in the target image to be detected. 8 . The detection device according to claim 7 , wherein the feature fusion module is used in the feature pyramid network for inputting the shallow feature map and the deep feature map to the target detection model. 9 . , respectively perform weighting processing on the shallow feature map and the deep feature map with different weights, and when the fusion feature map is determined, the feature fusion module is specifically used for: Inputting the shallow feature map and the deep feature map to the feature pyramid network for feature splicing, and determining the splicing feature map; Inputting the splicing feature map to the convolution layer of the feature pyramid network, performing convolution processing on the splicing feature map, and determining the first target feature map; Inputting the first target feature map to the pooling layer of the feature pyramid network, performing a global average pooling process on the first target feature map, and determining a second target feature map; Inputting the second target feature map to the activation layer of the feature pyramid network, performing activation processing on the second target feature map, and determining the first target weight and the second target weight; The first target weight and the second target weight are input to the fusion layer of the feature pyramid network, and the shallow feature map is paired with the first target weight and the second target weight is used for the The deep feature map is weighted to determine the fusion feature map. 9. An electronic device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the Communication between the memories is performed through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the method for detecting a target image according to any one of claims 1 to 6. 10. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the target image according to any one of claims 1 to 6 when the computer program is run by a processor steps of the detection method.

Images