CN116403064A - Picture processing method, model, basic block structure, device and medium - Google Patents

Abstract

The invention relates to the field of computer vision, and discloses a picture processing method, model, basic block structure, equipment and medium. The method includes: obtaining n branch features extracted from the target image under n branches, where n is a positive integer; calculating branch feature channel weight value vectors corresponding to each branch, and each branch feature channel weight value vector is used to represent the corresponding branch The channel importance corresponding to each channel of the feature; use the branch feature channel weight value vector to perform channel-level feature weighting on the corresponding branch features, and obtain n channel weighted branch features; fuse the n channel weighted branch features to obtain the target The output features of the image. Based on the technical solution provided by the present invention, it is possible to enhance the expressive ability of channels in the network that can improve performance, and at the same time suppress the expressive ability of channels that have little influence on the final result, thereby improving the image processing effect.

Description

Image processing method, model, basic block structure, equipment and medium

technical field

The invention relates to the field of computer vision, in particular to a picture processing method, model, basic block structure, equipment and media.

Background technique

In recent years, deep learning has been widely used in computer vision fields such as image classification, image segmentation, and object detection.

When using a deep learning model to learn pictures, it is often necessary to deepen or widen the direction of the network to enhance the learning ability of the model, but this method inevitably introduces too many picture features.

In the face of a large number of image features, how to select discriminative image features from them needs to provide a solution.

Contents of the invention

In view of this, the present invention provides a picture processing method, model, basic block structure, equipment and medium, and designs an attention mechanism to improve the picture processing performance by weighting the channels of picture features.

In a first aspect, the present invention provides an image processing method, the method comprising:

Obtain n branch features extracted from the target image under n branches, where n is a positive integer;

Calculating the branch feature channel weight value vector corresponding to each branch, each of the branch feature channel weight value vectors is used to represent the channel importance corresponding to each channel of the corresponding branch feature;

Using the branch feature channel weight value vector, performing channel-level feature weighting on the corresponding branch features to obtain n channel weighted branch features;

The n channel weighted branch features are fused to obtain the output features of the target picture.

In a second aspect, the present invention provides an attention model, which includes:

The input module is used to obtain n branch features extracted from the target picture under n branches, and n is a positive integer;

The channel weight value calculation module is used to calculate the branch feature channel weight value vector corresponding to each branch, and each branch feature channel weight value vector is used to represent the channel importance corresponding to each channel of the corresponding branch feature;

A feature weighting module, configured to use the branch feature channel weight value vector to perform channel-level feature weighting on the corresponding branch features to obtain n channel-weighted branch features;

An output module, configured to fuse the n channel weighted branch features to obtain the output features of the target picture.

In a third aspect, the present invention provides a first basic block structure, which includes: the attention model and the addition module as described in the above aspect;

The attention model is used to output the output features of the target picture when the input target picture is extracted under the branch features of n branches, and n is a positive integer;

The adding module is configured to add the output feature of the target picture to the target picture to obtain an output result of the first basic block structure for the target picture.

In a fourth aspect, the present invention provides a second basic block structure, which includes: the attention model and the addition module as described in the above aspect;

The attention model is used to output the output features of the target picture when the input target picture is extracted under the branch features of n branches, and n is a positive integer;

The adding module is configured to add the output feature of the target picture and the result of batch normalization of the target picture to obtain the output result of the second basic block structure for the target picture.

In a fifth aspect, the present invention provides a computer device, including: a memory and a processor, the memory and the processor are connected to each other, computer instructions are stored in the memory, and the processor executes the computer instructions to perform the above-mentioned first aspect or the image processing method in any implementation manner corresponding thereto.

In a sixth aspect, the present invention provides a computer-readable storage medium, where computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the image processing in the above-mentioned first aspect or any corresponding implementation manner method.

The technical solutions provided by the embodiments of the present invention can at least have the following beneficial effects:

In the case where there are n branch features under n branches extracted from the target image, the branch feature channel weight value vector is calculated for each branch, and the branch feature channel weight value vector is used to represent the importance of the channel corresponding to each channel of the corresponding branch feature degree, use the branch feature channel weight value vector to carry out channel-level feature weighting on the branch features to obtain n channel weighted branch features, add the n channel weighted branch features to obtain the output features of the target image, and thus based on attention The mechanism weights channels with different branch features, enhances the expressive ability of channels that improve performance, and at the same time suppresses the expressive ability of channels that have little impact on performance, and finally improves the performance of output features based on channel-weighted branch features. When the output feature is applied to computer vision tasks such as classification, it can guarantee the processing effect of the task.

Description of drawings

In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings that need to be used in the specific implementation or description of the prior art. Obviously, the accompanying drawings in the following description The drawings show some implementations of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a schematic flow diagram of an image processing method according to an embodiment of the present invention;

FIG. 2 is a schematic flowchart of another image processing method according to an embodiment of the present invention;

FIG. 3 is a schematic flowchart of another image processing method according to an embodiment of the present invention;

Fig. 4 is a schematic structural diagram of an attention model according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of an image processing process according to an embodiment of the present invention;

6 is a schematic structural diagram of a first basic block structure according to an embodiment of the present invention;

7 is a schematic structural diagram of a second basic block structure according to an embodiment of the present invention;

Fig. 8 is a schematic structural diagram of a classification network according to an embodiment of the present invention;

Fig. 9 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present invention.

First, a brief introduction to the concept of the attention mechanism involved in the present invention:

Deep learning has achieved great success in solving computer vision problems such as image classification, image segmentation, and object detection. In recent years, many excellent deep learning models have emerged.

In recent years, the attention model (a type of deep learning model) (Attention Model) has been widely used in various types of deep learning tasks such as natural language processing, image recognition and speech recognition, and is the most noteworthy in deep learning technology. Therefore, the deep learning model based on the attention mechanism has received extensive attention in recent years and has become an important research direction.

In human vision, the attention mechanism is manifested as the human visual attention mechanism, which is a brain signal processing mechanism unique to human vision. Human vision quickly scans the global image to obtain the target area that needs to be focused on, which is generally referred to as the focus of attention, and then invests more attention resources in this area to obtain more detailed information about the target that needs to be focused on. And suppress other useless information. This is a means for humans to use limited attention resources to quickly screen out high-value information from a large amount of information. It is a survival mechanism formed in the long-term evolution of humans. The human visual attention mechanism has greatly improved the efficiency and efficiency of visual information processing. accuracy.

For deep learning, the design of the attention mechanism, in layman's terms, is to focus on important points and ignore other unimportant factors. The judgment of the importance depends on different network structures or application scenarios.

With the continuous development of deep learning technology, deep learning models emerge in endlessly, but in order to further improve the accuracy, researchers often design in the direction of deepening or widening the network. It is undeniable that as the network becomes deeper or wider, the learning ability of the model continues to increase, but the amount of computation and parameters of the model also increase rapidly, which is not conducive to deployment in practical applications. At the same time, as the number of model layers increases, It is also inevitable to introduce a lot of noise (that is: many useless features), too many features usually not only will not improve the ability of the network model, but will confuse the classifier, thereby reducing the recognition ability of the network.

Therefore, only a limited number of discriminative features can be selected to achieve good discrimination and maximize the ability of the model. The attention mechanism (attention mechanism) shows a huge advantage in feature selection ability and is widely adopted.

Based on this, the embodiment of the present invention provides an image processing method, which aims to design a better attention model structure, to enhance the expressive ability of channels in the network that can improve performance, and at the same time suppress The expressive ability of the channel can further improve the image processing effect.

According to an embodiment of the present invention, a picture processing method is provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the steps shown in the flow chart Although a logical order is shown, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

In this embodiment, a picture processing method is provided, which can be used in a computer device. FIG. 1 is a flowchart of a picture processing method according to an embodiment of the present invention. As shown in FIG. 1 , the process includes the following steps:

Step S101, acquiring n branch features extracted from n branches of the target picture, where n is a positive integer.

Among them, the target picture is a picture that needs to be characterized for feature identification, and the output feature is selected. The embodiment of the present invention does not limit the specific content of the target picture, the specific application scenarios of the finally selected output features, and the like. For example: applying the output features to the task of pedestrian re-identification. In the task of pedestrian re-identification, the expression of pedestrian features and the screening of discriminative features directly determine whether the target pedestrian can be correctly identified, which is an important task of pedestrian re-identification. links.

In this embodiment, feature extraction is performed on the target picture under n branches respectively, and n branch features under n branches are obtained, where n is a positive integer.

Among them, each branch is an independent computing module containing multiple convolutional layers. Each branch usually has different convolution kernel sizes and the number of convolutional layers, so that the branch features extracted under different branches can have different receptive fields.

Wherein, n can be further limited to a positive integer greater than 1, that is, branch features are extracted under multi-branch, and the image processing method based on the attention mechanism proposed in the embodiment of the present invention is applied to a multi-branch scene. It can be understood that the image processing method based on the attention mechanism proposed in the embodiment of the present invention can also be applied to a single-branch scenario, but in a multi-branch scenario, the performance improvement effect of the technical solution proposed in the embodiment of the present invention is even better. significantly. In the following embodiments, a multi-branch scenario is mainly used for description.

Step S102, calculating branch feature channel weight value vectors corresponding to each branch, and each branch feature channel weight value vector is used to represent the channel importance corresponding to each channel of the corresponding branch feature.

Among them, channel is a dimension used to describe image features, and the branch features under each branch can have multiple different channels.

In this embodiment, n branch feature channel weight value vectors corresponding to n branches are respectively calculated, and each branch feature channel weight value vector is used to represent the channel importance corresponding to each channel of the corresponding branch feature.

Exemplarily, there are two branches: branch 1 and branch 2, and two branch feature channel weight value vectors corresponding to the two branches are calculated: branch feature channel weight value vector 1 and branch feature channel weight value vector 2. Among them, the branch feature channel weight value vector 1 is used to represent the channel importance corresponding to each channel of the branch feature under branch 1; the branch feature channel weight value vector 2 is used to represent the channel importance corresponding to each channel of the branch feature under branch 2.

Step S103 , using the branch feature channel weight value vectors to perform channel-level feature weighting on the corresponding branch features to obtain n channel-weighted branch features.

In this embodiment, after n branch feature channel weight value vectors are obtained, channel-level feature weighting is performed on corresponding branch features using the branch feature channel weight value vectors to obtain n channel-weighted branch features.

Exemplarily, there are two branches: branch 1 and branch 2. Use branch feature channel weight value vector 1 to perform channel-level feature weighting on branch feature 1 under branch 1 to obtain channel-weighted branch feature 1; use branch feature channel weight Value vector 2 performs channel-level feature weighting on branch feature 2 under branch 2 to obtain channel-weighted branch feature 2.

Step S104, fusing the n channel weighted branch features to obtain the output features of the target picture.

In this embodiment, all channel weighted branch features are fused to obtain the final output features of the target picture.

One way of fusing the n channel weighted branch features may be to add the n channel weighted branch features.

To sum up, in the image processing method provided by this embodiment, when the target image extracts n branch features under n branches, the branch feature channel weight value vector is calculated for each branch, and the branch feature channel weight value The vector is used to represent the channel importance corresponding to each channel of the corresponding branch feature. Using the branch feature channel weight value vector, the branch feature is weighted at the channel level to obtain n channel weighted branch features, and the n channel weighted branch features are calculated. Add them together to get the output features of the target image, so as to weight the channels of different branch features based on the attention mechanism, enhance the expressive ability of the channel that improves the performance, and suppress the expressive ability of the channel that has little impact on the performance, and finally improve Based on the performance of the output feature obtained by channel-weighted branch features, the output feature can guarantee the processing effect of the task when it is applied to computer vision tasks such as classification.

In this embodiment, a picture processing method is provided, which can be used in a computer device. FIG. 2 is a flowchart of a picture processing method according to an embodiment of the present invention. As shown in FIG. 2 , the process includes the following steps:

Step S201, acquiring branch features extracted from n branches of the target picture, where n is a positive integer.

For details, refer to step S101 in the embodiment shown in FIG. 1 , and details are not repeated here.

Step S202, performing feature compression on n branch features to obtain compressed features.

Wherein, the compressed feature is a feature obtained by compressing n branch features. It can be understood that in subsequent processing steps, from the perspective of calculation amount, the calculation amount of compressed features is less than that of n individual branch features.

Step S203, based on the compressed feature, calculate the compressed feature channel weight value vector, which is used to represent the channel importance corresponding to each channel of the compressed feature.

In this embodiment, after the compressed features corresponding to the n branch features are obtained, a compressed feature channel weight value vector representing the channel importance corresponding to each channel of the compressed feature is calculated.

Exemplarily, there are 3 channels in the compressed feature: channel 1, channel 2, and channel 3, then calculate the channel weights of the compressed feature that can represent the channel importance of channel 1, the channel importance of channel 2, and the channel importance of channel 3 a vector of values.

In an optional implementation, the calculation process of compressing features includes: adding n branch features to obtain fusion features corresponding to n branch features; or connecting n branch features based on channel dimensions to obtain Fusion features corresponding to n branch features.

In this embodiment, the n branch features can be compressed in a fusion manner. Specifically, the n branch features can be added directly, or the n branch features can be connected based on the channel dimension, so as to ensure that the n branch features Features are effectively compressed.

In an optional implementation, when the number of compressed feature channels is m, and m is a positive integer, calculating the compressed feature channel weight value vector based on the compressed feature includes: splitting the compressed feature at the channel level , to obtain m channel feature maps; calculate statistical information of each channel feature map under various statistics, and obtain m channel statistic vectors; based on the m channel statistic vectors, calculate and obtain compressed feature channel weight value vectors.

Wherein, the statistic information under the multi-statistic refers to: performing statistics on the channel feature map from the perspective of multiple statistic, and the obtained statistic information includes the corresponding information of the multiple statistic.

In this embodiment, the compressed features are first split according to the dimension of the channel, and then the statistics information under the multi-statistics is calculated for each channel feature map, and the obtained channel statistics vector can be used to reflect the channel feature map characteristics, and then use the statistics vector of each channel to calculate the weight value vector of the compressed feature channel, so as to complete the accurate calculation of the weight value vector of the compressed feature channel by evaluating the importance of the channel through multiple probabilities in the branch.

In an optional implementation manner, based on the m channel statistic vectors, calculating the compressed feature channel weight value vector includes: respectively inputting the m channel statistic vectors into at least one fully connected layer, and outputting m channels respectively Corresponding channel importance; the channel importance corresponding to the m channels is arranged according to the channel, and the compressed feature channel weight value vector is obtained.

In this embodiment, channel statistics under multi-statistics are learned through at least one fully connected layer, and the channel importance corresponding to each channel is obtained. The channel importance of all channels is the compressed feature after being arranged according to the channel. Channel weight value vector, so that under the design of multi-statistics, the importance of channels can be accurately determined according to multi-statistics.

In an optional implementation manner, the statistics include at least one of the following: mean, variance, coefficient of variation, skewness, peak value, maximum value, minimum value, median, and quartile.

In this embodiment, statistics from different angles are used to comprehensively reflect the data distribution characteristics of each channel feature map. Quantitative information can reflect the importance of channels in more detail.

Step S204, based on the compressed feature channel weight value vector, calculate branch feature channel weight value vectors corresponding to each branch.

In this embodiment, after the compressed feature channel weight value vector is calculated, based on this information, the channel importance corresponding to each channel of the branch feature is calculated to obtain n branch feature channel weight value vectors.

Step S205, using the branch feature channel weight value vectors to perform channel-level feature weighting on the corresponding branch features to obtain n channel-weighted branch features.

For details, please refer to step S103 in the embodiment shown in FIG. 1 , which will not be repeated here.

Step S206, fusing the n channel weighted branch features to obtain the output features of the target picture.

For details, please refer to step S104 in the embodiment shown in FIG. 1 , which will not be repeated here.

To sum up, the image processing method based on the attention mechanism provided by this embodiment first calculates the fusion feature channel weight value vector representing the channel importance corresponding to each channel of the fusion feature, and then based on the fusion feature channel weight value vector, Calculate the branch feature channel weight value vector corresponding to each branch, so as to accurately evaluate the importance of the channel from the fine-grained feature comparison of the fusion feature.

In this embodiment, a picture processing method is provided, which can be used in computer equipment. FIG. 3 is a flow chart of the picture processing method according to the embodiment of the present invention. As shown in FIG. 3, the above process step S204 can be replaced as follows step:

Step S301, based on the compressed feature channel weight value vector, perform channel-level feature weighting on the compressed feature to obtain weighted compressed features.

In this embodiment, channel-level feature weighting is performed according to the corresponding relationship between compressed feature channel weight value vectors and compressed feature channels to obtain weighted compressed features.

In an optional implementation manner, the weighted compressed features further include the following weighting process: performing feature weighting of local spatial positions on the compressed features that have undergone channel-level feature weighting to obtain weighted compressed features.

In this embodiment, after the first weighting of the channel level is performed on the compressed features, the second weighting of the local spatial position is carried out, and the subsequent calculation is performed using the compressed features after double weighting. By adding the second weighting The feature weighting of the local spatial position further improves the performance of compressed features.

In an optional implementation manner, the feature weighting of the local spatial position is performed on the compressed features that have completed the channel-level feature weighting, and the weighted compressed features are obtained, including:

(1) For any spatial position pixel in the compressed feature with channel-level feature weighting, calculate the local neighbor relationship vector corresponding to the spatial position pixel, and the local neighbor relationship vector is used to represent the spatial position pixel and the neighborhood spatial position pixel. The correlation of the neighborhood spatial loxel is the spatial loxel around the spatial loxel.

In this embodiment, after the first channel-level feature weighting is completed, for each spatial position pixel in the compressed feature, the correlation between it and the surrounding spatial position pixels is calculated to obtain each spatial position pixel The corresponding local neighbor relation vector.

In an optional implementation manner, for any spatial position pixel in the compressed feature that has completed channel-level feature weighting, calculating the local neighbor relationship vector corresponding to the spatial position pixel includes: for the target spatial position pixel, respectively calculating the target The correlation between the spatial position pixel and the k neighborhood spatial position pixels obtains k local neighbor relationship scalars of the target spatial position pixel, k is a positive integer; the k local neighbor relationship scalars of the target spatial position pixel are spliced, The local neighbor relationship vector corresponding to the target spatial position pixel is obtained.

Wherein, the target spatial location pixel is any spatial location pixel in the compressed feature.

In this embodiment, the calculation process of the local neighbor relationship vector of the target spatial position pixel is as follows: first determine the k spatial position pixels around it as k neighborhood spatial position pixels, and then calculate the target spatial position pixel and each The correlation between the pixels in the neighborhood space position is used to obtain k local neighbor relationship scalars, and then the k local neighbor relationship scalars are spliced in columns to obtain the local neighbor relationship vector of the target spatial position pixel, thereby accurately generating the spatial The local neighbor relationship vector corresponding to the loxel.

In an optional implementation manner, before calculating the local neighbor relationship vector corresponding to the spatial position pixel, the method further includes: selecting a target from various local neighborhood relationships for the spatial position pixel through the local neighborhood selection network The local neighborhood relationship, the local neighborhood relationship is used to define the relationship between the spatial position pixel and the neighborhood spatial position pixel; the compressed feature that satisfies the selected target local neighborhood relationship with the spatial position pixel is used as the spatial position pixel’s neighbor Domain space loxel.

Among them, the local neighborhood selection network is a neural network with a local neighborhood relationship matching function.

In this embodiment, at least one local neighborhood relationship is defined in advance, and the local neighborhood selection network is used to match the target local neighborhood relationship for the spatial position pixel, and then the corresponding neighborhood of the spatial position pixel is determined through the target local neighborhood relationship spatial position pixels, so as to select a reasonable neighborhood spatial position pixel for each spatial position pixel, and the selection of the neighborhood spatial position pixels is adaptive and variable.

(2) Normalize the local neighbor relationship vectors corresponding to each spatial position pixel to obtain the normalized local neighbor relationship vectors corresponding to each spatial position pixel.

In this embodiment, for each spatial position pixel in the compressed feature, the correlation between the local neighbor relation vector and different neighborhood spatial position pixels is normalized, and finally the corresponding spatial position pixel The normalized local neighbor relation vector of .

Among them, the normalization process can be realized through the Softmax function.

(3) Use the normalized local neighbor relationship vector to perform feature weighting on the corresponding spatial position pixels to obtain weighted compressed features.

In this embodiment, for each spatial position pixel in the compressed feature, its corresponding normalized local neighbor relationship vector is used for feature weighting. After the feature weighting of all spatial position pixels is completed, the weighted Compression features.

In an optional implementation manner, the normalized local neighbor relationship vector is used to perform feature weighting on the corresponding spatial position pixels to obtain the weighted compressed features, including: extracting the target spatial position pixel for the target spatial position pixel The vector of the pixel in the neighborhood spatial position of the pixel in the channel dimension constitutes the local area feature of the target spatial position pixel; the local neighbor relationship vector corresponding to the target spatial position pixel is weighted with the local area feature of the target spatial position pixel to obtain the target space The local area weighted features of the position pixels; the local weighted features of all the spatial position pixels are spliced to obtain the weighted compressed features.

Wherein, the target spatial location pixel is any spatial location pixel in the compressed feature.

In this embodiment, the calculation process of using the normalized local neighbor relationship vector to perform feature weighting on the target spatial position pixel is as follows: for the k neighboring spatial position pixels around the target spatial position pixel, extract according to the dimension of the channel The vectors of these k neighborhood spatial position pixels are obtained to obtain the extracted local area features, and the normalized local neighbor relationship vector corresponding to the target spatial position pixels is used to perform matrix with the local area feature corresponding to the target spatial position pixels Multiplication for accurate weighted fusion.

Step S302 , downsampling the weighted compressed features in the dimensions of height and width to obtain the downsampled compressed features.

In this embodiment, the dimension of the weighted compressed feature is height×width×channel, which is down-sampled by height×width to obtain the compressed feature after downsampling of 1×channel dimension, so as to facilitate the subsequent channel dimension. deal with.

Step S303, split the down-sampled compressed features into n groups of split features, each group of split features corresponds to a branch.

In this embodiment, the downsampled compressed features are evenly split into n groups, and the split features in each group can be used to train the channel importance of the corresponding branch.

Step S304, perform feature restoration on the n groups of split features respectively, and obtain n branch feature channel weight value vectors corresponding to the n branches respectively.

In this embodiment, since the split feature has been split, its scale cannot be used to characterize the channel importance corresponding to each channel of the branch feature. Therefore, the feature restoration is performed on the n groups of split features, and the restoration is the same as the branch feature The channel weight value vector matches the scale to obtain the branch feature channel weight value vector under the corresponding branch.

In an optional implementation manner, feature restoration is performed on n groups of split features respectively, and n branch feature channel weight value vectors corresponding to n branches are respectively obtained, including: inputting n groups of split features into at least one full The connection layer is output to obtain n feature channel weight value vectors of the preliminary branches; the weight value vectors of the feature channel of the n preliminary branches are compared and weighted between branches, and n branch feature channel weight value vectors are obtained.

In this embodiment, the split feature is learned through at least one fully connected layer, and the learned information is used as the weight value vector of the feature channel of the preliminary branch, and the weight value vector of the feature channel of the preliminary branch is further subjected to the comparative weighting process between branches , and use the obtained information as the final branch feature channel weight value vector, thereby increasing the fine-grained feature comparison between branch features and accurately evaluating the importance of channels in all directions.

In an optional implementation, when the number of compressed feature channels is m, and m is a positive integer, the comparison and weighting process between branches is performed on the n preparatory branch feature channel weight value vectors to obtain n branch features Channel weight value vectors, including: respectively extracting the channel weight values of the n preparatory branch feature channel weight value vectors on the same channel to obtain m reorganized channel weight value vectors corresponding to m channels respectively; The value vectors are normalized respectively to obtain normalized m reorganized channel weight value vectors; use the normalized m reorganized channel weight value vectors to perform feature Replace to get n branch feature channel weight value vectors.

In this embodiment, for all the pre-branch feature channel weight value vectors, elements corresponding to the same channel are extracted, and after m channel extraction is completed, m reorganized channel weight value vectors are obtained, and m reorganized channel weight value vectors are obtained. The channel weight value vectors are normalized, and then the normalized m reorganized channel weight value vectors are restored to the original positions in the n preliminary branch feature channel weight value vectors, so that the pre-branch feature channel weights between branches The horizontal comparison of value vectors realizes the comparison feature weighting between branches.

To sum up, the image processing method based on the attention mechanism provided by this embodiment, based on the compressed feature channel weight value vector, performs channel-level feature weighting on the compressed features, obtains the weighted compressed features, and then weights the compressed features The features are down-sampled in the dimensions of height and width to obtain the compressed features after down-sampling, split the compressed features after the down-sampling into n groups of split features, and restore the n groups of split features to obtain the corresponding N branch feature channel weight value vectors, so that the channel importance of the channel in each branch is calculated through feature compression and restoration.

It can be understood that the above method embodiments may be implemented individually or in combination, which is not limited in the present invention.

An attention model is also provided in this embodiment, and the attention model is used to implement the above embodiments and preferred implementation manners, and what has been described will not be repeated.

This embodiment provides an attention model, as shown in Figure 4, the attention model includes:

The input module 401 is used to obtain n branch features extracted from the target picture under n branches, where n is a positive integer;

The channel weight value calculation module 402 is used to calculate the branch feature channel weight value vector corresponding to each branch, and each branch feature channel weight value vector is used to represent the channel importance corresponding to each channel of the corresponding branch feature;

The feature weighting module 403 is configured to use the branch feature channel weight value vector to perform channel-level feature weighting on the corresponding branch features to obtain n channel-weighted branch features;

The output module 404 is configured to fuse the features of the n channel weighted branches to obtain the output features of the target picture.

In some optional implementation manners, the channel weight calculation module 402 includes:

A feature compression unit is used to perform feature compression on n branch features to obtain compressed features;

The compressed feature channel weight value calculation unit is used to calculate the compressed feature channel weight value vector based on the compressed feature, and the compressed feature channel weight value vector is used to represent the channel importance corresponding to each channel of the compressed feature;

The branch feature channel weight value calculation unit is configured to calculate the branch feature channel weight value vector corresponding to each branch based on the compressed feature channel weight value vector.

In some optional implementation manners, the branch feature channel weight value calculation unit includes:

The weighted calculation subunit is used to perform channel-level feature weighting on the compressed features based on the compressed feature channel weight value vector to obtain weighted fusion features;

The downsampling subunit is used to downsample the weighted compressed features in the dimensions of height and width to obtain the downsampled compressed features;

The feature splitting subunit is used to split the compressed features after downsampling into n groups of split features, each group of split features corresponds to a branch;

The feature reduction sub-unit is used to perform feature reduction on n groups of split features respectively, and obtain n branch feature channel weight value vectors corresponding to n branches respectively.

In some optional embodiments, the feature reduction unit is used for:

Input n groups of split features into at least one fully connected layer respectively, and output n preparatory branch feature channel weight value vectors;

The comparison and weighting process between branches is performed on the n preparatory branch feature channel weight value vectors to obtain n branch feature channel weight value vectors.

In some optional implementations, when the number of channels of the compressed feature is m, and m is a positive integer, the feature reduction unit is used for:

Extracting the channel weight values of the n preparatory branch feature channel weight value vectors on the same channel respectively, and obtaining m reorganized channel weight value vectors corresponding to the m channels respectively;

Normalize each reorganized channel weight value vector respectively to obtain normalized m reorganized channel weight value vectors;

Using the normalized m reorganized channel weight value vectors, perform feature replacement on the n preparatory branch feature channel weight value vectors to obtain n branch feature channel weight value vectors.

In some optional implementation manners, the weighted calculation subunit is used for:

The feature weighting of the local spatial position is performed on the compressed features that have completed the channel-level feature weighting, and the weighted compressed features are obtained.

In some optional implementation manners, the weighted calculation subunit is used for:

For any spatial position pixel in the compressed feature with channel-level feature weighting, calculate the local neighbor relationship vector corresponding to the spatial position pixel, and the local neighbor relationship vector is used to represent the correlation between the spatial position pixel and the neighborhood spatial position pixel , the neighborhood spatial location pixel is the spatial location pixel around the spatial location pixel;

Normalizing the local neighbor relationship vectors corresponding to each spatial position pixel to obtain the normalized local neighbor relationship vectors corresponding to each spatial position pixel;

Use the normalized local neighbor relationship vector to perform feature weighting on the corresponding spatial position pixels to obtain weighted compressed features.

In some optional implementation manners, the weighted calculation subunit is used for:

Through the local neighborhood selection network, a target local neighborhood relationship is selected from a variety of local neighborhood relationships for the spatial location pixel, and the local neighborhood relationship is used to define the relationship between the spatial location pixel and the neighborhood spatial location pixel;

The compressed feature that satisfies the selected target local neighborhood relationship with the spatial position pixel is used as the neighborhood spatial position pixel of the spatial position pixel.

In some optional implementation manners, the weighted calculation subunit is used for:

For the target spatial position pixel, respectively calculate the correlation between the target spatial position pixel and k neighborhood spatial position pixels, and obtain k local neighbor relationship scalars of the target spatial position pixel, k is a positive integer;

The k local neighbor relationship scalars of the target spatial position pixel are spliced to obtain the local neighbor relationship vector corresponding to the target spatial position pixel.

In some optional implementation manners, the weighted calculation subunit is used for:

For the target spatial position pixel, extract the vector of the neighborhood spatial position pixel of the target spatial position pixel in the channel dimension to form the local area feature of the target spatial position pixel;

Perform feature weighting on the local neighbor relationship vector corresponding to the target spatial position pixel and the local area feature of the target spatial position pixel to obtain the local area weighted feature of the target spatial position pixel;

The local weighted features of all spatial position pixels are spliced to obtain weighted compressed features.

In some optional implementation manners, when the number of compressed feature channels is m, and m is a positive integer, the compressed feature channel weight value calculation unit includes:

The channel splitting subunit is used to split the compressed features at the channel level to obtain m channel feature maps;

A multi-statistic statistics subunit is used to calculate the statistics information of each channel feature map under multiple statistics, and obtain m channel statistics vectors;

The fusion feature channel weight value calculation subunit is used to calculate and obtain the compressed feature channel weight value vector based on the m channel statistic vectors.

In some optional implementation manners, the compressed feature channel weight value calculation subunit is used for:

Input the m channel statistics vectors into at least one fully connected layer, and output the channel importance corresponding to the m channels respectively;

The channel importance corresponding to the m channels is arranged according to the channel, and the compressed feature channel weight value vector is obtained.

In some optional embodiments, the statistics include at least one of the following:

Mean, variance, coefficient of variation, skewness, peak, maximum, minimum, median, and quartiles.

In some optional implementation manners, the channel weight value calculation module 402 also includes a feature fusion unit, a feature fusion unit, for:

Add the n branch features to obtain the fusion features corresponding to the n branch features;

or,

The n branch features are connected based on the channel dimension to obtain the fusion features corresponding to the n branch features.

The further functional descriptions of the above-mentioned modules and units are the same as those of the above-mentioned corresponding embodiments, and will not be repeated here.

Combining the image processing methods described in the above embodiments, the process performed in the attention model can be divided into four stages: 1. Feature compression; 2. Feature splitting; 3. Feature screening; 4. Feature weighting.

Next, with reference to FIG. 5 , an exemplary description will be given to the above embodiment.

(1) Extraction stage of multi-branch features

The feature input of multi-branch picture features is obtained from any feature extraction module, and the present invention does not limit the specific implementation form of the feature extraction module used.

It is understandable that in the feature extraction stage, each branch plays a different role, and each branch usually achieves different functions. For example, each branch provides features of different receptive fields, which can provide richer features in the fusion stage. However, when more branches (usually means a wider network) provide rich features, there is no doubt that a lot of noise is introduced. Redundant features will not only improve the performance of the network in more cases, but also usually damage the classification. classification ability of the device. Therefore, it is necessary to design a mechanism to remove redundant features and retain the features of optimal discrimination. The attention model in the following stage can achieve this effect.

In Figure 5, it is assumed that each branch input feature map is c×H×W, and there are 4 branches in total.

(2) Feature compression stage

At this stage, the input multi-branch feature maps will be fused. There are two types of fusion operations: a) adding all feature maps; b) all feature maps are connected together in the channel dimension (concat), and the fused features use F Indicates that the dimension of F is C×H×W.

When the channel numbers of the four branch feature maps are the same, they can be added directly. When the channel numbers of the four branch feature maps are different, they can be connected together in the channel dimension.

(3) Feature splitting stage

The fused features contain a lot of information. How to filter out the most effective feature map channels from this information while suppressing the channels that do not contribute much to the final output requires designing an effective channel selection mechanism, namely the attention mechanism. .

The dimension of the fused feature F is C×H×W, where C represents the number of channels after fusion, H represents the height after fusion, and W represents the width after fusion. The usual attention mechanism represents the importance of the channel by traversing each channel C and calculating the mean value of the corresponding channel feature map H×W. After calculating the mean value of all C channels, a vector is obtained. Carry out multi-layer fully connected layer calculations to learn the importance of the corresponding channels, and finally weight them.

The present invention proposes an attention mechanism of multi-parameter probability statistics, which can truly realize multi-branch channel-level feature selection.

The first step is to split the fusion feature F according to the dimension of channel C. The second step is to do multi-parameter probability statistics for each channel. The traditional attention mechanism uses the mean value to count the importance of the feature map, but the variance and coefficient of variation , skewness and other statistics can more accurately reflect the characteristics of the channel feature map. It is too general to use the mean value as the training base value of channel importance alone. Therefore, this invention uses a variety of statistics to calculate the importance of each channel feature map.

Below, a variety of statistics are introduced:

1. Mean: used to describe the average value of the data.

为第i个像素值，M为像素值的数量，/>

为平均值。in,

is the i-th pixel value, M is the number of pixel values, />

is the average value.

2. Variance: used to reflect the fluctuation and stability of the data.

为第i个像素值，N为像素值的数量，/>

为平均值，/>

为方差。in,

is the i-th pixel value, N is the number of pixel values, />

is the average value, />

is the variance.

3. Coefficient of variation: The ratio of the standard deviation to the mean is called the coefficient of variation, which is a dimensionless quantity used to describe the relative dispersion of data.

为平均值，/>

为标准差，/>

为变异系数。in,

is the average value, />

is the standard deviation, />

is the coefficient of variation.

4. Skewness: used to describe the symmetry of the data.

为第i个像素值，M为像素值的数量，/>

为平均值，/>

为偏度。in,

is the i-th pixel value, M is the number of pixel values, />

is the average value, />

is the skewness.

5. Kurtosis: It is used to describe the steepness of the sample data distribution shape relative to the normal distribution.

为第i个像素值，M为像素值的数量，/>

为平均值，/>

为峰度。in,

is the i-th pixel value, M is the number of pixel values, />

is the average value, />

is the kurtosis.

6. Maximum (Maximum): used to represent the maximum pixel value of the feature map.

7. Minimum: The minimum pixel value used to represent the feature map.

8. Median: It is the median of the pixel values of the feature map, which is used to reflect the center position of the numerical distribution of the feature map.

9. Quartiles: Arrange the pixel values of the feature map from small to large, and divide the arranged values into four equal parts. The first, second, and third quartiles are 25%, 50%, and 75, respectively. The value of the % position can reflect the degree of dispersion of the value distribution of the feature map.

These statistics can be obtained by calculating the value of each channel of the feature map, and can be used to describe the numerical distribution of the feature map.

For each channel, calculate the multi-statistic information of each channel, and form a new channel statistics vector P, which can fully reflect the detailed distribution of the channel data, and is used to fully reflect the importance of the channel.

Through a fully connected layer, the channel statistics vector P of multi-statistic features is learned, and a number is output, which is used to represent the nonlinear combination of multi-statistic features, reflecting the importance of the channel. The usual attention mechanism directly calculates the average value of each channel and recombines it into a vector. The multi-statistic feature proposed by the present invention can reflect the importance of the channel in more detail. However, there are multiple statistics. Which one is the statistic? It is not known to reflect the importance of the channel, so the nonlinear combination of these statistics can be dynamically learned based on the fully connected learning mechanism, so as to learn the importance of the channel.

After traversing all channels, the above operations are performed, and finally the weight value learned by each channel is obtained. The weight values are arranged according to the channel to generate the fusion feature channel weight value vector V, and the dimension of V is C×1.

Use the fusion feature channel weight value vector V to weight the fusion features. The weighting method is to perform one-to-one correspondence multiplication according to the channel correspondence between V and the fusion feature to perform weighting.

The multi-probability channel weighting in the branch is completed above, and further local space weighting can be performed below.

First, obtain the fusion features weighted by the multi-probability channel in the previous step. Next, we will continue to process this fusion feature.

1) Location generation, establishing a local neighborhood selection network, the present invention pre-defines local neighborhood relationships of k shapes. For fusion features, a local neighborhood selection network can be designed to achieve local area selection of the fusion features through 2 global downsampling and 2 fully connected layers.

2) Calculation of the local neighbor relationship vector: For the fusion feature i corresponding to each position, calculate the correlation between it and the surrounding k positions. Assuming the size of the feature map is H x W x C, then k positions in a local area around position i can be expressed as P_i = {j_1, j_2, ..., j_k}. For each position j_k, the feature vectors at positions i and j can be mapped into d-dimensional vectors f_i and f_j by two 1x1 convolutional layers. Then, concatenate these two vectors to get a 2d-dimensional vector h_{ij} = [f_i, f_j], and input it into a small neural network for processing to generate a local neighbor relationship scalar, which is used to represent Correlation between position i and position j.

Generation of local neighbor relationship vectors: For each position i, the correlation w_{ij} between i and the surrounding k positions is concatenated by column, and a k*1 column vector is obtained as a local neighbor relationship vector.

3) Normalization of the local neighbor relationship vector w_{ij}: For each position i, use the Softmax function to normalize the correlation between the k positions around it, and still get a k * 1 column vector.

4) Feature weighting: weight the local neighbor relationship vector w_{ij} and the input feature map at position i to obtain a C-dimensional local area weighted feature, where the weighting method of each dimension is:

，得到位置i的局部区域加权特征，其维度为1*C，遍历所有H*W个位置，完成所有空间位置的局部区域加权。Traversing each position i of the fusion feature map, the k positions in the local area around the i position can be expressed as P_i = {j_1, j_2, ..., j_k}, according to the dimension of the channel, extract the k positions Vector, the dimension of the fusion feature map is H*W*C, after the extraction is completed, the extracted local area feature T_i is obtained, and its dimension is k*C. In the i-th position space, the local neighbor relationship vector w_{ij} (its dimension is k*1) is matrix multiplied with the local area feature T_i (its dimension is k*c) to realize weighted fusion, namely

, to obtain the local area weighted feature of position i, whose dimension is 1*C, traverse all H*W positions, and complete the local area weighting of all spatial positions.

Output feature map: The local area weighted features of all positions are spliced into an H*W*C tensor as the output feature map. The output feature map is the fusion feature of multi-probability channel weighting and local space weighting in the completed branch.

、/>

、/>

、/>

，每个组的维度为C/4维，下一步使用每个组的拆分特征/>

对每个分支的通道重要性进行训练。After the multi-probability channel weighting and local space weighting fusion features in the completed branch are down-sampled in the height and width dimensions, they are split into 4 parts on average, and each part corresponds to a group, called

, />

, />

, />

, the dimension of each group is C/4 dimension, and the next step is to use the split feature of each group />

Channel importance is trained on each branch.

(4) Feature screening stage

，建立多个分支，每个分支相互独立，用来计算输入特征图的通道的重要程度，即注意力机制。示例性的，每个分支包含2个全连接层，具体结构如下全连接层->整流 ->全连接层 ->整流，该结构对输入特征/>

进行学习，获取对应输入特征图通道的重要性，通过该操作可以获得预备分支特征通道权重值向量I_i。First according to the split feature of each group

, establish multiple branches, each branch is independent of each other, and is used to calculate the importance of the channel of the input feature map, that is, the attention mechanism. Exemplarily, each branch contains 2 fully connected layers, and the specific structure is as follows: fully connected layer -> rectification -> fully connected layer -> rectification, this structure has an impact on the input features />

Carry out learning to obtain the importance of the channel of the corresponding input feature map, and through this operation, the weight value vector I _i of the feature channel of the preliminary branch can be obtained.

Extract the elements at the corresponding positions for all I _i and recombine them into a new reorganized channel weight value vector Vi. The meaning of extracting the element at the corresponding position is: given that the dimension of I _i is C×1, there are a total of N feature maps from I ₁ to I _N , corresponding to N branches, traversing I ₁ to I _N , and taking the first of them respectively elements (that is, traverse all vectors I _i to take the value of the first element), form a new vector V _i , and the dimension of V _i is N×1. In this example, the dimension of V _i is 4×1.

Next, each reorganized vector V _i is normalized by the softmax function. Finally, replace the original feature vector I _i with the normalized features one by one, that is, finally restore the normalized features to their original positions, and obtain branch feature channel weight value vectors W ₁ to W _N .

(5) Feature weighting stage

The input features are weighted using the trained and filtered channel importance features W ₁ to W _N . The weighting method includes: in the first step, W ₁ to W _N are respectively multiplied with the corresponding input feature maps b ₁ to b _N at the pixel level according to the correspondence of the channels, and the channel-weighted vectors Wb ₁ to Wb _N are obtained; the second In the second step, the weighted features of all channels are added and the final result is output. This stage can be expressed by the following formula:

代表哈达玛积；N为分支数量，/>

为第i个分支对应通道权重值，/>

为第i个分支对应分支特征；/>

为输出特征。in,

Represents Hadamard product; N is the number of branches, />

is the channel weight value corresponding to the i-th branch, />

is the branch feature corresponding to the i-th branch; />

is the output feature.

It is understandable that the process described above can also be summarized as: splitting the target image into multiple branches; fusing the multi-branch features; performing multi-parameter probability statistical attention learning within the branch on the fused features, and obtaining channel-level weighted The final fusion features; through the local neighborhood selection network, the fusion features weighted at the channel level are weighted again in the spatial local position; the fusion features after the double weighting are completed; the fusion features after the downsampling are subjected to feature splitting , feature restoration; perform attention reorganization, comparison, and weighting between branches on the restored channel weight value vectors, and obtain the channel weight value vectors corresponding to each branch that has been processed; Perform weighted fusion to obtain the output features of the target image.

Based on the above attention model, the basic network layer can be constructed. In the embodiment of the present invention, two typical network layers are proposed: the first basic block structure and the second basic block structure.

(1) The first basic block structure

The first basic block structure includes: the attention model and the addition module of the above embodiment; the attention model is used to output the output of the target picture in the case of the branch features extracted under the n branches of the input target picture feature, n is a positive integer; the addition module is used to add the output feature of the target picture and the target picture to obtain the output result of the first basic block structure for the target picture.

Exemplarily, as shown in Figure 6, the feature weight of the target image is lifted through convolution kernels of different sizes to obtain multiple branch features, and the multiple branch features are input into the attention model to obtain the output features of the target image, and then the target image The output features of the image and the target image are added to obtain the final output result.

(2) The second basic block structure

The second basic block structure includes: the attention model and the addition module of the above embodiment; the attention model is used to output the output of the target picture in the case of the branch features extracted under the n branches of the input target picture feature, n is a positive integer; the addition module is used to add the output feature of the target picture and the result of the target picture after batch normalization to obtain the output result of the second basic block structure for the target picture.

Exemplarily, as shown in Figure 7, the feature weight of the target image is lifted through convolution kernels of different sizes to obtain multiple branch features, and the multiple branch features are input into the attention model to obtain the output features of the target image, and then the target image The output features of the image and the results of the target image after batch normalization are added to obtain the final output result.

It can be understood that, as shown in Figure 6 and Figure 7, the attention model designed by the present invention is plug-and-play, that is, the structure corresponding to the multi-branch is directly inserted into the output position of the multi-branch, and finally the weighted feature map.

Exemplarily, the basic block structure provided by the embodiment of the present invention can be used for image classification, and its specific use is shown in Figure 8. The specific structure of the network can include: convolution -> batch normalization, rectification -> maximum pooling -> Second basic block structure -> first basic block structure -> first basic block structure -> average pooling -> convolution -> second basic block structure -> first basic block structure -> first basic block structure - >First basic block structure -> global average pooling -> fully connected layer -> fully connected layer -> normalization -> classification probability.

It can be understood that, the embodiment of the present invention does not limit the specific application of the basic block structure, and FIG. 8 is only an exemplary illustration.

An embodiment of the present invention also provides a computer device having the attention model shown in FIG. 4 above.

Please refer to FIG. 9. FIG. 9 is a schematic structural diagram of a computer device provided in an optional embodiment of the present invention. As shown in FIG. 9, the computer device includes: one or more processors 10, memory 20, and a Interfaces of various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively connected to each other using different buses and may be mounted on a common motherboard or otherwise as desired. The processor may process instructions executed within the computer device, including instructions stored in or on the memory, to display graphical information of a GUI on an external input/output device such as a display device coupled to an interface. In some alternative implementations, multiple processors and/or multiple buses may be used with multiple memories and multiple memories, if desired. Likewise, multiple computing devices may be connected, with each device providing some of the necessary operations (eg, as a server array, a set of blade servers, or a multi-processor system). A processor 10 is taken as an example in FIG. 9 .

Processor 10 may be a central processing unit, a network processor or a combination thereof. Wherein, the processor 10 may further include a hardware chip. The aforementioned hardware chip may be an application specific integrated circuit, a programmable logic device or a combination thereof. The aforementioned programmable logic device may be complex programmable logic device, field programmable logic gate array, general array logic or any combination thereof.

Wherein, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes and implements the methods shown in the above-mentioned embodiments.

The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; created data, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some optional implementations, the memory 20 may optionally include memories that are remotely located relative to the processor 10, and these remote memories may be connected to the computer device through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state disk; the memory 20 may also include a combination of the above types of memories.

The computer device also includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware or firmware, or can be recorded in a storage medium, or can be downloaded through the network for original storage. Computer code on a remote storage medium or a non-transitory machine-readable storage medium to be stored on a local storage medium so that the methods described herein can be stored on a computer using a general purpose computer, a special purpose processor, or programmable or dedicated hardware Such software processing on storage media. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium may also include a combination of the above types of memories. It can be understood that a computer, processor, microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, processor or hardware, the above-mentioned implementation Example method.

Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention. within the bounds of the requirements.

Claims (20)

1. A picture processing method, is characterized in that, described method comprises: Obtain n branch features extracted from the target image under n branches, where n is a positive integer; Calculating the branch feature channel weight value vector corresponding to each branch, each of the branch feature channel weight value vectors is used to represent the channel importance corresponding to each channel of the corresponding branch feature; Using the branch feature channel weight value vector, performing channel-level feature weighting on the corresponding branch features to obtain n channel weighted branch features; The n channel weighted branch features are fused to obtain the output features of the target picture. 2. The method according to claim 1, wherein the calculation of branch feature channel weight value vectors corresponding to each branch comprises: Perform feature compression on n branch features to obtain compressed features; Based on the compressed feature, calculate a compressed feature channel weight value vector, and the compressed feature channel weight value vector is used to represent the channel importance corresponding to each channel of the compressed feature; Based on the compressed feature channel weight value vector, a branch feature channel weight value vector corresponding to each branch is calculated. 3. The method according to claim 2, wherein the calculation of the branch feature channel weight value vector corresponding to each branch based on the compressed feature channel weight value vector includes: Based on the compressed feature channel weight value vector, perform channel-level feature weighting on the compressed feature to obtain weighted compressed features; Downsampling the weighted compression feature in the dimensions of height and width to obtain the downsampled compression feature; The compressed features after the downsampling are split into n groups of split features, each group of split features corresponds to a branch; Perform feature restoration on the n groups of split features respectively to obtain n branch feature channel weight value vectors corresponding to the n branches respectively. 4. The method according to claim 3, wherein said splitting features of said n groups are respectively subjected to feature restoration to obtain n branch feature channel weight value vectors corresponding to n branches respectively, including: Input the split features of n groups into at least one fully connected layer respectively, and output n preparatory branch feature channel weight value vectors; Perform comparison and weighting processing between branches on the n preparatory branch feature channel weight value vectors to obtain the n branch feature channel weight value vectors. 5. The method according to claim 4, wherein, when the number of channels of the compressed feature is m, and m is a positive integer, the n preparatory branch feature channel weight value vectors are carried out between branches. Contrast weighting processing to obtain the n branch feature channel weight value vectors, including: Extracting the channel weight values of the n preparatory branch feature channel weight value vectors on the same channel respectively, and obtaining m reorganized channel weight value vectors corresponding to the m channels respectively; Normalize each reorganized channel weight value vector respectively to obtain normalized m reorganized channel weight value vectors; Using the normalized m reorganized channel weight value vectors, perform feature replacement on the n preparatory branch feature channel weight value vectors to obtain the n branch feature channel weight value vectors. 6. method according to claim 3, is characterized in that, the compression feature after described weighting also comprises following weighting process: The feature weighting of the local spatial position is performed on the compressed features that have been weighted at the channel level to obtain the weighted compressed features. 7. The method according to claim 6, wherein the feature weighting of the local spatial position is performed on the compressed features that have completed the feature weighting of the channel level, and the compressed features after the weighting are obtained, including: For any spatial position pixel in the compressed feature weighted by the channel level, calculate the local neighbor relationship vector corresponding to the spatial position pixel, and the local neighbor relationship vector is used to represent the spatial position pixel and the neighborhood a correlation between spatially located pixels, said neighborhood spatially located pixels being spatially located pixels surrounding said spatially located pixel; Normalizing the local neighbor relationship vectors corresponding to each spatial position pixel to obtain the normalized local neighbor relationship vectors corresponding to each spatial position pixel; Using the normalized local neighbor relationship vectors, perform feature weighting on corresponding spatial position pixels to obtain the weighted compressed features. 8. The method according to claim 7, wherein, before calculating the local neighbor relationship vector corresponding to the spatial position pixel for any spatial position pixel in the compressed feature of the feature weighting of the channel level, The method also includes: Through the local neighborhood selection network, a target local neighborhood relationship is selected from a variety of local neighborhood relationships for the spatial location pixel, and the local neighborhood relationship is used to define the relationship between the spatial location pixel and the neighborhood spatial location pixel. relationship between The compressed feature that satisfies the selected target local neighborhood relationship with the spatial position pixel is used as the neighborhood spatial position pixel of the spatial position pixel. 9. The method according to claim 7, wherein, for any one spatial position pixel in the compressed feature of the feature weighting of the completed channel level, calculating the local neighbor relationship vector corresponding to the spatial position pixel comprises: For the target spatial position pixel, respectively calculate the correlation between the target spatial position pixel and k neighborhood spatial position pixels to obtain k local neighbor relationship scalars of the target spatial position pixel, k is a positive integer; The k local neighbor relationship scalars of the target spatial position pixel are spliced to obtain a local neighbor relationship vector corresponding to the target spatial position pixel. 10. The method according to claim 7, characterized in that, using the normalized local neighbor relationship vector to perform feature weighting on corresponding spatial position pixels to obtain the weighted compressed features, comprising : For the target spatial position pixel, extract the vector of the neighborhood spatial position pixel of the target spatial position pixel in the channel dimension to form the local area feature of the target spatial position pixel; Performing feature weighting on the local neighbor relationship vector corresponding to the target spatial position pixel and the local area feature of the target spatial position pixel to obtain the local area weighted feature of the target spatial position pixel; The local weighted features of all spatial position pixels are concatenated to obtain the weighted compressed features. 11. The method according to claim 2, wherein, when the channel number of the compressed feature is m, and m is a positive integer, the compressed feature channel weight value vector is calculated based on the compressed feature, include: Splitting the compressed features at the channel level to obtain m channel feature maps; Calculate the statistics information of each channel feature map under various statistics, and obtain m channel statistics vectors; Based on the m channel statistic vectors, the compressed feature channel weight value vector is obtained by calculation. 12. The method according to claim 11, wherein the calculation of the compressed feature channel weight value vector based on the m channel statistic vectors comprises: Input the m channel statistic vectors into at least one fully connected layer respectively, and output the channel importance corresponding to the m channels respectively; The channel importances corresponding to the m channels are arranged according to the channels to obtain the compressed feature channel weight value vector. 13. The method according to claim 11, wherein the statistic comprises at least one of the following: Mean, variance, coefficient of variation, skewness, peak, maximum, minimum, median, and quartiles. 14. The method according to claim 2, wherein the calculation process of the compressed features comprises: The n branch features are added to obtain the fusion feature corresponding to the n branch features. 15. The method according to claim 2, wherein the calculation process of the compressed features comprises: The n branch features are connected based on the channel dimension to obtain fusion features corresponding to the n branch features. 16. A kind of attention model, is characterized in that, described attention model comprises: The input module is used to obtain n branch features extracted from the target picture under n branches, and n is a positive integer; The channel weight value calculation module is used to calculate the branch feature channel weight value vector corresponding to each branch, and each branch feature channel weight value vector is used to represent the channel importance corresponding to each channel of the corresponding branch feature; A feature weighting module, configured to use the branch feature channel weight value vector to perform channel-level feature weighting on the corresponding branch features to obtain n channel-weighted branch features; An output module, configured to fuse the n channel weighted branch features to obtain the output features of the target picture. 17. A first basic block structure, characterized in that the first basic block structure comprises: an attention model as claimed in claim 16, an addition module; The attention model is used to output the output features of the target picture when the input target picture is extracted under the branch features of n branches, and n is a positive integer; The adding module is configured to add the output feature of the target picture to the target picture to obtain an output result of the first basic block structure for the target picture. 18. A second basic block structure, characterized in that the second basic block structure comprises: an attention model as claimed in claim 16, an addition module; The attention model is used to output the output features of the target picture when the input target picture is extracted under the branch features of n branches, and n is a positive integer; The adding module is configured to add the output feature of the target picture and the result of batch normalization of the target picture to obtain the output result of the second basic block structure for the target picture. 19. A computer device, comprising: A memory and a processor, the memory and the processor are connected in communication with each other, computer instructions are stored in the memory, and the processor performs any one of claims 1 to 15 by executing the computer instructions The image processing method described above. 20. A computer-readable storage medium, wherein computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to make a computer perform the image processing according to any one of claims 1 to 15 method.

Images