JP2022058173A - Image classification method and apparatus - Google Patents

Abstract

To provide an image classification method and an apparatus for recognizing and classifying image data.SOLUTION: A method comprises a step of, by incorporating a residual mechanism into an attention model, combining contextual information within an attention mechanism without increasing a parameter and assisting the attention model to further accurately extract a feature of interest for an image classification task, inputting image data into a residual attention mechanism model to improve efficiency and accuracy of image classification when recognizing and classifying the image data.SELECTED DRAWING: Figure 1

Description

One or more embodiments of the present specification relate to the art of image recognition, in particular to image classification methods and equipment.

As the degree of informationization in society has increased, images have gradually replaced text and have become an important medium for the transmission and storage of information by humans. The disorder and enormous amount of information contained in images poses great challenges in the processing of image information. How to effectively classify images and extract useful information that we need has become a hot issue in the field of computer vision.

However, with the development of society, the amount of image data has increased exponentially, and the range of application of images has continued to expand. Far from meeting the requirement to classify chaotic image data perfectly and efficiently, the efficiency and accuracy of conventional image classification methods can be improved.

In view of this, an object of one or more embodiments herein is to propose image classification methods and devices for solving the problem of low efficiency and accuracy of image classification.

Based on the above object, one or more embodiments of the present specification are image classification methods.
Establishing a residual network model and replacing the standard convolution at the original edge of the residual network model with a perforated convolution to generate a perforated residual network backbone.
Generating the weight layer of the residual network model based on the channel attention module and the spatial attention module of the attention mechanism model,
To generate a residual attention mechanism model composed of the residual network main trunk with holes and the weight layer, and to train the residual attention mechanism model.
Provided is an image classification method including inputting image data into the residual attention mechanism model and recognizing and classifying the image data.

In some embodiments, generating the weight layer of the residual network model described above can be done.
A channel attention weight layer and a spatial attention weight layer are generated based on the channel attention module and the spatial attention module, and the channel attention weight layer and the spatial attention weight layer are sequentially arranged in series. Including that.

In some embodiments, matrix addition between the channel attention module and the residual network edge is performed to generate the channel attention weight layer.

In some embodiments, prior to matrix addition between the channel attention module and the residual network edge,
It further includes performing a deconvolution operation on the channel attention module.

In some embodiments, replacing the standard convolution at the original edge of the residual network model described above with a perforated convolution is possible.
It involves replacing the standard convolution at the original edge with the convolution layer of the series linear rectification activation function for perforated convolution series batch normalization.

Based on the same concept, one or more embodiments of the present specification are image classification devices.
A trunk module that establishes a residual network model and replaces the standard convolution at the original edge of the residual network model with a perforated convolution to generate a perforated residual network backbone.
A weight module that generates a weight layer of the residual network model based on the channel attention module and the spatial attention module of the attention mechanism model, and
A generation module that generates a residual attention mechanism model composed of the residual network main trunk with holes and the weight layer, and trains the residual attention mechanism model.
Further provided is an image classification device including a classification module for inputting image data into the residual attention mechanism model and recognizing and classifying the image data.

In some embodiments, the weighting module produces a weighting layer for the residual network model.
A channel attention weight layer and a spatial attention weight layer are generated based on the channel attention module and the spatial attention module, and the channel attention weight layer and the spatial attention weight layer are sequentially arranged in series. Including that.

In some embodiments, the weight module performs matrix addition between the channel attention module and the residual network edge to generate the channel attention weight layer.

In some embodiments, the weighting module is prior to performing matrix addition between the channel attention module and the residual network edge.
It further includes performing a deconvolution operation on the channel attention module.

In some embodiments, the trunk module replaces the standard convolution at the original edge of the residual network model with a perforated convolution.
It involves replacing the standard convolution at the original edge with the convolution layer of the series linear rectification activation function for perforated convolution series batch normalization.

As can be seen from the above description, the image classification method and device according to one or more embodiments of the present specification establishes a residual network model and perforates the standard convolution at the original edge of the residual network model. To generate the core of the residual network with holes, and to generate the weight layer of the residual network model based on the channel attention module and the spatial attention module of the attention mechanism model, and the hole. Generate a residual attention mechanism model composed of the residual network main trunk and the weight layer, train the residual attention mechanism model, and input the image data into the residual attention mechanism model. , Includes recognizing and classifying the image data. One or more embodiments of the specification incorporate a residual mechanism into the attention model, combining contextual information within the attention mechanism without increasing parameters, and more features of interest to the image classification task. By assisting the attention model to be extracted accurately, we improved the efficiency and accuracy of image classification. The training time of the attention mechanism model improved by this technical proposal is shortened to about half of the original time, and the training efficiency is greatly improved.

In order to more clearly explain one or more embodiments or technical proposals in the prior art of the present specification, the drawings that need to be used in the description of the examples or the prior art will be briefly introduced below, but it is clear. Indeed, the drawings described below are merely one or more embodiments of the specification, and those skilled in the art may obtain other drawings from these drawings without paying creative labor. can.
FIG. 1 is a schematic flow diagram of an image classification method according to one or more embodiments of the present specification. FIG. 2 is a schematic diagram of the application principle of the residual network model according to one or more embodiments of the present specification. FIG. 3 is a schematic diagram of a residual block of a perforated residual network main trunk according to one or more embodiments of the present specification. FIG. 4 is a structural schematic diagram of an attention mechanism model according to one or more embodiments of the present specification. FIG. 5 is a schematic structural diagram of a residual attention mechanism model (Dirated-CBAM) according to one or more embodiments of the present specification. FIG. 6 is a schematic structural diagram of the residual channel attention module according to one or more embodiments of the present specification. FIG. 7 is a schematic structural diagram of another residual channel attention module according to one or more embodiments herein. FIG. 8 is a schematic structural diagram of an image classification device according to one or more embodiments of the present specification.

In order to further clarify the purposes, technical proposals and advantages of the present specification, the present specification will be described in more detail with reference to the drawings together with specific examples below.

It should be explained that, unless otherwise defined, the technical or scientific terms used in the examples herein have the usual meanings understood by one of ordinary skill in the art. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or materiality, but merely to distinguish between different components. Similar terms such as "include" or "include" include elements, members or method steps in which the elements, members or method steps described prior to the term include the elements, members or method steps listed after the term, and their equivalents. However, it does not exclude other elements, members or method steps. Similar terms such as "connect" and "connect" are not limited to physical or mechanical connections and may include electrical connections, either directly or indirectly. "Upper", "lower", "left", "right", etc. only indicate relative positional relationships, and if the absolute position to be explained changes, the relative positional relationships may also change accordingly. ..

As described in the Background Technology section, image segmentation is specifically a computer's determination of image types using input data with the assistance of relevant algorithms, a research target detection task. It has relatively high academic research and scientific and technological application value as an important basis for image segmentation tasks, etc., and most of the research work in the field of computer vision is related to image segmentation tasks. With the leap of deep learning, image classification technology has been significantly improved at both the hardware and software levels, reaching levels that exceed the human eye's ability to discriminate images in many existing big datasets. More and more researchers are beginning to focus on research in the field of image classification and related computer vision.

As a popular research direction in computer vision, image object classification is video intelligent analysis in the security defense field, pedestrian detection, face recognition, retrograde detection in the traffic monitoring field, vehicle counting, traffic scene object recognition, number plate detection and recognition. , Object recognition count in the field of logistics management statistics, product recognition classification, product quality evaluation, and image search based on picture content in the field of album intelligent analysis, automatic album clustering, human body image detection, object image detection, etc. Widely applied in the field.

However, as the amount of image data increases and the range of applications continues to expand, existing network structures and algorithms meet the requirement to classify different types, different properties and disordered image data perfectly and efficiently. Far from it. Therefore, researchers need to further study and improve the convolutional neural network architecture in order to improve the efficiency and accuracy of image classification.

In view of the above situation, we incorporated the residual mechanism into the attention model, used the residual edge in the attention model to perform the equivalent mapping in the attention module, and inside the attention mechanism without increasing the parameters. We have improved the efficiency and accuracy of image classification by combining the contextual information of the image classification task and assisting the attention model to more accurately extract features of interest to the image classification task. The training time of the attention mechanism model improved by this technical proposal is shortened to about half of the original time, and the training efficiency is greatly improved.

With reference to FIG. 1, which is a schematic flow diagram of an image classification method according to an embodiment of the present specification, the image classification method specifically includes the following steps 101 to 104, as shown in FIG.

Step 101 is to establish a residual network model and replace the standard convolution at the original edge of the residual network model with a perforated convolution to generate a perforated residual network backbone.

になり、即ち、元エッジ（図中のエッジ）に対しては、残差エッジに入力された特徴図内の情報が精緻化されて残差ネットワーク効果が最適化されるように、標準畳み込みの畳み込み層を構築することになる。ここで、標準畳み込みとは、一般的な畳み込みであり、数学上で通俗的に言えば、入力行列と畳み込みカーネル（畳み込みカーネも行列である）とが、対応する要素を乗算して合計を求めたものであるため、１回の畳み込みの結果出力は１つの数値となり、最後に入力された入力行列全体を遍歴すると、最終的に１つの結果行列が得られる。一般的な畳み込みの２次元畳み込みカーネルとしては、３＊３の畳み込みカーネルが最もよくあるが、ネットワークの設計に応じて、５＊５又は７＊７のものを設計してもよい。

That is, for the original edge (edge in the figure), the standard convolution is performed so that the information in the feature diagram input to the residual edge is refined and the residual network effect is optimized. You will build a convolutional layer. Here, the standard convolution is a general convolution, and mathematically speaking, the input matrix and the convolution kernel (the convolution carne is also a matrix) multiply the corresponding elements to obtain the sum. Therefore, the output of the result of one convolution becomes one numerical value, and when the entire input matrix input at the end is iterated, one result matrix is finally obtained. As a general convolutional two-dimensional convolution kernel, a 3 * 3 convolution kernel is most common, but a 5 * 5 or 7 * 7 convolution kernel may be designed depending on the network design.

Next, perforated convolution is a extended convolution, also known as an inflated convolution, which increases the model's receptive field by injecting holes into a standard convolution kernel. Compared to general convolution, extended convolution increases the expansion factor parameter, which refers to the number of intervals between points in the convolution kernel. Assuming the expansion ratio is set within a typical convolution, the expansion ratio value is 1, indicating that the points of the convolution kernel are adjacent, but in a perforated convolution, the expansion ratio is For example, if the expansion ratio is 2 instead of 1, it indicates that the points of the convolution kernel are separated by 1 pixel, that is, the 3 * 3 convolution kernel with a hole convolution having an expansion ratio of 2 and the standard convolution. It has the same receptive field as the 5 * 5 convolution kernel of. In the case of the residual network, the contour information of the image is generally extracted by the feature diagram of the input image obtained in the previous period, but in the case of the perforated convolution, the receptive field brought about by it is expanded. According to the characteristic, it is possible to more preferably screen useful information of the initial feature diagram, and features of the image contour extracted at the early stage, the characteristic diagram of the edge, and the detailed image information extracted at the later stage. Since the images can be combined and the image information can be more preferably collected and integrated, the effect of network image classification is improved.

Step 102 is to generate the weight layer of the residual network model based on the channel attention module and the spatial attention module of the attention mechanism model.

The purpose of this step is to set the channel attention module and the spatial attention module in the attention mechanism model as the weight layer of the residual network model. Here, the attention mechanism model (CBAM, Structural Block Attention Module) is an attention mechanism module in which a space (spatial) and a channel (channel) are combined as shown in FIG. 4, and the "attention mechanism module" in the circle is used. “X” represents a dot product operation for each element (element-wise) of the matrix. In this specific embodiment, the feature diagram obtained by performing the maximum pooling operation and the average pooling operation on the channel using the channel attention mechanism in the CBAM model is input to the multi-layer perceptron (shard MLP). , The element-by-element addition operation can be used for the two obtained feature diagrams, and the feature diagram output from the convolution layer can be non-linearized by the sigmoid activation function to expand the ability to express channel attention. Therefore, a more effective channel weight can be obtained.

In a specific embodiment, since it is based on the channel attention module and the spatial attention module, two weight layers are generated and set in the perforated residual network main trunk. As the setting method, the two modules may be taken out as a weight layer as they are, or the weight layer may be formed by further adjusting the modules after taking out the modules. For example, for a channel attention module. Alternatively, the corresponding weight layer may be generated by the matrix addition process of the residual edge of the standard convolution and the channel attention module, or the matrix addition with the residual edge with holes similar to the previous step is performed. By doing so, the corresponding weight layer may be generated. Next, regarding the channel attention weight layer, the two generated weight layers may be in a parallel relationship or a serial relationship. In the serial relationship, the preceding weight layer may be a channel attention weight layer or a spatial attention weight layer.

Step 103 is to generate a residual attention mechanism model composed of the holed residual network main trunk and the weight layer, and train the residual attention mechanism model.

The purpose of this step is to connect the generated main trunk and the weight layer to generate and train a residual attention mechanism model. Here, as shown in FIG. 5, which is a structural schematic diagram of the residual attention mechanism model (Dilated-CBAM) in a specific embodiment, “x” in the circle indicates a dot product operation for each element of the matrix. , And "+" in the circle represents the operation of each element of matrix addition. Then, model training for image classification is performed on the Directed-CBAM model.

In a specific application scene, the image classification effect of the Laid-CBAM model is verified, and the Cifar-10 data set (one of the image data sets, which is a labeled data set like the CIFAR-100, and more. Cifar-10 data, as shown in Table 1, to train a training set (data set derived from a large scale of 80 million small pictures) to utilize an optimized Laid-CBAM model. The test set of the set verified the accuracy and convergence ability of the classification of image data of the same nature by the network and weight obtained by training. Here, Train ac represents the success rate of classification by model in the Cifar-10 dataset training set, Test acc represents the success rate of classification by model in the test set of Cifar-10 dataset, and EPOCH is. , Representing a model period or period, the process from one complete dataset passing through the neural network once to returning once is called a single epoch. Among them, the models are, in sequence, an 18-layer residual network model (ResNet-18), an existing CBAM model, a CBAM model embedded holed convolutional experimental model, and a channel attention module in the CBAM model. The Directed-CBAM model framework, which is the original channel attention module, and the Laid-CBAM model frame, which is the residual channel attention module in which the channel attention module combines the channel attention module and the residual network edge. The work and the channel attention module are the Dilated-CBAM model framework, which is a holed residual channel attention module that combines the channel attention module and the holed residual network edge, and the holed residual network main trunk. The Laid-CBAM model framework in which the perforated convolution in is replaced by a group conv, the Laid-CBAM model framework in which the ELU activation function is embedded, and the Laid-CBAM in which the SELU activation function is embedded. It becomes a model framework. For the Laid-CBAM model, if the channel attention module in it is a residual channel attention module that combines the channel attention module and the residual network edge (ie, the data in row 5 of the table). ), The accuracy rate of classification in the training set has reached 98.7%, the accuracy rate of classification in the test set has reached 93.5%, while the convergence speed is only 10 cycles. I understand.

Step 104 is to input the image data into the residual attention mechanism model and recognize and classify the image data.

The purpose of this step is to input the image to be recognized into the trained residual attention mechanism model and to recognize and classify the image by the residual attention mechanism model. Here, the image data may be obtained by an external device such as a video camera or a camera, may be obtained by the user via an external network, and may be obtained by the system or the server itself. It may be the one stored in the database of.

Regarding the recognized classification result, the recognition classification result may be processed in the form of storage, presentation or reprocessing, and the classification result here is a type to which a single image specifically belongs, or a plurality of images. It may be the result of classification processing between images. It is possible to flexibly select the output method of the recognition classification result specifically according to various application scenes and implementation needs.

For example, in the case of an application scene in which the method according to this embodiment is executed on a single device, the recognition classification result is output as it is so as to be displayed on the display component (display, projector, etc.) of the current device. Currently, it is possible for the operator of the device to directly visually recognize the contents of the recognition classification result from the display component.

As another example, in the case of an application scene in which the method according to this embodiment is executed on a system composed of a plurality of devices, the recognition classification result can be input to any data communication method (wired connection, NFC, Bluetooth (registered). It is possible to transmit to a predetermined device as another receiving side in the system via (trademark), wifi, cellular mobile network, etc., and allow the predetermined device that has received the recognition classification result to perform subsequent processing on it. Is. Optionally, the predetermined device may be a predetermined server, and the server is usually set in the cloud, and can store and distribute the recognition classification result as a data processing and storage center. Here, the receiving side of the distribution is a terminal device, and the owner or operator of the terminal device is the current user, the mechanism or individual who owns the image, the organization related to the presentation of the image, the individual, the website, etc. Can be.

As a further example, in the case of an application scene in which the method according to this embodiment is executed on a system composed of a plurality of devices, the recognition classification result is directly transferred to a predetermined terminal device via an arbitrary data communication method. It is possible to transmit and the terminal device may be one or more listed in the previous paragraph.

The image classification method provided by applying one or more embodiments herein establishes a residual network model and replaces the standard convolution at the original edge of the residual network model with a perforated convolution. , Generating a holed residual network trunk, generating a weight layer of the residual network model based on the channel attention module and spatial attention module of the attention mechanism model, and the holed residual. The residual attention mechanism model composed of the network main trunk and the weight layer is generated, the residual attention mechanism model is trained, and the image data is input to the residual attention mechanism model, and the image is described. Includes recognizing and classifying data. One or more embodiments of the specification incorporate a residual mechanism into the attention model, combining contextual information within the attention mechanism without increasing parameters, and more features of interest to the image classification task. By assisting the attention model to be extracted accurately, we improved the efficiency and accuracy of image classification. The training time of the attention mechanism model improved by this technical proposal is shortened to about half of the original time, and the training efficiency is greatly improved.

It should be explained that the method according to one or more embodiments herein may be performed by a single device, such as a single computer or server. The method according to this embodiment may be applied to a distributed scene and completed by collaboration between a plurality of devices. In the case of such a distributed scene, of these plurality of devices, one device performs only one or more steps within the method according to one or more embodiments herein. It may be performed and the plurality of devices interact with each other to complete the above method.

Although the specific embodiments of the present specification have been described above, other embodiments are also included in the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than in the examples, and the desired result can be achieved. Also, the procedures described in the drawings do not necessarily require the particular order or sequence shown to obtain the desired result. In some implementations, multitasking and concurrency may also be possible or advantageous.

In a selective embodiment of the present specification, in order to maximize the effect of image recognition, it is possible to generate the weight layer of the residual network model described above.
A channel attention weight layer and a spatial attention weight layer are generated based on the channel attention module and the spatial attention module, and the channel attention weight layer and the spatial attention weight layer are sequentially arranged in series. Including that.

Here, arranging sequentially in series means an arrangement method of the channel attention module and the spatial attention module in the structural schematic diagram shown in FIG. In a specific application scene, the feature diagram is first processed by the channel attention module, the processing result is input to the spatial attention module and processed, and then the matrix addition of the output result and the residual edge is performed. As for the arrangement method of the channel attention module and the spatial attention module, the spatial attention module may be serialized so as to precede the channel attention module, or both modules may be arranged in parallel. You may.

In a selective embodiment of the present specification, in order to more accurately weight the region of the image by having the receptive field extract multi-scale contextual information, the channel attention module and the residual network edge are used. The matrix addition is performed to generate the channel attention weight layer, and the residual network edge here is the residual edge in the current existing residual network.

As shown in FIG. 6, in a specific embodiment, the basic channel attention module of the Laid-CBAM model mimics the CBAM model and extracts the global features of the channel through average pooling and maximum pooling, and the resulting features. Each figure is input to the multi-layer perceptron, the relationship between different channels is calculated, the channel weight matrix is output, and then the matrix addition operation of the residual edge channel weight matrix in the residual network model is performed. In FIG. 6, the “+” in the circle represents an operation for each element of matrix addition, and the “S” -shaped curve in the circle represents an activation function such as a sigmoid.

In a specific application scene, the image is stored and calculated in the form of a numerical matrix so that one channel corresponds to one matrix and the spatial attention module has an effect on the matrix corresponding to each channel. I have to. From a mathematical point of view, the spatial attention module does not apply the residual mechanism to the Laid-CBAM model because there is no concatenation problem of contextual information in the same matrix. That is, as the spatial attention module in the Laid-CBAM model, the spatial attention module in the current CBAM model is used as it is.

In a selective embodiment of the present specification, the feature diagram at the residual edge and the channel attention module are output by integrating the changes in image size during image feature extraction and re-magnifying the image size. Before performing the matrix addition between the channel attention module and the residual network edge, the channel attention module is performed so that the operation for each element of the matrix addition with the feature diagram can be performed more flexibly. Further includes performing a deconvolution operation on the object.

FIG. 7 shows FIG. 6 with a deconvolution operation added. The single circle in the figure represents a deconvolution operation. In this specific application scene, a deconvolution operation is performed in order to better adapt the matrix that requires addition and improve the accuracy, but in other application scenes, it is not always necessary to perform deconvolution. not.

In a selective embodiment of the present specification, the standard convolution at the original edge of the residual network model described above is replaced with a perforated convolution in order to extract the contour of the image more accurately and to increase the calculation speed and the convergence speed. That is
It involves replacing the standard convolution at the original edge with the convolution layer of the series linear rectification activation function for perforated convolution series batch normalization.

Based on the same concept, one or more embodiments herein are as shown in FIG.
A trunk module 801 that establishes a residual network model and replaces the standard convolution at the original edge of the residual network model with a perforated convolution to generate a perforated residual network backbone.
A weight module 802 that generates a weight layer of the residual network model based on the channel attention module and the spatial attention module of the attention mechanism model, and
A generation module 803 that generates a residual attention mechanism model composed of the residual network main trunk with holes and the weight layer, and trains the residual attention mechanism model.
Further provided are image classification devices including a classification module 804 that inputs image data into the residual attention mechanism model and recognizes and classifies the image data.

As one optional embodiment, the weighting module 802 may generate a weighting layer for the residual network model.
A channel attention weight layer and a spatial attention weight layer are generated based on the channel attention module and the spatial attention module, and the channel attention weight layer and the spatial attention weight layer are sequentially arranged in series. Including that.

As one selective embodiment, the weight module 802 performs matrix addition between the channel attention module and the residual network edge to generate the channel attention weight layer.

As one selective embodiment, before the weight module 802 performs matrix addition between the channel attention module and the residual network edge,
It further includes performing a deconvolution operation on the channel attention module.

As one optional embodiment, the trunk module 801 replaces the standard convolution at the original edge of the residual network model with a perforated convolution.
It involves replacing the standard convolution at the original edge with the convolution layer of the series linear rectification activation function for perforated convolution series batch normalization.

For convenience of explanation, the functions are described individually by dividing them into various modules when the above devices are described. Of course, when implementing one or more embodiments herein, the functionality of each module may be implemented in the same software and / or hardware.

The equipment according to the above embodiment is for realizing the corresponding method in the above-mentioned embodiment, and has a beneficial effect of the embodiment of the corresponding method, but will not be described repeatedly here.

Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and means that the scope of the present disclosure (including claims) is limited to these examples. is not. Based on the ideas of the present disclosure, the technical features of the above embodiments or different embodiments may be combined with each other, the steps may be realized in any order, and the present specification as described above. Although one or more examples of the book also have many other variants in various embodiments, they are not described in detail for the sake of brevity.

Further, in order to simplify the description and discussion and not obscure one or more embodiments herein, the drawings provided are publicly known with integrated circuit (IC) chips and other components. Power / ground connection may or may not be shown. It should be noted that, in order to avoid obscuring one or more embodiments of the specification, the device may be shown in the form of a block diagram, which also takes into account the following facts: That is, the details of the embodiments of the equipment according to these block diagrams are highly dependent on the platform on which one or more embodiments of the present specification are to be implemented (ie, these details are included). , Should be entirely within the understanding of those skilled in the art). Where specific details (eg, circuits) are provided for illustration purposes of the exemplary embodiments of the present disclosure, it will be apparent to those of skill in the art that these specific details are absent or these specifics. One or more embodiments of the present specification can be implemented even if the details change. Therefore, these statements should be considered descriptive rather than restrictive.

Although the present disclosure has been described based on specific embodiments of the present disclosure, many of the substitutions, modifications and variations of these embodiments will be apparent to those skilled in the art according to the above description. For example, other memory architectures (eg, dynamic RAM (DRAM)) are also applicable to the examples discussed.

One or more embodiments of the specification are intended to cover all of these substitutions, modifications and modifications contained within the broad scope of the appended claims. Accordingly, any omissions, amendments, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be within the scope of this disclosure.

Claims (4)

It is an image classification method
Establishing a residual network model and replacing the standard convolution at the original edge of the residual network model with a perforated convolution to generate a perforated residual network backbone.
Generating the weight layer of the residual network model based on the channel attention module and the spatial attention module of the attention mechanism model,
To generate a residual attention mechanism model composed of the residual network main trunk with holes and the weight layer, and to train the residual attention mechanism model.
It includes inputting image data into the residual attention mechanism model and recognizing and classifying the image data.
Replacing the standard convolution at the original edge of the residual network model described above with a perforated convolution
It involves replacing the standard convolution at the original edge with the convolution layer of the series linear rectification activation function for perforated convolution series batch normalization.
Generating the weight layer of the residual network model described above
A channel attention weight layer and a spatial attention weight layer are generated based on the channel attention module and the spatial attention module, and the channel attention weight layer and the spatial attention weight layer are sequentially arranged in series. Including that
An image classification method comprising performing matrix addition between the channel attention module and a residual network edge to generate the channel attention weight layer. Before performing the matrix addition between the channel attention module and the residual network edge,
The method of claim 1, further comprising performing a deconvolution operation on the channel attention module. It is an image classification device
A trunk module that establishes a residual network model and replaces the standard convolution at the original edge of the residual network model with a perforated convolution to generate a perforated residual network backbone.
A weight module that generates a weight layer of the residual network model based on the channel attention module and the spatial attention module of the attention mechanism model, and
A generation module that generates a residual attention mechanism model composed of the residual network main trunk with holes and the weight layer, and trains the residual attention mechanism model.
Includes a classification module that inputs image data into the residual attention mechanism model and recognizes and classifies the image data.
It is possible that the trunk module replaces the standard convolution at the original edge of the residual network model with a perforated convolution.
It involves replacing the standard convolution at the original edge with the convolution layer of the series linear rectification activation function for perforated convolution series batch normalization.
It is possible that the weight module produces a weight layer for the residual network model.
A channel attention weight layer and a spatial attention weight layer are generated based on the channel attention module and the spatial attention module, and the channel attention weight layer and the spatial attention weight layer are sequentially arranged in series. Including that
The weight module is an image classification device characterized in that a matrix addition between the channel attention module and a residual network edge is performed to generate the channel attention weight layer. Before the weight module performs matrix addition between the channel attention module and the residual network edge,
The device according to claim 3, further comprising performing a deconvolution operation on the channel attention module.

Images