CN114972834B - Image classification method and device of multi-level multi-classifier - Google Patents

Abstract

The present application discloses a multi-level multi-classifier image classification method and device. The method includes: obtaining a feature extraction classifier to perform pre-classification and feature extraction on the target image, and obtain the feature of each convolutional layer in each category. The first feature probability of the map and the feature map; based on the first feature probability, the feature maps of each convolutional layer in the various categories are probability accumulated to obtain each convolution in the various categories The first heat map corresponding to the layer; the first heat map is processed into a uniform size, and spliced into the first multi-level heat map of each category; the first multi-level heat map is input to the multi-level classifier to obtain The classification probability of each layer category output by the multi-level classifier; output the category of the target image according to the quantitative relationship between the maximum probability of the classification probability of the bottom layer and the preset threshold.

Description

Image classification method and device of multi-level multi-classifier

technical field

The present application relates to the field of image classification, in particular to a multi-level and multi-classifier image classification method and device.

Background technique

With the advancement and development of science and technology, it has become a common way to use neural networks for image recognition or image classification. In the existing image classification methods, it is generally to use various categories of classified images to train a neural network. When predicting, input the picture to be predicted into the neural network, and the neural network will directly output the probability of each category, and select the category with the highest probability as the predicted output.

However, when the probabilities of all categories output by the neural network are relatively low, directly selecting the category with the highest probability as the output will lead to classification errors, and how to accurately output the category of the image to be predicted has become a problem that needs to be solved now.

Contents of the invention

The present application discloses a multi-level multi-classifier image classification method and device, in order to solve the problem of high misidentification caused by the current image classification method when the probabilities of all categories are low.

In order to solve the above problems, the application adopts the following technical solutions:

In the first aspect, the embodiment of the present application discloses an image classification method with multi-level and multi-classifiers. The method includes: obtaining a feature extraction classifier to perform pre-classification and feature extraction on the target image, and each convolution in each category obtained The feature map of the layer and the first feature probability of the feature map; based on the first feature probability, the feature maps of each convolutional layer in the various categories are probabilistically accumulated to obtain each of the categories. The first heat map corresponding to each convolutional layer; the first heat map is processed into a uniform size, and spliced into the first multi-level heat map of each category; the first multi-level heat map is input to the multi-level classification device to obtain the classification probability of each layer category output by the multi-level classifier; according to the classification of the bottom layer

The quantitative relationship between the maximum probability in the class probability and the preset threshold, and output the class of the target image.

In the second aspect, the embodiment of the present application discloses a multi-level multi-classifier image classification device, the device includes: an acquisition module, which is used to obtain the pre-classification and feature extraction of the target image by the feature extraction classifier. The feature map of each convolutional layer in the feature map and the first feature probability of the feature map; the accumulation module is used to calculate the probability of the feature map of each convolution layer in each category based on the first feature probability Accumulate to obtain the first thermal map corresponding to each convolutional layer in each category; the splicing module is used to process the first thermal map into a uniform size and splicing it into the first multi-level thermal map of each category ; The input module is used to input the first multi-level heat map into the multi-level classifier to obtain the classification probability of each layer category output by the multi-level classifier; the output module is used for according to the classification probability of the bottom layer The quantitative relationship between the maximum probability of , and the preset threshold, outputs the category of the target image.

The embodiment of the present application discloses that the technical solution adopted by the present application can achieve the following beneficial effects:

The embodiment of the present application discloses a multi-level and multi-classifier image classification method. The target image is pre-classified and feature extracted by the feature extraction classifier, and the feature extraction result is processed into the first heat map, and the first heat map is stitched together. After the first multi-level heat map is obtained, the first multi-level heat map is input into the multi-level multi-classifier, and the target image is subdivided by the multi-level multi-classifier to obtain the classification probability of each layer category, and according to the classification of the bottom layer The relationship between the maximum probability in the probability and the preset threshold value is used to output the category of the target image. Through this multi-level and multi-classification method, while improving the accuracy of the target image classification, it can better solve the problem of the current image classification method. A problem with low probability of all classes leading to high false misses.

Description of drawings

FIG. 1 is a schematic flow diagram of a multi-level multi-classifier image classification method disclosed in an embodiment of the present application;

FIG. 2 is a schematic flow diagram of another multi-level multi-classifier image classification method disclosed in the embodiment of the present application;

3 is a schematic diagram of a feature extraction classifier and a multi-level multi-classifier pre-training process disclosed in the embodiment of the present application;

Fig. 4 is a schematic structural diagram of a multi-level multi-classifier image classification device disclosed in an embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

A multi-level and multi-classifier image classification method and device provided in the embodiments of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

FIG. 1 is a schematic flowchart of a multi-level and multi-classifier image classification method provided by an embodiment of the present application. The method can be executed by an electronic device. In other words, the method can be executed by software or hardware installed in the electronic device. As shown in Figure 1, a multi-level multi-classifier image classification method disclosed in the embodiment of the present application may include the following steps:

S110: Acquiring the feature extraction classifier to perform pre-classification and feature extraction on the target image, and obtain the feature map of each convolutional layer in each category and the first feature probability of the feature map.

After the target image is input into the feature extraction classifier, the convolutional layer in the feature extraction classifier extracts the features of the target image. After extracting the features, the pooling layer compresses the features of the target image, and then extracts the main features of the target image. Finally, the fully connected layer connects all the features, and sends the output value to the classifier, and the classifier performs classification, so that the output layer obtains the feature map corresponding to each convolution layer in each category of the target image and the first feature map of the feature map. Feature probability, there are many types of classifiers, such as softmax classifiers. Specifically, through the softmax function, the output value of the fully connected layer can be converted into a probability distribution in the range [0, 1] and the sum is 1, that is, the first A characteristic probability.

It should be noted that the feature extraction classifier is equivalent to a convolutional neural network, which includes: input layer, convolutional layer, pooling layer, fully connected layer and output layer.

S120: Based on the first feature probability, perform probability accumulation of feature maps of each convolutional layer in each category to obtain a first heat map corresponding to each convolutional layer in each category.

S130: Processing the first heat map into a uniform size, and splicing it into the first multi-level heat map of each category.

By inputting the target image into the feature extraction classifier, the weight corresponding to each feature map in multiple convolutional layers (that is, the first feature probability) can be obtained, and the weights of the feature maps corresponding to each convolutional layer can be accumulated to obtain the second A heat map, processing the first heat map into a uniform size, in an achievable way, multiple first heat maps can be pooled using ROIPooling (Region of Interest Pooling, interest area pooling) The graphs are processed to a uniform size, so that the first heat map can be spliced into the first multi-level heat map of each category.

S140: Input the first multi-level heat map into the multi-level classifier, and obtain the classification probability of each class output by the multi-level classifier.

S150: Output the category of the target image according to the quantitative relationship between the maximum probability of the classification probabilities at the bottom layer and a preset threshold.

After inputting the combined first multi-level heat map into the multi-level classifier, the multi-level classifier has multiple layers of classifiers, each layer of classifier is equivalent to a convolutional neural network, and the first multi-level heat map passes through each layer The classification probability of each layer category of the target image can be obtained through the processing of the classifier, and the category with the highest probability can be output according to the relationship between the maximum probability of the classification probability at the bottom layer and the number of preset thresholds. Through the method of multi-level and multi-classification, it is possible to avoid the problem of wrong and missed recognition caused when the probability of obtaining the sub-classification of the target image is low.

The embodiment of the present application discloses a multi-level and multi-classifier image classification method. The target image is pre-classified and feature extracted by the feature extraction classifier, and the feature extraction result is processed into the first heat map, and the first heat map is stitched together. After the first multi-level heat map is obtained, the first multi-level heat map is input into the multi-level multi-classifier, and the target image is subdivided by the multi-level multi-classifier to obtain the classification probability of each layer category, and according to the classification of the bottom layer The relationship between the maximum probability in the probability and the preset threshold value is used to output the category of the target image. Through this multi-level and multi-classification method, while improving the accuracy of the target image classification, it can better solve the problem of the current image classification method. A problem with low probability of all classes leading to high false misses.

In an achievable manner, the first multi-level heat map is input to the multi-level classifier, and the classification probability of each class output by the multi-level classifier is obtained, including:

The first multi-level heat map is input to each layer classifier in the multi-layer classifier to obtain the first classification probability of each layer category.

Based on the first feature probability, the first classification probability and the first feature probability of each layer category are probabilistically accumulated to obtain the second classification probability.

The second classification probability is normalized to obtain the classification probability of each layer category.

After the multi-level classifier receives the first multi-level heat map, the probability of each category of the current level can be obtained, that is, the first classification probability, based on the first feature probability obtained by the pre-classification operation of the target image by the feature extraction classifier , the probability of the first classification probability and the first feature probability of each category of the current level are accumulated to obtain the probability of the target image at the current level, that is, the second classification probability, and then by normalizing the second classification probability, it can be Get the classification probability of each layer class output by the multi-level classifier.

After obtaining the classification probability of each layer category output by the multi-level multi-classifier, the category of the target image can be judged and output according to the quantitative relationship between the maximum probability of the bottom-level classification probability and the preset threshold, a method that can be realized In the way, according to the quantitative relationship between the maximum probability of the lowest classification probability and the preset threshold, the category of the target image is output, including:

When the maximum probability is greater than the preset threshold, the category corresponding to the maximum probability is output as the category of the target image.

When the maximum probability is not greater than the preset threshold, judge whether the current level is the highest level, if so, output the category corresponding to the maximum probability as the category of the target image; otherwise, combine the classification output by the previous classifier Probability, recalculate the third classification probability of each category in the bottom layer, wherein, when the maximum probability in the third classification probability is greater than the preset threshold, output the category corresponding to the maximum probability in the third classification probability as the target image category.

According to the classification probabilities of each category in each layer obtained by the multi-level classifier, we can first judge from the bottom layer, select the maximum probability in each category, and judge whether the maximum probability is greater than the preset threshold. If so, it means that the classifier has relatively high The category is judged with high confidence, and the category with the highest probability can be directly selected as the output. Otherwise, judge based on the classification probabilities output by the previous layer, recalculate the probability of each category at the bottom layer (that is, the third classification probability), and output the third classification probability when the maximum probability of the third classification probability is greater than the preset threshold The category corresponding to the maximum probability in the classification probability is used as the category of the target image.

Fig. 2 is a schematic flow chart of another image classification method with multi-level and multi-classifiers. As shown in Fig. 2, the embodiment of the present application discloses another image classification method with multi-level and multi-classifiers. The method includes the following steps:

S210: Input the target image into the feature extraction classifier for feature extraction and category judgment, and obtain the probability p _i (i=1Λk ₁ ) of each category, where k ₁ is the number of categories of the feature extraction classifier.

S220: Calculating heat maps of multiple convolutional layers corresponding to each category.

S230: Using ROI Pooling to process multiple heat maps into a uniform size, and combine them into a multi-level heat map.

S240: Input the multi-level heat map of each category into each layer classifier in the multi-level classifier, and obtain the probability p _ij (i=1Λk ₁ , j=1Λk ₂ ) of each category at the current level, where k ₂ is the current The number of categories in the hierarchy.

S250: Accumulate the probability of each category of the current level to obtain

S260: Perform normalization processing on p _j to obtain the final classification probability p' _j ,

S270: According to the classification probability of each layer, select the maximum probability p among the classification probabilities of the bottom layer.

S280: Determine whether the maximum probability p is greater than a preset threshold h;

If yes, directly select the category with the largest probability as the output;

Otherwise, judge based on the classification probabilities output by the previous layer, and recalculate the probability p _m of each category at the bottom layer, p _m =p _n p _mn , where p _mn is the probability of each category at the bottom layer at the moment, and p _n is the probability of each category at the bottom layer The category probability of the classification, p _m is the recalculated probability of each category at the bottom layer, and p _m is normalized to obtain

S290: Determine whether the current level is the highest level, and if so, directly output the probability of the highest category.

For example, the bottom-level classification can be cat (0.4), dog (0.1), flower (0.4), grass (0.1), and the preset threshold is 0.5, which cannot be distinguished at this time, while the upper-level classification is animal (0.9), plant (0.1 ), then according to p _m =p _n p _mn recalculate to get cat (0.9*0.4=0.36), dog (0.9*0.1=0.09), flower (0.1*0.4=0.04), grass (0.1*0.1=0.01), according to After normalization, cat (0.72), dog (0.18), flower (0.08), and grass (0.02) are obtained. At this time, if the probability of cat is greater than the preset threshold of 0.5, the output category is cat.

In order to better ensure the accuracy of the prediction, in an achievable way, the target image is pre-classified and feature extracted by the feature extraction classifier, and the feature map and feature map of each convolutional layer in each category are obtained. Before the first feature probabilities of the graph, the method also includes:

Pretrained feature extraction classifiers and multi-level classifiers.

Through the pre-training of the feature extraction classifier and multi-level classifier, it can output more accurate categories when actually predicting the target image.

In an achievable manner, the pre-training feature extraction classifier and multi-level classifier may include the following steps:

The training images of the target category are input to the convolutional neural network in the feature extraction classifier, and the feature vectors of the feature maps of each layer of the convolutional neural network are output.

The feature vectors of the last n convolutional neural network layers are selected for global average pooling, and the pooled feature vectors are spliced into the first feature vector, where n is an integer greater than 1.

Fully connect the first feature vector to the output layer and perform softmax classification.

By concatenating the feature vectors of the last few layers of convolutional neural network layers into the first feature vector, the feature extraction classifier can be pre-trained based on the feature vectors of the last few layers of convolutional neural networks, and when the pre-training is completed , to perform feature extraction and pre-classification on the training images.

Based on the pre-trained feature extraction classifier, feature extraction is performed on the training image to obtain the feature map of each convolutional neural network layer and the second feature probability of the feature map.

Based on the second feature probability, the feature maps of each convolutional neural network layer corresponding to the target category are probabilistically accumulated to obtain a second heat map corresponding to each convolutional neural network layer.

Process the second heat map into a uniform size and stitch it into a multi-level heat map.

The multi-level heat map is input to each layer classifier in the multi-level classifier, and each layer classifier in the multi-level classifier is trained.

Based on the pre-trained feature extraction classifier, the weight (and the second feature probability) corresponding to each feature map in multiple convolutional layers can be obtained, the weight of the target category is selected, and the feature map of each convolutional layer is The weights are accumulated to obtain the second heat map of each convolutional layer, and the second heat map is spliced to obtain a multi-level heat map. Then, based on this heat map, the classifiers of each layer can be trained separately.

Specifically, Fig. 3 shows a schematic diagram of the pre-training process of the feature extraction classifier and the multi-level multi-classifier. After inputting the image into the feature extraction classifier, after being processed by a multi-layer convolutional neural network layer, several layers of convolution are selected. The feature vector extracted by the neural network is subjected to global average pooling, and the feature vector after the global average pooling is spliced to obtain the first feature vector, and then the full connection is performed, and the output value after the full connection is performed through softmax. Classification, and then can realize the pre-training of the feature extraction classifier, as well as the pre-classification and feature extraction of the image. Based on the trained feature extraction classifier, the weight corresponding to each feature map in multiple convolutional layers can be obtained, the weight of the target category of the selected image, and the weights of the feature maps of each convolutional layer can be accumulated. Get the heat map of each convolutional layer (that is, the second heat map mentioned above), stitch multiple heat maps into a multi-level heat map and input it into a multi-level multi-classifier, based on this multi-level heat map , multiple convolutional neural networks in a multi-level multi-classifier can be trained, that is, a classifier for each layer can be trained.

Based on the above-mentioned image classification method with multi-level and multi-classifiers, the embodiment of the present application discloses a multi-level and multi-classifier image classification device 400, as shown in FIG. 4 , the device includes:

The acquiring module 410 is configured to acquire the feature map of each convolutional layer in each category and the first feature probability of the feature map obtained by performing pre-classification and feature extraction on the target image by the feature extraction classifier.

The accumulation module 420 is configured to, based on the first feature probability, perform probability accumulation of the feature maps of each convolutional layer in each category to obtain a first heat map corresponding to each convolutional layer in each category.

The splicing module 430 is configured to process the first heat map into a uniform size and splice it into first multi-level heat maps of each category.

The input module 440 is used to input the first multi-level heat map into the multi-level classifier to obtain the classification probability of each class output by the multi-level classifier.

The output module 450 is configured to output the category of the target image according to the quantitative relationship between the maximum probability of the lowest classification probability and a preset threshold.

The embodiment of the present application discloses a multi-level multi-classifier image classification device, which pre-classifies the target image and extracts features through the feature extraction classifier, and processes the feature extraction results into the first heat map, and stitches the first heat map After creating the first multi-level heat map, input the first multi-level heat map into the multi-level multi-classifier. Through the multi-level multi-classifier, the target image can be subdivided to obtain the classification probability of each layer category, and according to the bottom layer The relationship between the maximum probability of the classification probability and the preset threshold value is used to output the category of the target image. Through this multi-level and multi-classification method, while improving the accuracy of the target image classification, the current image classification can be better solved. method leads to a high rate of false-miss identification problems when all class probabilities are low.

In a possible implementation manner, the input module 440 is used for:

The first multi-level heat map is input to each layer classifier in the multi-layer classifier to obtain the first classification probability of each layer category.

Based on the first feature probability, the first classification probability and the first feature probability of each layer category are probabilistically accumulated to obtain the second classification probability.

The second classification probability is normalized to obtain the classification probability of each layer category.

After the multi-level classifier receives the first multi-level heat map, the probability of each category of the current level can be obtained, that is, the first classification probability, based on the first feature probability obtained by the pre-classification operation of the target image by the feature extraction classifier , the probability of the first classification probability and the first feature probability of each category of the current level are accumulated to obtain the probability of the target image at the current level, that is, the second classification probability, and then by normalizing the second classification probability, it can be Get the classification probability of each layer class output by the multi-level classifier.

After obtaining the classification probability of each layer category output by the multi-level multi-classifier, the category of the target image can be judged and output according to the quantitative relationship between the maximum probability of the bottom-level classification probability and the preset threshold, a method that can be realized In a manner, the output module 450 can be used to:

When the maximum probability is greater than the preset threshold, the category corresponding to the maximum probability is output as the category of the target image.

When the maximum probability is not greater than the preset threshold, judge whether the current level is the highest level, if so, output the category corresponding to the maximum probability as the category of the target image; otherwise, combine the classification output by the previous classifier Probability, recalculate the third classification probability of each category in the bottom layer, wherein, when the maximum probability in the third classification probability is greater than the preset threshold, output the category corresponding to the maximum probability in the third classification probability as the target image category.

In this way, it can be judged according to the quantitative relationship between the maximum probability of each category at the bottom layer and the preset threshold, combined with the output probability of the previous layer, until the maximum probability is greater than the preset threshold, and the category with the highest probability is output.

In a manner that can be implemented, the device also includes:

The pre-training module is used to pre-train the feature extraction classifier and the multi-level classifier before the acquisition module 410 .

In a further technical solution, the pre-training module is used for:

The training images of the target category are input to the convolutional neural network in the feature extraction classifier, and the feature vectors of the feature maps of each layer of the convolutional neural network are output.

The feature vectors of the last n convolutional neural network layers are selected for global average pooling, and the pooled feature vectors are spliced into the first feature vector, where n is an integer greater than 1.

Fully connect the first feature vector to the output layer and perform softmax classification.

Based on the pre-trained feature extraction classifier, feature extraction is performed on the training image to obtain the feature map of each convolutional neural network layer and the second feature probability of the feature map.

Based on the second feature probability, the feature maps of each convolutional neural network layer corresponding to the target category are probabilistically accumulated to obtain a second heat map corresponding to each convolutional neural network layer.

Process the second heat map into a uniform size and stitch it into a multi-level heat map.

The multi-level heat map is input to each layer classifier in the multi-level classifier, and each layer classifier in the multi-level classifier is trained.

Based on the pre-trained feature extraction classifier, the weight (and the second feature probability) corresponding to each feature map in multiple convolutional layers can be obtained, the weight of the target category is selected, and the feature map of each convolutional layer is The weights are accumulated to obtain the second heat map of each convolutional layer, and the second heat map is spliced to obtain a multi-level heat map. Then, based on this heat map, the classifiers of each layer can be trained separately.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

The above-mentioned embodiments of this application focus on the differences between the various embodiments. As long as the different optimization features of the various embodiments are not contradictory, they can be combined to form a better embodiment. Considering the simplicity of the text, here No longer.

The above descriptions are only examples of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may occur in this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims (2)

1. An image classification method of a multi-level multi-classifier, the method comprising:

the method comprises the steps of obtaining a feature extraction classifier to pre-classify a target image and extracting features, and obtaining a feature image of each convolution layer in each class and a first feature probability of the feature image;

based on the first feature probability, carrying out probability accumulation on the feature map of each convolution layer in each category to obtain a first thermodynamic diagram corresponding to each convolution layer in each category;

processing the first thermodynamic diagram into uniform size and splicing the first thermodynamic diagram into a first multi-level thermodynamic diagram of each category;

inputting the first multi-level thermodynamic diagram into a multi-level classifier to obtain the classification probability of each layer of class output by the multi-level classifier;

outputting the category of the target image according to the quantity relation between the maximum probability in the lowest-layer classification probability and a preset threshold value;

the step of inputting the first multi-level thermodynamic diagram into a multi-level classifier to obtain the classification probability of each layer of class output by the multi-level classifier comprises the following steps:

inputting the first multi-level thermodynamic diagram into each layer of classifier in the multi-level classifier to obtain a first classification probability of each layer of class;

based on the first feature probability, carrying out probability accumulation on the first classification probability of each layer of category and the first feature probability to obtain a second classification probability;

normalizing the second classification probability to obtain the classification probability of each layer of class;

wherein outputting the category of the target image according to the quantitative relation between the maximum probability of the lowest-layer classification probability and a preset threshold value comprises:

outputting a category corresponding to the maximum probability as the category of the target image under the condition that the maximum probability is larger than the preset threshold value;

judging whether the current layer is the highest layer or not under the condition that the maximum probability is not greater than the preset threshold value, and if so, outputting a category corresponding to the maximum probability as the category of the target image;

otherwise, calculating third classification probability of each class in the bottommost layer again by combining the classification probability output by the classifier of the previous layer, wherein the class corresponding to the maximum probability in the third classification probability is output as the class of the target image under the condition that the maximum probability in the third classification probability is larger than the preset threshold value;

before the feature extraction classifier performs pre-classification and feature extraction on the target image, and the obtained feature map of each convolution layer in each class and the first feature probability of the feature map, the method further includes:

pre-training the feature extraction classifier and the multi-level classifier;

wherein said pre-training said feature extraction classifier and said multi-level classifier comprises:

inputting training images of target categories into the convolutional neural network in the feature extraction classifier, and outputting feature vectors of feature graphs of each convolutional neural network layer;

selecting the feature vectors of the last n convolutional neural network layers to carry out global average pooling, and splicing the pooled feature vectors into a first feature vector, wherein n is an integer greater than 1;

fully connecting the first feature vector with an output layer and classifying softmax;

performing feature extraction on the training image based on the pre-trained feature extraction classifier to obtain a feature map of each convolutional neural network layer and a second feature probability of the feature map;

based on the second feature probability, carrying out probability accumulation on the feature map of each convolutional neural network layer corresponding to the target category to obtain a second thermodynamic diagram corresponding to each convolutional neural network layer;

processing the second thermodynamic diagram into uniform size and splicing the uniform size into a multi-level thermodynamic diagram;

and inputting the multi-level thermodynamic diagram into each layer of classifier in the multi-level classifier, and training each layer of classifier in the multi-level classifier.

2. An image classification apparatus of a multi-level multi-classifier, the apparatus comprising:

the acquisition module is used for acquiring a feature image of each convolution layer in each class and a first feature probability of the feature image, wherein the feature image is obtained by pre-classifying a target image by the feature extraction classifier and extracting features;

the accumulation module is used for carrying out probability accumulation on the feature graphs of each convolution layer in each category based on the first feature probability to obtain a first thermodynamic diagram corresponding to each convolution layer in each category;

the splicing module is used for processing the first thermodynamic diagrams into uniform sizes and splicing the first thermodynamic diagrams into first multi-level thermodynamic diagrams of various types;

the input module is used for inputting the first multi-level thermodynamic diagram into a multi-level classifier to obtain the classification probability of each layer of class output by the multi-level classifier;

the output module is used for outputting the category of the target image according to the quantity relation between the maximum probability in the lowest-layer classification probability and a preset threshold value;

wherein, the input module is used for:

inputting the first multi-level thermodynamic diagram into each layer of classifier in the multi-level classifier to obtain a first classification probability of each layer of class;

based on the first feature probability, carrying out probability accumulation on the first classification probability of each layer of category and the first feature probability to obtain a second classification probability;

normalizing the second classification probability to obtain the classification probability of each layer of class;

wherein, output module is used for:

outputting a category corresponding to the maximum probability as the category of the target image under the condition that the maximum probability is larger than the preset threshold value;

judging whether the current layer is the highest layer or not under the condition that the maximum probability is not greater than the preset threshold value, and if so, outputting a category corresponding to the maximum probability as the category of the target image;

otherwise, calculating third classification probability of each class in the bottommost layer again by combining the classification probability output by the classifier of the previous layer, wherein the class corresponding to the maximum probability in the third classification probability is output as the class of the target image under the condition that the maximum probability in the third classification probability is larger than the preset threshold value;

the pre-training module is used for pre-training the feature extraction classifier and the multi-level classifier before the acquisition module;

wherein, the pre-training module is used for:

inputting training images of target categories into the convolutional neural network in the feature extraction classifier, and outputting feature vectors of feature graphs of each convolutional neural network layer;

selecting the feature vectors of the last n convolutional neural network layers to carry out global average pooling, and splicing the pooled feature vectors into a first feature vector, wherein n is an integer greater than 1;

fully connecting the first feature vector with an output layer and classifying softmax;

performing feature extraction on the training image based on the pre-trained feature extraction classifier to obtain a feature map of each convolutional neural network layer and a second feature probability of the feature map;

based on the second feature probability, carrying out probability accumulation on the feature map of each convolutional neural network layer corresponding to the target category to obtain a second thermodynamic diagram corresponding to each convolutional neural network layer;

processing the second thermodynamic diagram into uniform size and splicing the uniform size into a multi-level thermodynamic diagram;

and inputting the multi-level thermodynamic diagram into each layer of classifier in the multi-level classifier, and training each layer of classifier in the multi-level classifier.