CN117253095A - Image classification system and method based on biased shortest distance criterion - Google Patents

Abstract

The invention discloses an image classification system and method based on a biased shortest distance criterion, and belongs to the technical field of image classification of artificial intelligence. The method comprises the following steps: s1, acquiring an image to be classified, and pre-classifying the image to be classified to obtain a sample class; the sample categories include: a first category and a second category; s2, constructing an image classification model, and reclassifying the sample category based on the image classification model to obtain a prediction category. According to the invention, the bad performances of the minority class in the image classification task are deeply analyzed from a new angle of the depth characteristic, a deflection shortest distance criterion is provided, a larger decision area is mapped for the minority class, and the classification accuracy of the minority class is improved.

Description

Image classification system and method based on biased shortest distance criterion

Technical Field

The invention belongs to the technical field of image classification of artificial intelligence, and particularly relates to an image classification system and method based on a biased shortest distance criterion.

Background

In the image classification task, the quality and characteristics of data used for training play a vital role in classification results, and unbalanced data used for training a neural network model can cause deviation of the final effect of the model and even failure in training. In recent years, various approaches have emerged to address the problem of balancing the characteristic information of unbalanced samples to alleviate the problem of data imbalance, including adding duplicate samples to minority classes, assigning more weights to minority classes during the training phase, and nesting collaborative learning to cooperatively train multiple experts to process representation and classifier learning simultaneously, etc. However, the previous methods tend to ignore the effect of the spatial information of the features on the image classification results, and we observe that in the test stage or practical application, the features of a few classes are sparsely mapped and assigned small decision regions in the training stage, and the sparsity of these classes makes it more difficult for their features to fall into the correct small regions, so they easily cross the boundaries and are misclassified. But despite crossing decision boundaries, many few features are still closer to the cluster centroid in their space.

In general, for the problem of unbalanced training data in the picture classification, not only more feature information of a minority class needs to be fully mined to relieve the dominant position of a majority class, but also the influence of space information of deep features on the problem needs to be focused. How to use the spatial information of the features to optimize the feature decision boundary so that a good image classification effect can still be achieved even under unbalanced data is of great importance.

Disclosure of Invention

The invention aims to solve the defects of the prior art, and provides an image classification system and method based on a biased shortest distance criterion, which are used for improving the accuracy of minority class data in classification results by utilizing spatial information of image features based on the Biased Shortest Distance (BSDC) criterion.

In order to achieve the above object, the present invention provides the following solutions: an image classification system based on biased shortest distance criteria, comprising: the model building module is used for building a model;

the pre-classifying module is used for collecting images to be classified and pre-classifying the images to be classified to obtain sample types; the sample categories include: a first category and a second category;

the model construction module is used for constructing an image classification model, and reclassifying the sample category based on the image classification model to obtain a prediction category.

Further preferably, the method for obtaining the sample category by the pre-classification module includes:

classifying the images to be classified based on a deep neural network, setting a probability threshold, and setting the images to be classified as the first category if the probability threshold is higher than the probability threshold; and below the probability threshold, the second category.

Further preferably, the image classification model adopts a depth residual network as a backbone network and a linear classifier, a bias shortest distance criterion is inserted after the full connection layer, prediction probability and characteristics are received from the BN layer, and the prediction class is output.

Further preferably, the model building module includes: a first calculation unit and a second calculation unit;

the first computing unit adopts a backbone network to extract the characteristics of the images to be classified and computes the mass center;

the second calculation unit is used for calculating the distance between the feature and the centroid, and obtaining the prediction class according to the shortest distance.

The invention also provides an image classification method based on the biased shortest distance criterion, which is applied to the classification system and comprises the following steps:

s1, acquiring an image to be classified, and pre-classifying the image to be classified to obtain a sample class; the sample categories include: a first category and a second category;

s2, constructing an image classification model, and reclassifying the sample category based on the image classification model to obtain a prediction category.

Further preferably, the method for obtaining the sample class includes:

classifying the images to be classified based on a deep neural network, setting a probability threshold, and setting the images to be classified as the first category if the probability threshold is higher than the probability threshold; and below the probability threshold, the second category.

Further preferably, the image classification model adopts a depth residual network as a backbone network and a linear classifier, a bias shortest distance criterion is inserted after the full connection layer, prediction probability and characteristics are received from the BN layer, and the prediction class is output.

Further preferably, the S2 includes:

s21, extracting the characteristics of the image to be classified by adopting a backbone network, and calculating the mass center;

s22, calculating the distance between the feature and the mass center, and obtaining the prediction class according to the shortest distance.

Compared with the prior art, the invention has the beneficial effects that:

according to the invention, the bad performances of the minority class in the image classification task are deeply analyzed from a new angle of the depth characteristic, a deflection shortest distance criterion is provided, a larger decision area is mapped for the minority class, and the classification accuracy of the minority class is improved. The invention can be easily combined with the method in the current popular image classification task and further improve the performance of the model.

Drawings

In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed in the embodiments are briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of an image classification method based on a biased shortest distance criterion according to an embodiment of the present invention;

FIG. 2 is a schematic diagram illustrating classification of samples by decision boundaries according to an embodiment of the present invention;

fig. 3 is a diagram showing the effect of the embodiment of the present invention in practical application.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.

Embodiment one:

the embodiment provides an image classification system based on a biased shortest distance criterion, which comprises: the model building module is used for building a model;

the pre-classification module is used for collecting images to be classified and pre-classifying the images to be classified to obtain sample categories; sample categories include: a first category and a second category.

In this embodiment, the first class is a majority class, and the second class is a minority class. Since there is a large difference in the number of sample data of different categories in different data sets, it is often not confident for a few categories for deep learning networks, which tends to result in a lower prediction probability. Therefore, it is necessary to estimate the minority class sample threshold by pre-classification first to determine which test samples belong to minority classes.

In the embodiment, the deep neural network is adopted to classify the images to be classified, and on the basis, a probability threshold value is set（/>A comparison value for determining whether the sample belongs to a minority class), above a probability threshold, being a first class; below the probability threshold, is the second category. The probability distribution of the minority class samples is different for different data sets, the set probability threshold +.>And also different.

The model construction module is used for constructing an image classification model, and reclassifying sample categories based on the image classification model to obtain prediction categories.

In this embodiment, the image classification model uses a depth residual network as a backbone network and a linear classifier, and a shortest distance deviation criterion is inserted after the full connection layer, and the prediction probability and the characteristics are received from the BN layer, and the prediction class is output. The new prediction class is partitioned by calculating the shortest distance between the feature point and each class centroid.

Specifically, the model building module includes: a first calculation unit and a second calculation unit;

the first computing unit extracts the characteristics of the image to be classified by adopting a backbone network and computes the mass center.

The centroid calculation method is as follows:

，

in the method, in the process of the invention,indicate->Centroid of class->Is->Sample number of class, ++>Is->The samples are in class->Obtained from the last layer of the backbone network.

The second calculation unit is used for calculating the distance between the feature and the mass center, and obtaining the prediction class according to the shortest distance.

Features and classesDistance between centroids->The calculation method of (2) is as follows:

，

in the method, in the process of the invention,indicating that the probability of the test sample is less than +.>Is characterized by (3).

Determining a new prediction class P according to the shortest distance:

，

where N represents the number of classes.

Taking binary classification as an example, the Shortest Distance Criterion (SDC) can be expressed as determining two given pointsAnd->Which of (the centroids of the two classes) is closer to an arbitrary point in space +.>(characteristics of the test sample) as shown in fig. 2 (a). When the point is->Is positioned at->And->It is equidistant from the perpendicular bisectors of (a). If the point is located on one side of the perpendicular bisector, it is close to a given point on that side. The perpendicular bisector serves as a new decision boundary for sample classification and divides the space into equal decision regions of the two classes. This approach adds a minority class of decision regions compared to the original region of the fully connected network rendering, making their features easier to fall within the correct region during the test phase.

Considering that the number of minority samples is not limited in practical application, the number may be close to or even equal to the number of majority samples, which may cause minority features to occupy a larger area in space due to sparsity, and thus the present embodiment proposes a Biased Shortest Distance Criterion (BSDC), as shown in fig. 2 (b), to bias the perpendicular bisector to move toward the majority, and provide a larger decision area for minority.

The principle of the Biased Shortest Distance Criterion (BSDC) will be described below in connection with a simple example: assume that there is one feature point A (from a minority class) and two given centroids in two dimensionsAnd->(from the majority class and minority class, respectively) as shown in fig. 2 (c). Because of the sparse mapping, point a falls to the left of the perpendicular bisector and is misclassified by SDC. However, if the perpendicular bisector can be transferred to most classes, point a will enter its decision area. Therefore, will->And->Multiplied by the same factor (less than 1) and then the centroids are scaled to the origin (point O in fig. 2 (c)) and to the new positionAnd->. Thus, the perpendicular bisector of the new centroid moves toward most classes as compared to the original centroid. In the BSDC proposed by the present invention, the feature of each sample is regarded as a point in M-dimensional space, where M is a parameter in the feature, each parameter representing a dimension.

The present invention multiplies the centroid by a factor in calculating the distance between two classes of features and the centroid from the SDC（/>For controlling the direction and distance of movement of the decision boundary) to achieve a movement of the perpendicular bisector, defined as follows:

，

where i ε {1,2} (i represents the identity of two points in FIG. 2), and δ <1.

The above example contains an implicit assumption, namely the origin (point) More approximate->If the origin is closer ++>When the centroid is multiplied by a factor->In this case, the perpendicular bisector will be oriented +>Moving as shown in fig. 2 (d). This contradicts the original intent of BSDC.To solve this problem, the present invention determines the distance between the centroid and the origin before multiplication. If the centroid of most classes is closer to the origin, +.>The assignment is less than 1. Conversely, let->Above 1, this results in the centroid being away from the origin, with the perpendicular bisector continuing toward the majority class of trajectories, as shown in FIG. 2 (e). Factor->The following can be defined:

，

in the method, in the process of the invention,is a superparameter controlling the shift,/->And->Representing the distances of the centroids of the majority class and minority class, respectively, from the origin.

For multi-class tasks, to reduce computational complexity, the shortest distance is calculated using the Top-n prediction class of the deep neural network as a candidate class for BSDC, since Top-n (e.g., n=3) predictions typically cover the correct class.

BSDC is focused on improving the performance of a minority class during the test phase, without conflicting with other modules, such as loss functions and sampling strategies. It can thus be combined with existing long tail methods and further boost them. In order to conveniently apply the invention, a heat exchange mechanism is designed for BSDC. That is, inserting BSDC after fully connected layer and receiving prediction probability and feature from BN layer (Batch Normalization, data normalization method, used before activating layer in deep neural network) can accelerate convergence speed in model training, make model training process more stable, avoid gradient explosion or gradient disappearance), then output new prediction probability, wherein BSDC replaces original probability with one-hot vector generated by reclassifying. This operation does not change the dimension of the original prediction probabilities and allows them to work normally in subsequent tasks.

Embodiment two:

as shown in fig. 1, the present embodiment provides an image classification method based on a biased shortest distance criterion, which includes the following steps:

s1, acquiring an image to be classified, and pre-classifying the image to be classified to obtain a sample class; sample categories include: a first category and a second category.

In this embodiment, the first class is a majority class, and the second class is a minority class. Since there is a large difference in the number of sample data of different categories in different data sets, it is often not confident for a few categories for deep learning networks, which tends to result in a lower prediction probability. Therefore, it is necessary to estimate the minority class sample threshold by pre-classification first to determine which test samples belong to minority classes.

In the embodiment, the deep neural network is adopted to classify the images to be classified, and on the basis, a probability threshold value is set（/>A comparison value for determining whether the sample belongs to a minority class), above a probability threshold, being a first class; below the probability threshold, is the second category. The probability distribution of the minority class samples is different for different data sets, the set probability threshold +.>And also different.

S2, constructing an image classification model, and reclassifying sample categories based on the image classification model to obtain prediction categories.

In this embodiment, the image classification model uses a depth residual network as a backbone network and a linear classifier, and a shortest distance deviation criterion is inserted after the full connection layer, and the prediction probability and the characteristics are received from the BN layer, and the prediction class is output. The new prediction class is partitioned by calculating the shortest distance between the feature point and each class centroid.

Specifically, S2 includes:

s21, extracting the characteristics of the image to be classified by adopting a backbone network, and calculating the mass center.

The centroid calculation method is as follows:

，

in the method, in the process of the invention,indicate->Centroid of class->Is->Sample number of class, ++>Is->The samples are in class->Obtained from the last layer of the backbone network.

S22, calculating the distance between the feature and the mass center, and obtaining a prediction class according to the shortest distance.

Features and classesDistance between centroids->The calculation method of (2) is as follows:

，

in the method, in the process of the invention,indicating that the probability of the test sample is less than +.>Is characterized by (3).

Determining a new prediction class P according to the shortest distance:

，

where N represents the number of classes.

Taking binary classification as an example, the Shortest Distance Criterion (SDC) can be expressed as determining two given pointsAnd->Which of (the centroids of the two classes) is closer to an arbitrary point in space +.>(characteristics of the test sample) as shown in fig. 2 (a). When the point is->Is positioned at->And->It is equidistant from the perpendicular bisectors of (a). If the point is located on one side of the perpendicular bisector, it is close to a given point on that side. The vertical bisector serves as a new decision boundary for sample classification and divides the space into two classes of phasesAnd an equal decision area. This approach adds a minority class of decision regions compared to the original region of the fully connected network rendering, making their features easier to fall within the correct region during the test phase.

Considering that the number of minority samples is not limited in practical application, the number may be close to or even equal to the number of majority samples, which may cause minority features to occupy a larger area in space due to sparsity, and thus the present embodiment proposes a Biased Shortest Distance Criterion (BSDC), as shown in fig. 2 (b), to bias the perpendicular bisector to move toward the majority, and provide a larger decision area for minority.

The principle of the Biased Shortest Distance Criterion (BSDC) will be described below in connection with a simple example: assume that there is one feature point A (from a minority class) and two given centroids in two dimensionsAnd->(from the majority class and minority class, respectively) as shown in fig. 2 (c). Because of the sparse mapping, point a falls to the left of the perpendicular bisector and is misclassified by SDC. However, if the perpendicular bisector can be transferred to most classes, point a will enter its decision area. Therefore, will->And->Multiplied by the same factor (less than 1) and then the centroids are scaled to the origin (point O in fig. 2 (c)) and to the new positionAnd->. Thus, the perpendicular bisector of the new centroid moves toward most classes as compared to the original centroid. In the BSDC proposed by the present invention, each sampleThe feature of the present is considered as a point in M-dimensional space, where M is a parameter in the feature, each parameter representing a dimension. In calculating the distance between two types of features and the centroid from SDC, the present invention multiplies the centroid by a factor +.>（/>For controlling the direction and distance of movement of the decision boundary) to achieve a movement of the perpendicular bisector, defined as follows:

，

where i ε {1,2} (i represents the identity of two points in FIG. 2), and δ <1.

The above example contains an implicit assumption, namely the origin (point) More approximate->If the origin is closer ++>When the centroid is multiplied by a factor->In this case, the perpendicular bisector will be oriented +>Moving as shown in fig. 2 (d). This contradicts the original intent of BSDC. To solve this problem, the present invention determines the distance between the centroid and the origin before multiplication. If the centroid of most classes is closer to the origin, +.>The assignment is less than 1. Conversely, let->Above 1, this results in the centroid being away from the origin, with the perpendicular bisector continuing toward the majority class of trajectories, as shown in FIG. 2 (e). Factor->The following can be defined:

，

in the method, in the process of the invention,is a superparameter controlling the shift,/->And->Representing the distances of the centroids of the majority class and minority class, respectively, from the origin.

For multi-class tasks, to reduce computational complexity, the shortest distance is calculated using the Top-n prediction class of the deep neural network as a candidate class for BSDC, since Top-n (e.g., n=3) predictions typically cover the correct class.

BSDC is focused on improving the performance of a minority class during the test phase, without conflicting with other modules, such as loss functions and sampling strategies. It can thus be combined with existing long tail methods and further boost them. In order to conveniently apply the invention, a heat exchange mechanism is designed for BSDC. That is, inserting BSDC after fully connected layer and receiving prediction probability and feature from BN layer (Batch Normalization, data normalization method, used before activating layer in deep neural network) can accelerate convergence speed in model training, make model training process more stable, avoid gradient explosion or gradient disappearance), then output new prediction probability, wherein BSDC replaces original probability with one-hot vector generated by reclassifying. This operation does not change the dimension of the original prediction probabilities and allows them to work normally in subsequent tasks.

Embodiment III:

as shown in tables 1 and 2, the comparison of the method of the present invention with several of the most advanced comparison methods shows that the performance of the present invention is excellent.

Table 1 shows the highest accuracy comparison of the inventive method with the original method on the ImageNet-LT and iNaturalist 2018 datasets. Table 2 shows the highest accuracy of the invention on CIFAR long tail data set compared to the original method.

TABLE 1

TABLE 2

。

Embodiment four:

in the medical field, doctors often analyze the severity of arthritis from a patient's knee image and are classified as 5 according to the Kellgren & Lawrence classification criteria:

grade 0, no change;

grade I, slight osteophyte;

stage II, obvious osteophyte, but without affecting joint clearance;

III, moderately narrowing joint space;

grade IV, the joint space is obviously narrowed, and the subchondral bone is hardened.

But in real life the number of data samples collected from the patient varies greatly from one class of sample data to another. When medical images are classified through artificial intelligence, unbalanced data sets can cause trouble to the model, so that the accuracy of the final classification result is low.

The total amount of medical image data collected in the actual scene reaches 6300, wherein the IV-level patient case samples are few, the case images are often misclassified during recognition, and the method provided by the invention can optimize the problem.

First byPre-experiment treatment, which is to use the existing probability of predicting samples by using the deep neural network to roughly determine sample types, and on the basis, the invention sets an initial threshold value after pre-experiment is carried out on the collected medical image data setWhen the probability of the sample is less than +.0.5 =>When it is identified as a minority class. According to the invention, the sample pictures are mapped into the feature space, each sample is mapped into a point, the Euclidean distance is used for calculating the distance between the feature point and the mass centers of various samples, and the new prediction class is re-divided. By biasing the decision boundary, the decision boundary is adjusted, adding a minority of larger decision areas. In order to reduce the calculation complexity, the invention performs experimental comparison on n different values when using Top-n prediction class of the deep neural network as candidate class of BSDC to calculate the shortest distance. The accuracy is highest when n=4 is selected for the present dataset. As shown in fig. 3, the method of the present invention was used to classify as class IV arthritis samples, but the images were incorrectly classified as class ii cases by the existing optimal model.

The above embodiments are merely illustrative of the preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, but various modifications and improvements made by those skilled in the art to which the present invention pertains are made without departing from the spirit of the present invention, and all modifications and improvements fall within the scope of the present invention as defined in the appended claims.

Claims (8)

1. An image classification system based on a biased shortest distance criterion, comprising: the model building module is used for building a model;

the pre-classifying module is used for collecting images to be classified and pre-classifying the images to be classified to obtain sample types; the sample categories include: a first category and a second category;

the model construction module is used for constructing an image classification model, and reclassifying the sample category based on the image classification model to obtain a prediction category.

2. The image classification system based on the biased shortest distance criterion of claim 1, wherein the method of the pre-classification module to obtain the sample class comprises:

classifying the images to be classified based on a deep neural network, setting a probability threshold, and setting the images to be classified as the first category if the probability threshold is higher than the probability threshold; and below the probability threshold, the second category.

3. The image classification system based on the biased shortest distance criterion according to claim 1, wherein the image classification model adopts a depth residual network as a backbone network and a linear classifier, the biased shortest distance criterion is inserted after the fully connected layer, the prediction probability and the feature are received from the BN layer, and the prediction class is output.

4. An image classification system based on a biased shortest distance criterion as claimed in claim 3, wherein said model building module comprises: a first calculation unit and a second calculation unit;

the first computing unit adopts a backbone network to extract the characteristics of the images to be classified and computes the mass center;

the second calculation unit is used for calculating the distance between the feature and the centroid, and obtaining the prediction class according to the shortest distance.

5. An image classification method based on a biased shortest distance criterion, the classification method being applied to the classification system of any of claims 1-4, comprising the steps of:

s1, acquiring an image to be classified, and pre-classifying the image to be classified to obtain a sample class; the sample categories include: a first category and a second category;

s2, constructing an image classification model, and reclassifying the sample category based on the image classification model to obtain a prediction category.

6. The method of claim 5, wherein the step of obtaining the sample class comprises:

classifying the images to be classified based on a deep neural network, setting a probability threshold, and setting the images to be classified as the first category if the probability threshold is higher than the probability threshold; and below the probability threshold, the second category.

7. The image classification method based on the biased shortest distance criterion according to claim 5, wherein the image classification model adopts a depth residual network as a backbone network and a linear classifier, the biased shortest distance criterion is inserted after the fully connected layer, the prediction probability and the feature are received from the BN layer, and the prediction class is output.

8. The method of image classification based on the biased shortest distance criterion as claimed in claim 7, wherein said S2 comprises:

s21, extracting the characteristics of the image to be classified by adopting a backbone network, and calculating the mass center;

s22, calculating the distance between the feature and the mass center, and obtaining the prediction class according to the shortest distance.