CN111325242A - Image classification method, terminal and computer storage medium - Google Patents

Abstract

The embodiment of the present application discloses an image classification method, which is applied to a terminal and includes: acquiring an image to be classified, using a pre-trained fine-grained classification model, classifying the to-be-classified image, and obtaining a classified image. The embodiments of the present application also provide a terminal and a computer storage medium at the same time.

Description

Image classification method, terminal and computer storage medium

technical field

The present application relates to a fine-grained image classification technology, and in particular, to an image classification method, a terminal, and a computer storage medium.

Background technique

Fine-grained image classification technology is a branch of image classification. Since its categories belong to the same large category, for example, dogs of different breeds belong to the same category of dogs, so the direct differences between their categories are relatively small, but there are background and There are still many differences between categories due to the diversity of shapes.

At present, image fine-grained classification methods can be roughly divided into the following branches: fine-tuning based on existing classification networks, methods based on fine-grained feature learning, methods based on the combination of target block detection and classification, and methods based on visual attention mechanism , however, the above methods are mainly improved on the basis of general classification algorithms. At present, the most commonly used loss function for classification algorithms is softmax, and softmax is not highly discriminative between classes for fine-grained classification; it can be seen that the existing The use of fine-grained image classification algorithms results in poor image classification accuracy.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image classification method, a terminal, and a computer storage medium, which can improve the accuracy of image classification.

The technical solution of the present application is realized as follows:

An embodiment of the present application provides a method for classifying images, and the method is applied to a terminal, including:

Get images to be classified;

Use a pre-trained fine-grained classification model to classify the to-be-classified image to obtain a classified image;

Wherein, the trained fine-grained classification model is obtained in the following manner:

According to the obtained image labels of the to-be-trained image set, the images of the to-be-trained image set are grouped to obtain a grouped to-be-trained image set; wherein, the image labels are used to characterize the category of the image;

Extracting feature vectors from the grouped images in the to-be-trained image set to obtain a feature vector group;

The fine-grained classification model is trained by using the feature vector group to determine the model parameters in the fine-grained classification model when the value of the loss function and the objective function is the smallest, and the trained fine-grained classification model is obtained.

An embodiment of the present application provides a terminal, where the terminal includes:

The acquisition module is used to acquire the images to be classified;

A classification module, configured to use a pre-trained fine-grained classification model to classify the to-be-classified image to obtain a classified image;

Wherein, the trained fine-grained classification model is obtained in the following manner:

According to the obtained image labels of the to-be-trained image set, the images of the to-be-trained image set are grouped to obtain a grouped to-be-trained image set; wherein, the image labels are used to characterize the category of the image;

Extracting feature vectors from the grouped images in the to-be-trained image set to obtain a feature vector group;

The fine-grained classification model is trained by using the feature vector group to determine the model parameters in the fine-grained classification model when the value of the loss function and the objective function is the smallest, and the trained fine-grained classification model is obtained.

An embodiment of the present application further provides a terminal, the terminal includes: a processor and a storage medium storing executable instructions of the processor, the storage medium depends on the processor to perform operations through a communication bus, and when the When the instructions are executed by the processor, the image classification method according to one or more of the above embodiments is executed.

Embodiments of the present application provide a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the images described in the above one or more embodiments. classification method.

The embodiments of the present application provide an image classification method, a terminal, and a computer storage medium. The method is applied to a terminal and includes: acquiring an image to be classified, using a pre-trained fine-grained classification model, and classifying the image to be classified, Obtaining classified images, wherein the trained fine-grained classification model is obtained in the following manner: according to the obtained image labels of the image set to be trained, the images of the image set to be trained are grouped, and the grouped image set to be trained is obtained; Among them, the image label is used to represent the category of the image, and the feature vector is extracted from the grouped images to be trained to obtain a feature vector group, and the feature vector group is used to train the fine-grained classification model to determine the loss in the fine-grained classification model. The model parameter when the value of the function and the objective function is the smallest, a trained fine-grained classification model is obtained; that is, in the embodiment of the present application, the images to be classified are classified by adopting the pre-trained fine-grained classification model, wherein , the trained fine-grained classification model is to group the images of the to-be-trained image set according to the image labels of the to-be-trained image set and extract the feature vector, and use the feature vector group to train the fine-grained classification model. The objective function is obtained, and the model parameters are obtained when its value is the smallest, so as to obtain the trained fine-grained classification model. In this way, by setting the loss function and the objective function in the model, the obtained trained fine-grained classification model is more optimized , and on this basis, when performing image classification, the accuracy of image classification is improved, thereby improving user experience.

Description of drawings

1 is a schematic flowchart of an optional image classification method provided by an embodiment of the present application;

2 is a schematic flowchart of an example of an optional image classification method provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram 1 of a terminal according to an embodiment of the present application;

FIG. 4 is a second schematic structural diagram of a terminal according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

Example 1

An embodiment of the present application provides an image classification method, which is applied to a terminal. FIG. 1 is a schematic flowchart of an optional image classification method provided by an embodiment of the present application. Referring to FIG. 1 , the above Image classification methods can include:

S101: Obtain an image to be classified;

At present, the fine-grained classification models of images can be roughly divided into the following branches: fine-tuning based on existing classification networks, methods based on fine-grained feature learning, methods based on the combination of target block detection and classification, and methods based on visual attention mechanism. method, in which the method based on the fine-tuning of the existing classification network usually uses the existing classification network (for example: MobileNet, Xception, etc.) to perform preliminary training on ImageNet to obtain a trained classification model, and then continue on the fine-grained dataset. Fine-tune the model to make the model more suitable for distinguishing sub-categories; the method based on fine-grained feature learning requires the combination of information obtained by two networks, one network is used to obtain the location information of the target, and the other is used to extract the abstract feature expression of the target; The fine-grained classification method based on the combination of detection and classification of target blocks draws on the idea of target detection. First, the target area of the image is framed by the target detection module, and then the fine-grained classification is performed based on the target area. The classification algorithm can be a traditional support vector. Compared with the general classification algorithm, the fine-grained classification algorithm based on the attention mechanism adds an attention mechanism, which makes the model pay more attention to the information expression of the target position.

Among them, the above methods are mainly improved on the basis of general classification algorithms. At present, the most commonly used loss function of classification algorithms is softmax, and softmax is not highly discriminative between classes for fine-grained classification, and has the following shortcomings: First, between classes The distance between the feature centers is short, which is easy to cause the problem of misclassification between classes; second, the intra-class features are not clustered enough, resulting in overlapping feature distributions between multiple classes, which will also cause misclassification between classes; third, The algorithm of adding the detection module will introduce complex operations, increase the computational cost and cause more time delay, resulting in lower accuracy of image classification.

In order to improve the accuracy of image classification, here, the terminal first obtains the image to be classified, wherein, the image to be classified may contain objects to be classified, such as: dog, car, tree and other objects, for the category of dog, car, tree Subdivide again to classify the dog, car, tree in the image to be classified.

S102: Using a pre-trained fine-grained classification model, classify the images to be classified, and obtain the classified images;

In order to classify the images to be classified, here, a pre-trained fine-grained classification model is used to classify the images to be classified, wherein the trained fine-grained classification model is obtained in the following manner:

According to the obtained image labels of the image set to be trained, the images of the image set to be trained are grouped to obtain the grouped image set to be trained;

Extract feature vectors from the grouped images to be trained to obtain feature vector groups;

The feature vector group is used to train the fine-grained classification model to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

Specifically, the terminal first obtains the image set to be trained and the image labels of the images in the to-be-trained image set, where the image labels are used to represent the category of the images; for example, for images classified according to the category of cars, the image labels can be classified according to the brand To classify the car, the image labels in the image set to be trained can include BYD, Audi, BMW, etc.

After the image labels of the to-be-trained image sets are obtained, the to-be-trained image sets are grouped according to the image labels of the to-be-trained image sets, and the grouped to-be-trained image sets can be obtained. Extract the feature vectors for the images in the image set, and then obtain the feature vectors of each group of images in the image set to be trained, that is, use each set of feature vectors to form a feature vector group.

After the feature vector group is obtained, the feature vector group is used to train the fine-grained classification model. Here, it should be noted that the fine-grained classification model includes not only the loss function, but also the objective function. In the process of training the model, adding The optimized objective function can make the intra-class features more concentrated and the inter-class features more distant, so that the trained fine-grained classification model can be more accurate in classifying categories.

Specifically, when the fine-grained classification model is trained, the model parameters with the smallest loss function and the objective function are determined through iteration. After the model parameters are determined, the trained fine-grained classification model can be obtained. .

In order to realize the grouping of the models to be trained and determine the feature vector group, in an optional embodiment, the images of the image set to be trained are grouped according to the obtained image labels of the image set to be trained, and the grouped images to be trained are obtained. Training image set, including:

Determining the images in the image set to be trained as the first image in sequence;

For the first image, select the second image and the third image from the set of images to be trained except the first image; wherein, the image label of the second image is the same as the image label of the first image, and the image label of the third image is the same as the image label of the first image. different from the image tag of the first image;

Using the first image, the second image and the third image to form a group to obtain a grouped image set to be trained;

Correspondingly, a feature vector is extracted from the images in the grouped images to be trained to obtain a feature vector group, including:

For the first image, the second image and the third image, a fine-grained classification model is used to extract the feature vector, respectively, to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image;

Using the feature vector of the first image, the feature vector of the second image and the feature vector of the third image, a feature vector group is formed.

The first image is first determined from the set of images to be trained, each image in the set of images to be trained can be sequentially determined as the first image, and then the first image, and the first image is selected from the set of images to be trained except the first image. For the second image and the third image, the selection rules are:

The image label of the first image is the same as the image label of the second image, and the image label of the first image is different from that of the third image. That is to say, first traverse the images in the image set to be trained, and determine each image as the first image. One image, then select the second image and the third image according to the above rules, so that the first image, the second image and the third image are used to form a group, and several groups of images to be trained can be obtained, that is, the grouped images to be trained set.

After the grouped image sets to be trained are obtained, each set of images to be trained includes three images, namely the first image, the second image and the third image. Here, the three images in each set of images to be trained are Extract the feature vectors respectively to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image, and use the feature vector of the first image, the feature vector of the second image and the feature vector of the third image A set of eigenvectors is formed, and several sets of eigenvectors can be obtained according to the above method, thereby obtaining a set of eigenvectors.

In order to make the classification accuracy of the fine-grained classification model obtained by training higher, the feature vector group needs to be screened. In an optional embodiment, feature vectors are extracted from the grouped images to be trained to obtain the features After the vector group, the feature vector group is used to train the fine-grained classification model to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and before the trained fine-grained classification model is obtained. Methods also include:

Select the groups that do not meet the preset conditions in the eigenvector group;

The groups that do not meet the preset conditions are deleted from the feature vector group to update the feature vector group.

Specifically, the groups that do not meet the preset conditions are selected from the feature vector group, and the groups that do not meet the preset conditions are deleted, so as to update the feature vector group, wherein the groups that do not meet the preset conditions can be There are various forms. Generally speaking, it is necessary to delete the group with larger intra-class features of the first image and the second image, and/or delete the first image and the third image with smaller inter-class features. The group is deleted, and this is not specifically limited in this embodiment of the present application.

In order to select the groups in the feature vector group that do not meet the preset conditions, in an optional embodiment, select the groups in the feature vector group that do not meet the preset conditions, including:

calculating a first distance value between the feature vector of the first image and the feature vector of the second image;

When the first distance value is greater than or equal to the first preset threshold, determine the group including the first image and the second image as a group that does not meet the preset condition, and select a group that does not meet the preset condition;

and / or,

calculating a second distance value between the feature vector of the first image and the feature vector of the third image;

When the second distance value is less than or equal to the second preset threshold, the group including the first image and the third image is determined as a group that does not meet the preset condition, and the group that does not meet the preset condition is selected.

Specifically, the distance between the feature vector of the first image and the feature vector of the second image can be calculated first, which is recorded as the first distance value, the first preset threshold is preset in the terminal, and the first distance value is compared with The size between the first preset thresholds, when the first distance value is greater than or equal to the first preset threshold, it means that the intra-class features of the first image and the second image are quite different, and here, the first image is needed The group that is less different from the intra-class feature of the second image, therefore, the feature vector group of the first image and the second image corresponding to the first distance value is determined as the group that does not meet the preset conditions, and the group Don't delete from the eigenvector group.

It is also possible to first calculate the distance between the feature vector of the first image and the feature vector of the third image, denoting it as the second distance value, preset the second preset threshold value in the terminal, and compare the second distance value with the second preset value. Set the size between the thresholds. When the second distance value is less than or equal to the second preset threshold, it means that the difference between the features of the first image and the third image is small, and here, what is needed is the first image and the third image. A group with a large difference in the inter-class features of the images, therefore, the feature vector group of the first image and the third image corresponding to the second distance value is determined as the group that does not meet the preset conditions, and the group is determined from the feature vector group. removed from the vector group.

In addition, the distance between the eigenvector of the first image and the eigenvector of the second image can also be calculated, denoted as the first distance value, and the distance between the eigenvector of the first image and the eigenvector of the third image can be calculated , denoted as the second distance value, set the first preset threshold and the second preset threshold in the terminal, compare the relationship between the first distance value and the first preset threshold, and compare the second distance value with the second preset threshold The relationship between the thresholds, when the first distance value is greater than or equal to the first preset threshold, and the second distance value is less than or equal to the second preset threshold, it means that the intra-class features of the first image and the second image differ greatly, and the The difference between the features of the first image and the third image is small, so the group corresponding to the first distance value and the second distance value is determined as the group that does not meet the preset condition, and the group is converted from the feature vector removed from the group.

Further, in order to delete the groups that do not meet the preset conditions, in an optional embodiment, select the groups that do not meet the preset conditions in the feature vector group, including:

Calculate the first distance value between the feature vector of the first image and the feature vector of the second image, and the second distance value between the feature vector of the first image and the feature vector of the third image;

When the difference between the second distance value and the first distance value is greater than the third preset threshold, the group including the first image, the second image and the third image is determined as a group that does not meet the preset conditions, and the selected group that does not meet the preset conditions.

First, the distance value between the vector feature of the first image and the feature vector of the second image is calculated, which is the first distance value, and the distance value between the feature vector of the first image and the feature vector of the third image is calculated, That is, the second distance value. After calculating the difference between the first distance value and the second distance value, a third preset threshold is preset in the terminal, and the relationship between the difference and the third preset threshold is compared. , when the difference is greater than the third preset threshold, it means that the inter-class feature distance between the first image and the third image in the group is much larger than the intra-class feature distance between the first image and the second image, thus ensuring the class The distance between features is a larger value, which makes the classification accuracy of the trained fine-grained classification model higher.

Therefore, the group is determined as a group that does not meet the preset conditions, and is selected and deleted to update the feature vector group. In this way, the intra-class features of the updated feature vector group are more concentrated, and the inter-class features are more distant. In turn, the accuracy of the trained fine-grained classification model is improved.

In order to train a more optimized fine-grained classification model, in an optional embodiment, a feature vector group is used to train the fine-grained classification model, so as to determine the best value of the loss function and the objective function in the fine-grained classification model. hours of model parameters to obtain a trained fine-grained classification model, including:

Use the feature vector group to train the fine-grained classification model to determine the model parameters with the smallest value of the loss function and the smallest value of the objective function in the fine-grained classification model, and obtain the trained fine-grained classification model;

or,

The feature vector group is used to train the fine-grained classification model to determine the model parameters when the sum of the value of the loss function and the value of the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

Here, the feature vector group can be used to train the fine-grained classification model, and the model parameters when the value of the loss function in the model is the smallest and the value of the newly added objective function is the smallest can be determined. After obtaining the model parameters, the training is completed. A trained fine-grained classification model is obtained.

It is also possible to train the fine-grained classification model by using the feature vector group, set the weight value for the loss function of the model, set the weight value for the objective function in the model, and determine the value of the function and the value of the objective function during training. After the weighted summation, the model parameters with the smallest weighted summation value are obtained to obtain a trained fine-grained classification model; usually, the same weight value can be selected for the loss function and the objective function, so only the loss function in the model needs to be determined. The model parameters when the sum of the value of , and the value of the objective function is the smallest, so as to obtain a trained fine-grained classification model.

In order to optimize the fine-grained classification model, an objective function is added to the model. In an optional embodiment, the objective function is a function related to the feature vector of the first image, the feature vector of the second image, the feature vector of the third image and the third preset threshold.

Here, through the objective function, when the fine-grained classification model is used for classification, the intra-class features are more gathered, and the inter-class features are more distant, so that the images to be recognized can be more accurately classified in the process of classification. The category of the object to be classified.

Examples are given below to illustrate the image classification method described in one or more of the above embodiments.

In practical applications, the loss function in the fine-grained classification model is usually based on the softmax method. The calculation formula of softmax is as follows:

x _i represents the abstract feature vector extracted by the ith sample based on the classification network, _yi represents the label of the ith sample, W represents the weight of the classification layer used to extract features from the fully convolutional network, and W _j represents The weight of the jth column (ie, class j), similarly b _j represents the paranoid term of the column, T is the matrix transpose operation, and N is the number of classes.

The above formula (1) does not include the optimization of the distance metric for the features belonging to the same category. On the basis of softmax, according to the construction of triples, the features of the same category are constrained to make the distance closer, and the features of different categories are constrained. Limits are also made to make them farther apart. The most complex module is to construct triples, and FIG. 2 is a schematic flowchart of an example of an optional image classification method provided by the embodiment of the present application. As shown in FIG. 2 , the process of constructing triples is as follows:

Anchor is the current input image Va, Positive is the positive sample Vp that is consistent with the Anchor input image label, Negative is the negative sample Vn that is different from the current input image label, and the process of constructing triples can be offline or online. The example adopts an online method to facilitate updating. According to the input data in each branch (N), it is processed by convolutional neural network (CNN, Convolutional Neural Networks) to obtain the feature vector of each image, which is randomly combined into triples, N*N*N sets of triples {Va, Vp, Vn} _i can be combined. However, not all triples are reasonable. Too many useless triples will cause the algorithm to fail to converge or to converge very slowly. Therefore, it is still necessary to filter the triples. N*N*N triples , only when the image labels of Va and Vp are consistent, and the image labels of Va and Vn are opposite, are valid triples that satisfy the description in FIG. 2 .

Since the purpose of the algorithm is to make the intra-class features more concentrated and the inter-class features more distant, the purpose of the algorithm optimization is to make the distance between Va and Vp smaller than the distance between Va and Vn, and the optimization objective function is described as follows:

In the above formula (2), a is margin, the purpose is to make the class spacing larger. According to the experimental experience, you can try 0.5 initially. The objective function combined with the loss can improve the accuracy of fine-grained classification.

The above examples can improve the inter-class discrimination and intra-class aggregation of the classification algorithm, and further improve the fine-grained classification effect. The above-mentioned image classification method can be applied to fine-grained classification and recognition.

An embodiment of the present application provides an image classification method, which is applied to a terminal, and includes: acquiring an image to be classified, using a pre-trained fine-grained classification model, classifying the image to be classified, and obtaining a classified image, The trained fine-grained classification model is obtained in the following manner: according to the obtained image labels of the image set to be trained, the images of the image set to be trained are grouped to obtain the grouped image set to be trained; wherein, the image labels are used for Characterize the category of the image, extract the feature vector from the grouped images to be trained, obtain the feature vector group, use the feature vector group to train the fine-grained classification model, and determine the value of the loss function and the objective function in the fine-grained classification model. The model parameter with the smallest value is used to obtain a trained fine-grained classification model; that is, in the embodiment of the present application, the images to be classified are classified by using a pre-trained fine-grained classification model, wherein the trained fine-grained classification model is The classification model is to group the images of the image set to be trained according to the image labels of the image set to be trained and extract the feature vector, and use the feature vector group to train the fine-grained classification model. The model parameters are obtained when the value is the smallest, so as to obtain a trained fine-grained classification model. In this way, by setting the loss function and the objective function in the model, the obtained trained fine-grained classification model is more optimized. When performing image classification, the accuracy of image classification is improved, thereby improving user experience.

Embodiment 2

FIG. 3 is a schematic structural diagram 1 of a terminal provided by an embodiment of the present application. As shown in FIG. 3 , an embodiment of the present application provides a terminal, including:

an acquisition module 31, configured to acquire images to be classified;

The classification module 32 is configured to use a pre-trained fine-grained classification model to classify the images to be classified, and obtain the classified images;

Among them, the trained fine-grained classification model is obtained in the following way:

According to the obtained image labels of the to-be-trained image set, the images of the to-be-trained image set are grouped to obtain a grouped to-be-trained image set; wherein, the image labels are used to characterize the category of the image;

Extract feature vectors from the grouped images to be trained to obtain feature vector groups;

The feature vector group is used to train the fine-grained classification model to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

Optionally, the terminal groups the images in the to-be-trained image set according to the acquired image labels of the to-be-trained image set, and obtains the grouped to-be-trained image set, including:

Determining the images in the image set to be trained as the first image in sequence;

For the first image, select the second image and the third image from the set of images to be trained except the first image; wherein, the image label of the second image is the same as the image label of the first image, and the image label of the third image is the same as the image label of the first image. different from the image tag of the first image;

Using the first image, the second image and the third image to form a group to obtain a grouped image set to be trained;

Correspondingly, the terminal extracts feature vectors from the images in the grouped images to be trained, and obtains a feature vector group, including:

For the first image, the second image and the third image, a fine-grained classification model is used to extract the feature vector, respectively, to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image;

Using the feature vector of the first image, the feature vector of the second image and the feature vector of the third image, a feature vector group is formed.

Optionally, the terminal is also used to:

After the feature vector is extracted from the grouped images to be trained, and the feature vector group is obtained, the fine-grained classification model is trained by using the feature vector group to determine the best value of the loss function and the objective function in the fine-grained classification model. Hourly model parameters, before obtaining the trained fine-grained classification model, select the groups in the feature vector group that do not meet the preset conditions;

The groups that do not meet the preset conditions are deleted from the feature vector group to update the feature vector group.

Optionally, the terminal selects the groups in the feature vector group that do not meet the preset conditions, including:

calculating a first distance value between the feature vector of the first image and the feature vector of the second image;

When the first distance value is greater than or equal to the first preset threshold, determine the group including the first image and the second image as a group that does not meet the preset condition, and select a group that does not meet the preset condition;

and / or,

calculating a second distance value between the feature vector of the first image and the feature vector of the third image;

When the second distance value is less than or equal to the second preset threshold, the group including the first image and the third image is determined as a group that does not meet the preset condition, and the group that does not meet the preset condition is selected.

Optionally, the terminal selects the groups in the feature vector group that do not meet the preset conditions, including:

Calculate the first distance value between the feature vector of the first image and the feature vector of the second image, and the second distance value between the feature vector of the first image and the feature vector of the third image;

When the difference between the second distance value and the first distance value is greater than the third preset threshold, the group including the first image, the second image and the third image is determined as a group that does not meet the preset conditions, and the selected group that does not meet the preset conditions.

Optionally, the terminal uses the feature vector group to train the fine-grained classification model, so as to determine the model parameters when the value of the loss function and the objective function in the fine-grained classification model is the smallest, and obtain the trained fine-grained classification model, including: :

Use the feature vector group to train the fine-grained classification model to determine the model parameters with the smallest value of the loss function and the smallest value of the objective function in the fine-grained classification model, and obtain the trained fine-grained classification model;

or,

The feature vector group is used to train the fine-grained classification model to determine the model parameters when the sum of the value of the loss function and the value of the objective function in the fine-grained classification model is the smallest, and the trained fine-grained classification model is obtained.

Optionally, the objective function is a function related to the feature vector of the first image, the feature vector of the second image, the feature vector of the third image and the third preset threshold.

In practical applications, the acquisition module 31 and the classification module 32 can be implemented by a processor located on the terminal, specifically a CPU, a microprocessor (MPU, Microprocessor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable Gate Array (FPGA, Field Programmable Gate Array) etc.

FIG. 4 is a second schematic structural diagram of a terminal provided by an embodiment of the present application. As shown in FIG. 4 , an embodiment of the present application provides a terminal 400, including:

The processor 41 and the storage medium 42 storing the executable instructions of the processor 41. The storage medium 42 relies on the processor 41 to perform operations through the communication bus 43. When the instructions are executed by the processor 41, Execute the image classification method described in the first embodiment.

It should be noted that, in practical application, various components in the terminal are coupled together through the communication bus 43 . It can be understood that the communication bus 43 is used to realize the connection communication between these components. In addition to the data bus, the communication bus 43 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the various buses are labeled as communication bus 43 in FIG. 4 .

An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image classification method according to the first embodiment. .

Wherein, the computer-readable storage medium may be a magnetic random access memory (ferromagnetic random access memory, FRAM), a read only memory (Read Only Memory, ROM), a programmable read only memory (Programmable Read-Only Memory, PROM), an erasable memory In addition to programmable read-only memory (ErasableProgrammable Read-Only Memory, EPROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), flash memory (Flash Memory), magnetic surface memory, optical disks, Or memory such as Compact Disc Read-Only Memory (CD-ROM).

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the protection scope of the present application.

Claims (10)

1. A method for classifying images is applied to a terminal and comprises the following steps:

acquiring an image to be classified;

classifying the images to be classified by adopting a pre-trained fine-grained classification model to obtain classified images;

the trained fine-grained classification model is obtained by adopting the following method:

grouping images of the image set to be trained according to the acquired image labels of the image set to be trained to obtain a grouped image set to be trained; wherein the image tag is used for characterizing the category of the image;

extracting a characteristic vector from the grouped images to be trained in the set to obtain a characteristic vector group;

and training the fine-grained classification model by adopting the feature vector group to determine a model parameter when the values of the loss function and the target function in the fine-grained classification model are minimum, so as to obtain the trained fine-grained classification model.

2. The method according to claim 1, wherein the grouping images of the image set to be trained according to the obtained image labels of the image set to be trained to obtain a grouped image set to be trained, comprises:

sequentially determining the images in the image set to be trained as first images;

selecting a second image and a third image from the image set to be trained except the first image aiming at the first image; wherein the image label of the second image is the same as the image label of the first image, and the image label of the third image is different from the image label of the first image;

forming a group by using the first image, the second image and the third image to obtain the grouped image set to be trained;

correspondingly, the extracting the feature vectors from the grouped images to be trained in the set to obtain a feature vector group includes:

extracting feature vectors of the first image, the second image and the third image by adopting the fine-grained classification model respectively to obtain the feature vector of the first image, the feature vector of the second image and the feature vector of the third image;

and forming the characteristic vector group by using the characteristic vector of the first image, the characteristic vector of the second image and the characteristic vector of the third image.

3. The method according to claim 2, wherein after extracting feature vectors from the grouped images in the image set to be trained to obtain a feature vector group, before training a fine-grained classification model by using the feature vector group to determine a model parameter when values of a loss function and an objective function in the fine-grained classification model are minimum to obtain the trained fine-grained classification model, the method further comprises:

selecting a group which does not meet a preset condition from the feature vector group;

and deleting the groups which do not meet the preset condition from the feature vector group so as to update the feature vector group.

4. The method of claim 3, wherein the selecting the group of the feature vector group that does not satisfy the predetermined condition comprises:

calculating a first distance value between the feature vector of the first image and the feature vector of the second image;

when the first distance value is larger than or equal to a first preset threshold value, determining a group containing the first image and the second image as the group which does not meet the preset condition, and selecting the group which does not meet the preset condition;

and/or the presence of a gas in the gas,

calculating a second distance value between the feature vector of the first image and the feature vector of the third image;

and when the second distance value is smaller than or equal to a second preset threshold value, determining the group comprising the first image and the third image as the group which does not meet the preset condition, and selecting the group which does not meet the preset condition.

5. The method of claim 3, wherein the selecting the group of the feature vector group that does not satisfy the predetermined condition comprises:

calculating a first distance value between the feature vector of the first image and the feature vector of the second image, and a second distance value between the feature vector of the first image and the feature vector of the third image;

and when the difference value between the second distance value and the first distance value is larger than a third preset threshold value, determining the group containing the first image, the second image and the third image as a group which does not meet a preset condition, and selecting the group which does not meet the preset condition.

6. The method of claim 1, wherein the training of the fine-grained classification model by using the feature vector group to determine a model parameter when values of a loss function and an objective function in the fine-grained classification model are minimum to obtain the trained fine-grained classification model comprises:

training a fine-grained classification model by adopting the feature vector group to determine a model parameter when a loss function value is minimum and a target function value is minimum in the fine-grained classification model, so as to obtain the trained fine-grained classification model;

or,

and training the fine-grained classification model by adopting the feature vector group to determine a model parameter when the sum of the value of the loss function and the value of the target function in the fine-grained classification model is minimum, so as to obtain the trained fine-grained classification model.

7. The method according to claim 5 or 6, wherein the objective function is a function related to the feature vector of the first image, the feature vector of the second image, the feature vector of the third image and the third preset threshold.

8. A terminal, comprising:

the acquisition module is used for acquiring an image to be classified;

the classification module is used for classifying the images to be classified by adopting a pre-trained fine-grained classification model to obtain classified images;

the trained fine-grained classification model is obtained by adopting the following method:

grouping images of the image set to be trained according to the acquired image labels of the image set to be trained to obtain a grouped image set to be trained; wherein the image tag is used for characterizing the category of the image;

extracting a characteristic vector from the grouped images to be trained in the set to obtain a characteristic vector group;

and training the fine-grained classification model by adopting the feature vector group to determine a model parameter when the values of the loss function and the target function in the fine-grained classification model are minimum, so as to obtain the trained fine-grained classification model.

9. A terminal, characterized in that the terminal comprises: a processor and a storage medium storing instructions executable by the processor to perform operations in dependence on the processor via a communication bus, the instructions when executed by the processor performing the method of classifying an image according to any one of claims 1 to 7.

10. A computer storage medium having stored thereon executable instructions which, when executed by one or more processors, perform the method of classifying an image of any one of claims 1 to 7.

Images