CN113688893A - Picture classification method and device, electronic equipment and storage medium - Google Patents
Picture classification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113688893A CN113688893A CN202110952977.3A CN202110952977A CN113688893A CN 113688893 A CN113688893 A CN 113688893A CN 202110952977 A CN202110952977 A CN 202110952977A CN 113688893 A CN113688893 A CN 113688893A
- Authority
- CN
- China
- Prior art keywords
- picture
- classified
- classification
- target
- trained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The embodiment of the invention discloses a method and a device for classifying pictures, electronic equipment and a storage medium. Wherein, the method comprises the following steps: acquiring an original picture to be classified, and converting the original picture to be classified into tensor data; tensor data of an original picture to be classified are input into a picture classification model trained in advance, and classification probabilities corresponding to the original picture to be classified under at least three preset picture categories are obtained; judging whether the original picture to be classified has a classification probability equal to or greater than a preset probability threshold, if so, determining the original picture to be classified as a target picture to be classified; determining a candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified, and judging whether the candidate picture category is a preset picture category to be confirmed; if yes, tensor data of the target picture to be classified are input into a pre-trained classifier, a target picture category of the target picture to be classified is obtained, and the picture classification precision is improved.
Description
Technical Field
The present invention relates to image processing technologies, and in particular, to a method and an apparatus for classifying pictures, an electronic device, and a storage medium.
Background
The picture classification can classify the pictures according to different characteristics reflected in the pictures so as to replace the visual interpretation of people.
Similar features exist among different types of pictures, the existing picture classification method adopts a neural network model to extract picture features, but the prior art cannot accurately identify the differences among the different types of pictures. For example, the picture categories are an elevation view, a plan view, and a cross-sectional view, and when picture classification is performed, the elevation view, the plan view, and the cross-sectional view are easily mistakenly classified with each other, which results in low picture classification accuracy.
Disclosure of Invention
The embodiment of the invention provides a method and a device for classifying pictures, electronic equipment and a storage medium, which are used for improving the precision of picture classification.
In a first aspect, an embodiment of the present invention provides a method for classifying pictures, where the method includes:
acquiring an original picture to be classified, and converting the original picture to be classified into tensor data;
inputting tensor data of the original picture to be classified into a picture classification model trained in advance to obtain classification probabilities corresponding to the original picture to be classified under at least three preset picture categories;
judging whether the original picture to be classified has a classification probability equal to or greater than a preset probability threshold, if so, determining the original picture to be classified as a target picture to be classified;
determining a candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified, and judging whether the candidate picture category is a preset picture category to be confirmed;
if so, inputting tensor data of the target picture to be classified into a pre-trained classifier to obtain a target picture category of the target classified picture.
In a second aspect, an embodiment of the present invention further provides an apparatus for classifying pictures, where the apparatus includes:
the data conversion module is used for acquiring an original picture to be classified and converting the original picture to be classified into tensor data;
the probability obtaining module is used for inputting tensor data of the original picture to be classified into a pre-trained picture classification model to obtain classification probabilities corresponding to the original picture to be classified under at least three preset picture categories;
the target picture determining module is used for judging whether the original picture to be classified has a classification probability which is equal to or greater than a preset probability threshold value, and if so, determining the original picture to be classified as the target picture to be classified;
the candidate class determining module is used for determining a candidate picture class of the target picture to be classified according to the classification probability of the target picture to be classified and judging whether the candidate picture class is a preset picture class to be confirmed;
and the target category classification module is used for inputting tensor data of the target picture to be classified into a pre-trained classifier to obtain a target picture category of the target classified picture if the target picture to be classified is the target picture.
In a third aspect, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the method for classifying pictures according to any embodiment of the present invention.
In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the method for picture classification according to any of the embodiments of the present invention.
According to the embodiment of the invention, tensor data are obtained by converting the original pictures to be classified. Tensor data are input into a pre-trained picture classification model, multiple picture categories are preset in the picture classification model, and classification probability that a picture to be classified belongs to each picture category can be output. And judging whether the classification probability is equal to or greater than a preset probability threshold, and if so, determining that the original picture to be classified is the target picture to be classified. According to the classification probability of the target picture to be classified, determining a candidate picture category to which the target picture to be classified possibly belongs, and judging whether the candidate picture category is a preset picture category to be confirmed, wherein the picture category to be confirmed is two picture categories with similar characteristics. If yes, tensor data of the target picture to be classified can be input into a pre-trained two-classifier, the two-classifier can only output classification probabilities of two picture categories, and the target picture category of the target classified picture is determined according to the classification probability. The problem of among the prior art, the picture classification that the characteristic is similar can't accurately distinguish is solved. And reducing the range of the original picture to be classified into a target picture to be classified by comparing the classification probability with a probability threshold. Through secondary classification, the precision of picture classification is effectively improved.
Drawings
Fig. 1 is a flowchart illustrating a method for classifying pictures according to a first embodiment of the present invention;
fig. 2 is a flowchart illustrating a method for classifying pictures according to a second embodiment of the present invention;
FIG. 3 is a block diagram of an apparatus for classifying pictures according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an apparatus for classifying pictures in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart illustrating a method for classifying pictures according to an embodiment of the present invention, where the method is applicable to classifying pictures and can be executed by a device for classifying pictures. As shown in fig. 1, the method specifically includes the following steps:
and 110, acquiring an original picture to be classified, and converting the original picture to be classified into tensor data.
The pre-collected picture to be classified is obtained and used as the original picture to be classified, for example, a building picture can be obtained. According to the resolution and the pixel value of the original picture to be classified, the original picture to be classified can be converted into tensor, and tensor data are obtained. Tensor data of an original picture to be classified can indicate the resolution, the pixel value of each pixel point and the like.
When the original picture to be classified is converted into tensor data, the original picture to be classified may be resized, and the original picture to be classified may be resized to a uniform preset size, for example, the preset size may be set to 224 × 224 or 224 × 320, and the preset size may be determined during training of the picture classification model. And determining the original picture to be classified after the size adjustment as the initial picture to be classified. The method can randomly horizontally cut the initial pictures to be classified, and take any one of the cut pictures. For example, the initial picture to be classified may be divided into two upper and lower blocks, and the upper block may be taken. And determining the initial to-be-classified picture obtained after cutting as a candidate to-be-classified picture, wherein the size of the candidate to-be-classified picture is consistent with that of the initial to-be-classified picture. The initial picture to be classified is not cut, or the part where the content to be classified is located in the initial picture to be classified is obtained by cutting, so that the influence of irrelevant content around the content to be recognized is reduced, and the classification precision is improved. And converting the initial picture to be classified or the candidate picture to be classified after cutting into tensor to obtain tensor data, wherein the obtained tensor data correspond to the original picture to be classified one by one.
And 120, inputting tensor data of the original picture to be classified into a pre-trained picture classification model to obtain classification probabilities corresponding to the original picture to be classified under at least three preset picture categories.
The image classification model is a pre-trained neural network model, for example, a residual neural network may be used as a network structure, and the model may include a full-connected layer and three convolutional layers. The image classification model can be provided with a plurality of image categories, and the image categories are more than or equal to three. The input data is tensor data of the picture to be classified, the output data is classification probability of the picture to be classified, and the classification probability is probability of possibility that the original picture to be classified belongs to various picture categories. For an original picture to be classified, the number of classification probabilities output by the picture classification model is the number of picture categories, and the sum of the classification probabilities of the original picture to be classified is 1. The higher the classification probability is, the higher the possibility that the picture belongs to the picture category corresponding to the classification probability is.
In this embodiment, the preset picture categories may include four categories, namely, an elevation view, a section view, a plan view, and a photograph. Therefore, for an original picture to be classified, the picture classification model can output four classification probabilities. For example, for an original picture to be classified, four classification probabilities are output, which are 5% of the classification probability of the elevation map, 10% of the classification probability of the cross-sectional map, 10% of the classification probability of the plan map, and 75% of the classification probability of the picture, so that the picture category of the original picture to be classified is the picture.
The method comprises the steps of presetting a probability threshold, and comparing each classification probability with the probability threshold after each classification probability of an original picture to be classified is obtained. And judging whether the original picture to be classified has at least one classification probability, wherein the value of the classification probability is greater than or equal to a preset probability threshold. And if so, determining the original picture to be classified as the target picture to be classified. After obtaining at least three classification probabilities of the original picture to be classified, determining a maximum probability value of the classification probabilities, that is, determining a maximum probability value of the at least three classification probabilities of the original picture to be classified. And judging whether the maximum probability value is equal to or greater than a preset probability threshold value or not, and comparing other classification probability values except the maximum probability value with the probability threshold value. And if the maximum probability value is equal to or greater than a preset probability threshold value, determining that the original picture to be classified is the target picture to be classified.
In this embodiment, optionally, after determining whether the original to-be-classified picture has a classification probability equal to or greater than a preset probability threshold, the method further includes: and if the classification probability which is equal to or greater than the preset probability threshold does not exist, determining that the original picture to be classified is an abnormal picture, and stopping classifying the abnormal picture.
Specifically, if there is no classification probability equal to or greater than a preset probability threshold in the classification probabilities of the original to-be-classified pictures, or the maximum probability value of the original to-be-classified pictures is smaller than the preset probability threshold, it is determined that the original to-be-classified pictures are not any of the preset picture categories, the original to-be-classified pictures are determined to be abnormal pictures, the original to-be-classified pictures are discarded, and the classification process of the original to-be-classified pictures is stopped. For example, if the preset probability threshold is 50%, there are four image categories, and the classification probabilities of the original image to be classified are 40%, 30%, 15%, and 15%, respectively, it is determined that the original image to be classified is an abnormal image. The method has the advantages that the original pictures to be classified which obviously do not belong to the preset picture category can be eliminated by comparing the classification probability with the probability threshold, so that the interference of abnormal pictures is avoided, and the picture classification precision is improved.
And step 140, determining a candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified, and judging whether the candidate picture category is a preset picture category to be confirmed.
The classification probabilities of the target pictures to be classified are compared, the classification probabilities are ranked from large to small, and the picture categories with the classification probabilities arranged in front are determined, for example, the picture categories corresponding to the classification probabilities arranged in the front two can be determined. And determining the picture category corresponding to the classification probability of the preset sorting position as a candidate picture category, for example, determining the picture category corresponding to the first two classification probabilities as a candidate picture category. That is, the picture categories corresponding to the classification probabilities of the first two bits may be the target picture categories of the target pictures to be classified.
One or more picture categories are preset as picture categories to be confirmed, for example, the picture categories include an elevation, a section, a plan and a photo, and the picture categories to be confirmed are the elevation and the section. The image categories with similar image characteristics can be used as the image categories to be confirmed, so that the image categories to be confirmed can be accurately judged conveniently, for example, similar characteristics exist between the elevation image and the section image, which easily causes mutual misclassification, and therefore, the elevation image and the section image are set as the image categories to be confirmed. And after the candidate picture categories of the target picture to be classified are obtained, determining whether the candidate picture categories are all picture categories to be confirmed. And if one picture type in the candidate picture types is not the picture type to be confirmed, the candidate picture type is not the preset picture type to be confirmed. For example, the picture category to be confirmed is a vertical view and a cross-sectional view, and the candidate picture category is a picture category corresponding to the classification probability of the first two sorted bits. And if the picture category corresponding to the first sorted classification probability is a section picture, and the picture category corresponding to the second sorted classification probability is an elevation picture, determining the candidate picture category as the picture category to be confirmed.
In this embodiment, optionally, determining the candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified includes: judging whether at least two classification probabilities in the classification probabilities of the target pictures to be classified are equal to or larger than a preset probability threshold; if so, determining a first picture category corresponding to the maximum classification probability in the classification probabilities of the target pictures to be classified and a second picture category corresponding to the second maximum classification probability; and determining the first picture category and the second picture category as candidate picture categories of the target picture to be classified.
Specifically, the picture category corresponding to the classification probability ranked at the preset rank may be determined as the candidate picture category, for example, the picture categories corresponding to the first two classification probabilities are used as the candidate picture categories. The candidate picture categories are at least two picture categories, and when the candidate picture categories are determined, the classification probability of the target picture to be classified can be compared with a preset probability threshold value. Whether at least two classification probabilities in the classification probabilities of the target pictures to be classified are equal to or greater than a preset probability threshold value or not can be judged, that is, whether the classification probabilities of the first two classification probabilities in the sequence are equal to or greater than the preset probability threshold value or not at least needs to be judged. It has been determined in step 130 that at least one of the classification probabilities of the target pictures to be classified is greater than or equal to the probability threshold, and thus, it can be determined whether the classification probability of the second bit of the target picture to be classified is equal to or greater than the probability threshold.
If only the first maximum classification probability is equal to or greater than the probability threshold, the candidate picture category does not need to be determined, and the picture category corresponding to the maximum classification probability is directly used as the target picture category of the target picture to be classified. If at least two classification probabilities of the target picture to be classified are equal to or greater than a preset probability threshold, namely the maximum classification probability and the second maximum classification probability are both certainly greater than or equal to the probability threshold, determining a first picture category corresponding to the maximum classification probability and a second picture category corresponding to the second maximum classification probability. In this embodiment, the picture categories to be confirmed are two picture categories, and therefore, only the picture categories with the first two classification probabilities can be determined. And determining the first picture category and the second picture category as candidate picture categories of the target pictures to be classified, and judging whether the candidate picture categories are all picture categories to be confirmed. The method has the advantages that the image categories to be confirmed are two, whether at least two classification probabilities in the classification probabilities are equal to or larger than a preset probability threshold value is judged, and if only one classification probability exists, the image category corresponding to the maximum classification probability is determined as the target image category no matter whether the image categories corresponding to the two classification probabilities are the image categories to be confirmed. Because the maximum value classification probability is the classification probability equal to or greater than the probability threshold, the image classification operation is reduced, and the image classification efficiency is improved. If at least two classification probabilities are equal to or greater than a preset probability threshold, the target picture category is further confirmed, and the picture classification precision is improved.
In this embodiment, optionally, after determining whether the candidate picture category is a preset picture category to be confirmed, the method further includes: if the candidate picture category is not the preset picture category to be confirmed, determining the maximum value classification probability of the target picture to be classified; and determining the picture category corresponding to the maximum classification probability as the target picture category of the target classification picture.
Specifically, the picture category to be confirmed is a picture category which is easily mistakenly classified with each other, if the candidate picture category is not the preset picture category to be confirmed, the maximum value classification probability in the classification probabilities of the target picture to be classified is directly determined, and the picture category corresponding to the maximum value classification probability is determined as the target picture category of the target classified picture. For example, the image category to be confirmed is an elevation and a section, the candidate image category is an elevation and a plan, the classification probability corresponding to the elevation is higher than that of the plan, and the possibility of the elevation being mistaken with the plan is low, so that the elevation can be directly used as the target image category. The beneficial effect of the setting is that when the candidate picture category is not the picture category to be confirmed, the target picture category can be directly determined, and the picture classification efficiency is improved.
And 150, if so, inputting tensor data of the target picture to be classified into a pre-trained classifier to obtain a target picture category of the target classified picture.
After determining that the candidate picture category is the preset picture category to be confirmed, if the picture category corresponding to the maximum value classification probability is taken as the target picture category of the target picture to be classified, a situation of picture classification error may occur, that is, the true target picture category may be the picture category corresponding to the second maximum value classification probability. Therefore, it is necessary to confirm again the candidate picture type which is the picture type to be confirmed. Tensor data of the target picture to be classified can be input into a pre-trained two-classifier, the network structures of the two-classifier and the picture classification model can be consistent, the difference between the two classifiers is that the picture classification model can output classification probabilities of at least three picture classes, the two-classifier can output classification probabilities of two picture classes, and the picture class corresponding to the classification probability output by the two-classifier is the picture class to be confirmed. And after receiving tensor data of the target picture to be classified, the two classifiers can output the classification probability of the target picture to be classified to two picture categories to be confirmed, compare the two classification probabilities, determine the picture category corresponding to the larger classification probability as the target picture category, and finish the picture classification process.
According to the technical scheme of the embodiment, tensor data are obtained by converting the original pictures to be classified. Tensor data are input into a pre-trained picture classification model, multiple picture categories are preset in the picture classification model, and classification probability that a picture to be classified belongs to each picture category can be output. And judging whether the classification probability is equal to or greater than a preset probability threshold, and if so, determining that the original picture to be classified is the target picture to be classified. According to the classification probability of the target picture to be classified, determining a candidate picture category to which the target picture to be classified possibly belongs, and judging whether the candidate picture category is a preset picture category to be confirmed, wherein the picture category to be confirmed is two picture categories with similar characteristics. If yes, tensor data of the target picture to be classified can be input into a pre-trained two-classifier, the two-classifier can only output classification probabilities of two picture categories, and the target picture category of the target classified picture is determined according to the classification probability. The problem of among the prior art, the picture classification that the characteristic is similar can't accurately distinguish is solved. And reducing the range of the original picture to be classified into a target picture to be classified by comparing the classification probability with a probability threshold. Through secondary classification, the precision of picture classification is effectively improved.
Example two
Fig. 2 is a flowchart illustrating a method for classifying pictures according to a second embodiment of the present invention, which is further optimized based on the second embodiment. As shown in fig. 2, the method specifically includes the following steps:
The method includes acquiring an original picture to be trained, for example, acquiring building pictures of different categories in advance, where the category of the picture may include a facade picture, a profile picture, a plan picture, and a photograph. Each original picture to be trained is converted into tensor data, and the tensor data can represent information such as resolution size and pixel value size of the original picture to be trained.
In this embodiment, optionally, converting the original to-be-trained picture into tensor data includes: adjusting the size of an original picture to be trained to a preset size to obtain an initial picture to be trained; randomly and horizontally cutting the initial picture to be trained, and determining the cut initial picture to be trained as a candidate picture to be trained; and converting the candidate to-be-trained picture into a tensor to obtain tensor data.
Specifically, because the size of the original to-be-trained picture is too large, if the original to-be-trained picture is directly input into the model, a huge memory usage rate will be generated, and therefore, when the original to-be-trained picture is converted into tensor data, the size of the original to-be-trained picture may be adjusted first, and the original to-be-trained picture is adjusted to a uniform preset size, for example, the preset size may be set to 224 × 224 or 224 × 320. In this embodiment, the preset size may be determined to be 224 × 320. And determining the original picture to be trained after the size is adjusted as the initial picture to be trained. The initial picture to be trained can be randomly and horizontally cut, and any cut picture is taken as a candidate picture to be trained. For example, the initial picture to be trained may be divided into two upper and lower blocks, and the upper block may be taken. And determining the initial picture to be trained obtained after cutting as a candidate picture to be trained, wherein the size of the candidate picture to be trained is consistent with that of the initial picture to be trained. And converting the cut candidate to-be-trained pictures into tensors to obtain tensor data, wherein the obtained tensor data correspond to the original to-be-trained pictures one by one. The beneficial effect who sets up like this lies in, reduces the memory rate of utilization, through the operation of cutting out of level, improves the training precision, and then improves picture classification's efficiency and precision.
And step 220, inputting tensor data of the original picture to be trained into the picture classification model to be trained to obtain an output result.
The image classification model to be trained may use Resnet101 (residual neural network) as a network structure, and include a full connection layer and three convolutional layers. The image classification model can output the classification probability that the original image to be trained is any image category, determine the maximum classification probability, and take the image category corresponding to the maximum classification probability as an output result.
In this embodiment, optionally, the inputting the tensor data of the original to-be-trained picture into the to-be-trained picture classification model includes: randomly extracting at least two candidate pictures to be trained, and fusing the extracted candidate pictures to be trained to obtain a target picture to be trained; converting a target picture to be trained into tensor data; and inputting tensor data of the candidate to-be-trained picture and tensor data of the target to-be-trained picture into the to-be-trained picture classification model.
Specifically, due to the limitation of computational power, the number of pictures input into the picture classification model, that is, the batch size, may be set to 80, for example, tensor data of 80 original pictures to be trained may be input at a time. The data may be processed in advance using the mix up method before entering the data into the model. The Mix up method is to fuse two candidate images to be trained, for example, the two candidate images to be trained may be randomly extracted, the transparency of the candidate images to be trained is reduced, and the extracted candidate images to be trained are fused to obtain a target image to be trained. Namely, the target picture to be trained has the characteristics of two candidate pictures to be trained. The extracted candidate pictures to be trained can be two different picture types, and a plurality of groups of candidate pictures to be trained can be extracted, wherein each group of candidate pictures to be trained comprises two pictures. After the target picture to be trained is obtained, tensor data of the target picture to be trained can be input into the picture classification model. The method has the advantages that the smoothness of the neighborhood can be improved through the mix up method, so that some error-prone samples are artificially created, a model learns a better feature space, and the image classification precision is improved.
And step 230, judging whether the output result is consistent with the preset target result, and if not, inputting the output result and the preset target result into a preset loss function.
After the output result is obtained, the output result is compared with a preset target result, whether the output result is consistent with the preset target result or not is judged, and if yes, the training of the current round is determined to be successful. And if the two results are not consistent, inputting the output result and a preset target result into a preset loss function. The preset target result is the picture category of each original picture to be trained, which is determined when the original picture to be trained is obtained. The loss function may be Cross entropy loss, Focal loss, or Label smoothing. The Label smoothening is used as a loss function, so that cluster between classes can be more compact, the distance between the classes is increased, the distance in the classes is reduced, and the picture classification precision is improved.
And 240, updating the neural network parameters of the image classification model according to the calculation result of the loss function so as to train the image classification model.
The neural network parameters of the training of the current round can be calculated by back propagation according to the calculation result of the loss function, for example, the neural network parameters can be convolutional layer parameters. And updating parameters of the image classification model until the loss function value meets the preset training requirement, and finishing the training of the image classification model. In this embodiment, the preset size of the picture classification model is 224 × 320, the loss function is Label Smoothing, the optimization function may be Adam (Adaptive moment estimation) or SGD + Momentum (random gradient descent algorithm + Momentum), and SGD + Momentum is preferentially adopted.
And 250, training the two classifiers according to a preset two classifier training rule.
The training mode of the two classifiers is consistent with that of the image classification model, only two image categories are identified by the two classifiers, and the two classifiers can be specially used for training image categories with similar characteristics. For example, the probability of the elevation and the profile being mutually mistaken is high, and therefore, a two-classifier can be trained for the elevation and the profile separately. The training of the two classifiers can be performed before or after the training of the image classifier, and in this embodiment, the training sequence of the two models is not limited.
The training process of the two classifiers comprises the steps of obtaining an original to-be-trained picture of the two classifiers and converting the original to-be-trained picture into tensor data. And inputting tensor data of the original picture to be trained into a classifier model to be trained to obtain an output result. And judging whether the output result is consistent with a preset target result, if not, inputting the output result and the preset target result into a preset loss function. And updating the neural network parameters of the two classifiers according to the calculation result of the loss function so as to train the two classifiers. The preset size of the two classifiers to the picture is 224 multiplied by 320, the loss function is Label smoothening, and the optimization function is SGD + Momentum. When data is input into the binary classifier, the data can be input in batch.
And step 260, acquiring an original picture to be classified, and converting the original picture to be classified into tensor data.
And step 211, if yes, inputting tensor data of the target picture to be classified into a pre-trained classifier to obtain a target picture category of the target classified picture.
The embodiment of the invention trains the image classification model and the two classifiers, converts the original image to be classified and obtains tensor data. Tensor data are input into a pre-trained picture classification model, multiple picture categories are preset in the picture classification model, and classification probability that a picture to be classified belongs to each picture category can be output. And judging whether the classification probability is equal to or greater than a preset probability threshold, and if so, determining that the original picture to be classified is the target picture to be classified. According to the classification probability of the target picture to be classified, determining a candidate picture category to which the target picture to be classified possibly belongs, and judging whether the candidate picture category is a preset picture category to be confirmed, wherein the picture category to be confirmed is two picture categories with similar characteristics. If yes, tensor data of the target picture to be classified can be input into a pre-trained two-classifier, the two-classifier can only output classification probabilities of two picture categories, and the target picture category of the target classified picture is determined according to the classification probability. The problem of among the prior art, the picture classification that the characteristic is similar can't accurately distinguish is solved. And reducing the range of the original picture to be classified into a target picture to be classified by comparing the classification probability with a probability threshold. Through secondary classification, the precision of picture classification is effectively improved.
EXAMPLE III
Fig. 3 is a block diagram of a device for classifying pictures according to a third embodiment of the present invention, which is capable of executing a method for classifying pictures according to any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 3, the apparatus specifically includes:
the data conversion module 301 is configured to acquire an original to-be-classified picture and convert the original to-be-classified picture into tensor data;
a probability obtaining module 302, configured to input tensor data of the original to-be-classified picture into a pre-trained picture classification model, so as to obtain classification probabilities corresponding to the original to-be-classified picture under at least three preset picture categories;
a target picture determining module 303, configured to determine whether the original to-be-classified picture has a classification probability equal to or greater than a preset probability threshold, and if so, determine that the original to-be-classified picture is a target to-be-classified picture;
a candidate category determining module 304, configured to determine a candidate image category of the target to-be-classified image according to the classification probability of the target to-be-classified image, and determine whether the candidate image category is a preset to-be-confirmed image category;
and a two-classifier classification module 305, configured to, if yes, input tensor data of the target to-be-classified picture into a pre-trained two-classifier to obtain a target picture category of the target classified picture.
Optionally, the apparatus further comprises:
the image to be trained conversion module is used for acquiring an original image to be classified, converting the original image to be classified into tensor data, and converting the original image to be trained into the tensor data;
the to-be-trained result output module is used for inputting tensor data of the original to-be-trained picture into a to-be-trained picture classification model to obtain an output result;
the loss function calculation module is used for judging whether the output result is consistent with a preset target result or not, and if not, inputting the output result and the preset target result into a preset loss function;
and the model updating module is used for updating the neural network parameters of the image classification model according to the calculation result of the loss function so as to train the image classification model.
Optionally, the to-be-trained image conversion module is specifically configured to:
adjusting the size of the original picture to be trained to a preset size to obtain an initial picture to be trained;
randomly and horizontally cutting the initial picture to be trained, and determining the cut initial picture to be trained as a candidate picture to be trained;
and converting the candidate to-be-trained picture into a tensor to obtain tensor data.
Optionally, the to-be-trained result output module is specifically configured to:
randomly extracting at least two candidate pictures to be trained, and fusing the extracted candidate pictures to be trained to obtain a target picture to be trained;
converting the target picture to be trained into tensor data;
and inputting the tensor data of the candidate to-be-trained picture and the tensor data of the target to-be-trained picture into a to-be-trained picture classification model.
Optionally, the apparatus further comprises:
and the abnormal picture determining module is used for determining that the original picture to be classified is an abnormal picture and stopping classifying the abnormal picture if the original picture to be classified has no classification probability which is equal to or greater than a preset probability threshold after judging whether the original picture to be classified has the classification probability which is equal to or greater than the preset probability threshold.
Optionally, the candidate category determining module 304 is specifically configured to:
judging whether at least two classification probabilities in the classification probabilities of the target pictures to be classified are equal to or larger than a preset probability threshold value or not;
if so, determining a first picture category corresponding to the maximum classification probability in the classification probabilities of the target pictures to be classified and a second picture category corresponding to the second maximum classification probability;
and determining the first picture category and the second picture category as candidate picture categories of the target picture to be classified.
Optionally, the apparatus further comprises:
the maximum value determining module is used for determining the maximum value classification probability of the target picture to be classified if the candidate picture category is not the preset picture category to be confirmed after judging whether the candidate picture category is the preset picture category to be confirmed;
and the category determining module is used for determining the picture category corresponding to the maximum classification probability as the target picture category of the target classified picture.
According to the embodiment of the invention, tensor data are obtained by converting the original pictures to be classified. Tensor data are input into a pre-trained picture classification model, multiple picture categories are preset in the picture classification model, and classification probability that a picture to be classified belongs to each picture category can be output. And judging whether the classification probability is equal to or greater than a preset probability threshold, and if so, determining that the original picture to be classified is the target picture to be classified. According to the classification probability of the target picture to be classified, determining a candidate picture category to which the target picture to be classified possibly belongs, and judging whether the candidate picture category is a preset picture category to be confirmed, wherein the picture category to be confirmed is two picture categories with similar characteristics. If yes, tensor data of the target picture to be classified can be input into a pre-trained two-classifier, the two-classifier can only output classification probabilities of two picture categories, and the target picture category of the target classified picture is determined according to the classification probability. The problem of among the prior art, the picture classification that the characteristic is similar can't accurately distinguish is solved. And reducing the range of the original picture to be classified into a target picture to be classified by comparing the classification probability with a probability threshold. Through secondary classification, the precision of picture classification is effectively improved.
Example four
Fig. 4 is a schematic structural diagram of an apparatus for classifying pictures according to a fourth embodiment of the present invention. The device for picture classification is an electronic device and fig. 4 shows a block diagram of an exemplary electronic device 400 suitable for use in implementing an embodiment of the invention. The electronic device 400 shown in fig. 4 is only an example and should not bring any limitation to the function and the scope of use of the embodiments of the present invention.
As shown in fig. 4, electronic device 400 is embodied in the form of a general purpose computing device. The components of electronic device 400 may include, but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 that couples the various system components (including the system memory 402 and the processing unit 401).
The system memory 402 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)404 and/or cache memory 405. The electronic device 400 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 406 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 403 by one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 407 generally perform the functions and/or methods of the described embodiments of the invention.
The electronic device 400 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), with one or more devices that enable a user to interact with the electronic device 400, and/or with any devices (e.g., network card, modem, etc.) that enable the electronic device 400 to communicate with one or more other computing devices. Such communication may be through input/output (I/O) interface 411. Also, the electronic device 400 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 412. As shown in FIG. 4, the network adapter 412 communicates with the other modules of the electronic device 400 over the bus 403. It should be appreciated that although not shown in FIG. 4, other hardware and/or software modules may be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 401 executes various functional applications and data processing by running a program stored in the system memory 402, for example, implementing a method for classifying pictures provided by the embodiment of the present invention, including:
acquiring an original picture to be classified, and converting the original picture to be classified into tensor data;
inputting tensor data of the original picture to be classified into a picture classification model trained in advance to obtain classification probabilities corresponding to the original picture to be classified under at least three preset picture categories;
judging whether the original picture to be classified has a classification probability equal to or greater than a preset probability threshold, if so, determining the original picture to be classified as a target picture to be classified;
determining a candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified, and judging whether the candidate picture category is a preset picture category to be confirmed;
if so, inputting tensor data of the target picture to be classified into a pre-trained classifier to obtain a target picture category of the target classified picture.
EXAMPLE five
The fifth embodiment of the present invention further provides a storage medium containing computer-executable instructions, where the storage medium stores a computer program, and when the computer program is executed by a processor, the method for classifying pictures provided in the fifth embodiment of the present invention is implemented, where the method includes:
acquiring an original picture to be classified, and converting the original picture to be classified into tensor data;
inputting tensor data of the original picture to be classified into a picture classification model trained in advance to obtain classification probabilities corresponding to the original picture to be classified under at least three preset picture categories;
judging whether the original picture to be classified has a classification probability equal to or greater than a preset probability threshold, if so, determining the original picture to be classified as a target picture to be classified;
determining a candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified, and judging whether the candidate picture category is a preset picture category to be confirmed;
if so, inputting tensor data of the target picture to be classified into a pre-trained classifier to obtain a target picture category of the target classified picture.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method for classifying pictures, comprising:
acquiring an original picture to be classified, and converting the original picture to be classified into tensor data;
inputting tensor data of the original picture to be classified into a picture classification model trained in advance to obtain classification probabilities corresponding to the original picture to be classified under at least three preset picture categories;
judging whether the original picture to be classified has a classification probability equal to or greater than a preset probability threshold, if so, determining the original picture to be classified as a target picture to be classified;
determining a candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified, and judging whether the candidate picture category is a preset picture category to be confirmed;
if so, inputting tensor data of the target picture to be classified into a pre-trained classifier to obtain a target picture category of the target classified picture.
2. The method according to claim 1, before obtaining an original picture to be classified and converting the original picture to be classified into tensor data, further comprising:
acquiring an original picture to be trained, and converting the original picture to be trained into tensor data;
inputting tensor data of the original picture to be trained into a picture classification model to be trained to obtain an output result;
judging whether the output result is consistent with a preset target result, if not, inputting the output result and the preset target result into a preset loss function;
and updating the neural network parameters of the image classification model according to the calculation result of the loss function so as to train the image classification model.
3. The method of claim 2, wherein converting the original picture to be trained into tensor data comprises:
adjusting the size of the original picture to be trained to a preset size to obtain an initial picture to be trained;
randomly and horizontally cutting the initial picture to be trained, and determining the cut initial picture to be trained as a candidate picture to be trained;
and converting the candidate to-be-trained picture into a tensor to obtain tensor data.
4. The method of claim 3, wherein inputting tensor data of the original picture to be trained into a picture classification model to be trained comprises:
randomly extracting at least two candidate pictures to be trained, and fusing the extracted candidate pictures to be trained to obtain a target picture to be trained;
converting the target picture to be trained into tensor data;
and inputting the tensor data of the candidate to-be-trained picture and the tensor data of the target to-be-trained picture into a to-be-trained picture classification model.
5. The method according to claim 1, after determining whether the original picture to be classified has a classification probability equal to or greater than a preset probability threshold, further comprising:
and if the classification probability which is equal to or greater than the preset probability threshold does not exist, determining that the original picture to be classified is an abnormal picture, and stopping classifying the abnormal picture.
6. The method according to claim 1, wherein determining the candidate picture category of the target picture to be classified according to the classification probability of the target picture to be classified comprises:
judging whether at least two classification probabilities in the classification probabilities of the target pictures to be classified are equal to or larger than a preset probability threshold value or not;
if so, determining a first picture category corresponding to the maximum classification probability in the classification probabilities of the target pictures to be classified and a second picture category corresponding to the second maximum classification probability;
and determining the first picture category and the second picture category as candidate picture categories of the target picture to be classified.
7. The method according to claim 1, further comprising, after determining whether the candidate picture category is a preset to-be-confirmed picture category:
if the candidate picture category is not the preset picture category to be confirmed, determining the maximum value classification probability of the target picture to be classified;
and determining the picture category corresponding to the maximum classification probability as the target picture category of the target classification picture.
8. An apparatus for classifying pictures, comprising:
the data conversion module is used for acquiring an original picture to be classified and converting the original picture to be classified into tensor data;
the probability obtaining module is used for inputting tensor data of the original picture to be classified into a pre-trained picture classification model to obtain classification probabilities corresponding to the original picture to be classified under at least three preset picture categories;
the target picture determining module is used for judging whether the original picture to be classified has a classification probability which is equal to or greater than a preset probability threshold value, and if so, determining the original picture to be classified as the target picture to be classified;
the candidate class determining module is used for determining a candidate picture class of the target picture to be classified according to the classification probability of the target picture to be classified and judging whether the candidate picture class is a preset picture class to be confirmed;
and the two-classifier classification module is used for inputting tensor data of the target picture to be classified into two pre-trained classifiers if the target picture to be classified is in the positive state, so as to obtain a target picture category of the target classified picture.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of picture classification as claimed in any one of claims 1 to 7 when executing the program.
10. A storage medium containing computer-executable instructions for performing the method of picture classification as claimed in any one of claims 1-7 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110952977.3A CN113688893A (en) | 2021-08-19 | 2021-08-19 | Picture classification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110952977.3A CN113688893A (en) | 2021-08-19 | 2021-08-19 | Picture classification method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113688893A true CN113688893A (en) | 2021-11-23 |
Family
ID=78580945
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110952977.3A Pending CN113688893A (en) | 2021-08-19 | 2021-08-19 | Picture classification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688893A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108665457A (en) * | 2018-05-16 | 2018-10-16 | 腾讯科技（深圳）有限公司 | Image-recognizing method, device, storage medium and computer equipment |
| CN109242042A (en) * | 2018-09-28 | 2019-01-18 | 百度在线网络技术（北京）有限公司 | Picture training sample method for digging, device, terminal and computer readable storage medium |
| CN109299715A (en) * | 2017-07-24 | 2019-02-01 | 图灵通诺（北京）科技有限公司 | The settlement method and device of image recognition technology based on convolutional neural networks |
| US20190073588A1 (en) * | 2017-09-06 | 2019-03-07 | Panasonic Intellectual Property Management Co., Ltd. | Learning apparatus, learning method, estimation apparatus, estimation method, and computer-readable storage medium |
| CN111104832A (en) * | 2018-10-29 | 2020-05-05 | 百度在线网络技术（北京）有限公司 | Image tag acquisition method, image tag acquisition device and electronic equipment |
-
2021
- 2021-08-19 CN CN202110952977.3A patent/CN113688893A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109299715A (en) * | 2017-07-24 | 2019-02-01 | 图灵通诺（北京）科技有限公司 | The settlement method and device of image recognition technology based on convolutional neural networks |
| US20190073588A1 (en) * | 2017-09-06 | 2019-03-07 | Panasonic Intellectual Property Management Co., Ltd. | Learning apparatus, learning method, estimation apparatus, estimation method, and computer-readable storage medium |
| CN108665457A (en) * | 2018-05-16 | 2018-10-16 | 腾讯科技（深圳）有限公司 | Image-recognizing method, device, storage medium and computer equipment |
| CN109242042A (en) * | 2018-09-28 | 2019-01-18 | 百度在线网络技术（北京）有限公司 | Picture training sample method for digging, device, terminal and computer readable storage medium |
| CN111104832A (en) * | 2018-10-29 | 2020-05-05 | 百度在线网络技术（北京）有限公司 | Image tag acquisition method, image tag acquisition device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 刘芳;刘玉坤;林森;郭文忠;徐凡;张白;: "基于改进型YOLO的复杂环境下番茄果实快速识别方法", 农业机械学报, no. 06 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11610384B2 (en) | Zero-shot object detection | |
| US10832096B2 (en) | Representative-based metric learning for classification and few-shot object detection | |
| CN109583332B (en) | Face recognition method, face recognition system, medium, and electronic device | |
| CN110472675B (en) | Image classification method, image classification device, storage medium and electronic equipment | |
| KR20210134528A (en) | Video processing method, apparatus, electronic device and storage medium and computer program | |
| US11804069B2 (en) | Image clustering method and apparatus, and storage medium | |
| WO2020005731A1 (en) | Text entity detection and recognition from images | |
| CN111368878B (en) | Optimization method based on SSD target detection, computer equipment and medium | |
| CN112906823B (en) | Target object recognition model training method, recognition method and recognition device | |
| CN113657274B (en) | Table generation method and device, electronic equipment and storage medium | |
| CN115578735B (en) | Text detection method and training method and device of text detection model | |
| CN114863229A (en) | Image classification method and training method and device of image classification model | |
| US20230096921A1 (en) | Image recognition method and apparatus, electronic device and readable storage medium | |
| EP4273737A1 (en) | Language labeling method and apparatus, and computer device and storage medium | |
| CN114495113A (en) | Text classification method and training method and device of text classification model | |
| CN113902899A (en) | Training method, target detection method, device, electronic device and storage medium | |
| CN113239883A (en) | Method and device for training classification model, electronic equipment and storage medium | |
| CN113223011A (en) | Small sample image segmentation method based on guide network and full-connection conditional random field | |
| CN111626291A (en) | Image visual relationship detection method, system and terminal | |
| CN113762455A (en) | Detection model training method, single character detection method, device, equipment and medium | |
| US9378466B2 (en) | Data reduction in nearest neighbor classification | |
| CN113688893A (en) | Picture classification method and device, electronic equipment and storage medium | |
| CN114419327B (en) | Image detection method and training method and device of image detection model | |
| CN111738290B (en) | Image detection method, model construction and training method, device, equipment and medium | |
| CN115170919A (en) | Image processing model training method, image processing device, image processing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |