CN116630694A - A target classification method, system and electronic equipment for more marked images - Google Patents

Abstract

The invention provides an object classification method, system and electronic equipment for an over-marked image, and relates to the technical field of object classification for an over-marked image. The method includes acquiring an image of many markers to be identified; inputting the image of many markers to be identified into a related marker determination model, determining the related markers of all targets in the image of many markers to be identified; determining the target to be identified according to a plurality of related markers Mostly label the types of all objects in the image; the relevant labels correspond to the object types one-to-one. The invention utilizes the principle of contrastive label disambiguation to train a classifier to obtain a multi-label determination model capable of accurately identifying relevant labels in unseen images, thereby improving the accuracy of image classification.

Description

A target classification method, system and electronic equipment for more marked images

technical field

The present invention relates to the technical field of object classification of overly marked images, in particular to an object classification method, system and electronic equipment of overly marked images.

Background technique

Multi-label image classification aims to deal with image classification problems, where each image is associated with multiple label information. At present, the problem of multi-label image classification has received extensive attention. However, multi-label image classification relies on accurate labeling of data, which is extremely difficult to achieve in real-world scenarios with limited resources. In order to alleviate the labeling pressure, the current approach is to assign multiple candidate labels to each image by non-professional labelers. The candidate labels include not only relevant labels that are beneficial to image classification, but also some noise labels. Learning from such a set of candidate labels for images is defined as an overly labeled image object classification problem.

There are two types of multi-label learning problems: the first type is to assign a confidence level to each candidate label, and iteratively update the label confidence level and classification model parameters during training. For example, the paper "Partial Multi-Label Learning" proposes to optimize the label ranking confidence matrix by considering label correlation and feature prototypes separately during the process of training classifiers. The paper "Feature-Induced Partial Multi-label Learning" introduces a feature-induced partial multi-label method by considering the low-rank characteristics of the label and feature space. The paper "Partial Multi-Label Learning with Meta Disambiguation" proposes to iteratively minimize the confidence-weighted ranking loss and use the performance of the model on the validation set to adaptively estimate the confidence of each candidate label to achieve the purpose of disambiguation. The second type is a two-stage training method, which obtains reliable labels from a set of candidate labels, and then uses these reliable labels to train a multi-label classifier. For example, the paper "Discriminative and Correlative Partial Multi-Label Learning" uses feature manifolds to induce high-confidence labels, and then trains multi-label classifiers. The paper "PartialMulti-Label Learning via Credible Label Elimination" induces classifiers by extracting credible labels using an iterative label propagation strategy. In terms of patents, the Chinese Invention Patent Application No. 202010412162.1 provides a multi-label learning method based on multi-subspace representation; the Chinese Invention Patent Application No. 202010412161.7 provides a multi-label learning method based on noise tolerance; The Chinese invention patent application with application number 202111369388.9 provides a patient screening label method based on multi-label learning; the Chinese invention patent application with application number 202010411579.6 provides a multi-label learning method based on global and local label relationships; the application number is The Chinese invention patent application of 202110717550.5 provides a multi-label learning method based on complementary label collaborative training; the Chinese invention patent application of application number 202010411580.9 provides a multi-label learning method with noise in the feature information, using low rank and sparse The idea of decomposition restores the correct feature information and effectively reduces the influence of noise feature information. Although these traditional methods have made significant progress, they generally learn based on manual features. When faced with the target classification problem of more labeled images, their representation ability and label correction ability are weak, and they cannot achieve good label disambiguation effect.

Contents of the invention

The purpose of the present invention is to provide a target classification method, system and electronic equipment for over-marked images, using the principle of contrastive label disambiguation to train the classifier to obtain related tags that can accurately identify related tags in unseen over-marked images Determine the model to improve the accuracy of image classification.

To achieve the above object, the present invention provides the following scheme:

An object classification method for overly labeled images, comprising:

Acquiring an image with many marks to be identified; the image with many marks to be identified includes at least one target;

Input the image of many markers to be identified into the relevant marker determination model, and determine the relevant markers of all targets in the image of multiple markers to be identified; The label disambiguation principle is used to train the classifier;

The types of all targets in the image with more tags to be identified are determined according to the plurality of related tags; the related tags correspond to the target types one by one.

Optionally, before obtaining the image to be classified, it also includes:

Acquiring multiple over-marked historical images; the over-marked historical images are marked with various marks; the type of the mark is a relevant mark or a noise mark;

Carrying out random data augmentation processing on the historical image with too many marks to obtain query view and key view of the historical image with too many marks;

Determine the tag-level embedding under the query view and the tag-level embedding under the key view; the tag-level embedding is in one-to-one correspondence with the multiple tags on the historical image with more tags;

According to the tag-level embedding under the query view and the tag-level embedding under the key view, the classifier is trained using the principle of contrastive tag disambiguation to obtain the relevant tag determination model.

Optionally, after determining the tag-level embedding under the query view and the tag-level embedding under the key view, further include:

Determine the sign of multiple tag-level embeddings under the query view;

Determines the signability of multiple label-level embeddings under the key view.

Optionally, according to the label-level embedding under the query view and the label-level embedding under the key view, the classifier is trained using the principle of contrastive label disambiguation to obtain the relevant label determination model, including:

Determine that the classifier is the classifier at the 0th iteration;

Obtain the initial positive type of each mark in the classifier as the positive type at the 0th iteration;

Obtain the initial negative prototype of each label in the classifier as the negative prototype of the 0th iteration;

Let the first iteration number i=1;

Let the second iteration number j=1;

Determine any tag-level embedding under any query-based view as the current tag-level embedding;

According to the positive and negative of the current label-level embedding, update the positive prototype at the i-1th iteration and the negative prototype at the i-1th iteration;

Calculate the similarity between the current tag-level embedding and the positive prototype at the i-1th iteration after the update as the first similarity;

Calculate the similarity between the current label-level embedding and the negative prototype at the i-1th iteration after the update as the second similarity;

According to the first similarity and the second similarity, determine the current label-level embedding according to the label vector predicted by the prototype;

Updating the pseudo-label embedded with the corresponding label at the current label level according to the label vector, and obtaining the pseudo-label embedded with the corresponding label at the current label level during the jth iteration;

Increase the value of the second iteration number j by 1, update the current tag-level embedding to a current tag-level embedding other than the current tag-level embedding under the same query view, and return to the step "according to the positive or negative of the current tag-level embedding, update The positive prototype at the i-1th iteration and the negative prototype at the i-1th iteration" until the second iteration number reaches the second iteration number threshold;

Embed multiple current label levels corresponding to the query view into the classifier at the i-1th iteration to obtain multiple category outputs;

Determine the classification loss function at the i-1th iteration according to a plurality of the category outputs and the pseudo-labels embedded in the corresponding labels at the current label level during multiple iterations;

Judging whether the classification loss function is less than a classification loss function threshold, and obtaining a first judgment result;

If the first judgment result is no, update the parameters of the classifier at the i-1th iteration to obtain the classifier at the ith iteration, increase the value of the first iteration number i by 1, and return to step " Let the second iteration times j=1";

If the first judgment result is yes, it is judged whether the first iteration number reaches the first iteration number threshold, and a second judgment result is obtained;

If the second judgment result is no, then it is determined that the classifier at the i-1th iteration is the classifier at the ith iteration, the value of the first iteration number i is increased by 1, and the step "make the second The number of iterations j = 1";

If the second judgment result is yes, the classifier at the i-1th iteration is determined to be the relevant marker determination model.

Optionally, before determining the classifier for the i-1th iteration and determining the model for the relevant markers, the method also includes:

Determine the tag-level embedding under the query view and the tag-level embedding under the key view as an embedding pool; the embedding pool also includes the tag-level embedding in the momentum tag-level embedding queue;

Determine any tag-level embedding with a positive positive or negative in the query view as the current positive tag-level embedding;

Determining that the positive label-level embedding corresponding to the same label as the positive label-level embedding in the embedding pool is the positive sample set corresponding to the current positive label-level embedding;

Determining that the current positive label-level embedding and the samples in the positive sample set corresponding to the current positive label-level embedding constitute a plurality of positive sample pairs;

Determine the comparative loss function corresponding to more than one marked historical image based on multiple positive sample pairs under the same query view;

Judging whether a plurality of the comparison loss functions are less than the comparison loss function threshold to obtain a third judgment result;

If the third judgment result is no, update the parameters of the classifier at the i-1th iteration to obtain the classifier at the 0th iteration, and return to the step "make the first iteration number i=1";

If the third judgment result is yes, the step of "determining the classifier at the i-1th iteration as the model for the relevant markers" is invoked.

An object classification system for mostly labeled images, comprising:

An image acquisition module with many marks to be identified is used to acquire an image with many marks to be identified; the image with many marks to be identified includes at least one target;

The relevant mark identification module is used to input the image of many marks to be identified into the relevant mark determination model, and determine the relevant marks of all targets in the image of many marks to be recognized; the relevant mark determination model is based on multiple partial Multi-label historical images, obtained by training the classifier using the principle of contrastive label disambiguation;

The target type determination module is used to determine the types of all targets in the image with more tags to be identified according to the plurality of related marks; the related marks correspond to the target types one by one.

An electronic device, optionally including a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned method of marking more images Target classification method.

Optionally, the memory is a readable storage medium.

According to the specific embodiments provided by the invention, the invention discloses the following technical effects:

The object classification method, system and electronic equipment of a multi-marked image provided by the present invention acquire the multi-marked image to be recognized; input the multi-marked image to be identified into a relevant mark determination model, and determine the multi-marked image to be recognized The related marks of all the targets in the image; according to the multiple related marks, the types of all the targets in the image to be identified are determined; the related marks correspond to the target types one by one. The invention utilizes the principle of contrastive label disambiguation to train a classifier to obtain a relevant label determination model capable of accurately identifying relevant labels in unseen images, thereby improving the accuracy of image classification.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without paying creative labor.

Fig. 1 is a flow chart of a target classification method for an over-marked image in Embodiment 1 of the present invention;

Fig. 2 is a frame diagram of a CPLD model in Embodiment 2 of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The purpose of the present invention is to provide a target classification method, system and electronic equipment for over-marked images, using the principle of contrastive label disambiguation to train the classifier to obtain related tags that can accurately identify related tags in unseen over-marked images Determine the model to improve the accuracy of image classification.

In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

Example 1

As shown in Figure 1, the present embodiment provides a target classification method for more marked images, including:

Step 101: Obtain an image of more markers to be identified.

Wherein, at least one target is included in the image to be identified with more marks.

Step 102: Input the image with many labels to be identified into the relevant label determination model, and determine the relevant labels of all objects in the image with many labels to be identified.

The relevant label determination model is obtained by training a classifier based on multiple over-labeled historical images using the principle of contrastive label disambiguation.

Step 103: Determine the types of all targets in the image with more tags to be identified according to the plurality of related tags; the related tags correspond to the target types one by one.

Before step 101, also include:

Step 104: Acquiring multiple over-marked historical images; multiple kinds of marks are marked on the over-marked historical images. The types of markers are correlation markers or noise markers.

Step 105: Perform random data augmentation processing on the over-marked historical image to obtain the query view and key view of the over-marked historical image.

Step 106: Determine the tag-level embedding under the query view and the tag-level embedding under the key view. Label-level embeddings correspond one-to-one to multiple labels on overlabelled historical images.

Step 107: According to the tag-level embedding under the query view and the tag-level embedding under the key view, the classifier is trained using the principle of contrastive tag disambiguation to obtain a related tag determination model.

After step 106, also include:

Step 108: Determine the positive or negative of multiple tag-level embeddings under the query view.

Step 109: Determine the positive or negative of multiple tag-level embeddings under the key view.

Step 107, comprising:

Step 1071: Determine the classifier as the classifier at the 0th iteration.

Step 1072: Obtaining the initial positive type of each label in the classifier is the positive type at the 0th iteration.

Step 1073: Obtain the initial negative prototype of each label in the classifier when it is the negative prototype of the 0th iteration.

Step 1074: Set the first iteration number i=1.

Step 1075: Set the second iteration number j=1.

Step 1076: Determine any tag-level embedding in any query-based view as the current tag-level embedding.

Step 1077: Update the positive prototype at the i-1th iteration and the negative prototype at the i-1th iteration according to the positive or negative of the current label-level embedding.

Step 1078: Calculate the first similarity between the current tag-level embedding and the positive prototype at the i-1th iteration after the update.

Step 1079: Calculate the similarity between the current tag-level embedding and the negative prototype at the i-1th iteration after the update as the second similarity.

Step 10710: According to the first similarity and the second similarity, determine the label vector predicted by the current label-level embedding according to the prototype.

Step 10711: Update the pseudo-label embedded with the corresponding label at the current label level according to the label vector, and obtain the pseudo-label embedded with the corresponding label at the current label level at the jth iteration.

Step 10712: Increase the value of the second iteration number j by 1, update the current tag-level embedding to a current tag-level embedding other than the current tag-level embedding under the same query view, and return to step 1077 until the second iteration count reaches the th Two iteration thresholds.

Step 10713: Embedding multiple current label levels corresponding to the query view into the classifier at the i-1th iteration to obtain multiple category outputs.

Step 10714: Determine the classification loss function at the i-1th iteration according to the outputs of multiple categories and the pseudo-labels embedded in the corresponding labels at the current label level during multiple iterations.

Step 10715: Judging whether the classification loss function is less than the threshold of the classification loss function, and obtaining the first judgment result; if the first judgment result is no, execute step 10716; if the first judgment result is yes, execute step 10717.

Step 10716: Update the parameters of the classifier at the i-1th iteration to obtain the classifier at the ith iteration, increase the value of the first iteration number i by 1, and return to step 1075.

Step 10717: Judging whether the first iteration count reaches the threshold of the first iteration count, and obtaining a second judgment result; if the second judgment result is no, execute step 10718; if the second judgment result is yes, execute step 10719.

Step 10718: Determine that the classifier at the i-1th iteration is the classifier at the ith iteration, increase the value of the first iteration number i by 1, and return to step 1075.

Step 10719: Determine the classifier at the i-1th iteration as the relevant marker determination model.

Before step 10719, also include:

Step 10720: Determine the tag-level embedding under the query view and the tag-level embedding under the key view as the embedding pool; the embedding pool also includes the tag-level embedding in the momentum tag-level embedding queue.

Step 10721: Determine any positive tag-level embedding in the query view as the current positive tag-level embedding.

Step 10722: Determine the positive label-level embedding corresponding to the same label as the positive label-level embedding in the embedding pool as the positive sample set corresponding to the current positive label-level embedding.

Step 10723: Determine that the current positive label-level embedding and the samples in the positive sample set corresponding to the current positive label-level embedding constitute a plurality of positive sample pairs.

Step 10724: Determine the comparative loss function corresponding to more than one labeled historical images based on multiple positive sample pairs under the same query view.

Step 10725: Judging whether multiple contrastive loss functions are all smaller than the contrastive loss function threshold to obtain a third judgment result; if the third judgment result is no, execute step 10726; if the third judgment result is yes, execute step 10727.

Step 10726: Update the parameters of the classifier at the i-1th iteration to obtain the classifier at the 0th iteration, and return to step 1074.

Step 10727: If the third judgment result is yes, call step 10719.

Example 2

As shown in FIG. 2 , an object classification method for mostly labeled images provided by this embodiment consists of two parts: a contrastive learning module and a prototype-based label disambiguation module. The method uses these two modules to construct a synergistic system framework: where contrastive learning aims to obtain high-quality representations, prototype-based label disambiguation uses the high-quality representations learned by contrastive learning to improve prototypes, and then updates pseudo Marking, to guide the prediction results of the model, to help contrastive learning to establish more accurate positive sample pairs. At the same time, the present invention uses a two-stage training strategy to make the comparative learning technique more reasonably applied to the present invention. The two modules depend on each other and cooperate with each other. As the training progresses, the model can gradually update the confidence of the marker, extract relevant markers, and reduce the attention to noise markers. Specific steps are as follows:

Step 1: More labeled image training data as input. definition and /> are the feature space and label space, respectively, where K represents the number of labels of interest. training dataset /> Consists of n samples, where Indicates the i-th image observed, /> Indicates the candidate label vector corresponding to the i-th image, y _i,j = 1 means that the label j is the mark of the i-th image, and vice versa.

Step 2: Get an augmented view of the image:

For the sake of brevity, the present invention omits the index i. For the input image x, the present invention uses two image augmentation methods to obtain the query view Aug _q (x) and the key view Aug _k (x) respectively.

Among them, the present invention uses the SimAugment data augmentation method in the paper "Supervised contrastive learning" for the query network, and uses the RandAugment data augmentation method in the paper "Randaugment: PracticalAutomated Data Augmentation With a Reduced Search Space" for the key network.

Step 3: Get tab-level embeds under both views:

For the query view, Enc(Aug _q (x))∈R ^d*h*w is obtained through its encoder network, where d, h, and w represent the dimension of the encoder output, the height and width of the feature map, respectively. Then use 1*1 convolution to reduce the dimensionality of the feature map to K to obtain the feature map corresponding to each category, flatten the class feature map to vector form and input a projection head to project the class vector To the comparison space required later, the output g(Aug _q (x))∈R ^{K *D} of the query network g(·) will be obtained at this time, where D is the dimension of the comparison space. To this end, the image-level feature map is decoupled into K D-dimensional label-level embeddings q _j ∈ R ^1*D , j ∈ {1,...,K}, each label-level embedding can be regarded as an image A representation vector in the context of the corresponding label, containing the feature information of the corresponding class. The key network g'( ) is the result of the moving average of the parameter momentum of the query network. The obtained g'(Aug _k (x))∈R ^K*D is prepared for the subsequent use of comparative learning, and each row k _j ∈R ^1*D , j∈{1,...,K} is similar to q _j , representing the label-level embedding obtained by decoupling the image-level representation by the key network.

Step 4: Use contrastive learning to obtain high-quality embedding representations:

Step 1: Judge the positive or negative of the label-level embedding.

For the augmented image Aug _q (x) above, after classifier Get the output f(Aug _q (x))∈[0,1] ^K of the classifier, where each item of f(Aug _q (x)) is the output obtained by the Sigmoid activation function.

use Judge the positiveness or negativeness of each label-level embedding. Among them, if /> Then it is judged that the j-th label-level embedding is a positive label-level embedding, and vice versa. Aggregating these label-level embeddings yields a set of positive/negative label-level embeddings PE(x)/NE(x) for the sample image. In the above formula, α is a hyperparameter, b _j and /> are the baseline probability of class j and the average baseline probability for all classes, respectively.

Step 2: Build a set of positive samples for each positive label-level embedding.

Combine the positive label-level embedding obtained by the query network and the key network with the historical embedding in the embedding queue to build an embedding pool A=B _q ∪B _k ∪queue, and A(q _j )=A\{q _j }, where B _q and _Bk denote the positive label-level embeddings of all sample images in the current batch corresponding to the query network and the key network. The present invention uses the queue queue to retain the positive label-level embedding obtained by the key network of the latest batch of samples.

Use P(q _j )={k _j |k _j ∈ A ^j (q _j )} to represent the positive sample set of positive label-level embedding q _j , where Denotes the label-level embeddings corresponding to all labels j in A(q _j ). The above formula shows that the set of positive samples for each positive label-level embedding is the same category as other positive label-level embeddings in the embedding pool.

Step 3: Establish a contrastive loss.

All positive label-level embeddings and their samples in the positive sample set constitute positive sample pairs, and a contrastive loss is established to obtain high-quality label-level embedding representations.

The formula for calculating the contrastive loss function for a single image sample is as follows:

Among them, τ represents the temperature parameter.

Step 5: Prototype-based label disambiguation:

For each class c∈{1,...,K}, the present invention uses a positive type and a negative prototype /> represent representative positive/negative label-level embedding features of class c, respectively.

Step 1: Update positive/negative prototypes.

In the current mini-batch, the positive/negative prototypes of the corresponding classes are updated using the positive/negative label-level embeddings of the samples. The update formula is as follows:

Step 2: Update pseudo tags.

Compute the similarity between the sample’s label-level embedding and the improved positive/negative prototypes to obtain the label vector z predicted by the prototypes. The pseudo-label s is gradually updated by means of moving average.

where φ∈(0,1) is a normal quantity. The above formula means that if the tag-level embedding q _c of a tag is found to be more similar to the positive prototype after calculating the similarity with the corresponding positive/negative prototype, then this tag is considered as a related tag of image x. As the model training progresses, the predictions of label-level embeddings obtained by corresponding prototypes will gradually agree. Therefore, the pseudo-labels of relevant labels will gradually stabilize at 1, while those of irrelevant labels will smoothly approach 0.

Step 3: Build a classification loss using the output of the classifier with the pseudo-labels.

The updated pseudo-labels are used with the output of the classifier to build a classification loss, and the improved pseudo-labels after improving the prototype using contrastive learning are used to guide the prediction of the model classifier. The classification loss calculation formula is as follows:

Step 6: Combining the classification loss and the comparison loss to form a total loss function, which is used as the objective function for training and optimizing the neural network. The total loss function is calculated as follows:

where λ is an adjustable hyperparameter.

Step 7: Use a two-stage training strategy:

The two-stage training strategy consists of a pre-disambiguation stage and a contrastive disambiguation stage.

Pre-disambiguation stage: Remove the comparative learning branch, that is, only use the query network and the prototype disambiguation strategy. At this time, only use as the objective function of the network.

Contrastive disambiguation stage: Using the total loss function, the entire system is trained.

Step 8: Multi-label prediction of unseen samples on test data using threshold δ:

For an unseen sample x, its associated label prediction result is

That is, for an unseen instance x, compare the predicted probability of the classifier with the threshold δ to determine whether the label is a relevant label. The model is evaluated using a multi-label evaluation metric.

The present invention is tested on the mainstream multi-label image classification data set VOC2007. The VOC2007 dataset contains images from 20 object categories, where each image contains an average of 2.5 categories of objects. The VOC2007 dataset contains a training dataset consisting of 5011 images and a testing dataset consisting of 4952 images.

In order to construct a more labeled data set, the present invention makes the ratio of the average size of the label set to the number of labels in the entire label space be q. In the experiment, for the VOC2007 data set, q takes 0.1.

Following many existing studies, the present invention uses mAP, OF1, CF1 as evaluation indicators. Among them, mAP, also known as the average precision of all classes or the mean of average precision, is obtained by comprehensively weighting the average accuracy rate (AP) of all categories of detection. CF1 and OF1 comprehensively consider the recall and precision of the overall and each category. Therefore, these three indicators are the most important and representative evaluation indicators among all measurement indicators.

In order to verify the effectiveness of the present invention, the present invention uses binary cross entropy loss BCE (binary crossentropy) to directly perform training on a mostly labeled image data set to form a benchmark method. In addition, two advanced multi-label image classification methods ASL (published in "Asymmetric loss for multi-label classification") and ML-GCN (published in "Multi-Label Image Recognition with Graph ConvolutionalNetworks") are added as the other two benchmarks for comparison method.

For the consistency of comparison, the present invention follows the research of ASL, adopts TresnetL "Tresnet: High performance gpu-dedicated architecture" pre-trained on ImageNet as the backbone of the present invention, and uses 224*224 as the resolution of the input image. Let α = 0.8 for the VOC2007 dataset. Using γ = 0.99 to update the prototype, the present invention makes it linearly decrease from 0.95 to 0.8 for the constant φ of the pseudo-label update. The temperature parameter τ=0.2, the weight factor of the loss function λ=0.1, and the present invention uses exponential moving average (EMA) to update the model parameters, and the attenuation parameter is 0.9997. .

The method proposed in the present invention is named CPLD (Contrastive Prototype-based Label Disambiguation), and the method comparison results are shown in Table 1. It can be seen that on the VOC2007 data set, the method of the present invention surpasses other methods in performance, which proves the effectiveness of the method of the present invention.

Table 1 Comparison table of model classification results

Method\Indicator mAP CF1 OF1 BCE 88.37 82.17 84.42 ML-GCN 80.31 68.13 72.81 ASL 87.88 79.78 81.62 CPLD 89.79 82.68 85.33

The invention provides a target classification method for more marked images, which directly uses the training data of more marked images to obtain a classification model capable of performing multi-mark prediction on unseen instances, thereby greatly reducing the marking cost.

In addition, the classification model in the present invention uses the binary cross-entropy loss function (BCE), and more advanced loss calculation methods such as FocalLoss "Focal Loss for Dense Object Detection" or ASL "Asymmetric Loss For Multi-Label Classification" can also be used. In addition, for the positive and negative judgment of the label-level embedding obtained by model decoupling, it is also possible to simply use a fixed threshold to judge the predicted probability of the classifier, or use softmax to obtain the predicted probability As thresholds, these improvements or other easily conceivable changes or replacements all belong to the protection scope of the present invention.

Example 3

In order to implement the method corresponding to the above-mentioned embodiment 1 to achieve corresponding functions and technical effects, a target classification system for more marked images is provided below, including:

The image acquisition module with many marks to be identified is used to acquire images with many marks to be identified; the image with many marks to be identified includes at least one target.

The relevant mark identification module is used to input the image of many mark to be identified into the relevant mark determination model, and determine the relevant marks of all targets in the image of many mark to be identified; the relevant mark determination model is based on multiple mark history images , obtained by training the classifier using the principle of contrastive label disambiguation.

The target type determination module is used to determine the types of all targets in the image to be identified with more tags according to the plurality of related marks; the related marks are in one-to-one correspondence with the target types.

Example 4

This embodiment provides an electronic device, including a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the object classification method for more marked images described in Embodiment 1. Wherein, the memory is a readable storage medium.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. As for the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for the related information, please refer to the description of the method part.

In this paper, specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; meanwhile, for those of ordinary skill in the art, according to the present invention Thoughts, there will be changes in specific implementation methods and application ranges. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims (8)

1. A target classification method for more marked images, characterized in that, comprising: Acquiring an image with many marks to be identified; the image with many marks to be identified includes at least one target; Input the image of many markers to be identified into the relevant marker determination model, and determine the relevant markers of all targets in the image of multiple markers to be identified; The label disambiguation principle is used to train the classifier; The types of all targets in the image with more tags to be identified are determined according to the plurality of related tags; the related tags correspond to the target types one by one. 2. the target classification method of a kind of partial mark image according to claim 1, is characterized in that, before obtaining image to be classified, also comprises: Acquiring multiple over-marked historical images; the over-marked historical images are marked with various marks; the type of the mark is a relevant mark or a noise mark; Carrying out random data augmentation processing on the historical image with too many marks to obtain query view and key view of the historical image with too many marks; Determine the tag-level embedding under the query view and the tag-level embedding under the key view; the tag-level embedding is in one-to-one correspondence with the multiple tags on the historical image with more tags; According to the tag-level embedding under the query view and the tag-level embedding under the key view, the classifier is trained using the principle of contrastive tag disambiguation to obtain the relevant tag determination model. 3. the target classification method of a kind of partial label image according to claim 2, it is characterized in that, after determining the label-level embedding under query view and the label-level embedding under key view, also comprise: Determine the sign of multiple tag-level embeddings under the query view; Determines the signability of multiple label-level embeddings under the key view. 4. the target classification method of a kind of partial label image according to claim 3, it is characterized in that, described according to the label level embedding under query view and the label level embedding under key view, utilize contrasting label disambiguation principle to The classifier is trained to obtain the relevant label determination model, including: Determine that the classifier is the classifier at the 0th iteration; Obtain the initial positive type of each mark in the classifier as the positive type at the 0th iteration; Obtain the initial negative prototype of each label in the classifier as the negative prototype of the 0th iteration; Let the first iteration number i=1; Let the second iteration number j=1; Determine any tag-level embedding under any query-based view as the current tag-level embedding; According to the positive and negative of the current label-level embedding, update the positive prototype at the i-1th iteration and the negative prototype at the i-1th iteration; Calculate the similarity between the current tag-level embedding and the positive prototype at the i-1th iteration after the update as the first similarity; Calculate the similarity between the current label-level embedding and the negative prototype at the i-1th iteration after the update as the second similarity; According to the first similarity and the second similarity, determine the current label-level embedding according to the label vector predicted by the prototype; Updating the pseudo-label embedded with the corresponding label at the current label level according to the label vector, and obtaining the pseudo-label embedded with the corresponding label at the current label level during the jth iteration; Increase the value of the second iteration number j by 1, update the current tag-level embedding to a current tag-level embedding other than the current tag-level embedding under the same query view, and return to the step "according to the positive or negative of the current tag-level embedding, update The positive prototype at the i-1th iteration and the negative prototype at the i-1th iteration" until the second iteration number reaches the second iteration number threshold; Embed multiple current label levels corresponding to the query view into the classifier at the i-1th iteration to obtain multiple category outputs; Determine the classification loss function at the i-1th iteration according to a plurality of the category outputs and the pseudo-labels embedded in the corresponding labels at the current label level during multiple iterations; Judging whether the classification loss function is less than a classification loss function threshold, and obtaining a first judgment result; If the first judgment result is no, update the parameters of the classifier at the i-1th iteration to obtain the classifier at the ith iteration, increase the value of the first iteration number i by 1, and return to step " Let the second iteration times j=1"; If the first judgment result is yes, it is judged whether the first iteration number reaches the first iteration number threshold, and a second judgment result is obtained; If the second judgment result is no, then it is determined that the classifier at the i-1th iteration is the classifier at the ith iteration, the value of the first iteration number i is increased by 1, and the step "make the second The number of iterations j = 1"; If the second judgment result is yes, the classifier at the i-1th iteration is determined to be the relevant marker determination model. 5. the target classification method of a kind of partial mark image according to claim 3, it is characterized in that, before the classifier when determining the i-1th iteration is the model for the relevant mark determination, it also includes: Determine the tag-level embedding under the query view and the tag-level embedding under the key view as an embedding pool; the embedding pool also includes the tag-level embedding in the momentum tag-level embedding queue; Determine any tag-level embedding with a positive positive or negative in the query view as the current positive tag-level embedding; Determining that the positive label-level embedding corresponding to the same label as the positive label-level embedding in the embedding pool is the positive sample set corresponding to the current positive label-level embedding; Determining that the current positive label-level embedding and the samples in the positive sample set corresponding to the current positive label-level embedding constitute a plurality of positive sample pairs; Determine the comparative loss function corresponding to more than one marked historical image based on multiple positive sample pairs under the same query view; Judging whether a plurality of the comparison loss functions are less than the comparison loss function threshold to obtain a third judgment result; If the third judgment result is no, update the parameters of the classifier at the i-1th iteration to obtain the classifier at the 0th iteration, and return to the step "make the first iteration number i=1"; If the third judgment result is yes, the step of "determining the classifier at the i-1th iteration as the model for the relevant markers" is invoked. 6. A target classification system for more marked images, characterized in that it comprises: An image acquisition module with many marks to be identified is used to acquire an image with many marks to be identified; the image with many marks to be identified includes at least one target; The relevant mark identification module is used to input the image of many marks to be identified into the relevant mark determination model, and determine the relevant marks of all targets in the image of many marks to be recognized; the relevant mark determination model is based on multiple partial Multi-label historical images, obtained by training the classifier using the principle of contrastive label disambiguation; The target type determination module is used to determine the types of all targets in the image with more tags to be identified according to the plurality of related marks; the related marks correspond to the target types one by one. 7. An electronic device, characterized in that it comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute any one of claims 1 to 5 An object classification method for mostly labeled images described in the item. 8. The electronic device according to claim 7, wherein the memory is a readable storage medium.