CN116977769A

CN116977769A - Label labeling method, image classification model construction method and image classification method

Info

Publication number: CN116977769A
Application number: CN202310151970.0A
Authority: CN
Inventors: 林志文; 鄢科
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-02-15
Filing date: 2023-02-15
Publication date: 2023-10-31

Abstract

The application relates to a label labeling method, a construction method of an image classification model, an image classification method, a device, equipment, a storage medium and a program product. The method involves artificial intelligence, comprising: the method comprises the steps of obtaining a sample image set to be marked, determining a marking sequence corresponding to the sample image set to be marked, and marking each sample image to be marked in the sample image set to be marked according to the marking sequence to obtain a marked sample image set. The method and the device have the advantages that the image types of the positive labels, the image types of the trusted negative labels and the image types of the untrusted labels carried by the sample images are quickly and accurately obtained by using ordered single label labeling, so that in the subsequent training process of the model by using labeled sample image sets, the model parameters are adjusted and optimized in a targeted mode according to the labeling labels and the image types represented by the labeling labels of the sample images, and the trained classification model can quickly and accurately identify the image types on the image.

Description

Label labeling method, image classification model construction method and image classification method

Technical Field

The present application relates to the field of artificial intelligence, and in particular, to a label labeling method, an image classification model construction method, an image classification apparatus, a computer device, a storage medium, and a computer program product.

Background

With the development of artificial intelligence technology and in different actual scenes in the field of image recognition, the requirements on the content of images and the recognition accuracy of all image categories in the images are increasingly improved, and an image classification task is presented. The object of the image classification task is to input a picture and output all image categories in the picture, and because of the condition that multiple image categories exist, when labeling a sample image, all the existing object categories in the image need to be labeled. And because all the categories in the image need to be marked when marking, the marking speed is obviously reduced under the conditions of more image categories and uneven distribution of different image categories in the image.

Conventionally, in order to improve the labeling speed in label classification, a labeling mode of single positive label appears, namely, when labeling is performed in a label task, a mode of only randomly labeling one target class in a graph and not labeling other classes is adopted, so that the consumption of labeling time is reduced, and the labeling speed is improved.

However, in the conventional single positive label method, only one image category is randomly marked, so that the problem of larger error of an image marking result still exists due to the fact that the randomness in the marking process is excessively depended, so that the accuracy of a model obtained by subsequent training according to a marked sample image set is lower, the error is larger, and the accuracy of image classification is still lower.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a label labeling method, an image classification model constructing method, an image classification method, an apparatus, a computer device, a storage medium, and a computer program product that can improve the accuracy of image classification.

In a first aspect, the present application further provides a label labeling method, where the method includes:

acquiring a sample image set to be marked;

determining a labeling sequence corresponding to the sample image set to be labeled;

according to the labeling sequence, orderly labeling the sample images to be labeled in the sample image set to be labeled to obtain a labeled sample image set;

each sample image in the marked sample image set carries a marked label, and the category represented by the marked label comprises an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label.

In a second aspect, the present application provides a method for constructing an image classification model. The method comprises the following steps:

acquiring a sample image set; each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises: the image category of the positive label, the image category of the trusted negative label and the image category of the untrusted label;

training the initial classification model according to each sample image, and obtaining a trained image classification model if the training ending condition is determined to be met; when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image categories of the positive label and the trusted negative label is smaller than a preset threshold value.

In a third aspect, the present application also provides an image classification method, the method comprising:

receiving an image classification request and acquiring an image to be classified corresponding to the image classification request;

classifying the images to be classified according to the trained image classification model to obtain at least one image category corresponding to the images to be classified;

The trained image classification model is obtained when the initial classification model is trained by using a sample image set and the training ending condition is met; each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label; when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image category of the positive label and the image category of the trusted negative label is smaller than a preset threshold value.

In a fourth aspect, the application further provides a label marking device. The device comprises:

the sample image set to be marked is obtained by the sample image set to be marked obtaining module;

the labeling sequence determining module is used for determining a labeling sequence corresponding to the sample image set to be labeled;

the label labeling module is used for sequentially labeling each sample image to be labeled in the sample image set to be labeled according to the labeling sequence to obtain a labeled sample image set; each sample image in the marked sample image set carries a marked label, and the category represented by the marked label comprises an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label.

In a fifth aspect, the application further provides a device for constructing the image classification model. The device comprises:

the sample image set obtaining module is used for obtaining a sample image set; each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises: the image category of the positive label, the image category of the trusted negative label and the image category of the untrusted label;

the image classification model obtaining module is used for training the initial classification model according to each sample image, and obtaining a trained image classification model if the training ending condition is determined to be met; when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image categories of the positive label and the trusted negative label is smaller than a preset threshold value.

In a sixth aspect, the present application further provides an image classification apparatus. The device comprises:

the image classification module is used for receiving the image classification request and acquiring an image to be classified corresponding to the image classification request;

The classification processing module is used for carrying out classification processing on the images to be classified according to the trained image classification model to obtain at least one image category corresponding to the images to be classified; the trained image classification model is obtained when the initial classification model is trained by using a sample image set and the training ending condition is met; each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label; when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image category of the positive label and the image category of the trusted negative label is smaller than a preset threshold value.

In a seventh aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:

Acquiring a sample image set to be marked;

In an eighth aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:

In a ninth aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:

In a tenth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

acquiring a sample image set to be marked;

In an eleventh aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

In a twelfth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

In a thirteenth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:

acquiring a sample image set to be marked;

In a fourteenth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:

In a fifteenth aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of:

In the label labeling method, the construction method of the image classification model, the image classification method, the device, the computer equipment, the storage medium and the computer program product, the sample image set to be labeled is obtained, the labeling sequence corresponding to the sample image set to be labeled is determined, and then according to the labeling sequence, each sample image to be labeled in the sample image set to be labeled is labeled with an ordered single label so as to obtain the labeled sample image set. The method comprises the steps that each sample image in a sample image set after labeling carries labeling labels, and the categories represented by the labeling labels specifically comprise image categories of positive labels, image categories of trusted negative labels and image categories of untrusted labels. The method can quickly and accurately obtain the image type of the positive label, the image type of the trusted negative label and the image type of the untrusted label carried by each sample image by adopting an orderly single label labeling mode, so that in the subsequent training process of the model by using the labeled sample image set, the specific adjustment optimization of model parameters is realized according to the labeling labels of each sample image and the image types represented by the labeling labels, and the trained classification model can concentrate on different image types on the image, thereby obtaining accurate and comprehensive image recognition results.

Drawings

FIG. 1 is an application environment diagram of a label labeling method, a construction method of an image classification model, and an image classification method in one embodiment;

FIG. 2 is a flow chart of a label labeling method in one embodiment;

FIG. 3 is a schematic illustration of labeling results of labeling a sample image to be labeled according to different labeling modes in one embodiment;

FIG. 4 is a schematic flow chart of orderly single-label labeling of each sample image to be labeled in a sub-image set in one embodiment;

FIG. 5 is a schematic illustration of labeling results of labeling sample images to be labeled according to a labeling order in one embodiment;

FIG. 6 is a flow chart of a label marking method in another embodiment;

FIG. 7 is a flow diagram of a method of constructing an image classification model in one embodiment;

FIG. 8 is a flow diagram of obtaining a trained image classification model in one embodiment;

FIG. 9 is a flow diagram of determining training loss values in one embodiment;

FIG. 10 is a flow chart of a method of constructing an image classification model according to another embodiment;

FIG. 11 is a schematic diagram of a flow chart for constructing an image classification model in one embodiment;

FIG. 12 is a flow diagram of a method of image classification in one embodiment;

FIG. 13 is a block diagram of a label marking apparatus in one embodiment;

FIG. 14 is a block diagram of an apparatus for constructing an image classification model in one embodiment;

FIG. 15 is a block diagram showing the structure of an image classification apparatus in one embodiment;

fig. 16 is an internal structural view of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The label labeling method, the image classification model construction method and the image classification method provided by the embodiment of the application relate to an artificial intelligence technology, and can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, network media, auxiliary driving and the like. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision. The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

The Computer Vision technology (CV) is a science of researching how to make a machine "look at", and more specifically, to replace a human eye with a camera and a Computer to perform machine Vision such as recognition, detection and measurement on a target, and further perform graphic processing, so that the Computer is processed into an image more suitable for the human eye to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data. Computer vision techniques typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D techniques, virtual reality, augmented reality, synchronous positioning, and map construction, among others, as well as common biometric recognition techniques such as face recognition, fingerprint recognition, and others. Machine Learning (ML) is a multi-domain interdisciplinary, and involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

With research and advancement of artificial intelligence technology, research and application of artificial intelligence technology is being developed in various fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned, automatic driving, unmanned aerial vehicles, robots, smart medical treatment, smart customer service, etc., and it is believed that with the development of technology, artificial intelligence technology will be applied in more fields and with increasing importance value.

The label labeling method, the image classification model construction method and the image classification method provided by the embodiment of the application relate to the technologies of computer vision technology, machine learning and the like in the artificial intelligence technology, and can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, portable wearable devices, aircrafts, etc., and the internet of things devices may be smart speakers, smart car devices, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be a separate physical server or may be a server cluster formed by a plurality of physical servers.

Further, the terminal 102 and the server 104 may be separately configured to execute the label labeling method, the image classification model construction method and the image classification method provided in the embodiments of the present application, and the terminal 102 and the server 104 may cooperatively execute the label labeling method, the image classification model construction method and the image classification method provided in the embodiments of the present application. For example, taking the label labeling method provided by the embodiment of the present application as an example, where the terminal 102 and the server 104 cooperatively execute the label labeling method, the server 102 obtains a sample image set to be labeled, determines a labeling sequence corresponding to the sample image set to be labeled, and then performs ordered single label labeling on each sample image to be labeled in the sample image set to be labeled according to the labeling sequence, so as to obtain a labeled sample image set. The sample image set to be labeled may be stored in a cloud storage of the server 104, or in a data storage system, or in a local storage of the terminal 102, and may be obtained from the server 104, or the data storage system, or the terminal 102 when label labeling processing is required. Each sample image in the marked sample image set carries a marked label, and the category represented by the marked label comprises the image category of a positive label, the image category of a trusted negative label and the image category of an untrusted label.

Similarly, taking the method for constructing the image classification model provided by the embodiment of the present application as an example, where the terminal 102 and the server 104 cooperatively execute the method, the server 104 trains the initial classification model according to each sample image by acquiring a sample image set, and if it is determined that the training end condition is met, a trained image classification model is obtained, where each sample image in the sample image set carries a labeling label, and the class represented by the labeling label includes: the method comprises the steps that when a training loss value is calculated in the training process, the weight of the image class of the unreliable label is smaller than that of the image class of the positive label and that of the image class of the trusted negative label, and the difference value of the weights of the image classes of the positive label and the trusted negative label is smaller than a preset threshold value. The image sample set may be stored in a cloud storage of the server 104, or in a data storage system, or in a local storage of the terminal 102, and may be obtained from the server 104, or the data storage system, or the terminal 102 when the image classification model needs to be built.

Similarly, taking the terminal 102 and the server 104 cooperatively execute the image classification method provided in the embodiment of the present application as an example, the server 104 receives the image classification request triggered by the terminal 102, and acquires the image to be classified corresponding to the image classification request. Further, the server 104 performs classification processing on the images to be classified according to the trained image classification model, obtains at least one image category corresponding to the images to be classified, and feeds back the image category of each obtained image to be classified to the terminal 102. The image to be classified may be stored in a cloud storage of the server 104, or in a data storage system, or in a local storage of the terminal 102, and may be acquired from the server 104, or the data storage system, or the terminal 102 when the image classification process is required. The trained image classification model is obtained by training the initial classification model by using a sample image set and meeting the training ending condition. Each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label. Further, when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image category of the positive label and the image category of the trusted negative label is smaller than a preset threshold value.

In one embodiment, as shown in fig. 2, a label labeling method is provided, and the method is applied to the server in fig. 1 for illustration, and specifically includes the following steps:

step S202, a sample image set to be marked is obtained.

Specifically, a sample image set to be marked is obtained by collecting each sample image to be marked. The sample image to be marked may specifically be an image including multiple object categories, and may be classified into different image categories, for example, a sample image including different object categories such as an automobile, a bicycle, a person, and a building, and may be classified into an automobile image category, a bicycle image category, a person image category, and a building image category.

Step S204, determining the labeling sequence corresponding to the sample image set to be labeled.

Before determining the labeling sequence corresponding to the sample image set to be labeled, the method further comprises the following steps: dividing the sample image set to be annotated into sub-image sets of preset batches, wherein each sub-image set of each batch comprises a plurality of sample images to be annotated.

Specifically, the sample image set to be marked may be divided into sub-image sets of preset batches according to the number of sample images in the sample image set to be marked and the required batch number (or the capacity of each batch), for example, the number of sample images in the sample image set to be marked D is K, and the required batch number is N, and then the sample image set to be marked may be divided into N batches including D1, D2, D3, … … and Dn.

The number of sample images in each batch can be the same, for example, K/N samples can be set to be different according to actual requirements. Similarly, for the sub-image sets of different batches, the labeling sequences of the sub-image sets of different batches are different, and then label labeling is performed according to the labeling sequences. Specifically, for example, the sub-image set of the lot D1 may be labeled according to the labeling order of O1, the sub-image set of the lot D2 may be labeled according to the labeling order of O2, and so On, the sub-image set of the lot Dn may be labeled according to the labeling order of On.

In one embodiment, as shown in fig. 3, a schematic diagram of labeling results of labeling sample images to be labeled according to different labeling modes is provided, and referring to fig. 3, for example, four categories of "person", "air line", "car" and "bike" are provided, and the labeling order can be the positive order of letters, namely, the order of a to z, the reverse order of letters, namely, the order of z to a, or the randomly set order.

The labeling order is "airland", "rake", "car" and "person" when the letter is in the positive order, and "person", "car", "rake" when the letter is in the negative order, and the labeling order is a randomly set order, and may be the order of "airland", "car", "person", "rake", or "car", "rake", "person", "airland", or the like.

Specifically, referring to fig. 3, when the sample image to be marked shown in fig. 3 needs to be completely marked, the image categories represented by the obtained marking labels include "person", "rake" and "car", and a conventional random marking mode (i.e. single positive label, which can be understood as a single positive label marking mode) is adopted, and the attention of the random marking mode to the image category of "person" is higher, the person image category is usually marked actively.

When labeling the sample image to be labeled in a random order of "car", "rake", "person" and "airland", only one image category needs to be labeled, the first image category "car" in the labeling order will be labeled.

Similarly, when labeling the sample image to be labeled in the alphabetical order labeling order of "airland", "bike", "car" and "person", since the image class of "airland" does not exist in the sample image to be labeled and the order of "bike" is located before "car" and "person", only the image class of "bike" is labeled, and the image class of "bike" is used as the image class of the positive label. However, when the sample image to be marked is marked by adopting the letter positive sequence marking sequence of ' airland ', ' bike ', ' carand ' person ', the image category of ' airland ' can be determined as the image category of the trusted negative label, and the image category of ' car ' and ' person ' can be determined as the image category of the untrusted label because the image category of ' airland ' does not exist in the sample image to be marked.

It can be understood that after orderly labeling of sample images to be labeled, three labeling label types exist for each sample image:

1. positive labels, namely marked single labels, represent that the image class of the sample image must exist, for example, if the image class of the positive labels of the sample image to be marked in fig. 3 is "rake", then the image class of "rake" must exist in the sample image to be marked.

2. The trusted negative label, that is, the image category before being marked as the positive label, indicates that the image category of the sample image to be marked does not exist, for example, the image category of the trusted negative label of the sample image to be marked in fig. 3 is "air land", and the labeling sequence is "air land", "rake", "car", "person", "air land" is located before the "rake" marked as the positive label, that is, the image category of "air land" does not exist in the sample image to be marked.

3. The unreliable label, that is, the image category after being marked with the positive label, the unreliable label indicates that whether the image category exists in the sample image is uncertain, for example, the image category of the unreliable label of the sample image to be marked in fig. 3 is "car", "person", and the image category of the unreliable label is uncertain whether the image category of "car" or "person" exists in the sample image to be marked because the marking sequence is "airland", "rake", "car", "person" is located after the "rake" marked with the positive label.

And S206, according to the labeling sequence, orderly labeling the sample images to be labeled in the sample image set to be labeled, and obtaining a labeled sample image set. Each sample image in the marked sample image set carries a marked label, and the category represented by the marked label comprises the image category of a positive label, the image category of a trusted negative label and the image category of an untrusted label.

Specifically, according to the labeling sequence corresponding to the sub-image sets of each batch, orderly single label labeling is carried out on each sample image to be labeled in the sub-image sets of each batch, and then according to each sample image labeled in the sub-image sets of each batch, a labeled sample image set is obtained.

Each sample image in the marked sample image set carries a marked label, and the category represented by the marked label comprises the image category of a positive label, the image category of a trusted negative label and the image category of an untrusted label.

Specifically, after labeling the sample images to be labeled, the sample images to be labeled are provided with three image categories represented by labeling labels, including an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label, by adopting the letter positive sequence labeling sequence of "airland", "bike", "car" and "person". The image class represented by the positive label can be understood as that the sample image must exist in the image class, for example, if the image class of the positive label of the sample image to be marked in fig. 3 is "rake", then the sample image to be marked must exist in the image class of "rake".

Likewise, the image category represented by the trusted negative label, that is, the image category before being marked as the positive label, can be understood that the image category of the sample image to be marked must not exist, for example, the image category of the trusted negative label of the sample image to be marked in fig. 3 is "air land", and since the marking sequence is "air land", "rake", "car", "person", "air land" is located before "rake" marked as the positive label, that is, the image category of "air land" must not exist in the sample image to be marked.

Likewise, the image category represented by the non-label can be understood as the image category of the sample image after being labeled with the positive label, and whether the image category exists is not determined, for example, the image category of the non-trusted label of the sample image to be labeled in fig. 3 is "car", "person", because the labeling sequence is "airland", "bike", "car", "person" is located after the "bike" labeled with the positive label, that is, whether the image category of "car" and "person" exists in the sample image to be labeled is not determined.

In the label labeling method, the sample image set to be labeled is obtained, the labeling sequence corresponding to the sample image set to be labeled is determined, and then according to the labeling sequence, each sample image to be labeled in the sample image set to be labeled is labeled in order to obtain the labeled sample image set. The method comprises the steps that each sample image in a sample image set after labeling carries labeling labels, and the categories represented by the labeling labels specifically comprise image categories of positive labels, image categories of trusted negative labels and image categories of untrusted labels. The method can quickly and accurately obtain the image type of the positive label, the image type of the trusted negative label and the image type of the untrusted label carried by each sample image by adopting an orderly single label labeling mode, so that in the subsequent training process of the model by using the labeled sample image set, the specific adjustment optimization of model parameters is realized according to the labeling labels of each sample image and the image types represented by the labeling labels, and the trained classification model can concentrate on different image types on the image, thereby obtaining accurate and comprehensive image recognition results.

In one embodiment, as shown in fig. 4, the step of orderly labeling each sample image to be labeled in the sub-image set specifically includes:

step S402, feature recognition is carried out on each sample image to be marked, and image categories included in each sample image to be marked are obtained.

Specifically, a trained recognition model may be adopted to perform feature recognition on each sample image to be marked, so as to obtain image types (may also be object types) included in each sample image, for example, different types of automobiles, bicycles, people, buildings and the like may be included in the sample images, and the sample images may be classified into an automobile image type, a bicycle image type, a person image type, a building image type and the like.

And step S404, orderly labeling the sample images to be labeled in the sub-image set according to the labeling sequence corresponding to the sub-image set, and determining the image category of the positive label from the image categories included in the sample images to be labeled.

Specifically, the sample image set to be marked can be divided into sub-image sets of preset batches, and the marking sequences of the sub-image sets are different, so that randomness in the marking process is improved, repeated occurrence of certain image types or label types is avoided, and error influence on marked sample images is reduced. Specifically, the image category of the positive label can be determined from the image categories included in the sample images to be marked by acquiring the marking sequence corresponding to each sub-image set and carrying out ordered single label marking on each sample image to be marked in the corresponding sub-image set according to the acquired standard sequence.

When the sample image to be marked is marked in an orderly marking mode, only the image type of the positive label in the sample image to be marked is required to be marked, and other image types can be determined according to the marking sequence, the image type of the positive label and the image type included in the sample image to be marked.

For example, taking the labeling order of the sample image set to be labeled as "orphan", "bike", "car" and "person" as an example, when labeling the sample image to be labeled according to the labeling order of "orphan", "bike", "car" and "person", the image category included in the sample image to be labeled is "bike", "car" and "person", because the image category of "orphan" is not included in the sample image to be labeled, the position of the image category of "bike" in the labeling order, that is, "bike" is located before "car" and "person", when labeling the sample image to be labeled, the "bike" is the image category of the positive label of the sample image to be labeled.

Step S406, determining the image category of the trusted negative label and the image category of the untrusted label in the sample image to be annotated based on the annotation order, the image category of the positive label, and the image category included in the sample image to be annotated.

Specifically, when the sample image to be marked is marked in an orderly marking mode, only the image type of the positive label in the sample image to be marked is required to be marked, and then other image types can be determined according to the marking sequence, the image type of the positive label and the image type included in the sample image to be marked, and specifically, the image type of the trusted negative label and the image type of the untrusted label can also be determined.

In one embodiment, as shown in fig. 5, a schematic diagram of a labeling result of labeling a sample image to be labeled according to a labeling order is provided, and referring to fig. 5, labeling the sample image to be labeled according to a labeling order of "airland", "rake", "car" and "person" is taken as an example, and the sample image to be labeled includes image categories of "rake", "car" and "person", and because the sample image to be labeled does not include the image category of "airland", the position of the image category of "rake" in the labeling order, that is, "rake" is located before "car" and "person", when labeling the sample image to be labeled, the "rake" is taken as the image category of the positive label of the sample image to be labeled.

Further, since the image category of "air land" is not identified, that is, the image category of "air land" is not included in the sample image to be annotated, the image category of "air land" is the image category of the trusted negative label. Meanwhile, since the image category having marked "bike" is the image category of the positive label, "car" and "person" after the image category of "bike" determine the image category of the unreliable label.

In this embodiment, feature recognition is performed on each sample image to be annotated, so as to obtain an image category included in each sample image to be annotated, and according to an annotation sequence corresponding to the sub-image set, each sample image to be annotated in the sub-image set is annotated with an ordered single label, and the image category of the positive label is determined from the image categories included in the sample image to be annotated. Further, the image category of the trusted negative label and the image category of the untrusted label in the sample image to be annotated are determined based on the annotation order, the image category of the positive label and the image category included in the sample image to be annotated. The method has the advantages that an orderly single-label labeling mode is adopted, the image types of the positive labels are only required to be labeled, the image types of the credible negative labels carried by the sample images and the image types of the unreliable labels can be further obtained, the labeling speed of the images and the labeling accuracy of the labels are improved, and therefore in the subsequent training process of the model by using the labeled sample image set, the model parameters can be adjusted and optimized in a targeted mode according to the labeling labels of the sample images and the image types represented by the labeling labels, and the trained classification model can be focused on different image types on the images, and accurate and comprehensive image recognition results are obtained.

In one embodiment, as shown in fig. 6, a label labeling method is provided, which specifically includes the following steps:

step S601, a sample image set to be marked is obtained.

In step S602, the sample image set to be annotated is divided into sub-image sets of preset batches, and each sub-image set of each batch includes a plurality of sample images to be annotated.

Step S603, determining the labeling sequence corresponding to each sub-image set in the sample image set to be labeled.

Step S604, performing feature recognition on each sample image to be annotated in the sub-image set to obtain image categories included in each sample image to be annotated in the sub-image set.

And step S605, orderly labeling the sample images to be labeled in the sub-image set according to the labeling sequence corresponding to the sub-image set, and determining the image type of the positive label from the image types included in the sample images to be labeled.

Step S606 determines the image category of the trusted negative label and the image category of the untrusted label in the sample image to be annotated based on the annotation order, the image category of the positive label, and the image category included in the sample image to be annotated.

Step S607, obtaining a labeled sample image set according to each sample image labeled in the sub-image set of each batch.

In the label labeling method, the sample image set to be labeled is obtained, and the labeling sequence corresponding to the sample image set to be labeled is determined, so that each sample image to be labeled in the sample image set to be labeled can be labeled orderly according to the labeling sequence. The method comprises the steps that each sample image after labeling carries labeling labels, the classes represented by the labeling labels comprise image classes of positive labels, image classes of trusted negative labels and image classes of untrusted labels, namely, the image classes of the positive labels, the image classes of the trusted negative labels and the image classes of the untrusted labels carried by each sample image can be obtained through an orderly label labeling mode, so that the model parameters of an initial classification model can be adjusted and optimized in a targeted mode according to the labeling labels of each sample image and the image classes represented by the labeling labels, and the trained image classification model can accurately identify different image classes on an image, and the classification accuracy of classification processing according to the obtained image classification model is further improved, and an accurate and comprehensive image identification result is obtained.

In one embodiment, as shown in fig. 7, a method for constructing an image classification model is provided, and the method is applied to the server in fig. 1 for illustration, and specifically includes the following steps:

step S702, a sample image set is obtained, each sample image in the sample image set carries a label tag, and the category represented by the label tag includes: the image category of positive labels, the image category of trusted negative labels, and the image category of untrusted labels.

The sample image set used for training the initial classification model can be specifically a sample image set subjected to label labeling, the sample image set subjected to label labeling comprises a plurality of sample images, and each sample image carries a label. The labeling label is used for representing the image type carried by the sample image, and the type represented by the labeling label specifically comprises the image type of the positive label, the image type of the trusted negative label and the image type of the untrusted label.

In one embodiment, before the sample image set is obtained, the sample set of the images to be annotated needs to be obtained, each sample of the images to be annotated in the sample image set to be annotated is annotated in order according to a preset certain or certain annotation sequence, and annotation labels are added to each sample image, namely the image category included in each sample image is determined.

The image category of the positive label is used for representing the image category which is necessarily present in the marked sample image, the image category of the trusted negative label represents the image category which is necessarily absent in the marked sample image, and the image category of the untrusted label cannot accurately determine whether the image category which is present in the marked sample image, namely, the image category belongs to the possible presence or the possible absence condition.

Specifically, labeling each sample image to be labeled in the sample image set to be labeled according to a certain labeling sequence (the positive sequence of letters, namely the sequence from a to z, the reverse sequence of letters, namely the sequence from z to a, the randomly set sequence, and the like), determining the image type of the positive label of a certain sample image to be labeled, namely the image type of the positive label in the sample image to be labeled, and further determining the image type of the trusted negative label and the image type of the untrusted label according to the labeling sequence, the image type of the positive label, and the possible image type obtained when the feature recognition is performed on the sample image to be labeled.

For example, taking labeling order of "airland", "bike", "car" and "person" as an example, when labeling a certain sample image to be labeled, feature recognition is first required to be performed on the sample image to be labeled, so as to determine possible image types included in the sample image to be labeled, for example, determine that the image types of "bike", "car" and "person" may exist in the sample image to be labeled, and when labeling the sample image to be labeled according to the labeling order of "airland", "bike", "car" and "person", the image type of the positive label of the sample image to be labeled is determined to be "bike", that is, the image type of only labeling "bike" is determined as the image type of the positive label.

Further, when feature recognition is performed, it is determined that the image class of "air land" does not exist in the sample image to be marked, and then the image class of "air land" is used as the image class of the trusted negative label in the sample image to be marked. Likewise, since the image category of "rake" is already marked as the image category of the positive label, the image categories of "car" and "person" that may exist subsequently are not marked, and since the marking is not needed, whether the two image categories actually exist in the sample image or not is also impossible, and the image categories of "car" and "person" are used as the image categories of the unreliable label. It can be understood that when the sample image to be marked is marked, only the image type of the positive label of the sample image is marked, but the image type of the trusted negative label and the image type of the untrusted label on the sample image can be determined according to the marking sequence, the image type of the positive label and the possible image type obtained when the feature recognition is performed on the sample image to be marked.

Step S704, training the initial classification model according to each sample image, if the training ending condition is determined to be met, obtaining a trained image classification model, and when the training loss value is calculated in the training process, the weight of the image class of the unreliable label is smaller than that of the image class of the positive label and that of the image class of the trusted negative label, and the difference value of the weights of the image classes of the positive label and the trusted negative label is smaller than a preset threshold value.

Specifically, training an initial classification model according to each marked sample image, and completing training the initial classification model when determining that the training ending condition is met, so as to obtain a trained image classification model. The initial classification model can be a basic deep learning network model or a convolutional neural network model, and is trained by using each marked sample image, and model parameters of the initial classification model are adjusted and updated until training ending conditions are met.

The training ending condition may be that a training loss value in a training process reaches a preset loss threshold value, or that training iteration times of an initial classification model reach preset times.

Further, taking the training ending condition as an example that the training loss value in the training process reaches the preset loss threshold, because the classes represented by the labeling labels carried by the sample images can include the image class of the positive label, the image class of the trusted negative label and the image class of the untrusted label, when the training loss value is calculated in the training process, the weight of the image class of the untrusted label is smaller than that of the image class of the positive label and that of the image class of the trusted negative label, the difference value between the weight of the image class of the positive label and that of the image class of the trusted negative label is smaller than the preset threshold, and further, the model parameters of the initial classification model are subjected to targeted adjustment and optimization based on the labeling labels of each sample image and that of the image class represented by the labeling labels, and when the training loss value is determined to reach the preset loss threshold, a trained image classification model is obtained, so that different image classes on the image are accurately identified according to the trained image classification model, and a precise and comprehensive image identification result is obtained.

In the method for constructing the image classification model, the initial classification model is trained according to each sample image by acquiring the sample image set, so that the trained image classification model is obtained when the condition that the training is finished is confirmed to be met. The sample image set comprises a plurality of sample images, each sample image carries a labeling label, and then according to the labeling label of each sample image, the actual image type of each sample image can be determined, and the actual image type can be the image type of a positive label, the image type of a trusted negative label or the image type of an untrusted label. When the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, the difference value of the weights of the image categories of the positive label and the trusted negative label is smaller than a preset threshold value, and further, the model parameters of the initial classification model can be adjusted and optimized in a targeted mode based on the labeling labels of all sample images and the weights of the image categories represented by the labeling labels, and the model parameters of the initial classification model are focused on different image categories on the images, so that the trained image classification model can accurately identify different image categories on the images, and the classification accuracy of classification processing according to the obtained image classification model is further improved, and a precise and comprehensive image identification result is obtained.

In one embodiment, as shown in fig. 8, the step of obtaining a trained image classification model, that is, training an initial classification model according to each sample image, and if it is determined that the training end condition is met, obtaining a trained image classification model specifically includes:

step S802, training the initial classification model according to each sample image, and obtaining a trained initial image classification model if the training ending condition is met.

Specifically, in the training process, each sample image is input into an initial classification model to obtain a prediction result of the initial classification model for carrying out image classification prediction on each sample image, and then weighting processing is carried out according to the prediction result of each sample image, the labeling labels carried by the sample image and the weights of each labeling label to determine a training loss value. If the training loss value meets the training ending condition, a trained initial image classification model is obtained, wherein the training ending condition can be that the training loss value in the training process of the initial classification model reaches a preset loss threshold value or the training iteration number of the initial classification model reaches a preset number of times.

According to the labeling labels carried by each sample image, the image types included on each sample image and the label types to which each image type belongs can be determined, for example, the image type of the positive label corresponds to which image type, the image type of the trusted negative label corresponds to which image type or image types, and the image type of the untrusted label corresponds to which image type or image types.

Specifically, after labeling each sample image to be labeled, obtaining a labeled sample image set, and then obtaining labeling labels of each labeled sample image to determine each image category on the sample image according to the labeling labels. The labeling labels of the labeled sample image include an image class of a positive label, an image class of a trusted negative label and an image class of an untrusted label, for example, labeling sequences of the sample image set are "airland", "rake", "car" and "person", in a sample image after labeling, the image class of the positive label is determined to be "rake", and according to the image class of the positive label, the labeling sequence and a possible image class in the sample image, the image class of the "airland" is the image class of the trusted negative label because the image class of the "airland" is not recognized, and meanwhile, the remaining "car" and "person" can be determined to be the image class of the untrusted label because the image class of the "rake" is the image class of the positive label.

Further, in the training process of the model, weights of image categories represented by different labeling labels are inconsistent, namely, the weights of the image categories of the unreliable labels are smaller than those of the image categories of the positive labels and those of the image categories of the trusted negative labels, and the difference between the weights of the image categories of the positive labels and those of the image categories of the trusted negative labels are smaller than a preset threshold, so that the weights of the image categories of the positive labels, the image categories of the trusted negative labels and those of the image categories of the unreliable labels are required to be determined respectively, and the training loss value is determined by weighting according to the prediction result of each sample image, the labeling labels carried by the sample image and the weights of each labeling label.

For example, taking labeling sequence of the sample image set to be labeled as "airland", "rake", "car" and "person" as examples, labeling each sample image to be labeled in the sample image set to be labeled, and obtaining a labeled sample image set. For a certain sample image in the marked sample image set, the image category of the positive label is "bike", the image category of the trusted negative label is "airland", and the image categories of the untrusted labels comprise "car" and "person".

Further, classifying and predicting each sample image in the marked sample image set through an initial classification model to obtain a corresponding prediction result, namely obtaining the probability of each image category belonging to 'airland', 'bike', 'person'.

The method comprises the steps that an initial classification model is trained by using image categories of trusted labels (including positive labels and trusted negative labels) of all image samples in a training process, and then weights of the image category 'bike' serving as the positive label and the image category 'airland' serving as the trusted negative label are set to be 1, so that the training loss value in the training process is calculated and obtained in the training process in the training loss value calculating process. Similarly, since the 'car' and the 'person' belong to the type of the unreliable image, by setting the weights of the two types to 0, calculation of the training loss value in the training process is not counted, and noise influence of the unreliable label in the model training process can be reduced.

Step S804, determining a pseudo tag sample from the sample image set based on the trained initial image classification model.

Specifically, through a trained initial image classification model, classifying each sample image in a sample image set to obtain class output probabilities corresponding to each sample image, and determining a pseudo tag sample from the sample image set according to the class output probabilities. The false label sample is obtained by determining the image category of the unreliable label from the sample image set and further screening based on the image category of the unreliable label.

The training method comprises the steps of training a trained initial image classification model, wherein the aim of improving model accuracy is to further train the trained initial image classification model, and an image sample utilized during training is to determine a pseudo tag sample from a sample image set according to class output probability. Specifically, each sample image in the sample image set is input into a trained initial image classification model, and classification processing is performed on each sample image through the trained initial image classification model, so that class output probabilities corresponding to each sample image are obtained. The class output probability can also be understood as a class prediction result of the trained initial image classification model on the sample image, that is, the image class of each sample image and the label to which each image class belongs are predicted, for example, the image class belonging to the positive label, the image class of the trusted negative label, the image class of the untrusted label, and the like.

Further, based on the class output probability, a pseudo positive label sample with the label being an unreliable label and the class output probability being greater than a first preset threshold value and a pseudo negative label sample with the label being an unreliable label and the class output probability being less than a second preset threshold value are determined from the sample image set. The first preset threshold value and the second preset threshold value are different, and the first preset threshold value is greater than the second preset threshold value, for example, the first preset threshold value is 0.9, and the second preset threshold value is 0.01, but the first preset threshold value and the second preset threshold value are not limited to some or some values of examples, but can be set and adjusted according to actual requirements, for example, the first preset threshold value is 0.95, the second preset threshold value is 0.02, and the like.

For example, the labeling order of the sample image set to be labeled is "airland", "rake", "car" and "person", for example, for a sample image after a certain label, the image class of the positive label is "rake", and the image class of the trusted negative label is "airland". The method comprises the steps of obtaining an initial image classification model, wherein the initial image classification model is trained, the class output probability of ' airplane ' is 0.01, the class output probability of ' bike ' is 0.96, the class output probability of ' car ' is 0.88, the class output probability of ' person ' is 0.92, and the image class of a positive label ' bike ' and the image class of a trusted negative label ' airplane ' are removed from the output class output probabilities because the image class of an unreliable label needs to be reserved, and the class output probabilities of the image class ' car ' and the person ' remain.

Further, the class output probability of the image class of the unreliable label, namely the class output probability of the car and the class output probability of the person, is respectively compared with a first preset threshold value and a second preset threshold value, and a pseudo positive label sample or a pseudo negative label sample is determined. Taking a first preset threshold value as 0.9, taking a second preset threshold value as 0.01 as an example, and comparing the class output probability of person with the first preset threshold value and the second preset threshold value respectively to obtain that the class output probability of person is larger than the first preset threshold value and meets the condition of pseudo positive label samples, and determining the image class person as the pseudo positive label samples. And because the class output probability of the person is larger than the second preset threshold value, the condition of the pseudo negative label sample is not met, and the image class of the person does not belong to the pseudo negative label sample.

Similarly, taking the first preset threshold value as 0.9 and the second preset threshold value as 0.01 as an example, when the class output probability of 'car' is 0.88 and the first preset threshold value and the second preset threshold value are respectively compared, the fact that the class output probability of 'car' is smaller than the first preset threshold value and larger than the second preset threshold value and does not meet the condition of the pseudo positive label sample or the pseudo negative label sample is known, and then the image class 'car' does not belong to the pseudo positive label sample or the pseudo negative label sample.

Step S806, training the trained initial image classification model according to the pseudo tag sample to obtain a trained image classification model.

Specifically, a pseudo tag sample set is obtained according to each pseudo positive tag sample and each pseudo negative tag sample, and then a trained initial image classification model can be trained according to the pseudo tag sample set obtained by the pseudo positive tag sample and the pseudo negative tag sample, so that a trained image classification model is obtained.

Further, when training the trained initial image classification model according to the pseudo tag sample set, it is also required to determine whether the training end condition is reached, and when determining that the training end condition is reached, obtain the trained image classification model. The training ending condition may be that the calculated training loss value reaches a preset loss threshold value in the training process of the trained initial image classification model, or may be that the training iteration number of the trained initial image classification model reaches a preset number threshold value.

In this embodiment, the initial classification model is trained according to each sample image, if the training end condition is satisfied, a trained initial image classification model is obtained, a pseudo tag sample is determined from a sample image set based on the trained initial image classification model, and then the trained initial image classification model is trained according to the pseudo tag sample, so as to obtain a trained image classification model. The training of the initial classification model by using the sample image is realized, so that a trained initial image classification model is obtained, the trained initial image classification model is secondarily trained based on the pseudo-label sample, the diversity and the comprehensiveness of the training sample in the model training process are improved, and the model precision of the obtained image classification model and the accuracy of subsequent image recognition and image classification are further improved through multi-layer training.

In one embodiment, as shown in fig. 9, the step of determining the training loss value, that is, performing weighting processing according to the prediction result of each sample image, the labeling label and the weight of each labeling label, specifically includes:

step S902, determining the category represented by the label of the prediction result according to the prediction result of the sample image and the label of the sample image.

Specifically, in the training process, each sample image is input into an initial classification model to obtain a prediction result of the initial classification model for performing image classification prediction on each sample image, and a labeling label obtained by labeling each sample image in advance is obtained.

Further, based on the prediction result of the sample image and the labeling label of the sample image, the category of the labeling label representation of the prediction result can be determined. For example, taking the labeling order of a class a, a class B and a class C as an example, the image class of the sample image is predicted to be the class C, and the image class of the positive label in the labeling label of the sample image is the class B, which indicates that after labeling the sample image, the class a is the image class of the trusted negative label, the class B is the image class of the positive label, and the class C is the image class of the untrusted label. And then, according to the labeling label of the sample image, namely the image type of the positive label is B type, the image type of the unreliable label is C type, the type C indicated by the prediction result can be determined, and the image type of the unreliable label is included.

Similarly, for example, taking the labeling order of a type, B type and C type as an example, the image type of the sample image is predicted to be B type, and the image type of the positive label in the labeling label of the sample image is also B type, which indicates that after labeling the sample image, the a type is the image type of the trusted negative label, the B type is the image type of the positive label, and the C type is the image type of the untrusted label. And then, according to the labeling label of the sample image, namely the image type of the positive label is the B type, the type B indicated by the prediction result is determined, and the label belongs to the image type of the positive label.

In step S904, the weight of the category indicated by the label is determined as the weight of the prediction result.

Specifically, after determining the category represented by the labeling label of the prediction result, the weight of the category represented by the labeling label is further obtained, and the weight of the category represented by the labeling label is determined as the weight of the prediction result. The weights of the categories represented by the labeling labels specifically comprise the weights of the image categories of the positive labels, the weights of the image categories of the trusted negative labels and the weights of the image categories of the untrusted labels.

Further, the difference between the weight of the image class of the positive tag and the weight of the image class of the trusted negative tag is smaller than a preset threshold, and the two may be set to the same value, for example, the weight of the image class of the positive tag and the weight of the image class of the trusted negative tag are both set to 1, while the weight of the image class of the untrusted tag is smaller than the weight of the image class of the positive tag and also smaller than the weight of the image class of the trusted negative tag, for example, the weight of the image class of the untrusted tag may be set to 0.

Specifically, when the image category represented by the prediction result is determined to be a positive label or a negative label, a first weight (for example, the first weight may be set to 1) corresponding to the image category of the positive label or the negative label is determined to be the weight of the image category represented by the prediction result. Similarly, when the image category represented by the prediction result is an untrusted label, a second weight (for example, the second weight may be set to 0) corresponding to the image category of the untrusted label is determined as the weight of the image category represented by the prediction result.

By setting the first weight to 1 and setting the second weight to 0, it can be ensured that in each training process, the positive label and the trusted negative label with the first weight of 1 are adopted, and the untrusted label with the second weight of 0 is not adopted, so that noise influence on the model caused by the untrusted label in the sample image in the training process is reduced.

Step S906, weighting each predicted result according to the weight of each predicted result, and determining a training loss value.

Specifically, according to the determined weights of the predicted results of the sample images, weighting the predicted results respectively to obtain training loss values for training the initial classification model by using the sample image set.

Further, the training loss value L is calculated by performing weighting processing using the following formula (1):

wherein L represents a training loss value, C represents the total number of image categories included in each sample image, specifically, the total number is determined by extracting features of each sample image, and by extracting features, it can be determined which categories the sample image can be specifically classified into or which object categories are included, for example, different object categories including an automobile, a bicycle, a person, a building and the like, and further, the sample image can be classified into an automobile image category, a bicycle image category, a person image category, a building image category and the like. i represents a certain image category, wi represents a weight corresponding to an i-category image, yi represents an actual image category determined when labeling a sample image, yi is a 01 vector, two results of 0 and 1 can be obtained, and can be represented as [ y0, y1, y2 … yi … yC ], if the actual image category is an image category of a positive label, yi is 1, for example, y0 is 1, the vector represented by yi is [1,0, … 0], if the actual image category is an image category of a trusted negative label, yi is 0, xi represents a probability that the sample image is subjected to classification prediction by using an initial classification image in a training process, and the obtained prediction result is the i-category image, and the obtained prediction result is in a range of (0, 1).

In this embodiment, a class represented by a label of a prediction result is determined according to the prediction result of a sample image and the label of the sample image, and a weight of the class represented by the label is determined as a weight of the prediction result, and then each prediction result is weighted according to the weight of each prediction result, so as to determine a training loss value. The training loss value in the training process is determined based on the labeling labels of the sample images and the weights of the image categories represented by the labeling labels, so that the model parameters of the initial classification model are adjusted and optimized in a targeted mode, and when the training loss value reaches the preset loss threshold value, a trained image classification model is obtained, so that different image categories on the images can be accurately identified according to the trained image classification model, and an accurate and comprehensive image identification result is obtained.

In one embodiment, as shown in fig. 10, a method for constructing an image classification model is provided, which specifically includes the following steps:

step S1001, acquiring a sample image set; each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises: the image category of positive labels, the image category of trusted negative labels, and the image category of untrusted labels.

Step S1002, training an initial classification model according to each sample image, and obtaining a prediction result of the initial classification model for performing image classification prediction on each sample image.

Step S1003, determining a category represented by the label of the prediction result according to the prediction result of the sample image and the label of the sample image.

Step S1004, determining the weight of the category represented by the label as the weight of the prediction result.

Step S1005, carrying out weighting processing on each prediction result according to the weight of each prediction result, and determining a training loss value, wherein when the training loss value is calculated in the training process, the weight of the image class of the unreliable label is smaller than that of the image class of the positive label and that of the image class of the trusted negative label, and the difference value of the weights of the image classes of the positive label and the trusted negative label is smaller than a preset threshold value.

Step S1006, if the training loss value is determined to meet the training ending condition, a trained initial image classification model is obtained.

Step S1007, classifying each sample image in the sample image set through the trained initial image classification model to obtain class output probability corresponding to each sample image.

Step S1008, based on the class output probability, determining a pseudo positive label sample with the label being an unreliable label and the class output probability being greater than a first preset threshold value and a pseudo negative label sample with the label being an unreliable label and the class output probability being less than a second preset threshold value from the sample image set.

Step S1009, training the trained initial image classification model according to each pseudo positive label sample and each pseudo negative label sample to obtain a trained image classification model.

In one embodiment, as shown in fig. 11, a construction flow of an image classification model is provided, and referring to fig. 11, it can be seen that the construction flow of the image classification model includes the following two stages:

stage 1, model training

Specifically, in the stage 1, namely a model training stage, training an initial classification model by acquiring a sample image set and according to each sample image in the sample image set, and obtaining a trained initial image classification model when determining that a training loss value meets a training ending condition.

Each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises: the method comprises the steps that when a training loss value is calculated in the training process, the weight of the image class of the unreliable label is smaller than that of the image class of the positive label and that of the image class of the trusted negative label, and the difference value of the weights of the image classes of the positive label and the trusted negative label is smaller than a preset threshold value.

In one embodiment, as can be seen from fig. 11, in the stage 1, the labeling sequence of the sample image set to be labeled is "airland", "rake", "car" and "person", and after labeling each sample image to be labeled in the sample image set to be labeled, a labeled sample image set is obtained. For a certain sample image in the marked sample image set, the image category of the positive label is "bike", the image category of the trusted negative label is "airland", and the image categories of the untrusted labels comprise "car" and "person".

Specifically, referring to fig. 11, it can be seen that, by performing classification prediction on each sample image in the labeled sample image set by using the initial classification model, a corresponding prediction result xi is obtained, that is, a probability xi that each image category belongs to "airland", "rake", "car" and "person" is obtained, for example, a probability that an image category of a certain sample image is obtained is 0.1, a probability that an image category of the certain sample image is obtained is 0.9, a probability that an image category of the certain sample image is obtained is 0.7, and a probability that an image category of the certain sample image is "person" is 0.8.

Further, since the labeling order of the sample image set to be labeled is "airland", "bike", "person", when labeling the sample image to be labeled, only the image category of "bike" is labeled as the positive label, that is, the image category of "bike" is determined as the positive label, the actual image category yi corresponding to "bike" is 1, likewise, the image category of "airland" is determined as the image category of the trusted negative label, the actual image category yi corresponding to "airland" is 0, and since the image category of "car" and "person" is determined as the image category of the untrusted label, the actual image category yi corresponding to the image category of "car" and "person" is not specifically limited, and? "to indicate.

In the stage 1, "car" and "person" belong to the unreliable image category, by setting the weights of the two categories to 0, calculation of the training loss value in the training process is not counted, noise influence caused by the unreliable label in the model training process can be reduced, and further, the corresponding actual image category yi of the two categories does not need to be used in the training loss calculation process, and further, specific limitation on the value of the corresponding actual image category yi is not needed, and the? The "reference number" or other reference number may be used for representative purposes.

Similarly, because the image categories of the credible labels (including the positive labels and the credible negative labels) of each image sample are required to be used in the training process, the initial classification model is trained, and then the weights of the image category 'bike' serving as the positive label and the image category 'airland' serving as the credible negative label are set to be 1, so that the training loss value in the training process is calculated and obtained in the training process.

Stage 2, pseudo tag generation, model secondary training

Specifically, in the stage 2, namely the generation of the pseudo tag and the secondary training of the model, a pseudo tag sample is determined from a sample image set based on a trained initial image classification model, and the trained initial image classification model is trained according to the pseudo tag sample to obtain a trained image classification model.

Specifically, through a trained initial image classification model, classifying each sample image in a sample image set to obtain class output probabilities corresponding to each sample image, and determining a pseudo positive label sample with an unreliable label and a class output probability greater than a first preset threshold value and a pseudo negative label sample with an unreliable label and a class output probability less than a second preset threshold value from the sample image set based on the class output probabilities. And finally, training the trained initial image classification model according to each pseudo positive label sample and each pseudo negative label sample to obtain a trained image classification model.

Further, referring to fig. 11, the labeling order of the sample image set to be labeled is "airland", "bike", "car" and "person", and each sample image in the sample image set after labeling is input into the trained initial image classification model to obtain a class output probability corresponding to each sample image, and a pseudo positive label sample and a pseudo negative label sample are determined from the sample image set based on the class output probability.

Specifically, for example, for a sample image after a certain label, the image class of the positive label is "bike", and the image class of the trusted negative label is "airland". The class output probability of "airland" obtained by using the trained initial image classification model is 0.01, the class output probability of "rake" is 0.96, the class output probability of "car" is 0.88, and the class output probability of "person" is 0.92.

Further, by removing the image class "bike" of the positive label, and the image class "airland" of the trusted negative label from the output class output probabilities, the image classes of the untrusted labels (i.e., image classes "car" and "person") remain. The method comprises the steps of comparing the class output probability of the image class of the unreliable label with a first preset threshold value and a second preset threshold value respectively to determine a pseudo positive label sample with the label being the unreliable label and the class output probability being greater than the first preset threshold value or a pseudo negative label sample with the label being the unreliable label and the class output probability being less than the second preset threshold value, wherein the class output probability of the image class of the unreliable label is further calculated according to the class output probability of the image class of the unreliable label, namely the class output probability of the car and the class output probability of the person.

Taking the first preset threshold value as 0.9 and the second preset threshold value as an example, and comparing the class output probability 0.88 of the car with the first preset threshold value and the second preset threshold value respectively, the fact that the class output probability of the car is smaller than the first preset threshold value and larger than the second preset threshold value and does not meet the condition of the pseudo positive label sample or the pseudo negative label sample is known, and then the image class car does not belong to the pseudo positive label sample or the pseudo negative label sample.

Similarly, taking the first preset threshold value as 0.9 and the second preset threshold value as 0.01 as an example, when the class output probability of person is 0.92 and the first preset threshold value and the second preset threshold value are respectively compared, the class output probability of person is larger than the first preset threshold value, and the condition of pseudo positive label samples is met, and then the image class person is determined to be the pseudo positive label sample. The class output probability of the person is also larger than the second preset threshold value, and the condition of the pseudo negative label sample is not met, so that the image class of the person does not belong to the pseudo negative label sample.

In one embodiment, as shown in fig. 12, an image classification method is provided, and the method is applied to the server in fig. 1 for illustration, and specifically includes the following steps:

step S1202, an image classification request is received, and an image to be classified corresponding to the image classification request is acquired.

Specifically, if an image classification request is received, the image classification request is analyzed to obtain an image to be classified corresponding to the image classification request. The images to be classified can be images corresponding to various different image categories, or images with various object categories, such as images with different categories of automobiles, bicycles, people, buildings and the like.

Step S1204, classifying the images to be classified according to a trained image classification model to obtain at least one image category corresponding to the images to be classified, wherein the trained image classification model is obtained by training an initial classification model by using a sample image set, each sample image in the sample image set carries a labeling label, and the categories represented by the labeling label comprise an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label; when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image category of the positive label and the image category of the trusted negative label is smaller than a preset threshold value.

Specifically, the image to be classified is classified by using the trained image classification model, so that at least one image category corresponding to the image to be classified is obtained, for example, the image category on the image to be classified is obtained as an automobile image category, and for example, the image category on the image to be classified is obtained as a human image category. It can be understood that by utilizing the trained image classification model to classify the image to be classified, a plurality of different image categories corresponding to the image to be classified can be determined, so that the image to be classified can be divided into a plurality of different image categories, and the accuracy of multi-category identification and multi-classification of the image under different actual scenes, such as scenes of image content identification, target object detection and the like, is improved.

In one embodiment, the trained image classification model is obtained by training the initial classification model with the sample image set and satisfying the training end condition. Specifically, the sample image set for the initial classification model may specifically be a sample image set after labeling, where the sample image set after labeling includes a plurality of sample images, and each sample image carries labeling labels. The labeling label is used for representing the image type carried by the sample image, and the type represented by the labeling label specifically comprises the image type of the positive label, the image type of the trusted negative label and the image type of the untrusted label.

Before obtaining the sample image set, the sample set of the image to be marked needs to be obtained, each sample of the image to be marked in the sample image set to be marked is marked orderly according to a preset certain marking sequence or a preset marking sequence, marking labels are added for each sample image, and therefore image types included in each sample image are determined.

Further, training the initial classification model according to the marked sample images, and completing training the initial classification model when the training ending condition is confirmed to be met, so as to obtain a trained image classification model. The initial classification model can be a basic deep learning network model or a convolutional neural network model, and is trained by using each marked sample image, and model parameters of the initial classification model are adjusted and updated until training ending conditions are met. When the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image category of the positive label and the image category of the trusted negative label is smaller than a preset threshold value.

In one embodiment, the classification models obtained by different training methods are applied to the MS COCO dataset (i.e., a common large dataset including scenes for object detection, image segmentation, and image description), and the following table 1 (i.e., an average precision average table of the models obtained by different training methods on the MS COCO dataset) is used for comparison and description:

TABLE 1 prediction results MAP table of model on MS COCO data set obtained by different training modes

Training mode	Prediction result MAP
		Large Loss Matter	61
Full labeling	78
		Single tag direct training	55
Orderly labeling bias training, and dividing a data set into 1 batch	65
		Orderly labeling bias training, and dividing a data set into 10 batches	68.5
Orderly labeling bias training, and dividing a data set into 20 batches	69.2

Specifically, referring to table 1, it can be seen that the training method of Large Loss Matter (namely Large Loss Matters in Weakly Supervised Multi-Label Classification, which is understood to be a training method using a large-loss sample in the weak-supervision multi-label classification) is 61 as a prediction result MAP (which is collectively referred to as Mean Average Precision, namely, average precision average value, used for evaluating model precision and prediction accuracy of a model, and that the larger the MAP is, the higher the model precision and the higher the prediction accuracy of a corresponding model are). Similarly, the initial model is trained to obtain a classification model by performing full labeling on the image samples to obtain labeled image samples (i.e., all image categories or object categories included in the samples are labeled for each image sample), the prediction result MAP of the classification model on the MS COCO dataset is 78, and the initial model is trained to obtain a classification model by performing single label labeling on the image samples (i.e., randomly labeling one image category for each image sample), and the prediction result MAP of the classification model on the MS COCO dataset is 55.

Further, the image classification model constructed by the image classification model construction method provided in the embodiment of the present application (i.e. the orderly labeling bias training mode recorded in table 1) has different prediction results MAP according to different batches of data set division in the orderly labeling process.

As can be seen from table 1, when the orderly labeling bias training method is adopted, the image sample data set is divided into 1 batch for orderly labeling, the labeled image sample data set is utilized to train the initial model, and the prediction result MAP of the obtained image classification model on the MS COCO data set is 65. And after the image sample data set is divided into 10 batches for orderly labeling, training an initial model by using the labeled image sample data set, wherein the prediction result MAP of the obtained image classification model on the MS COCO data set is 68.5, and similarly, after the image sample data set is divided into 20 batches for orderly labeling, training the initial model by using the labeled image sample data set, and the prediction result MAP of the obtained image classification model on the MS COCO data set is 69.2. Therefore, the more batches of the image sample are divided, the marking sequence is different for each batch, so that the marking process and the marked image sample data set have larger randomness, and the data error brought by the marking process and the marked image sample data set is smaller, so that the prediction result of the trained image classification model has higher MAP (mean value of average accuracy).

It can be understood that, the image classification model constructed by the image classification model construction method provided by the embodiment of the application has a prediction result MAP higher than the classification model trained by Large Loss Matter and the classification model trained by the single label labeling mode. The classification model obtained by training in the full labeling mode needs to consume a great deal of time and resources to label all categories on each image sample, and the labeling time, training time and resource consumption are far greater than those of the construction method of the image classification model provided by the embodiment of the application, so that the construction method of the image classification model provided by the embodiment of the application consumes less resources and can ensure the MAP of a higher prediction result while considering the comprehensive resource consumption and the MAP of the prediction result.

In the image classification method, the image classification request is received, the image to be classified corresponding to the image classification request is obtained, and then the image to be classified is classified according to the trained image classification model, so that at least one image category corresponding to the image to be classified is obtained. The trained image classification model is obtained by training an initial classification model by using a sample image set and is obtained when the training end condition is met, each sample image in the sample image set used for training the initial classification model carries a labeling label, and the class represented by the labeling label comprises the image class of a positive label, the image class of a trusted negative label and the image class of an untrusted label. Further, when the training loss value is calculated in the training process of the initial classification model, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, the difference value between the image category of the positive label and that of the image category of the trusted negative label is smaller than a preset threshold value, and further, the model parameters of the initial classification model are subjected to targeted adjustment and optimization based on the labeling labels of all sample images and the weights of the image categories represented by the labeling labels, and when the training loss value is determined to reach the preset loss threshold value, a trained image classification model is obtained, so that the accuracy of different image categories on the identified images according to the trained image classification model is improved, and a precise and comprehensive image identification result is obtained.

It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a label labeling device for realizing the label labeling method, a construction device of an image classification model and an image classification device of the image classification method. The implementation scheme of the device for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in the embodiments of the device for constructing one or more image classification models, the device for labeling labels, and the device for classifying images provided below can be referred to the above limitations of the method for labeling labels, the method for constructing the image classification model, and the method for classifying images, which are not described herein.

In one embodiment, as shown in fig. 13, there is provided a label marking apparatus comprising: a sample image set to be annotated obtaining module 1302, an annotation order determining module 1304, and a label annotating module 1306, wherein:

the sample image set to be annotated obtaining module 1302 is configured to obtain a sample image set to be annotated.

The labeling order determining module 1304 is configured to determine a labeling order corresponding to the sample image set to be labeled.

The label labeling module 1306 is configured to perform ordered single label labeling on each sample image to be labeled in the sample image set to obtain a labeled sample image set according to a labeling sequence, where each sample image in the labeled sample image set carries a label, and a category represented by the label includes an image category of a positive label, an image category of a trusted negative label, and an image category of an untrusted label.

In the label labeling device, the sample image set to be labeled is obtained, the labeling sequence corresponding to the sample image set to be labeled is determined, and then each sample image to be labeled in the sample image set to be labeled is sequentially labeled with a single label according to the labeling sequence, so that the labeled sample image set is obtained. The method comprises the steps that each sample image in a sample image set after labeling carries labeling labels, and the categories represented by the labeling labels specifically comprise image categories of positive labels, image categories of trusted negative labels and image categories of untrusted labels. The method can quickly and accurately obtain the image type of the positive label, the image type of the trusted negative label and the image type of the untrusted label carried by each sample image by adopting an orderly single label labeling mode, so that in the subsequent training process of the model by using the labeled sample image set, the specific adjustment optimization of model parameters is realized according to the labeling labels of each sample image and the image types represented by the labeling labels, and the trained classification model can concentrate on different image types on the image, thereby obtaining accurate and comprehensive image recognition results.

In one embodiment, a label labeling device is provided, and the label labeling device further comprises an image set dividing module, wherein the image set dividing module is used for dividing a sample image set to be labeled into sub-image sets of preset batches, and each sub-image set of each batch comprises a plurality of sample images to be labeled.

In one embodiment, the tag labeling module is further configured to: according to the labeling sequence corresponding to the sub-image sets of each batch, respectively labeling each sample image to be labeled in the sub-image sets of each batch in an ordered single label manner; and obtaining a marked sample image set according to each sample image marked in the sub-image set of each batch.

In one embodiment, the tag labeling module is further configured to: performing feature recognition on each sample image to be marked to obtain image categories included in each sample image to be marked; according to the labeling sequence corresponding to the sub-image set, orderly labeling each sample image to be labeled in the sub-image set, and determining the image category of the positive label from the image categories included in the sample image to be labeled; and determining the image category of the trusted negative label and the image category of the untrusted label in the sample image to be marked based on the marking sequence, the image category of the positive label and the image category included in the sample image to be marked.

In one embodiment, as shown in fig. 14, there is provided a construction apparatus of an image classification model, including: a sample image set acquisition module 1402 and an image classification model acquisition module 1404, wherein:

a sample image set obtaining module 1402, configured to obtain a sample image set; each sample image in the sample image set carries a labeling label, and the category represented by the labeling label comprises: the image category of positive labels, the image category of trusted negative labels, and the image category of untrusted labels.

An image classification model obtaining module 1404, configured to train the initial classification model according to each sample image, and obtain a trained image classification model if it is determined that the training end condition is satisfied; when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image categories of the positive label and the trusted negative label is smaller than a preset threshold value.

In the device for constructing the image classification model, the initial classification model is trained according to each sample image by acquiring the sample image set, so that the trained image classification model is obtained when the condition that the training is finished is confirmed to be met. The sample image set comprises a plurality of sample images, each sample image carries a labeling label, and then according to the labeling label of each sample image, the actual image type of each sample image can be determined, and the actual image type can be the image type of a positive label, the image type of a trusted negative label or the image type of an untrusted label. When the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, the difference value of the weights of the image categories of the positive label and the trusted negative label is smaller than a preset threshold value, and further, the model parameters of the initial classification model can be adjusted and optimized in a targeted mode based on the labeling labels of all sample images and the weights of the image categories represented by the labeling labels, and the model parameters of the initial classification model are focused on different image categories on the images, so that the trained image classification model can accurately identify different image categories on the images, and the classification accuracy of classification processing according to the obtained image classification model is further improved, and a precise and comprehensive image identification result is obtained.

In one embodiment, the image classification model obtaining module is further configured to: training the initial classification model according to each sample image, and obtaining a trained initial image classification model if the training ending condition is met; determining a pseudo tag sample from the sample image set based on the trained initial image classification model; and training the trained initial image classification model according to the pseudo tag sample to obtain a trained image classification model.

In one embodiment, the image classification model obtaining module is further configured to: obtaining a prediction result of the initial classification model for carrying out image classification prediction on each sample image; weighting according to the prediction result of each sample image, the labeling label and the weight of each labeling label, and determining a training loss value; and if the training loss value is determined to meet the training ending condition, obtaining a trained initial image classification model.

In one embodiment, the image classification model obtaining module is further configured to: classifying each sample image in the sample image set through the trained initial image classification model to obtain class output probability corresponding to each sample image; and determining a pseudo tag sample from the sample image set according to the class output probability.

In one embodiment, the image classification model obtaining module is further configured to: based on the class output probability, determining a pseudo positive label sample with the label being an unreliable label and the class output probability being greater than a first preset threshold value and a pseudo negative label sample with the label being an unreliable label and the class output probability being less than a second preset threshold value from the sample image set; and training the trained initial image classification model according to each pseudo positive label sample and each pseudo negative label sample to obtain a trained image classification model.

In one embodiment, the image classification model obtaining module is further configured to: determining the category represented by the labeling label of the prediction result according to the prediction result of the sample image and the labeling label of the sample image; determining the weight of the category represented by the label as the weight of the prediction result; and weighting each predicted result according to the weight of each predicted result, and determining a training loss value.

In one embodiment, as shown in fig. 15, there is provided an image classification apparatus including: an image to be classified obtaining module 1502 and a classification processing module 1504, wherein:

the image to be classified obtaining module 1502 is configured to receive an image classification request, and obtain an image to be classified corresponding to the image classification request.

The classification processing module 1504 is configured to perform classification processing on an image to be classified according to a trained image classification model, to obtain at least one image category corresponding to the image to be classified, where the trained image classification model is obtained by training an initial classification model with a sample image set, and when a training end condition is met, each sample image in the sample image set carries a label, and a category represented by the label includes an image category of a positive label, an image category of a trusted negative label, and an image category of an untrusted label; when the training loss value is calculated in the training process, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, and the difference value of the weights of the image category of the positive label and the image category of the trusted negative label is smaller than a preset threshold value.

In the image classification device, the image classification request is received, the image to be classified corresponding to the image classification request is obtained, and then the image to be classified is classified according to the trained image classification model, so that at least one image category corresponding to the image to be classified is obtained. The trained image classification model is obtained by training an initial classification model by using a sample image set and is obtained when the training end condition is met, each sample image in the sample image set used for training the initial classification model carries a labeling label, and the class represented by the labeling label comprises the image class of a positive label, the image class of a trusted negative label and the image class of an untrusted label. Further, when the training loss value is calculated in the training process of the initial classification model, the weight of the image category of the unreliable label is smaller than that of the image category of the positive label and that of the image category of the trusted negative label, the difference value between the image category of the positive label and that of the image category of the trusted negative label is smaller than a preset threshold value, and further, the model parameters of the initial classification model are subjected to targeted adjustment and optimization based on the labeling labels of all sample images and the weights of the image categories represented by the labeling labels, and when the training loss value is determined to reach the preset loss threshold value, a trained image classification model is obtained, so that the accuracy of different image categories on the identified images according to the trained image classification model is improved, and a precise and comprehensive image identification result is obtained.

The label labeling device, the image classification model constructing device and each module in the image classification device can be realized by all or part of software, hardware and combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 16. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing a sample image set to be annotated, an annotation sequence corresponding to the sample image set to be annotated, an annotated sample image set, an annotation label carried by the sample image, categories represented by the annotation label (comprising an image category of a positive label, an image category of a trusted negative label and an image category of an untrusted label), initial classification models of weights of categories represented by different annotation labels (comprising weights of image categories of the untrusted label, weights of image categories of the trusted negative label and weights of image categories of the positive label), trained image classification models, images to be classified and at least one image category corresponding to the images to be classified. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a label labeling method, a construction method of an image classification model, and an image classification method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 16 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. A method of labeling labels, the method comprising:

acquiring a sample image set to be marked;

2. The method of claim 1, further comprising, prior to said determining the labeling order corresponding to the set of sample images to be labeled: dividing the sample image set to be annotated into sub-image sets of preset batches, wherein each batch of sub-image sets comprises a plurality of sample images to be annotated;

and according to the labeling sequence, orderly labeling the sample images to be labeled in the sample image set to be labeled to obtain a labeled sample image set, wherein the method comprises the following steps of:

according to the labeling sequence corresponding to the sub-image sets of each batch, respectively labeling each sample image to be labeled in the sub-image sets of each batch in an ordered single label manner;

and obtaining a marked sample image set according to each sample image marked in the sub-image set of each batch.

3. The method according to claim 2, wherein the orderly single-label labeling mode is performed on each sample image to be labeled in the sub-image set, and the method comprises the following steps:

Performing feature recognition on each sample image to be marked to obtain an image category included in each sample image to be marked;

according to the labeling sequence corresponding to the sub-image set, orderly labeling each sample image to be labeled in the sub-image set, and determining the image category of the positive label from the image categories included in the sample image to be labeled;

and determining the image category of the trusted negative label and the image category of the untrusted label in the sample image to be marked based on the marking sequence, the image category of the positive label and the image category included in the sample image to be marked.

4. A method of constructing an image classification model, the method comprising:

5. The method of claim 4, wherein training the initial classification model based on each of the sample images, and if it is determined that the training end condition is satisfied, obtaining a trained image classification model comprises:

training the initial classification model according to each sample image, and obtaining a trained initial image classification model if the training ending condition is met;

determining a pseudo tag sample from the sample image set based on the trained initial image classification model;

and training the trained initial image classification model according to the pseudo tag sample to obtain a trained image classification model.

6. The method of claim 5, wherein training the initial classification model based on each of the sample images, if a training end condition is satisfied, obtaining a trained initial image classification model comprises:

obtaining a prediction result of the initial classification model for carrying out image classification prediction on each sample image;

weighting according to the prediction result of each sample image, the labeling label and the weight of each labeling label to determine a training loss value;

And if the training loss value is determined to meet the training ending condition, obtaining a trained initial image classification model.

7. The method of claim 6, wherein the weighting to determine the training loss value based on the prediction result of each sample image, the label, and the weight of each label comprises:

determining the category represented by the labeling label of the prediction result according to the prediction result of the sample image and the labeling label of the sample image;

determining the weight of the category represented by the label as the weight of the prediction result;

and weighting each predicted result according to the weight of each predicted result, and determining a training loss value.

8. The method of claim 5 or 6, wherein the determining a pseudo tag sample from the sample image set based on the trained initial image classification model comprises:

classifying each sample image in the sample image set through the trained initial image classification model to obtain class output probability corresponding to each sample image;

and determining a pseudo tag sample from the sample image set according to the class output probability.

9. The method according to claim 8, wherein:

and determining a pseudo tag sample from the sample image set according to the class output probability, wherein the determining comprises the following steps:

based on the class output probability, determining a pseudo positive label sample, wherein the labeling label is an unreliable label and the class output probability is greater than a first preset threshold value, and a pseudo negative label sample, wherein the labeling label is an unreliable label and the class output probability is less than a second preset threshold value, from the sample image set;

training the trained initial image classification model according to the pseudo tag sample to obtain a trained image classification model, wherein the training comprises the following steps:

and training the trained initial image classification model according to each pseudo positive label sample and each pseudo negative label sample to obtain a trained image classification model.

10. A method of classifying images, the method comprising:

11. A label marking apparatus, the apparatus comprising:

12. An apparatus for constructing an image classification model, the apparatus comprising:

13. An image classification apparatus, the apparatus comprising:

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 10 when the computer program is executed.

15. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 10.