CN115937639A

CN115937639A - Labeling method of training sample, model training method, device, equipment and medium

Info

Publication number: CN115937639A
Application number: CN202211739045.1A
Authority: CN
Inventors: 刘聪毅
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-12-30
Filing date: 2022-12-30
Publication date: 2023-04-07

Abstract

The disclosure provides a labeling method of a training sample, a model training method, a model training device, equipment and a medium, relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, image processing and computer vision, and can be applied to scenes such as face recognition. The specific implementation scheme is as follows: obtaining a labeled sample and an unlabeled sample; inputting the labeled sample into a target task model to obtain a first task characteristic of the labeled sample according to the target task model; inputting the label-free sample into the target task model to obtain a second task characteristic and a task processing result of the label-free sample according to the target task model; and determining the task labeling information of the unlabeled sample according to the first task characteristic, the second task characteristic and the task processing result.

Description

Labeling method of training sample, model training method, device, equipment and medium

Technical Field

The disclosure relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, image processing and computer vision, and can be applied to scenes such as face recognition.

Background

The model training can be generally divided into three stages, namely, sample marking, feature extraction and model training. The larger the number of training samples of model training, the more accurate the labeling information of the training samples, and the higher the accuracy of the constructed model.

For sample labeling, a manual labeling mode is generally adopted at present. However, the operation of manually labeling a large number of training samples is complicated and error-prone, resulting in a decrease in efficiency and reliability of labeling of the training samples. At the present stage, the sample labeling is realized through a sample labeling model, and although the efficiency is improved, the accuracy is still low.

Disclosure of Invention

The disclosure provides a labeling method, a model training method, a device, equipment and a medium for training samples.

According to a first aspect of the present disclosure, there is provided a method for labeling a training sample, including:

obtaining a labeled sample and an unlabeled sample;

inputting the labeled sample into a target task model to obtain a first task characteristic of the labeled sample according to the target task model;

inputting the label-free sample into the target task model to obtain a second task characteristic and a task processing result of the label-free sample according to the target task model;

and determining the task labeling information of the unlabeled sample according to the first task characteristic, the second task characteristic and the task processing result.

According to a second aspect of the present disclosure, there is provided a model training method, comprising:

obtaining a labeled sample and an unlabeled sample;

determining task labeling information of the unlabeled sample according to the first task characteristic, the second task characteristic and the task processing result;

and performing model training on a pre-training model by adopting the labeled sample and the unlabeled sample of which the task labeling information is determined.

According to a third aspect of the present disclosure, there is provided an annotating device of a training sample, comprising:

the acquisition module is used for acquiring a labeled sample and an unlabeled sample;

the first characteristic determining module is used for inputting the labeled sample into a target task model so as to obtain a first task characteristic of the labeled sample according to the target task model;

the second characteristic determining module is used for inputting the label-free sample into the target task model so as to obtain a second task characteristic and a task processing result of the label-free sample according to the target task model;

and the labeling module is used for determining the task labeling information of the label-free sample according to the first task characteristic, the second task characteristic and the task processing result.

According to a fourth aspect of the present disclosure, there is provided a model training apparatus comprising:

the labeling module is used for determining task labeling information of the label-free sample according to the first task characteristic, the second task characteristic and the task processing result;

and the training module is used for carrying out model training on a pre-training model by adopting the labeled sample and the unlabeled sample of which the task labeling information is determined.

According to a fifth aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method provided by the first aspect or the second aspect.

According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method provided by the first or second aspect.

According to a seventh aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method provided according to the first or second aspect.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a flowchart of a method for labeling a training sample according to an exemplary embodiment of the present disclosure;

FIG. 2 is a flowchart of task annotation information for determining unlabeled exemplars provided by an exemplary embodiment of the present disclosure;

FIG. 3 is a flowchart of another method for labeling training samples according to an exemplary embodiment of the present disclosure;

FIG. 4 is a flowchart of a model training method provided by an exemplary embodiment of the present disclosure;

FIG. 5 is a block diagram of an apparatus for labeling training samples according to an exemplary embodiment of the present disclosure;

FIG. 6 is a block diagram of a model training apparatus according to an exemplary embodiment of the present disclosure;

fig. 7 is a block diagram of an electronic device according to an exemplary embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a flowchart of a method for labeling a training sample according to an exemplary embodiment of the present disclosure, including:

step 101, obtaining a labeled sample and an unlabeled sample.

And the unlabeled sample is a training sample to be labeled. The marking information of the task to be marked of the training sample is related to the purpose of the training sample, and if the training sample is used for training a living body recognition model, the marking information of the task to be marked of the training sample comprises a living body recognition result; if the training sample is used for training a target detection model, the to-be-labeled task labeling information of the training sample comprises a target detection result; and so on.

The labeled sample is a training sample labeled with task labeling information. The task labeling information, that is, the label of the labeled sample, may be artificially labeled or may be labeled based on artificial intelligence, and this is not particularly limited in the embodiments of the present disclosure. The task marking information of the labeled sample is generally matched with the to-be-marked task marking information of the unlabeled sample, and if the to-be-marked task marking information comprises a living body identification result, the task marking information of the labeled sample comprises the living body identification result; and if the task marking information to be marked comprises the target detection result, the task marking information of the labeled sample comprises the target detection result.

And 102, inputting the labeled sample into a target task model to obtain a first task characteristic of the labeled sample according to the target task model.

The target task model is a pre-trained model with task processing capacity, a task which can be processed by the target task model is matched with the labeling information of a task to be labeled of a label-free sample, and if the labeling information of the task to be labeled comprises a living body recognition result, the target task model is a living body recognition model; and if the to-be-annotated task annotation information comprises a target detection result, the target task model is a target detection model.

And 103, inputting the unlabelled sample into the target task model to obtain a second task characteristic of the unlabelled sample according to the target task model and obtain a task processing result according to the second task characteristic.

And step 104, determining task labeling information of the label-free sample according to the first task characteristic, the second task characteristic and the task processing result.

In the embodiment of the disclosure, the unlabeled samples are labeled by means of the first living body identification features of a small number of labeled samples, and the relatively reliable second living body identification features are generated, so that the accuracy and efficiency of living body labeling on the unlabeled samples can be improved.

In one embodiment, the number of target task models is 1. In step 102, the labeled sample is input into the target task model, and the first task characteristic of the labeled sample is directly obtained according to the target task model. As can be appreciated, the first task characteristic relates to a task that can be processed by the target task model.

In one embodiment, the number of target task models is at least 2. In step 102, the labeled samples are respectively input into the at least 2 target task models, and each target task model performs feature extraction on the labeled samples to obtain an initial first task feature. If n target task models exist, n initial first task features are obtained, and n is a positive integer greater than 2. And determining the weighted result of each initial first task characteristic as the first task characteristic of the labeled sample.

In the embodiment of the disclosure, the first task feature with the label sample is determined according to the multiple target task models, so that the error risk of a single target task model can be avoided, and the accuracy of the first task feature is improved.

The weighting coefficients of the initial first task features may be set according to the requirement, and may be set to be the same, for example, the weighting coefficients of the initial first task features are all 1/n; may also be arranged differently.

In one implementation, the weight coefficient of each initial first task feature is determined according to the accuracy of each target task model, and the weight coefficient of each first initial feature is positively correlated with the accuracy of the target task model with which the first initial feature is obtained.

For example, assume that the number of target task models is 3, which are: a target task model a, a target task model b and a target task model c; the accuracy of the target task model a is P1, and an initial first task characteristic a is obtained; the accuracy of the target task model b is P2, and an initial first task characteristic b is obtained; the accuracy of the target task model c is P3, and an initial first task characteristic c is obtained; if P1 > P2 > P3, the weight coefficient ω 1 of the first task feature a > the weight coefficient ω 2 of the first task feature b > the weight coefficient ω 3 of the first task feature c, and ω 1+ ω 2+ ω 3=1.

The higher the accuracy of the target task model is, the higher the accuracy of the output first task feature is, and the weight coefficient of the initial first task feature is determined according to the accuracy of the target task model, so that the first task feature of the labeled sample is determined, and the accuracy of the first task feature of the labeled sample can be further improved.

In one embodiment, in step 103, the unlabeled sample is input into a target task model, and the second task characteristics of the unlabeled sample and the task processing result (output of the target task model) are obtained according to the target task model.

As can be understood, the task processing result is related to the task that can be processed by the target task model; if the target task model is a living body recognition model, the task processing result is a living body recognition result, for example, the living body recognition confidence is included; and if the target task model is the target detection model, the task processing result is the target detection result.

In one embodiment, in step 103, the unlabeled samples are respectively input into at least 2 target task models, and each target task model performs feature extraction on the unlabeled sample to obtain an initial second task feature. If n target task models exist, n initial second task features are obtained, and n is a positive integer greater than 2. And determining the weighted result of each initial second task characteristic as the second task characteristic of the unlabeled sample.

In the embodiment of the disclosure, the second task characteristics of the unlabeled sample are determined according to the multiple target task models, so that the error risk of a single target task model can be avoided, and the accuracy of the second task characteristics is improved.

The weight coefficients of the initial second task features can be set according to requirements and can be set to be the same, for example, the weight coefficients of the initial second task features are all 1/n; may also be arranged differently.

In one implementation, the weight coefficient of each initial second task feature is determined according to the accuracy of each target task model, and the weight coefficient of each second initial feature is positively correlated with the accuracy of the target task model with the obtained second initial feature.

For example, assume that the number of target task models is 3, which are: a target task model a, a target task model b and a target task model c; the accuracy of the target task model a is P1, and an initial first task characteristic a is obtained; the accuracy of the target task model b is P2, and an initial second task characteristic b is obtained; the accuracy of the target task model c is P3, and an initial second task characteristic c is obtained; if P1 > P2 > P3, the weight coefficient ω 4 of the second task feature a > the weight coefficient ω 5 of the second task feature b > the weight coefficient ω 6 of the second task feature c, and ω 4+ ω 5+ ω 6=1.

The higher the accuracy of the target task model is, the higher the accuracy of the output second task features is, the weight coefficient of the initial second task features is determined according to the accuracy of the target task model, and then the second task features of the labeled samples are determined, so that the accuracy of the second task features of the labeled samples can be further improved.

In one embodiment, referring to fig. 2, step 104 comprises:

and step 104-1, determining initial labeling information of the unlabeled sample according to the task processing result.

In step 104-1, the target task model trained in advance is used for carrying out initial labeling on the unlabeled sample, so as to obtain initial labeling information of the unlabeled sample.

For example, assuming that the target task model is a living body recognition model, the output task processing result is represented by a living body confidence level, if the living body confidence level is greater than a confidence level threshold, the initial tagging information is determined to be a living body, and if the living body confidence level is less than or equal to the confidence level threshold, the initial tagging information is determined to be a non-living body (attack). Wherein, the confidence threshold is determined according to the actual situation.

And step 104-2, judging whether the initial marking information is reliable or not according to the first task characteristic and the second task characteristic.

In the embodiment of the disclosure, the reliability of the initial labeling information obtained by labeling the unlabeled sample by the target task model is verified by means of the task labeling information of the labeled sample, so that the accuracy of automatically labeling the unlabeled sample can be improved.

In one embodiment, whether the initial annotation information is reliable is determined according to the similarity between the first task characteristic and the second task characteristic. If the similarity is greater than the similarity threshold, which indicates that the initial annotation information determined in step 104-1 is reliable, executing step 104-3; and if the similarity is less than or equal to the similarity threshold, which indicates that the initial annotation information determined in the step 104-1 is unreliable, executing a step 104-4.

The first task feature and the second task feature extracted by the target task model are related to the tasks which can be processed by the target task model, and the relevance of the first task feature and the second task feature to the individual attributes of the sample is not large, so that whether the initial annotation information is reliable or not can be determined according to the similarity of the first task feature and the second task feature. Whether the initial marking information is reliable or not is determined based on the similarity, the accuracy is high, and the method is easy to implement.

In one embodiment, whether the initial annotation information is reliable is determined based on the first task characteristic of a plurality of labeled exemplars, which may be 2, 3, 4, or even more. Specifically, the method comprises the following steps: and calculating the similarity between the second task characteristics and the first task characteristics of each labeled sample, and determining whether the initial labeling information is reliable or not according to the task labeling information of the labeled sample with the highest similarity. If the task labeling information of the labeled sample with the highest similarity is the same as the initial labeling information or the similarity of the task labeling information and the initial labeling information is greater than a similarity threshold, determining that the initial labeling information is reliable; and if the task labeling information of the labeled sample with the highest similarity is different from the initial labeling information, or the similarity of the task labeling information and the initial labeling information is less than or equal to a similarity threshold, determining that the initial labeling information is unreliable.

The probability that the task labeling information of the labeled sample with the highest similarity is the same as the task labeling information of the unlabeled sample is high, whether the initial labeling information of the unlabeled sample is reliable or not is judged by means of the task labeling information of the labeled sample with the highest similarity, and then the task labeling information of the unlabeled sample is generated according to the reliable initial labeling information, so that the accuracy and the efficiency of labeling the unlabeled sample can be improved.

And step 104-3, marking the initial marking information as the task marking information of the unlabeled sample.

And step 104-4, determining that the initial marking information is unreliable.

In one embodiment, for unlabeled exemplars determined to be unreliable for initial labeling information, the unlabeled exemplars and their initial labeling information are sent to a target object, such as an experienced annotator, who verifies the initial labeling information or re-labels it.

In one embodiment, the unlabeled samples determined to be unreliable in initial labeling information are labeled, so that the unlabeled samples are prevented from participating in subsequent model training and influencing the accuracy of the model training.

In the embodiment of the disclosure, the unlabeled samples are labeled by means of a small amount of task labeling information of the labeled samples, so that relatively reliable task labeling information is generated, and the accuracy and efficiency of labeling the unlabeled samples can be improved. The label-free sample marked with the task marking information is used as a training sample for model training, so that the noise in the training sample can be reduced, and the accuracy and efficiency of model training are improved.

Face recognition is widely used for identity verification in daily life, and face living body recognition is always an important component of a face recognition system: once the attack picture or video breaks through the living human face recognition, serious consequences such as identity forgery and property loss can be caused. Therefore, it is very meaningful to continuously improve and optimize the face living body recognition model.

The following describes a process of performing living body identification labeling on a unlabeled sample by taking a target task model as a living body identification model as an example and referring to fig. 3.

S1, inputting the N labeled samples into the K living body recognition models respectively to obtain a first task characteristic of each labeled sample.

Wherein K is a positive integer greater than 1.

Each labeled sample is labeled with living body labeling information (task labeling information), and the living body labeling information comprises two types, one type is a living body and the other type is a non-living body.

And respectively inputting K living body identification models for each labeled sample to obtain K initial first task features, performing weighting processing on the K initial first task features, and determining the result of the weighting processing as the first task feature F of the labeled sample. And correspondingly obtaining N first task characteristics F when N labeled samples exist.

S2, inputting the unlabeled sample into the living body recognition model to obtain a first task characteristic and a living body confidence coefficient of the unlabeled sample.

And inputting the unlabeled samples into K living body recognition models to obtain K initial second task characteristics and living body confidence degrees (task processing results). And performing weighting processing on the K initial first task features, and determining the result of the weighting processing as a second task feature f of the unlabeled sample. And performing weighting processing on the K initial live confidence coefficients, and determining the result of the weighting processing as the live confidence coefficient p of the unlabeled sample. Wherein the live confidence p of the unlabeled exemplar is the unlabeled exemplar

And S3, determining initial labeling information of the label-free sample according to the confidence coefficient of the living body.

If the confidence coefficient of the living body is greater than the confidence coefficient threshold t, the initial labeling information of the unlabeled sample is the living body; if the confidence coefficient of the living body is less than 1-t, the initial labeling information of the unlabeled sample is 'non-living body'; and if the living body confidence coefficient is more than or equal to 1-t and less than or equal to t, determining that the initial annotation information cannot be determined according to the living body confidence coefficient.

And S4, respectively calculating the similarity S of the N first task features F and the second task features F.

And S5, determining the task marking information of the labeled sample corresponding to the first task feature F with the maximum similarity.

And S6, if the task labeling information of the labeled sample determined in the S5 is the same as the initial labeling information of the unlabeled sample determined in the S3, determining that the initial labeling information of the unlabeled sample is reliable.

For example, if the task annotation information of the labeled sample determined in S5 is "living body" and the initial annotation information of the unlabeled sample is also "living body", both are the same, it is determined that the initial annotation information of the unlabeled sample is reliable, and the task annotation information of the unlabeled sample is labeled "living body", and the unlabeled sample can be used for training the living body recognition model.

And if the task marking information of the labeled sample determined in the S5 is living body and the initial marking information of the unlabeled sample is non-living body, determining that the initial marking information of the unlabeled sample is unreliable, discarding the unlabeled sample and not used for model training.

Fig. 4 is a flowchart of a model training method provided in an exemplary embodiment of the present disclosure, where the model training method includes the following steps:

step 401, obtaining a labeled sample and an unlabeled sample.

And 402, inputting the labeled sample into the target task model to obtain a first task characteristic of the labeled sample according to the target task model.

And 403, inputting the unlabeled sample into the target task model to obtain a second task characteristic and a task processing result of the unlabeled sample according to the target task model.

And step 404, determining task labeling information of the label-free sample according to the first task characteristic, the second task characteristic and the task processing result.

The specific implementation manner of steps 401 to 404 is similar to that of steps 101 to 104, and is not described herein again.

And 405, performing model training on the pre-training model by adopting the labeled sample and the unlabeled sample with the determined task labeling information.

In step 405, the unlabeled samples with the determined task labeling information become labeled samples, and model training is performed on the pre-training model based on the labeled samples to obtain a task model with an expected purpose. It will be appreciated that the intended use of the task model is in relation to task annotation information.

In step 405, refer to the description of the related art for a specific implementation manner of model training, which is not described herein again.

In the embodiment of the disclosure, a small amount of task labeling information of labeled samples is used for labeling unlabeled samples to generate relatively reliable task labeling information, and the samples labeled with the task labeling information are used for model training, so that training samples labeled with errors in the task labeling information can be prevented from participating in model training as much as possible, noise in the training samples is reduced, and the accuracy of the task model obtained by training is improved.

The pre-training model can be selected according to actual requirements.

In one embodiment, a target task model is employed as the pre-training model. And during model training, performing model training on the target task model by adopting the labeled sample and the unlabeled sample with the determined task labeling information to optimize the target task model.

In the embodiment of the disclosure, the label-free samples are labeled through the target task model to expand and enrich the training samples, and then the training samples are adopted to carry out iterative training on the target task model, so as to achieve the purposes of optimizing the target task model and improving the accuracy of the target task model.

Corresponding to the embodiments of the labeling method and the model training method of the training sample, the disclosure also provides embodiments of a labeling device and a model training device of the training sample.

Fig. 5 is a schematic block diagram of an apparatus for labeling a training sample according to an exemplary embodiment of the present disclosure, where the apparatus for labeling a training sample includes:

an obtaining module 51, configured to obtain a labeled sample and an unlabeled sample;

a first feature determining module 52, configured to input the labeled sample into a target task model, so as to obtain a first task feature of the labeled sample according to the target task model;

a second feature determining module 53, configured to input the unlabeled sample into the target task model, so as to obtain a second task feature and a task processing result of the unlabeled sample according to the target task model;

and the labeling module 54 is configured to determine task labeling information of the unlabeled sample according to the first task feature, the second task feature, and the task processing result.

Optionally, the number of the target task models is at least two; the first feature determination module includes:

the first input unit is used for respectively inputting the labeled samples into at least two target task models so as to perform feature extraction on the labeled samples by each target task model to obtain initial first task features;

a first determining unit, configured to determine a weighted result of each of the initial first task features as the first task feature of the labeled sample.

Optionally, the weight coefficient of each initial first task feature is positively correlated with the accuracy of the target task model for obtaining the initial first task feature.

Optionally, the number of the target task models is at least two; the second feature determination module includes:

the second input unit is used for respectively inputting the unlabeled samples into at least two target task models so as to perform feature extraction on the unlabeled samples by each target task model to obtain initial second task features;

and a second determining unit, configured to determine a weighted result of each of the initial second task features as a second task feature of the unlabeled sample.

Optionally, the weight coefficient of each second initial feature is positively correlated with the accuracy of the target task model for obtaining the second initial feature.

Optionally, the target task model is a living body recognition model; the first task feature is a first living identification feature; the second task feature is a second living body identification feature; the task processing result comprises a living body recognition confidence;

the labeling module is specifically configured to:

and determining task labeling information of the unlabeled sample according to the first living body identification characteristic, the second living body identification characteristic and the second living body identification characteristic.

Optionally, the labeling module includes:

a third determining unit, configured to determine initial labeling information of the unlabeled exemplar according to the task processing result;

the judging unit is used for judging whether the initial labeling information is reliable or not according to the first task characteristic and the second task characteristic and calling the labeling unit under the condition of judging that the initial labeling information is reliable;

and the labeling unit is used for determining the initial labeling information as the task labeling information of the unlabeled sample.

Optionally, the number of labeled samples is at least two;

the judgment unit is specifically configured to:

calculating the similarity between the second task characteristics and the first task characteristics of each labeled sample;

and judging whether the initial labeling information is reliable or not according to the task labeling information of the labeled sample corresponding to the first task characteristic with the highest similarity.

Fig. 6 is a schematic block diagram of a model training apparatus according to an exemplary embodiment of the present disclosure, where the model training apparatus includes:

an obtaining module 61, configured to obtain a labeled sample and an unlabeled sample;

a first feature determining module 62, configured to input the labeled sample into a target task model, so as to obtain a first task feature of the labeled sample according to the target task model;

a second feature determining module 63, configured to input the unlabeled sample into the target task model, so as to obtain a second task feature and a task processing result of the unlabeled sample according to the target task model;

the labeling module 64 is configured to determine task labeling information of the unlabeled sample according to the first task feature, the second task feature, and the task processing result;

and the training module 65 is configured to perform model training on the pre-training model by using the labeled samples and the unlabeled samples for which the task labeling information is determined.

Optionally, the pre-training model comprises a target task model; the training module is specifically configured to:

and performing model training on the target task model by adopting the labeled samples and the unlabeled samples of which the task labeling information is determined so as to optimize the target task model.

For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the disclosure. One of ordinary skill in the art can understand and implement without inventive effort.

In the technical scheme of the disclosure, the processes of collection, storage, use, processing, transmission, provision, disclosure and the like of the training samples accord with the regulations of related laws and regulations, and do not violate the customs of public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 7 illustrates a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 7, the device 700 comprises a computing unit 701 which may perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data required for the operation of the device 700 can be stored. The calculation unit 701, the ROM 702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, or the like; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, optical disk, or the like; and a communication unit 709 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Computing unit 701 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 701 performs the respective methods and processes described above, such as a labeling method of a training sample or a model training method. For example, in some embodiments, the labeling method or model training method of the training samples may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of a computer program may be loaded onto and/or installed onto device 700 via ROM 702 and/or communications unit 709. When loaded into RAM703 and executed by the computing unit 701, may perform one or more steps of the above described method of labeling of training samples or method of model training. Alternatively, in other embodiments, the computing unit 701 may be configured in any other suitable way (e.g., by means of firmware) to perform the labeling method or the model training method of the training samples.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

The computer readable storage medium provided by the embodiments of the present disclosure is a non-transitory computer readable storage medium having computer instructions, where the computer instructions are configured to cause the computer to perform the method provided by any one of the above embodiments.

The computer program product provided by the embodiment of the present disclosure includes a computer program, and the computer program realizes the method provided by any one of the above embodiments when being executed by a processor.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A labeling method of a training sample comprises the following steps:

obtaining a labeled sample and an unlabeled sample;

2. The labeling method of training samples according to claim 1, wherein the number of the target task models is at least two; the inputting the labeled sample into a target task model to obtain a first task characteristic of the labeled sample according to the target task model includes:

respectively inputting the labeled samples into at least two target task models, and performing feature extraction on the labeled samples by each target task model to obtain initial first task features;

and determining the weighted result of each initial first task characteristic as the first task characteristic of the labeled sample.

3. The method for labeling training samples as claimed in claim 2, wherein the weighting factor of each initial first task feature is positively correlated with the accuracy of the target task model for obtaining the initial first task feature.

4. The labeling method of training samples according to claim 1, wherein the number of the target task models is at least two; the inputting the unlabeled sample into the target task model to obtain a second task characteristic of the unlabeled sample according to the target task model includes:

respectively inputting the unlabeled samples into at least two target task models, and performing feature extraction on the unlabeled samples by each target task model to obtain initial second task features;

and determining the weighted result of each initial second task characteristic as the second task characteristic of the unlabeled sample.

5. The method for labeling training samples as claimed in claim 4, wherein the weighting factor of each second initial feature is positively correlated with the accuracy of the target task model for obtaining the second initial feature.

6. The labeling method of training samples according to claim 1, wherein the target task model is a living body recognition model; the first task feature is a first living identification feature; the second task feature is a second living body identification feature; the task processing result comprises a living body recognition confidence;

determining task labeling information of the unlabeled sample according to the first task characteristic, the second task characteristic and the task processing result, wherein the task labeling information comprises:

7. The method for labeling training samples according to any one of claims 1 to 6, wherein the determining task labeling information of the unlabeled sample according to the first task feature, the second task feature and the task processing result comprises:

determining initial labeling information of the unlabeled sample according to the task processing result;

judging whether the initial labeling information is reliable or not according to the first task characteristic and the second task characteristic;

and under the condition that the initial labeling information is determined to be reliable, labeling the initial labeling information as the task labeling information of the unlabeled sample.

8. The method for labeling training samples according to claim 7, wherein the number of labeled samples is at least two;

the judging whether the initial labeling information is reliable according to the first task characteristic and the second task characteristic comprises the following steps:

9. A model training method, comprising:

obtaining a labeled sample and an unlabeled sample;

10. The model training method of claim 9, wherein the pre-training model comprises a target task model; the model training of the pre-training model by adopting the labeled samples and the unlabeled samples with the determined task labeling information comprises the following steps:

11. A labeling apparatus for training samples, comprising:

12. The apparatus for labeling training samples according to claim 11, wherein the number of the target task models is at least two; the first feature determination module includes:

13. The apparatus for labeling training samples as claimed in claim 12, wherein the weighting factor of each initial first task feature is positively correlated to the accuracy of the target task model for obtaining the initial first task feature.

14. The apparatus for labeling training samples according to claim 11, wherein the number of the target task models is at least two; the second feature determination module includes:

the second input unit is used for respectively inputting the label-free samples into at least two target task models so as to perform feature extraction on the label-free samples by each target task model to obtain initial second task features;

15. The apparatus for labeling training samples as claimed in claim 14, wherein the weighting factor of each second initial feature is positively correlated to the accuracy of the target task model for obtaining the second initial feature.

16. The apparatus for labeling training samples according to claim 11, wherein the target task model is a living body recognition model; the first task feature is a first living identification feature; the second task feature is a second living body identification feature; the task processing result comprises a living body recognition confidence;

the labeling module is specifically configured to:

17. The apparatus for labeling training samples according to any one of claims 11 to 16, wherein the labeling module comprises:

a third determining unit, configured to determine initial labeling information of the unlabeled sample according to the task processing result;

18. The apparatus for labeling training samples according to claim 17, wherein the number of labeled samples is at least two;

the judgment unit is specifically configured to:

19. A model training apparatus comprising:

20. The model training apparatus of claim 19, wherein the pre-training model comprises a target task model; the training module is specifically configured to:

and performing model training on the target task model by adopting the labeled sample and the unlabeled sample of which the task labeling information is determined so as to optimize the target task model.

21. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

22. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-10.

23. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-10.