WO2023040156A1 - 基于人脸图像的人脸质量评估方法、装置、设备及介质 - Google Patents

基于人脸图像的人脸质量评估方法、装置、设备及介质 Download PDF

Info

Publication number
WO2023040156A1
WO2023040156A1 PCT/CN2022/072370 CN2022072370W WO2023040156A1 WO 2023040156 A1 WO2023040156 A1 WO 2023040156A1 CN 2022072370 W CN2022072370 W CN 2022072370W WO 2023040156 A1 WO2023040156 A1 WO 2023040156A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
target
quality
face image
model
Prior art date
Application number
PCT/CN2022/072370
Other languages
English (en)
French (fr)
Inventor
李佼
戴磊
刘玉宇
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2023040156A1 publication Critical patent/WO2023040156A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks

Definitions

  • the present application relates to the technical field of artificial intelligence, and in particular to a face image-based face quality assessment method, device, equipment and medium.
  • Face quality detection of face images is an essential link in the face recognition process. It can filter out some face images with poor face quality, so that the face recognition module can work more efficiently.
  • the inventor realizes that the existing face quality assessment models generate an image quality score based on facial features such as facial feature clarity, lighting quality, and image resolution.
  • Such face quality assessment models are usually trained using classification models. Divide the face image into several files to distinguish the quality of the face.
  • the training set used for training usually uses the actual collected face image, the virtual generated face image, and the face image returned in the business process At least one kind of human face image in training, there is the problem that the acquisition cost of human face image is very high.
  • the face confidence level of the face recognition module is used to score the face image and make it into a data set to train the face quality evaluation model.
  • the data label produced in this way cannot express the quality of the face very well. It is regarded as judging whether a face image can get a reasonable score under a specific face recognition module, and the trained face quality evaluation model does not have a generalization effect under other face recognition modules.
  • the main purpose of this application is to provide a face image-based face quality assessment method, device, equipment and medium, aiming at solving the problem that the face quality assessment model trained by the classification model has a very high cost of acquiring face images
  • using the face confidence of the face recognition module to score the face image and make it into a data set to train the face quality assessment model does not have a technical problem of generalization effect.
  • the application proposes a method for evaluating the quality of a face based on a face image, the method comprising:
  • the target training sample trains the model based on the neural network, and uses the trained model as the target face quality assessment model;
  • the quality score is used as the target face quality score corresponding to the face image to be evaluated.
  • the application also proposes a face quality assessment device based on a face image, the device comprising:
  • An image acquisition module configured to acquire a human face image to be evaluated
  • the face quality score evaluation module is used to input the face image to be evaluated into the target face quality evaluation model for face quality evaluation to obtain a quality score, wherein multiple different face recognition models generate multiple Target training samples, using a plurality of target training samples to train the model based on the neural network, and using the trained model as the target face quality evaluation model;
  • a target face quality score determining module configured to use the quality score as the target face quality score corresponding to the face image to be evaluated.
  • the present application also proposes a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the following method steps when executing the computer program:
  • the target training sample trains the model based on the neural network, and uses the trained model as the target face quality assessment model;
  • the quality score is used as the target face quality score corresponding to the face image to be evaluated.
  • the present application also proposes a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following method steps are implemented:
  • the target training sample trains the model based on the neural network, and uses the trained model as the target face quality evaluation model;
  • the quality score is used as the target face quality score corresponding to the face image to be evaluated.
  • the face image-based face quality evaluation method, device, equipment and medium of the present application wherein the method obtains the face image to be evaluated; input the face image to be evaluated into the target face quality evaluation model for face Quality assessment, obtain quality score, wherein, generate a plurality of target training samples based on a plurality of different face recognition models, adopt a plurality of described target training samples to train the model based on the neural network, and use the trained model as the
  • the target face quality evaluation model the quality score is used as the target face quality score corresponding to the face image to be evaluated, and the target face quality evaluation model is obtained based on multiple face recognition models and neural network training , there is no need to use the face quality evaluation model trained by the classification model, thereby avoiding the problem of very high cost of face image acquisition; and using multiple face recognition models to participate in the training, which improves the generalization effect of the target face quality evaluation model .
  • Fig. 1 is the schematic flow chart of the face quality evaluation method based on face image of an embodiment of the present application
  • Fig. 2 is a schematic block diagram of the structure of a face quality assessment device based on a face image according to an embodiment of the present application
  • FIG. 3 is a schematic block diagram of a computer device according to an embodiment of the present application.
  • a kind of face quality evaluation method based on face image is provided in the embodiment of the present application, and described method comprises:
  • S2 Input the face image to be evaluated into the target face quality evaluation model for face quality evaluation to obtain a quality score, wherein multiple target training samples are generated based on multiple different face recognition models, and multiple The target training sample trains the model based on the neural network, and uses the trained model as the target face quality evaluation model;
  • the face image to be evaluated is obtained; the face image to be evaluated is input into the target face quality evaluation model for face quality evaluation, and the quality score is obtained, wherein, based on a plurality of different face recognition models Generate a plurality of target training samples, adopt a plurality of said target training samples to train the model based on the neural network, use the trained model as the target face quality evaluation model; use the quality score as the to-be-evaluated
  • the target face quality score corresponding to the face image is realized, and the target face quality evaluation model is obtained based on multiple face recognition models and neural network training.
  • the face image to be evaluated can be obtained from the user input, the face image to be evaluated can also be obtained from the database, and the face image to be evaluated can also be obtained from a third-party application system.
  • the face image to be evaluated is a face image that needs face quality evaluation.
  • a face image is a digital image containing a human face.
  • the face image to be evaluated is input into the target face quality evaluation model to evaluate the face quality score, that is, a score of 0-1 is evaluated, which may include 0 or 1.
  • the face quality sub-labels of face images are generated, and then the target training samples are generated according to the face images and face quality sub-labels, and each target training sample is used to train the initial model.
  • the target human face quality evaluation model is determined, wherein the initial model is a model obtained based on a neural network.
  • the target face quality evaluation model For S3, obtain the face quality score output by the target face quality evaluation model, and use the obtained face quality score as the target face quality score corresponding to the face image to be evaluated. That is to say, the target face quality score is a score of 0-1, which may include 0 or 1.
  • the step of using the quality score as the target face quality score corresponding to the face image to be evaluated it also includes: acquiring a face quality score threshold; when the target face quality score When greater than the human face quality sub-threshold, it is determined that the target human face quality evaluation result corresponding to the human face image to be evaluated is qualified; when the target human face quality is less than or equal to the human face quality sub-threshold, It is determined that the target face quality evaluation result corresponding to the face image to be evaluated is unqualified.
  • the above-mentioned input of the human face image to be evaluated into the target human face quality evaluation model to evaluate the face quality score, before the step of obtaining the quality score includes:
  • S22 Perform face quality score label generation and training sample generation according to each of the face image pairs to be processed and each of the face recognition models, to obtain the target training samples;
  • S24 Determine the face quality evaluation module of the initial model that has reached the convergence state as the target face quality evaluation model.
  • the training samples are generated according to a plurality of face image pairs to be processed and a plurality of face recognition models, the generalization effect of the trained target face quality evaluation model is improved. ; and because the initial model is not a classification model, there is no need to use the face quality assessment model trained by the classification model, thereby avoiding the problem that the acquisition cost of the face image is very high.
  • multiple face image pairs to be processed and multiple face recognition models input by the user can be obtained, or multiple face image pairs to be processed and multiple face recognition models can be obtained from the database.
  • model, and a plurality of face image pairs to be processed and a plurality of face recognition models can also be obtained from a third-party application system.
  • the face image pairs to be processed include: the face image to be trained and the standard face image.
  • the face image to be trained is the image used to train the initial model.
  • a standard face image is a face image with qualified face quality.
  • the standard face images in each pair of face images to be processed may be the same face image.
  • the face images to be processed are respectively input into each of the face recognition models for forward prediction, and each face quality sub-label is generated according to the forward prediction result, and the face quality sub-label and the person to be trained are Face image generation training samples are used as target training samples. That is to say, the target training sample and the face image to be processed are trained one by one.
  • Each of the target training samples includes: a face image sample and a face quality score calibration value.
  • the face image sample is a face image.
  • the face quality score calibration value is an accurate calibration result of the face quality score of the face image sample.
  • the loss value of the initial model is calculated by using a cross entropy function.
  • the convergence state includes: the loss value of the initial model reaches the first convergence condition or the iteration number of the initial model reaches the second convergence condition.
  • the first convergence condition means that the magnitude of the loss value of the initial model calculated twice adjacently satisfies the Lipschitz condition (Lipschitz continuous condition).
  • the number of iterations of the initial model refers to the number of times the initial model is trained, that is, the number of iterations increases by 1 after being trained once.
  • the second convergence condition is a specific numerical value.
  • the initial model that has reached the convergence state is a model that has reached the expected training target, therefore, the face quality assessment module of the initial model that has reached the convergence state can be determined as the target face quality Evaluate the model.
  • the above-mentioned steps of generating face quality sub-labels and training samples according to each of the face image pairs to be processed and each of the face recognition models to obtain the target training samples include:
  • S221 Obtain one of the plurality of face image pairs to be processed as a target face image pair
  • S222 Input the target face image pair into each of the face recognition models to calculate the face distance score to obtain a plurality of target face distance scores;
  • S223 Generate a face quality score label according to a plurality of target face distance scores
  • S224 Generate a training sample according to the face quality score label and the target face image pair, and obtain the target training sample corresponding to the target face image pair;
  • S225 Repeat the step of acquiring one of the multiple to-be-processed human-face image pairs as a target human-face image pair until the acquisition of the to-be-processed human-face image pair is completed.
  • each face recognition model is used to calculate a face distance score for the target face image pair, and then a training sample is generated according to each face distance score and the target face image pair, because The training samples synthesize the face distance scores output by each face recognition model, thereby improving the generalization effect of the trained target face quality assessment model.
  • one of the face image pairs to be processed is acquired from the plurality of face image pairs to be processed, and the acquired face image pair to be processed is used as a target face image pair.
  • the target face image pair is input to each of the face recognition models to calculate the face distance score, that is, each of the face recognition models outputs a person for the target face image pair.
  • the face distance score using the face distance score output by each of the face recognition models for the target face image pair as a target face distance score.
  • the corresponding face quality score label is used as the face quality score calibration value of the target training sample corresponding to the target face image.
  • step S221 to step S225 are repeatedly executed until the acquisition of the face image pairs to be processed is completed. That is to say, each pair of face images to be processed corresponds to one target training sample.
  • the above-mentioned target face image pair includes: a face image to be trained and a standard face image, and the step of generating a face quality sub-label according to a plurality of target face distance scores includes:
  • S2231 Perform variance calculation according to multiple target face distance scores to obtain the face quality score label.
  • the variance is performed according to a plurality of target face distance scores, and the variance is used as the face quality score label, so that the face quality score label integrates the face distance scores output by the forward prediction of each face recognition model, so that The generalization effect of the trained target face quality assessment model is improved.
  • S n is the nth described target human face distance score in a plurality of described target human face distance scores, is the average value of multiple target human face distance scores, and P is the number of target human face distance scores.
  • At least one of the network structure, the number of network layers, and the amount of parameters of the above two face recognition models is different;
  • the step of inputting the target face image to each of the face recognition models to calculate the face distance score to obtain a plurality of target face distance scores includes:
  • S2222 Acquire a face distance score output by each face recognition model as the target face distance score.
  • At least one of the network structure, the number of network layers, and the amount of parameters between any two face recognition models in this embodiment is different, so that each of the face recognition models can cover various types of faces
  • the recognition model is conducive to making the training samples integrate the face distance scores output by more types of face recognition models, which further improves the generalization effect of the trained target face quality evaluation model;
  • the face distance score output by the recognition model forward is used as the target face distance score, which improves calculation efficiency.
  • At least one of the network structure, the number of network layers, and the amount of parameters of the two face recognition models is different, which means that any two of the multiple face recognition models used to generate training samples At least one of the network structure, the number of network layers, and the amount of parameters is different between the face recognition models.
  • the network structure of the face recognition model may be a CNN network or a Transformer network.
  • the CNN network is also a convolutional neural network, such as Resnet (deep residual network), VGG (Visual Geometry Group), Mobilenet (depth-level separable convolutional network).
  • the Transformer network is a network that uses Encoder (encoding) and Decoder (decoding), such as the Bert (Bidirectional Encoder Representations from Transformers) model.
  • the target face image pair is respectively input into each of the face recognition models for forward output, that is to say, the face recognition model only performs forward feature extraction and does not perform back propagation, so that the saved The intermediate values required for post-feedback are reduced, thereby reducing the requirements for memory space and computing resources, and improving computing efficiency.
  • the target face distance score is the distance score between the face image to be trained in the target face image pair and the standard face image in the target face image pair.
  • the above-mentioned initial model also includes: a human face key feature quality assessment module, wherein the input of the human face key feature quality assessment module is connected to the output of the convolution unit of the human face quality assessment module ;
  • Each of the target training samples includes: a human face image sample, a human face quality score calibration value and a human face key feature quality score calibration value.
  • the key feature quality evaluation module of the human face is added to the initial model as auxiliary training, which further improves the generalization effect of the trained target human face quality evaluation model.
  • the input terminal of the human face key feature quality evaluation module is connected with the last convolution block of the convolution unit of the human face quality evaluation module, that is to say, the human face key feature quality evaluation module is obtained from the The last convolution block of the convolution unit of the face quality assessment module acquires feature data.
  • the face key feature quality assessment module includes: convolutional layer, average pooling layer, convolutional layer, and fully connected layer.
  • the face key feature quality evaluation module is used to predict the quality score of the face key features in the face image according to the input feature data.
  • the key features of the face that is, the features of the feature points of the face.
  • the feature points of the face also known as the key points of the face, are the key areas of the face, such as eyebrows, eyes, nose, mouth, and facial contours.
  • the above-mentioned step of using a plurality of the target training samples to train the initial model until reaching a state of convergence includes:
  • S231 Obtain one of the target training samples from a plurality of target training samples as a target sample
  • S235 Calculate the loss value according to the predicted face quality score, the predicted face key feature quality score, the face quality score calibration value of the target sample, and the face key feature quality score calibration value to obtain the target loss value;
  • S237 Repeat the step of obtaining one of the target training samples from the plurality of target training samples as a target sample until the convergence state is reached.
  • a plurality of target training samples are used to train the face key feature quality evaluation module and the face quality evaluation module. Because the face key points and face quality have many commonalities in the face attributes, the relief is reduced. Over-fitting is prevented, which helps to improve the generalization effect of the face quality assessment module.
  • one target training sample is acquired from a plurality of target training samples, and the acquired target training sample is used as a target sample.
  • Both the first loss function and the second loss function use a cross entropy function.
  • update the network parameters of the initial model according to the target loss value that is, update the network parameters of the face quality evaluation module and the face key feature quality evaluation module of the initial model.
  • step S231 to step S237 are repeatedly executed until the convergence state is reached, and the face quality evaluation module of the initial model that has reached the convergence state is determined as the target face quality evaluation model.
  • the present application also proposes a kind of face quality assessment device based on face image, and described device comprises:
  • An image acquisition module 100 configured to acquire a human face image to be evaluated
  • the face quality evaluation module 200 is configured to input the face image to be evaluated into the target face quality evaluation model for face quality evaluation to obtain a quality score, wherein multiple different face recognition models are used to generate multiple A target training sample, using a plurality of the target training samples to train the model based on the neural network, using the trained model as the target face quality assessment model;
  • the target face quality score determination module 300 is configured to use the quality score as the target face quality score corresponding to the face image to be evaluated.
  • the face image to be evaluated is obtained; the face image to be evaluated is input into the target face quality evaluation model for face quality evaluation, and the quality score is obtained, wherein, based on a plurality of different face recognition models Generate a plurality of target training samples, adopt a plurality of said target training samples to train the model based on the neural network, use the trained model as the target face quality evaluation model; use the quality score as the to-be-evaluated
  • the target face quality score corresponding to the face image is realized, and the target face quality evaluation model is obtained based on multiple face recognition models and neural network training.
  • the above-mentioned device also includes: a model training module;
  • the model training module is configured to acquire a plurality of face image pairs to be processed and a plurality of the face recognition models, and perform human face recognition according to each of the face image pairs to be processed and each of the face recognition models. Face quality score label generation and training sample generation, obtain described target training sample, adopt a plurality of described target training samples to train initial model, until reaching convergent state, will reach the human face of described initial model of described convergent state.
  • the quality evaluation module is determined as the target human face quality evaluation model.
  • the above-mentioned model training module includes: a target training sample generation submodule;
  • the target training sample generation submodule is used to obtain one of the plurality of face image pairs to be processed as a target face image pair, and input the target face image pair into each of the face recognition models Carry out the face distance score calculation, obtain a plurality of target face distance scores, generate a face quality sub-label according to a plurality of the target face distance scores, and repeatedly perform the centering from a plurality of the face images to be processed The step of acquiring a target human face image pair until the acquisition of the to-be-processed human face image pair is completed.
  • the above-mentioned target human face image pair includes: a human face image to be trained and a standard human face image
  • the target training sample generation submodule includes: a target training sample generation unit
  • the target training sample generating unit is configured to perform variance calculation according to a plurality of target face distance scores to obtain the face quality score label, and according to the target face image pair corresponding to the face to be trained
  • the image and the face quality score label generate a training sample, and the target training sample corresponding to the target face image pair is obtained.
  • At least one of the network structure, the number of network layers, and the amount of parameters of the above two face recognition models is different;
  • the target training sample generation submodule also includes: a target face distance score determination unit;
  • the target face distance score determining unit is configured to input the target face image pair into each of the face recognition models, and obtain the face distance score output by each face recognition model as the Target face distance score.
  • the above-mentioned initial model also includes: a human face key feature quality assessment module, wherein the input of the human face key feature quality assessment module is connected to the output of the convolution unit of the human face quality assessment module ;
  • Each of the target training samples includes: a human face image sample, a human face quality score calibration value and a human face key feature quality score calibration value.
  • the above-mentioned model training module includes: a model update submodule
  • the model update submodule is used to obtain one of the target training samples from a plurality of target training samples as a target sample, input the face image sample of the target sample into the initial model, and obtain the initial
  • the predicted value of the human face quality score output by the human face quality evaluation module of the model is obtained, and the predicted value of the human face key feature quality output by the human face key feature quality evaluation module of the initial model is obtained, according to the human face quality Calculate the loss value of the predicted value, the predicted value of the key feature quality of the human face, the calibrated value of the human face quality of the target sample, and the calibrated value of the key feature quality of the human face to obtain the target loss value, and according to the target loss updating the network parameters of the initial model, and repeating the step of obtaining one of the target training samples from a plurality of the target training samples as the target sample until the convergence state is reached.
  • an embodiment of the present application also provides a computer device, which may be a server, and its internal structure may be as shown in FIG. 3 .
  • the computer device includes a processor, memory, network interface and database connected by a system bus. Among them, the processor designed by the computer is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, computer programs and databases.
  • the memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the database of the computer equipment is used to store data such as the face quality evaluation method based on the face image.
  • the network interface of the computer device is used to communicate with an external terminal via a network connection.
  • the face quality assessment method based on a face image comprises: obtaining a face image to be evaluated; inputting the face image to be evaluated into a target face quality evaluation model for face quality evaluation to obtain a quality score, wherein, a plurality of target training samples are generated based on a plurality of different face recognition models, a plurality of the target training samples are used to train the model obtained based on the neural network, and the trained model is used as the target face quality assessment model; The quality score is used as the target face quality score corresponding to the face image to be evaluated.
  • the face image to be evaluated is obtained; the face image to be evaluated is input into the target face quality evaluation model for face quality evaluation, and the quality score is obtained, wherein, based on a plurality of different face recognition models Generate a plurality of target training samples, adopt a plurality of said target training samples to train the model based on the neural network, use the trained model as the target face quality evaluation model; use the quality score as the to-be-evaluated
  • the target face quality score corresponding to the face image is realized, and the target face quality evaluation model is obtained based on multiple face recognition models and neural network training.
  • An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a face quality assessment method based on a face image is implemented, including the steps of: obtaining the face image to be evaluated Facial image; the human face image to be evaluated is input into the target human face quality evaluation model to carry out the human face quality evaluation, and obtains the quality score, wherein, a plurality of target training samples are generated based on a plurality of different face recognition models, using A plurality of the target training samples train the model based on the neural network, and use the trained model as the target face quality evaluation model; use the quality score as the target person corresponding to the face image to be evaluated. Face quality score.
  • the face quality evaluation method based on the face image performed above obtains the face image to be evaluated; the face image to be evaluated is input into the target face quality evaluation model for face quality evaluation to obtain a quality score,
  • a plurality of target training samples are generated based on a plurality of different face recognition models, a plurality of the target training samples are used to train the model obtained based on the neural network, and the trained model is used as the target face quality assessment model;
  • the quality score is used as the target face quality score corresponding to the face image to be evaluated, and the target face quality evaluation model is obtained based on multiple face recognition models and neural network training without using classification model training. Face quality evaluation model, thus avoiding the problem of very high cost of face image acquisition; and using multiple face recognition models to participate in training, improving the generalization effect of the target face quality evaluation model.
  • the computer-readable storage medium may be non-volatile or volatile.
  • Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in various forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
  • SRAM Static RAM
  • DRAM Dynamic RAM
  • SDRAM Synchronous DRAM
  • SSRSDRAM Double Data Rate SDRAM
  • ESDRAM Enhanced SDRAM
  • SLDRAM Synchronous Link (Synchlink) DRAM
  • SLDRAM Synchronous Link (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

本申请涉及人工智能技术领域，揭示了一种基于人脸图像的人脸质量评估方法、装置、设备及介质，其中方法包括：将待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为目标人脸质量评估模型；将质量分值作为待评估的人脸图像对应的目标人脸质量分。实现了基于多个人脸识别模型和神经网络训练得到目标人脸质量评估模型，不需要采用分类模型训练的人脸质量评估模型，从而避免了人脸图像的获取成本非常高昂的问题；而且采用多个人脸识别模型参与训练，提高了目标人脸质量评估模型的泛化效果。

Description

基于人脸图像的人脸质量评估方法、装置、设备及介质
本申请要求于2021年09月17日提交中国专利局、申请号为202111094851.3，发明名称为“基于人脸图像的人脸质量评估方法、装置、设备及介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及到人工智能技术领域，特别是涉及到一种基于人脸图像的人脸质量评估方法、装置、设备及介质。
背景技术
人脸图像的人脸质量检测是人脸识别流程里必不可少的环节，它可以过滤掉一些人脸质量不好的人脸图像，使得人脸识别模块可以更有效率的工作。发明人意识到现有的人脸质量评估模型都是基于面部特征清晰度、照明质量和图像分辨率等面部特征产生一个图像质量得分，这样的人脸质量评估模型通常会使用分类模型去训练，将人脸图像分为几个档来区分人脸质量的好坏程度，用于训练的训练集通常会使用实际采集的人脸图像、虚拟生成的人脸图像、业务流程中回流的人脸图像中的至少一种人脸图像进行训练，存在人脸图像的获取成本非常高昂的问题。还有利用人脸识别模块的人脸置信度给人脸图像打分制作成数据集训练人脸质量评估模型，这种方式制作的数据标签并不能很好的表达人脸质量的好坏，只能看做是判断人脸图像在特定的人脸识别模块下能否得到一个合理的分数，训练出来的人脸质量评估模型在其它人脸识别模块下并不具有泛化效果。
技术问题
本申请的主要目的为提供一种基于人脸图像的人脸质量评估方法、装置、设备及介质，旨在解决采用分类模型训练的人脸质量评估模型，存在人脸图像的获取成本非常高昂的问题，而利用人脸识别模块的人脸置信度给人脸图像打分制作成数据集训练人脸质量评估模型，不具有泛化效果的技术问题。
技术解决方案
为了实现上述发明目的，本申请提出一种基于人脸图像的人脸质量评估方法，所述方法包括：
获取待评估的人脸图像；
将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
本申请还提出了一种基于人脸图像的人脸质量评估装置，所述装置包括：
图像获取模块，用于获取待评估的人脸图像；
人脸质量分评估模块，用于将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
目标人脸质量分确定模块，用于将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
本申请还提出了一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现如下方法步骤：
获取待评估的人脸图像；
将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
本申请还提出了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如下方法步骤：
获取待评估的人脸图像；
将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
有益效果
本申请的基于人脸图像的人脸质量评估方法、装置、设备及介质，其中方法通过获取待评估的人脸图像；将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分，实现了基于多个人脸识别模型和神经网络训练得到目标人脸质量评估模型，不需要采用分类模型训练的人脸质量评估模型，从而避免了人脸图像的获取成本非常高昂的问题；而且采用多个人脸识别模型参与训练，提高了目标人脸质量评估模型的泛化效果。
附图说明
图1为本申请一实施例的基于人脸图像的人脸质量评估方法的流程示意图；
图2为本申请一实施例的基于人脸图像的人脸质量评估装置的结构示意框图；
图3为本申请一实施例的计算机设备的结构示意框图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
本发明的实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
参照图1，本申请实施例中提供一种基于人脸图像的人脸质量评估方法，所述方法包括：
S1：获取待评估的人脸图像；
S2：将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
S3：将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
本实施例通过获取待评估的人脸图像；将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得 到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分，实现了基于多个人脸识别模型和神经网络训练得到目标人脸质量评估模型，不需要采用分类模型训练的人脸质量评估模型，从而避免了人脸图像的获取成本非常高昂的问题；而且采用多个人脸识别模型参与训练，提高了目标人脸质量评估模型的泛化效果。
对于S1，可以获取用户输入的待评估的人脸图像，也可以从数据库中获取待评估的人脸图像，还可以从第三方应用系统中获取待评估的人脸图像。
待评估的人脸图像，是需要进行人脸质量评估的人脸图像。人脸图像是包含人脸的数字图像。
对于S2，将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量分的评估，也就是评估出一个0-1的分数，可以包括0，也可以包括1。
基于多个人脸识别模型生成人脸图像的人脸质量分标签，然后根据人脸图像和人脸质量分标签生成目标训练样本，采用各个目标训练样本对初始模型进行训练，根据训练结束的初始模型确定所述目标人脸质量评估模型，其中，初始模型是基于神经网络得到的模型。
对于S3，获取所述目标人脸质量评估模型输出的人脸质量分，将获取的人脸质量分作为所述待评估的人脸图像对应的目标人脸质量分。也就是说，目标人脸质量分是一个0-1的分数，可以包括0，也可以包括1。
可选的，所述将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分的步骤之后，还包括：获取人脸质量分阈值；当所述目标人脸质量分大于所述人脸质量分阈值时，确定所述待评估的人脸图像对应的目标人脸质量评估结果为合格；当所述目标人脸质量分小于或等于所述人脸质量分阈值时，确定所述待评估的人脸图像对应的目标人脸质量评估结果为不合格。
在一个实施例中，上述将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量分的评估，得到质量分值的步骤之前，包括：
S21：获取多个待处理的人脸图像对和多个所述人脸识别模型；
S22：根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本；
S23：采用多个所述目标训练样本对初始模型进行训练，直至达到收敛状态；
S24：将达到所述收敛状态的所述初始模型的人脸质量评估模块确定为所述目标人脸质量评估模型。
本实施例因训练样本是根据多个所述待处理的人脸图像对和多个所述人脸识别模型进行训练样本生成的，从而提高了训练出的目标人脸质量评估模型的泛化效果；又因为所述初始模型不是分类模型，不需要采用分类模型训练的人脸质量评估模型，从而避免了人脸图像的获取成本非常高昂的问题。
对于S21，可以获取用户输入的多个待处理的人脸图像对和多个所述人脸识别模型，也可以从数据库中获取多个待处理的人脸图像对和多个所述人脸识别模型，还可以从第三方应用系统中获取多个待处理的人脸图像对和多个所述人脸识别模型。
待处理的人脸图像对包括：待训练的人脸图像和标准人脸图像。待训练的人脸图像是用于训练初始模型的图像。标准人脸图像，是人脸质量合格的人脸图像。
可以理解的是，各个所述待处理的人脸图像对中的标准人脸图像可以是相同的人脸图像。
对于S22，将所述待处理的人脸图像分别输入各个所述人脸识别模型进行前 向预测，根据前向预测结果生成各个人脸质量分标签，根据人脸质量分标签和待训练的人脸图像生成训练样本作为目标训练样本。也就是说，所述目标训练样本与所述待处理的人脸图像对一一训练。
每个所述目标训练样本包括：人脸图像样本和人脸质量分标定值。人脸图像样本，是人脸图像。
在同一个所述目标训练样本中，人脸质量分标定值是对人脸图像样本的人脸质量分的准确标定结果。
对于S23，采用多个所述目标训练样本对初始模型进行训练的方法步骤在此不做赘述。
其中，在采用多个所述目标训练样本对初始模型进行训练时，采用交叉熵函数计算初始模型的损失值。
收敛状态包括：初始模型的损失值达到第一收敛条件或初始模型的迭代次数达到第二收敛条件。
所述第一收敛条件是指相邻两次计算初始模型的损失值的大小满足lipschitz条件(利普希茨连续条件)。
初始模型的迭代次数是指所述初始模型被训练的次数，也就是说，被训练一次，迭代次数增加1。
第二收敛条件是一个具体数值。
对于S24，达到所述收敛状态的所述初始模型是已经达到预期训练目标的模型，因此，可以将达到所述收敛状态的所述初始模型的人脸质量评估模块确定为所述目标人脸质量评估模型。
在一个实施例中，上述根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本的步骤，包括：
S221：从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对；
S222：将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数；
S223：根据多个所述目标人脸距离分数生成人脸质量分标签；
S224：根据所述人脸质量分标签和所述目标人脸图像对生成训练样本，得到所述目标人脸图像对对应的所述目标训练样本；
S225：重复执行所述从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对的步骤，直至完成所述待处理的人脸图像对的获取。
本实施例实现了先采用每个所述人脸识别模型对所述目标人脸图像对计算一个人脸距离分数，然后根据各个人脸距离分数和所述目标人脸图像对生成训练样本，因为训练样本综合了各个人脸识别模型输出的人脸距离分数，从而提高了训练出的目标人脸质量评估模型的泛化效果。
对于S221，从多个所述待处理的人脸图像对中获取其中一个所述待处理的人脸图像对，将获取的所述待处理的人脸图像对作为目标人脸图像对。
对于S222，将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，也就是说，每个所述人脸识别模型针对所述目标人脸图像对输出一个人脸距离分数，将每个所述人脸识别模型针对所述目标人脸图像对输出的人脸距离分数作为一个目标人脸距离分数。
对于S223，首先根据多个所述目标人脸距离分数进行方差计算，将计算得到的方差作为所述目标人脸图像对对应的人脸质量分标签.
对于S224，将所述目标人脸图像对对应的所述待训练的人脸图像，作为所述目标人脸图像对对应的所述目标训练样本的人脸图像样本；将所述目标人脸图像对对应的所述人脸质量分标签，作为所述目标人脸图像对对应的所述目标训练样本的人脸质量分标定值。
对于S225，重复执行步骤S221至步骤S225，直至完成所述待处理的人脸图像对的获取。也就是说，每个所述待处理的人脸图像对对应一个所述目标训练样本。
在一个实施例中，上述目标人脸图像对包括：待训练的人脸图像和标准人脸图像，所述根据多个所述目标人脸距离分数生成人脸质量分标签的步骤，包括：
S2231：根据多个所述目标人脸距离分数进行方差计算，得到所述人脸质量分标签。
本实施例通过根据多个所述目标人脸距离分数进行方差，将方差作为人脸质量分标签，使人脸质量分标签综合了各个人脸识别模型前向预测输出的人脸距离分数，从而提高了训练出的目标人脸质量评估模型的泛化效果。
对于S2231，所述人脸质量分标签的计算公式Q为：
Figure PCTCN2022072370-appb-000001
其中，S n是多个所述目标人脸距离分数中的第n个所述目标人脸距离分数，
Figure PCTCN2022072370-appb-000002
是多个所述目标人脸距离分数的平均值，P是所述目标人脸距离分数的数量。
在一个实施例中，上述两个所述人脸识别模型的网络结构、网络层数、参数量中的至少一种不相同；
所述将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数的步骤，包括：
S2221：将所述目标人脸图像对分别输入各个所述人脸识别模型；
S2222：获取每个所述人脸识别模型前向输出的人脸距离分数作为所述目标人脸距离分数。
本实施例的任意两个所述人脸识别模型之间的网络结构、网络层数、参数量中的至少一种不相同，从而使各个所述人脸识别模型可以涵盖各种类型的人脸识别模型，有利于使训练样本综合了更多类型的人脸识别模型输出的人脸距离分数，进一步提高了训练出的目标人脸质量评估模型的泛化效果；通过获取每个所述人脸识别模型前向输出的人脸距离分数作为所述目标人脸距离分数，提高了计算效率。
两个所述人脸识别模型的网络结构、网络层数、参数量中的至少一种不相同，是指用于生成训练样本的多个所述人脸识别模型中的任意两个所述人脸识别模型之间的网络结构、网络层数、参数量中的至少一种不相同。
所述人脸识别模型的网络结构可以是CNN网络、也可以是Transformer网络。CNN网络也就是卷积神经网络，比如，Resnet(深度残差网络)、VGG(Visual Geometry Group)、Mobilenet(深度级可分离卷积网络)。Transformer网络也就是采用Encoder(编码)和Decoder(解码)的网络，比如，Bert(Bidirectional Encoder Representations from Transformers)模型。
对于S2221，将所述目标人脸图像对分别输入各个所述人脸识别模型进行前向输出，也就是说，所述人脸识别模型只做前向特征提取，不做反向传播，使保存后反馈所需要的中间值减少，从而减少了内存空间的需求和计算资源的需求， 提高了计算效率。
对于S2222，获取每个所述人脸识别模型前向输出的人脸距离分数所述目标人脸图像对对应的一个目标人脸距离分数。
可以理解的是，目标人脸距离分数，是所述目标人脸图像对中的待训练的人脸图像和所述目标人脸图像对中的标准人脸图像之间的距离分数。
在一个实施例中，上述初始模型还包括：人脸关键特征质量评估模块，其中，所述人脸关键特征质量评估模块的输入端与所述人脸质量评估模块的卷积单元的输出端连接；
每个所述目标训练样本包括：人脸图像样本、人脸质量分标定值和人脸关键特征质量分标定值。
本实施例将人脸关键特征质量评估模块加入初始模型作为辅助训练，进一步提高了训练出的目标人脸质量评估模型的泛化效果。
其中，所述人脸关键特征质量评估模块的输入端与所述人脸质量评估模块的卷积单元的最后一个卷积块连接，也就是说，所述人脸关键特征质量评估模块从所述人脸质量评估模块的卷积单元的最后一个卷积块获取特征数据。
人脸关键特征质量评估模块依次包括：卷积层、平均池化层、卷积层、全连接层。
人脸关键特征质量评估模块，用于根据输入的特征数据预测出人脸图像中的人脸关键特征的质量分。
人脸关键特征，也就是人脸的特征点的特征。人脸的特征点，又称为人脸关键点，人脸面部的关键区域，比如，眉毛、眼睛、鼻子、嘴巴、脸部轮廓等。
在一个实施例中，上述采用多个所述目标训练样本对初始模型进行训练，直至达到收敛状态的步骤，包括：
S231：从多个所述目标训练样本中获取其中一个所述目标训练样本作为目标样本；
S232：将所述目标样本的人脸图像样本输入所述初始模型；
S233：获取所述初始模型的所述人脸质量评估模块输出的人脸质量分预测值；
S234：获取所述初始模型的所述人脸关键特征质量评估模块输出的人脸关键特征质量分预测值；
S235：根据所述人脸质量分预测值、所述人脸关键特征质量分预测值和所述目标样本的人脸质量分标定值及人脸关键特征质量分标定值进行损失值计算，得到目标损失值；
S236：根据所述目标损失值更新所述初始模型的网络参数；
S237：重复执行所述从多个所述目标训练样本中获取其中一个所述目标训练样本作为目标样本的步骤，直至达到所述收敛状态。
本实施例采用多个所述目标训练样本对人脸关键特征质量评估模块和所述人脸质量评估模块进行训练，因人脸关键点和人脸质量在人脸属性中有很多共通性，减轻了过拟合，有助于提升所述人脸质量评估模块的泛化效果。
对于S231，从多个所述目标训练样本中获取一个所述目标训练样本，将获取的所述目标训练样本作为目标样本。
对于S232，将所述目标样本的人脸图像样本输入所述初始模型进行质量分预测。
对于S233，获取所述初始模型的所述人脸质量评估模块输出的质量分作为人脸质量分预测值。
对于S234，获取所述初始模型的所述人脸关键特征质量评估模块输出的质量分作为人脸关键特征质量分预测值。
对于S235，将所述人脸质量分预测值和所述目标样本的人脸质量分标定值输入第一损失函数进行损失值计算，得到第一损失值；将所述人脸关键特征质量分预测值和所述目标样本的人脸关键特征质量分标定值输入第二损失函数进行损失值计算，得到第二损失值；将第一损失值和第二损失值进行相加，得到目标损失值。
第一损失函数、第二损失函数均采用交叉熵函数。
对于S236，根据所述目标损失值更新所述初始模型的网络参数，也就是更新所述初始模型的所述人脸质量评估模块和所述人脸关键特征质量评估模块的网络参数。
对于S237，重复执行步骤S231至步骤S237，直至达到所述收敛状态，将达到所述收敛状态的所述初始模型的人脸质量评估模块确定为所述目标人脸质量评估模型。
参照图2，本申请还提出了一种基于人脸图像的人脸质量评估装置，所述装置包括：
图像获取模块100，用于获取待评估的人脸图像；
人脸质量分评估模块200，用于将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
目标人脸质量分确定模块300，用于将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
本实施例通过获取待评估的人脸图像；将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分，实现了基于多个人脸识别模型和神经网络训练得到目标人脸质量评估模型，不需要采用分类模型训练的人脸质量评估模型，从而避免了人脸图像的获取成本非常高昂的问题；而且采用多个人脸识别模型参与训练，提高了目标人脸质量评估模型的泛化效果。
在一个实施例中，上述装置还包括：模型训练模块；
所述模型训练模块，用于获取多个待处理的人脸图像对和多个所述人脸识别模型，根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本，采用多个所述目标训练样本对初始模型进行训练，直至达到收敛状态，将达到所述收敛状态的所述初始模型的人脸质量评估模块确定为所述目标人脸质量评估模型。
在一个实施例中，上述模型训练模块包括：目标训练样本生成子模块；
所述目标训练样本生成子模块，用于从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对，将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数，根据多个所述目标人脸距离分数生成人脸质量分标签，重复执行所述从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对的步骤，直至完成所述待处理的人脸图像对的获取。
在一个实施例中，上述目标人脸图像对包括：待训练的人脸图像和标准人脸 图像，所述目标训练样本生成子模块包括：目标训练样本生成单元；
所述目标训练样本生成单元，用于根据多个所述目标人脸距离分数进行方差计算，得到所述人脸质量分标签，根据所述目标人脸图像对对应的所述待训练的人脸图像和所述人脸质量分标签生成训练样本，得到所述目标人脸图像对对应的所述目标训练样本。
在一个实施例中，上述两个所述人脸识别模型的网络结构、网络层数、参数量中的至少一种不相同；
所述目标训练样本生成子模块还包括：目标人脸距离分数确定单元；
所述目标人脸距离分数确定单元，用于将所述目标人脸图像对分别输入各个所述人脸识别模型，获取每个所述人脸识别模型前向输出的人脸距离分数作为所述目标人脸距离分数。
在一个实施例中，上述初始模型还包括：人脸关键特征质量评估模块，其中，所述人脸关键特征质量评估模块的输入端与所述人脸质量评估模块的卷积单元的输出端连接；
每个所述目标训练样本包括：人脸图像样本、人脸质量分标定值和人脸关键特征质量分标定值。
在一个实施例中，上述模型训练模块包括：模型更新子模块；
所述模型更新子模块，用于从多个所述目标训练样本中获取其中一个所述目标训练样本作为目标样本，将所述目标样本的人脸图像样本输入所述初始模型，获取所述初始模型的所述人脸质量评估模块输出的人脸质量分预测值，获取所述初始模型的所述人脸关键特征质量评估模块输出的人脸关键特征质量分预测值，根据所述人脸质量分预测值、所述人脸关键特征质量分预测值和所述目标样本的人脸质量分标定值及人脸关键特征质量分标定值进行损失值计算，得到目标损失值，根据所述目标损失值更新所述初始模型的网络参数，重复执行所述从多个所述目标训练样本中获取其中一个所述目标训练样本作为目标样本的步骤，直至达到所述收敛状态。
参照图3，本申请实施例中还提供一种计算机设备，该计算机设备可以是服务器，其内部结构可以如图3所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设计的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于储存基于人脸图像的人脸质量评估方法等数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种基于人脸图像的人脸质量评估方法。所述基于人脸图像的人脸质量评估方法，包括：获取待评估的人脸图像；将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
本实施例通过获取待评估的人脸图像；将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；将所述质量 分值作为所述待评估的人脸图像对应的目标人脸质量分，实现了基于多个人脸识别模型和神经网络训练得到目标人脸质量评估模型，不需要采用分类模型训练的人脸质量评估模型，从而避免了人脸图像的获取成本非常高昂的问题；而且采用多个人脸识别模型参与训练，提高了目标人脸质量评估模型的泛化效果。
本申请一实施例还提供一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现一种基于人脸图像的人脸质量评估方法，包括步骤：获取待评估的人脸图像；将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
上述执行的基于人脸图像的人脸质量评估方法，通过获取待评估的人脸图像；将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分，实现了基于多个人脸识别模型和神经网络训练得到目标人脸质量评估模型，不需要采用分类模型训练的人脸质量评估模型，从而避免了人脸图像的获取成本非常高昂的问题；而且采用多个人脸识别模型参与训练，提高了目标人脸质量评估模型的泛化效果。
所述计算机可读存储介质可以是非易失性，也可以是易失性。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的和实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可以包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双速据率SDRAM(SSRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、装置、物品或者方法不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、装置、物品或者方法所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、装置、物品或者方法中还存在另外的相同要素。
以上所述仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种基于人脸图像的人脸质量评估方法，其中，所述方法包括：
    获取待评估的人脸图像；
    将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
    将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
  2. 根据权利要求1所述的基于人脸图像的人脸质量评估方法，其中，所述将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值的步骤之前，包括：
    获取多个待处理的人脸图像对和多个所述人脸识别模型；
    根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本；
    采用多个所述目标训练样本对初始模型进行训练，直至达到收敛状态；
    将达到收敛状态的所述初始模型确定为所述目标人脸质量评估模型。
  3. 根据权利要求2所述的基于人脸图像的人脸质量评估方法，其中，所述根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本的步骤，包括：
    从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对；
    将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数；
    根据多个所述目标人脸距离分数生成人脸质量分标签；
    根据所述人脸质量分标签和所述目标人脸图像对生成训练样本，得到所述目标人脸图像对对应的所述目标训练样本；
    重复执行所述从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对的步骤，直至完成所述待处理的人脸图像对的获取。
  4. 根据权利要求3所述的基于人脸图像的人脸质量评估方法，其中，所述目标人脸图像对包括：待训练的人脸图像和标准人脸图像，所述根据多个所述目标人脸距离分数生成人脸质量分标签的步骤，包括：
    根据多个所述目标人脸距离分数进行方差计算，得到所述人脸质量分标签。
  5. 根据权利要求3所述的基于人脸图像的人脸质量评估方法，其中，两个所述人脸识别模型的网络结构、网络层数、参数量中的至少一种不相同；
    所述将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数的步骤，包括：
    将所述目标人脸图像对分别输入各个所述人脸识别模型；
    获取每个所述人脸识别模型前向输出的人脸距离分数作为所述目标人脸距离分数。
  6. 根据权利要求2所述的基于人脸图像的人脸质量评估方法，其中，所述初始模型还包括：人脸关键特征质量评估模块，其中，所述人脸关键特征质量评估模块的输入端与所述人脸质量评估模块的卷积单元的输出端连接；
    每个所述目标训练样本包括：人脸图像样本、人脸质量分标定值和人脸关键特征质量分标定值。
  7. 根据权利要求6所述的基于人脸图像的人脸质量评估方法，其中，所述采用多个所述目标训练样本对初始模型进行训练，直至达到收敛状态的步骤，包括：
    从多个所述目标训练样本中获取其中一个所述目标训练样本作为目标样本；
    将所述目标样本的人脸图像样本输入所述初始模型；
    获取所述初始模型的所述人脸质量评估模块输出的人脸质量分预测值；
    获取所述初始模型的所述人脸关键特征质量评估模块输出的人脸关键特征质量分预测值；
    根据所述人脸质量分预测值、所述人脸关键特征质量分预测值和所述目标样本的人脸质量分标定值及人脸关键特征质量分标定值进行损失值计算，得到目标损失值；
    根据所述目标损失值更新所述初始模型的网络参数；
    重复执行所述从多个所述目标训练样本中获取其中一个所述目标训练样本作为目标样本的步骤，直至达到所述收敛状态。
  8. 一种基于人脸图像的人脸质量评估装置，其中，所述装置包括：
    图像获取模块，用于获取待评估的人脸图像；
    人脸质量分评估模块，用于将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
    目标人脸质量分确定模块，用于将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
  9. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其中，所述处理器执行所述计算机程序时实现如下方法步骤：
    获取待评估的人脸图像；
    将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
    将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
  10. 根据权利要求9所述的计算机设备，其中，所述将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值的步骤之前，包括：
    获取多个待处理的人脸图像对和多个所述人脸识别模型；
    根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本；
    采用多个所述目标训练样本对初始模型进行训练，直至达到收敛状态；
    将达到收敛状态的所述初始模型确定为所述目标人脸质量评估模型。
  11. 根据权利要求10所述的计算机设备，其中，所述根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本的步骤，包括：
    从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对；
    将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数；
    根据多个所述目标人脸距离分数生成人脸质量分标签；
    根据所述人脸质量分标签和所述目标人脸图像对生成训练样本，得到所述目标人脸图像对对应的所述目标训练样本；
    重复执行所述从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对的步骤，直至完成所述待处理的人脸图像对的获取。
  12. 根据权利要求11所述的计算机设备，其中，所述目标人脸图像对包括：待训练的人脸图像和标准人脸图像，所述根据多个所述目标人脸距离分数生成人脸质量分标签的步骤，包括：
    根据多个所述目标人脸距离分数进行方差计算，得到所述人脸质量分标签。
  13. 根据权利要求11所述的计算机设备，其中，两个所述人脸识别模型的网络结构、网络层数、参数量中的至少一种不相同；
    所述将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数的步骤，包括：
    将所述目标人脸图像对分别输入各个所述人脸识别模型；
    获取每个所述人脸识别模型前向输出的人脸距离分数作为所述目标人脸距离分数。
  14. 根据权利要求10所述的计算机设备，其中，所述初始模型还包括：人脸关键特征质量评估模块，其中，所述人脸关键特征质量评估模块的输入端与所述人脸质量评估模块的卷积单元的输出端连接；
    每个所述目标训练样本包括：人脸图像样本、人脸质量分标定值和人脸关键特征质量分标定值。
  15. 一种计算机可读存储介质，其上存储有计算机程序，其中，所述计算机程序被处理器执行时实现如下方法步骤：
    获取待评估的人脸图像；
    将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值，其中，基于多个不同的人脸识别模型生成多个目标训练样本，采用多个所述目标训练样本对基于神经网络得到模型进行训练，将训练后的模型作为所述目标人脸质量评估模型；
    将所述质量分值作为所述待评估的人脸图像对应的目标人脸质量分。
  16. 根据权利要求15所述的计算机可读存储介质，其中，所述将所述待评估的人脸图像输入目标人脸质量评估模型进行人脸质量评估，得到质量分值的步骤之前，包括：
    获取多个待处理的人脸图像对和多个所述人脸识别模型；
    根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本；
    采用多个所述目标训练样本对初始模型进行训练，直至达到收敛状态；
    将达到收敛状态的所述初始模型确定为所述目标人脸质量评估模型。
  17. 根据权利要求16所述的计算机可读存储介质，其中，所述根据每个所述待处理的人脸图像对和各个所述人脸识别模型进行人脸质量分标签生成和训练样本生成，得到所述目标训练样本的步骤，包括：
    从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对；
    将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数；
    根据多个所述目标人脸距离分数生成人脸质量分标签；
    根据所述人脸质量分标签和所述目标人脸图像对生成训练样本，得到所述目 标人脸图像对对应的所述目标训练样本；
    重复执行所述从多个所述待处理的人脸图像对中获取一个作为目标人脸图像对的步骤，直至完成所述待处理的人脸图像对的获取。
  18. 根据权利要求17所述的计算机可读存储介质，其中，所述目标人脸图像对包括：待训练的人脸图像和标准人脸图像，所述根据多个所述目标人脸距离分数生成人脸质量分标签的步骤，包括：
    根据多个所述目标人脸距离分数进行方差计算，得到所述人脸质量分标签。
  19. 根据权利要求17所述的计算机可读存储介质，其中，两个所述人脸识别模型的网络结构、网络层数、参数量中的至少一种不相同；
    所述将所述目标人脸图像对分别输入各个所述人脸识别模型进行人脸距离分数计算，得到多个目标人脸距离分数的步骤，包括：
    将所述目标人脸图像对分别输入各个所述人脸识别模型；
    获取每个所述人脸识别模型前向输出的人脸距离分数作为所述目标人脸距离分数。
  20. 根据权利要求16所述的计算机可读存储介质，其中，所述初始模型还包括：人脸关键特征质量评估模块，其中，所述人脸关键特征质量评估模块的输入端与所述人脸质量评估模块的卷积单元的输出端连接；
    每个所述目标训练样本包括：人脸图像样本、人脸质量分标定值和人脸关键特征质量分标定值。
PCT/CN2022/072370 2021-09-17 2022-01-17 基于人脸图像的人脸质量评估方法、装置、设备及介质 WO2023040156A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111094851.3 2021-09-17
CN202111094851.3A CN113792682B (zh) 2021-09-17 2021-09-17 基于人脸图像的人脸质量评估方法、装置、设备及介质

Publications (1)

Publication Number Publication Date
WO2023040156A1 true WO2023040156A1 (zh) 2023-03-23

Family

ID=78878816

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/072370 WO2023040156A1 (zh) 2021-09-17 2022-01-17 基于人脸图像的人脸质量评估方法、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN113792682B (zh)
WO (1) WO2023040156A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117011698A (zh) * 2023-06-25 2023-11-07 重庆市地理信息和遥感应用中心(重庆市测绘产品质量检验测试中心) 多维度和多模型的地表全覆盖解译样本集评价方法
CN117275076A (zh) * 2023-11-16 2023-12-22 厦门瑞为信息技术有限公司 基于特征的人脸质量评估模型的构建方法及应用
CN117611516A (zh) * 2023-09-04 2024-02-27 北京智芯微电子科技有限公司 图像质量评估、人脸识别、标签生成及确定方法和装置

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113792682B (zh) * 2021-09-17 2024-05-10 平安科技（深圳）有限公司 基于人脸图像的人脸质量评估方法、装置、设备及介质
CN114344542A (zh) * 2022-02-22 2022-04-15 珠海横琴润霖生物科技有限公司 一种空气预警消毒系统
CN117372405A (zh) * 2023-10-31 2024-01-09 神州通立电梯有限公司 人脸图像质量评估方法、装置、存储介质及设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110751043A (zh) * 2019-09-19 2020-02-04 平安科技（深圳）有限公司 基于人脸可见性的人脸识别方法、装置及存储介质
CN110879981A (zh) * 2019-11-14 2020-03-13 深圳市华付信息技术有限公司 人脸关键点质量评估方法、装置、计算机设备及存储介质
CN111738243A (zh) * 2020-08-25 2020-10-02 腾讯科技（深圳）有限公司 人脸图像的选择方法、装置、设备及存储介质
CN112686234A (zh) * 2021-03-22 2021-04-20 杭州魔点科技有限公司 人脸图像质量评估方法、电子设备和存储介质
CN113792682A (zh) * 2021-09-17 2021-12-14 平安科技（深圳）有限公司 基于人脸图像的人脸质量评估方法、装置、设备及介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110866471A (zh) * 2019-10-31 2020-03-06 Oppo广东移动通信有限公司 人脸图像质量评价方法及装置、计算机可读介质、通信终端

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110751043A (zh) * 2019-09-19 2020-02-04 平安科技（深圳）有限公司 基于人脸可见性的人脸识别方法、装置及存储介质
CN110879981A (zh) * 2019-11-14 2020-03-13 深圳市华付信息技术有限公司 人脸关键点质量评估方法、装置、计算机设备及存储介质
CN111738243A (zh) * 2020-08-25 2020-10-02 腾讯科技（深圳）有限公司 人脸图像的选择方法、装置、设备及存储介质
CN112686234A (zh) * 2021-03-22 2021-04-20 杭州魔点科技有限公司 人脸图像质量评估方法、电子设备和存储介质
CN113792682A (zh) * 2021-09-17 2021-12-14 平安科技（深圳）有限公司 基于人脸图像的人脸质量评估方法、装置、设备及介质

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117011698A (zh) * 2023-06-25 2023-11-07 重庆市地理信息和遥感应用中心(重庆市测绘产品质量检验测试中心) 多维度和多模型的地表全覆盖解译样本集评价方法
CN117011698B (zh) * 2023-06-25 2024-05-03 重庆市地理信息和遥感应用中心(重庆市测绘产品质量检验测试中心) 多维度和多模型的地表全覆盖解译样本集评价方法
CN117611516A (zh) * 2023-09-04 2024-02-27 北京智芯微电子科技有限公司 图像质量评估、人脸识别、标签生成及确定方法和装置
CN117275076A (zh) * 2023-11-16 2023-12-22 厦门瑞为信息技术有限公司 基于特征的人脸质量评估模型的构建方法及应用
CN117275076B (zh) * 2023-11-16 2024-02-27 厦门瑞为信息技术有限公司 基于特征的人脸质量评估模型的构建方法及应用

Also Published As

Publication number Publication date
CN113792682A (zh) 2021-12-14
CN113792682B (zh) 2024-05-10

Similar Documents

Publication Publication Date Title
WO2023040156A1 (zh) 基于人脸图像的人脸质量评估方法、装置、设备及介质
CN109783655B (zh) 一种跨模态检索方法、装置、计算机设备和存储介质
CN112035672B (zh) 一种知识图谱补全方法、装置、设备以及存储介质
CN111160467B (zh) 一种基于条件随机场和内部语义注意力的图像描述方法
CN109992664B (zh) 争议焦点的标注分类方法、装置、计算机设备和存储介质
CN113593611B (zh) 语音分类网络训练方法、装置、计算设备及存储介质
CN110704601A (zh) 利用问题-知识引导的渐进式时空注意力网络解决需要常识的视频问答任务的方法
CN112257858A (zh) 一种模型压缩方法及装置
CN113516152B (zh) 一种基于复合图像语义的图像描述方法
CN111353049A (zh) 数据更新方法、装置、电子设备及计算机可读存储介质
CN111210382B (zh) 图像处理方法、装置、计算机设备和存储介质
CN111368118A (zh) 一种图像描述生成方法、系统、装置和存储介质
CN112699215B (zh) 基于胶囊网络与交互注意力机制的评级预测方法及系统
CN111125356A (zh) 一种文本分类方法及系统
CN116434741A (zh) 语音识别模型训练方法、装置、计算机设备及存储介质
CN113128527A (zh) 基于变换器模型和卷积神经网络的图像场景分类方法
CN110377618B (zh) 裁决结果分析方法、装置、计算机设备和存储介质
US20220188636A1 (en) Meta pseudo-labels
CN114547347A (zh) 基于卷积神经网络的时序知识图谱补全方法、装置及设备
CN114266905A (zh) 基于Transformer结构的图像描述生成模型方法、装置和计算机设备
CN111027681B (zh) 时序数据处理模型训练方法、数据处理方法、装置及存储介质
CN111552810B (zh) 实体抽取与分类方法、装置、计算机设备和存储介质
WO2023155546A1 (zh) 结构数据的生成方法、装置、设备、介质及程序产品
CN111368056A (zh) 一种古诗词生成方法和装置
WO2022178950A1 (zh) 预测语句实体的方法、装置和计算机设备

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE