CN112183190A - Human face quality evaluation method based on local key feature recognition - Google Patents

Human face quality evaluation method based on local key feature recognition Download PDF

Info

Publication number
CN112183190A
CN112183190A CN202010831673.7A CN202010831673A CN112183190A CN 112183190 A CN112183190 A CN 112183190A CN 202010831673 A CN202010831673 A CN 202010831673A CN 112183190 A CN112183190 A CN 112183190A
Authority
CN
China
Prior art keywords
face
human face
local key
features
key feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010831673.7A
Other languages
Chinese (zh)
Inventor
张诗辉
金梦醒
刘恒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Yiwei Technology Co ltd
Original Assignee
Hangzhou Yiwei Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Yiwei Technology Co ltd filed Critical Hangzhou Yiwei Technology Co ltd
Priority to CN202010831673.7A priority Critical patent/CN112183190A/en
Publication of CN112183190A publication Critical patent/CN112183190A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Abstract

The invention aims to disclose a human face quality evaluation method based on local key feature recognition, which comprises the following steps of; compared with the prior art, the method has the advantages that the multitask output judgment is added on the basis of the original face detection, the related face local key features are judged on the basis of not adding model parameters, the face recognition of the face local key features is carried out, the eye and the eyebrow are taken as the core, an attention mechanism is added to extract the face local key features, the face local key feature extraction pairing is carried out by combining face contour features, the high-quality evaluation of the face local key features is realized, the accuracy and the efficiency of the face recognition are better improved by matching with a deep learning algorithm and a face recognition technology, and the purpose of the method is realized.

Description

Human face quality evaluation method based on local key feature recognition
Technical Field
The invention relates to a face quality evaluation method, in particular to a face quality evaluation method based on local key feature recognition.
Background
At present, most face recognition equipment in the market does not support local key feature recognition, so that the detection recognition rate is low under special conditions (the face is shielded, sunglasses are worn on the face, and the like).
Therefore, a face quality evaluation method based on local key feature recognition is particularly needed to solve the existing problems.
Disclosure of Invention
The invention aims to provide a human face quality evaluation method based on local key feature recognition, aiming at the defects of the prior art, the high-quality evaluation of the local key features of the human face is realized, and the accuracy and the efficiency of the human face recognition are better improved by matching a deep learning algorithm and a human face recognition technology.
The technical problem solved by the invention can be realized by adopting the following technical scheme:
a human face quality evaluation method based on local key feature recognition is characterized by comprising the following steps:
step 1: acquiring an image through a camera device, entering the next step if the image is a human face image, and re-acquiring the image through the camera device if the image is not the human face image;
step 2: detecting whether the obtained face image has face local key features, if not, entering a step 3, and if so, entering a step 4;
and step 3: comparing the face image with a face database through a face algorithm, releasing if the comparison is passed, and returning to the step 1 if the comparison is not passed;
and 4, step 4: and (3) comparing the face image with a face local key feature database through a face local key feature algorithm, releasing if the comparison is passed, and returning to the step 1 if the comparison is not passed.
In an embodiment of the present invention, in step 2, it is detected whether the obtained face image has local key features of a face, and through a network model structure based on a feature pyramid and a multitask Loss function, Loss is defined as follows:
Figure BDA0002638232730000026
dividing the loss function into three loss function combinations, namely a two-classification softmax function of whether the loss function is a human face or not
Figure BDA0002638232730000027
Face detection frame coordinate smooth function
Figure BDA0002638232730000028
And whether it is a dichotomized softmax function of the mask
Figure BDA0002638232730000029
Figure BDA00026382327300000210
Y in (1)iIn order to be the GT and the GT,
Figure BDA00026382327300000211
for model test results, the same applies to tiAnd miAre all the GT bodies,
Figure BDA00026382327300000212
and
Figure BDA00026382327300000213
and (5) reasoning output for the model.
Wherein the content of the first and second substances,
Figure BDA00026382327300000214
a binary softmax function of whether the region is a face:
Figure BDA0002638232730000021
Figure BDA00026382327300000215
for the face frame coordinate function, smooth is usedL1The function of the function is that of the function,
Figure BDA0002638232730000022
wherein the content of the first and second substances,
Figure BDA00026382327300000216
and (4) using a softmax function as a classification function of whether the human face has the features.
In one embodiment of the invention, the local key features of the human face are that the eyes and the eyebrows of the data set picture are marked again on the basis of the features of the whole face, an attention mechanism is added to the network, depth feature extraction is emphasized on the eyes and the eyebrows, new human face features are generated by combining the features of the whole face and the eyebrows of the eyes, and finally the human face features are based on the features of a previous layer FC of the classification network trained by L-Softmax,
L-Softmax of
Figure BDA0002638232730000023
Is deformed into
Figure BDA0002638232730000024
Further, the attribute mechanism is to perform weight summation on the overall features and the local features of the picture, increase the weight for the region with large contribution, decrease the weight for the region with small contribution, combine the characteristics of the softmax function, output the interval [0,1], and is suitable for being used in the distribution of the weights to obtain a weight calculation formula:
Figure BDA0002638232730000025
and calculating to obtain:
Figure BDA0002638232730000031
further, the features are mapped into n +1 dimensions through a softmax function, each dimension represents a category, and a one-dimensional convolution is used for replacing a full-link layer to be classified and branched.
In an embodiment of the present invention, the comparison evaluation index is a general (MAP) index, and the formula is as follows:
Figure BDA0002638232730000032
p is the accuracy; r is the recall rate;
during calculation, the coincidence rate of the prediction frame and the marking frame is set as a, and the prediction frame with a being more than or equal to 0.5 is regarded as a positive example:
Figure BDA0002638232730000033
in an embodiment of the invention, by adopting the human face quality evaluation method based on local key feature recognition, the human face detection speed is less than 30ms, the human face feature extraction speed is less than 200ms, the human face detection accuracy is greater than 99.3%, and the human face recognition accuracy is greater than 92%.
Compared with the prior art, the human face quality evaluation method based on local key feature recognition adds multi-task output judgment on the basis of original human face detection, judges related human face local key features on the basis of not adding model parameters, carries out human face recognition on the human face local key features, extracts the human face local key features by taking eyes and eyebrows as a core and adding an attention mechanism, and combines facial contour features to carry out extraction pairing on the recognized human face local key features, thereby realizing the purpose of the invention.
The features of the present invention will be apparent from the accompanying drawings and from the detailed description of the preferred embodiments which follows.
Drawings
FIG. 1 is a schematic flow chart of a human face quality evaluation method based on local key feature recognition according to the present invention;
FIG. 2 is a schematic diagram of a local key feature model structure of a human face according to the present invention;
FIG. 3 is a schematic diagram of the network structure with and without Attention according to the present invention;
FIG. 4 is a schematic structural diagram of an Attention module according to the present invention;
fig. 5 is a schematic structural diagram of a ResBlock layer according to the present invention;
FIG. 6 is a structural diagram of the one-dimensional convolution according to the present invention.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further explained below by combining the specific drawings.
Examples
As shown in fig. 1 to 6, the method for evaluating the face quality based on local key feature recognition of the present invention includes the following steps:
step 1: acquiring an image through a camera device, entering the next step if the image is a human face image, and re-acquiring the image through the camera device if the image is not the human face image;
step 2: detecting whether the obtained face image has face local key features, if not, entering a step 3, and if so, entering a step 4;
and step 3: comparing the face image with a face database through a face algorithm, releasing if the comparison is passed, and returning to the step 1 if the comparison is not passed;
and 4, step 4: and (3) comparing the face image with a face local key feature database through a face local key feature algorithm, releasing if the comparison is passed, and returning to the step 1 if the comparison is not passed.
In this embodiment, in step 2, whether the obtained face image has local key features of the face is detected, and the Loss is defined as follows by using a network model structure based on a feature pyramid and a multitask Loss function:
Figure BDA0002638232730000043
wherein the content of the first and second substances,
Figure BDA0002638232730000044
a binary softmax function of whether the region is a face:
Figure BDA0002638232730000041
Figure BDA0002638232730000045
for the face frame coordinate function, smooth is usedL1The function of the function is that of the function,
Figure BDA0002638232730000042
wherein the content of the first and second substances,
Figure BDA0002638232730000046
and (4) using a softmax function as a classification function of whether the human face has the features.
In the embodiment, the local key features of the human face are that the eyes and the eyebrows of the data set picture are marked again on the basis of the features of the whole face, an attention mechanism is added to the network, depth feature extraction is emphasized on the eyes and the eyebrows, new human face features are generated by combining the features of the whole face and the eyebrows of the eyes, and finally the human face features are based on the features of the FC (fiber channel) in the front layer of the classification network trained by L-Softmax,
L-Softmax of
Figure BDA0002638232730000051
Is deformed into
Figure BDA0002638232730000052
Further, the attribute mechanism is to perform weight summation on the overall features and the local features of the picture, increase the weight for the region with large contribution, decrease the weight for the region with small contribution, combine the characteristics of the softmax function, output the interval [0,1], and is suitable for being used in the distribution of the weights to obtain a weight calculation formula:
Figure BDA0002638232730000053
and calculating to obtain:
Figure BDA0002638232730000054
resblock layer: (1 × 1 convolution + relu +1 × 1 convolution +3 × 3 convolution) × 8(12 or 16), the first 1 × 1 convolution is to make more detailed features of the original extended learning image for the channel, the second 1 × 1 convolution is to compress the number of channels in proportion, reduce the size of parameters and accelerate calculation, then obtain the output feature diagram of the layer through one 3 × 3 convolution, and the final output result of the layer is obtained by adding the input layer and the output layer. The Resblock layer may be repeated 8, 12 or 16 times.
A characteristic pyramid: convolutional neural networks of different levels can realize the extraction from low-level image features to high-level features. As shown in a structure diagram, the last four features of different levels of the model are fused, so that the last output feature simultaneously contains information of a low level and a high level, and the accuracy of network identification and position judgment is improved.
One-dimensional convolution: features are mapped into n +1 dimensions through a softmax function, each dimension represents a category, and a one-dimensional convolution is used for replacing a full-link layer to be classified and branched, so that the detection and classification precision is guaranteed, network parameters are effectively reduced, and the inference time is shortened.
In this embodiment, the comparison evaluation index is a general (MAP) index, and the formula is as follows:
Figure BDA0002638232730000055
p is the accuracy; r is the recall rate;
during calculation, the coincidence rate of the prediction frame and the marking frame is set as a, and the prediction frame with a being more than or equal to 0.5 is regarded as a positive example:
Figure BDA0002638232730000061
in this embodiment, by using the method for evaluating the face quality based on local key feature recognition, the face detection speed is less than 30ms, the face feature extraction speed is less than 200ms, the face detection accuracy is greater than 99.3%, and the face recognition accuracy is greater than 92%.
The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are merely illustrative of the principles of the present invention, but that various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined by the appended claims and their equivalents.

Claims (7)

1. A human face quality evaluation method based on local key feature recognition is characterized by comprising the following steps:
step 1: acquiring an image through a camera device, entering the next step if the image is a human face image, and re-acquiring the image through the camera device if the image is not the human face image;
step 2: detecting whether the obtained face image has face local key features, if not, entering a step 3, and if so, entering a step 4;
and step 3: comparing the face image with a face database through a face algorithm, releasing if the comparison is passed, and returning to the step 1 if the comparison is not passed;
and 4, step 4: and (3) comparing the face image with a face local key feature database through a face local key feature algorithm, releasing if the comparison is passed, and returning to the step 1 if the comparison is not passed.
2. The method for evaluating the quality of the human face based on the local key feature recognition according to claim 1, wherein the step 2 detects whether the obtained human face image has the local key features of the human face, and the Loss is defined as follows through a network model structure based on a feature pyramid and a multitask Loss function:
Figure FDA0002638232720000011
wherein the content of the first and second substances,
Figure FDA0002638232720000012
a binary softmax function of whether the region is a face:
Figure FDA0002638232720000013
Figure FDA0002638232720000014
for the face frame coordinate function, smooth is usedL1The function of the function is that of the function,
Figure FDA0002638232720000015
wherein the content of the first and second substances,
Figure FDA0002638232720000016
and (4) using a softmax function as a classification function of whether the human face has the features.
3. The human face quality evaluation method based on local key feature recognition according to claim 1, wherein the local key features of the human face are that the eyes and the eyebrows of the data set picture are marked again on the basis of the full-face features, an attention mechanism is added to the network, depth feature extraction is emphasized on the eyes and the eyebrows, new human face features are generated by combining the full-face and the eye eyebrow features, and finally the human face features are based on the features of the FC (first layer) of the classification network trained by L-Softmax,
L-Softmax of
Figure FDA0002638232720000021
Is deformed into
Figure FDA0002638232720000022
4. The method for evaluating the face quality based on the local key feature recognition according to claim 3, wherein the attention mechanism is to sum the weights of the overall features and the local features of the picture, increase the weight for the region with large contribution and decrease the weight for the region with small contribution, and in combination with the characteristics of the softmax function, the output interval is [0,1], and is suitable for being used in the distribution of the weights to obtain the weight calculation formula:
Figure FDA0002638232720000023
and calculating to obtain:
Figure FDA0002638232720000024
5. the method for evaluating the quality of the human face based on the local key feature recognition according to claim 2, characterized in that the features are mapped into n +1 dimensions by a softmax function, each dimension represents a category, and a one-dimensional convolution is used to replace a full-connected layer for classification branching.
6. The method for evaluating the quality of the human face based on the local key feature recognition according to claim 1, wherein the compared evaluation index is a general (MAP) index, and the formula is as follows:
Figure FDA0002638232720000025
p is the accuracy; r is the recall rate;
during calculation, the coincidence rate of the prediction frame and the marking frame is set as a, and the prediction frame with a being more than or equal to 0.5 is regarded as a positive example:
Figure FDA0002638232720000026
7. the method for evaluating the quality of a human face based on local key feature recognition according to claim 1, wherein the human face quality evaluating method based on local key feature recognition is adopted, the human face detection speed is less than 30ms, the human face feature extraction speed is less than 200ms, the human face detection accuracy is greater than 99.3%, and the human face recognition accuracy is greater than 92%.
CN202010831673.7A 2020-08-18 2020-08-18 Human face quality evaluation method based on local key feature recognition Pending CN112183190A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010831673.7A CN112183190A (en) 2020-08-18 2020-08-18 Human face quality evaluation method based on local key feature recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010831673.7A CN112183190A (en) 2020-08-18 2020-08-18 Human face quality evaluation method based on local key feature recognition

Publications (1)

Publication Number Publication Date
CN112183190A true CN112183190A (en) 2021-01-05

Family

ID=73919410

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010831673.7A Pending CN112183190A (en) 2020-08-18 2020-08-18 Human face quality evaluation method based on local key feature recognition

Country Status (1)

Country Link
CN (1) CN112183190A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110232369A (en) * 2019-06-20 2019-09-13 深圳和而泰家居在线网络科技有限公司 A kind of face identification method and electronic equipment
CN110610129A (en) * 2019-08-05 2019-12-24 华中科技大学 Deep learning face recognition system and method based on self-attention mechanism
CN111178183A (en) * 2019-12-16 2020-05-19 深圳市华尊科技股份有限公司 Face detection method and related device
CN111523431A (en) * 2020-04-16 2020-08-11 支付宝(杭州)信息技术有限公司 Face recognition method, device and equipment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110232369A (en) * 2019-06-20 2019-09-13 深圳和而泰家居在线网络科技有限公司 A kind of face identification method and electronic equipment
CN110610129A (en) * 2019-08-05 2019-12-24 华中科技大学 Deep learning face recognition system and method based on self-attention mechanism
CN111178183A (en) * 2019-12-16 2020-05-19 深圳市华尊科技股份有限公司 Face detection method and related device
CN111523431A (en) * 2020-04-16 2020-08-11 支付宝(杭州)信息技术有限公司 Face recognition method, device and equipment

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
DMITRY YASHUNIN等: "MaskFace: multi-task face and landmark detector", 《ARXIV》 *
HEFEI LING等: "Self Residual Attention Network For Deep Face Recognition", 《DIGITAL OBJECT IDENTIFIER》 *
JIANKANG DENG等: "ArcFace: Additive Angular Margin Loss for Deep Face Recognition", 《2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 *
YONG LI等: "Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 *
陈云霁 等著: "《智能计算系统》", 30 April 2020 *

Similar Documents

Publication Publication Date Title
CN109614921B (en) Cell segmentation method based on semi-supervised learning of confrontation generation network
CN110059586B (en) Iris positioning and segmenting system based on cavity residual error attention structure
CN110781829A (en) Light-weight deep learning intelligent business hall face recognition method
CN108961675A (en) Fall detection method based on convolutional neural networks
CN111832443B (en) Construction method and application of construction violation detection model
CN106650670A (en) Method and device for detection of living body face video
CN113627229B (en) Target detection method, system, device and computer storage medium
CN109919252A (en) The method for generating classifier using a small number of mark images
KR102593835B1 (en) Face recognition technology based on heuristic Gaussian cloud transformation
CN112668522A (en) Human body key point and human body mask combined detection network and method
CN111126155A (en) Pedestrian re-identification method for generating confrontation network based on semantic constraint
CN110659702A (en) Calligraphy copybook evaluation system and method based on generative confrontation network model
CN111507467A (en) Neural network model training method and device, computer equipment and storage medium
CN109741351A (en) A kind of classification responsive type edge detection method based on deep learning
CN112818774A (en) Living body detection method and device
CN113011307A (en) Face recognition identity authentication method based on deep residual error network
CN110855474B (en) Network feature extraction method, device, equipment and storage medium of KQI data
CN116597197A (en) Long-tail target detection method capable of adaptively eliminating negative gradient of classification
CN112183190A (en) Human face quality evaluation method based on local key feature recognition
CN113012167B (en) Combined segmentation method for cell nucleus and cytoplasm
CN111696117B (en) Loss function weighting method and device based on skeleton perception
CN111222463A (en) Motor vehicle face detection method based on cascade type convolution network
CN111274894A (en) Improved YOLOv 3-based method for detecting on-duty state of personnel
Ni et al. A proposed license plate classification method based on convolutional neural network
CN114897901B (en) Battery quality detection method and device based on sample expansion and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination