CN111860456A - Mask face recognition method - Google Patents
Mask face recognition method Download PDFInfo
- Publication number
- CN111860456A CN111860456A CN202010773738.7A CN202010773738A CN111860456A CN 111860456 A CN111860456 A CN 111860456A CN 202010773738 A CN202010773738 A CN 202010773738A CN 111860456 A CN111860456 A CN 111860456A
- Authority
- CN
- China
- Prior art keywords
- mask
- face
- recognition
- current
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000013528 artificial neural network Methods 0.000 claims abstract description 22
- 230000007246 mechanism Effects 0.000 claims abstract description 22
- 230000006978 adaptation Effects 0.000 claims abstract description 10
- 238000001514 detection method Methods 0.000 claims description 29
- 238000003384 imaging method Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 10
- 238000012549 training Methods 0.000 claims description 10
- 230000001815 facial effect Effects 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 230000015654 memory Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000012544 monitoring process Methods 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 4
- 230000007787 long-term memory Effects 0.000 description 4
- 238000010606 normalization Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 208000032443 Masked facies Diseases 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 210000000697 sensory organ Anatomy 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention belongs to the technical field of artificial intelligence, and discloses a mask face recognition method, which comprises the following steps: s1: acquiring a mask face image data set according to the non-mask face image data; s2: establishing a mask recognition model by using a mask face image data set based on a neural network, a confidence coefficient adaptation mechanism and a multi-frame clustering prediction mechanism; s3: and acquiring a field video in real time, and identifying by using a mask identification model according to the field video to obtain an identification result. The invention solves the problems of low recognition effect, low recognition efficiency, complex structure of a face recognition neural network and great influence by scenes in the prior art.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to a mask face recognition method.
Background
Social security is more and more emphasized, identity authentication of mask molecules becomes particularly important, and although some video monitoring devices are installed in various places, due to the fact that video monitoring environments are complex, the mask molecules have the reasons of crowd density, motion dispersion and the like and the reasons of imaging environments and the like, great practical challenges are brought to face identity identification of the mask molecules under huge monitoring.
With the development of economy, the mask face recognition demand also becomes remarkable. Many head manufacturers develop face recognition algorithms specifically under the mask. However, the algorithm research results of various manufacturers are different, and the recognition accuracy is also different. But basically applied to relatively static close-range recognition scenes such as gate machines, face attendance and the like.
The prior art has the following defects:
in the field of face recognition, wearing a mask/face mask belongs to large-area face shielding and is a long-known problem, and the difficulty is mainly reflected in the following four points:
1) the face recognition algorithm mainly judges the identity according to the facial features of the face, and when the mask is worn for recognition, the algorithm cannot accurately detect the face position and locate key points of five sense organs, so that the recognition effect is greatly reduced.
2) The deep learning technology used by the face recognition algorithm depends on massive training data, a large number of mask wearing photos are difficult to collect in a short period of time and are marked manually, and the recognition efficiency is low;
3) the face recognition neural network has a complex structure and comprises multiple modules, the wearing of the mask is not only a face comparison module, but also a plurality of modules such as face detection and tracking can be influenced, and great interference influence is brought to the whole design.
4) At present, mask identification in a monitoring scene is influenced by scene complexity and diversity, and particularly, imaging fuzzy complex scenes bring great challenges to mask visual identification.
Disclosure of Invention
The present invention aims to solve at least one of the above technical problems to a certain extent.
Therefore, the invention aims to provide a mask face recognition method, which is used for solving the problems of low recognition effect, low recognition efficiency, complex face recognition neural network structure and large influence of scenes in the prior art.
The technical scheme adopted by the invention is as follows:
a mask face recognition method comprises the following steps:
s1: acquiring a mask face image data set according to the non-mask face image data;
s2: establishing a mask recognition model by using a mask face image data set based on a neural network, a confidence coefficient adaptation mechanism and a multi-frame clustering prediction mechanism;
s3: and acquiring a field video in real time, and identifying by using a mask identification model according to the field video to obtain an identification result.
Further, in step S1, the MaskGAN method is used to operate on the non-mask face image data using various and interactive face images, so as to obtain a mask face image data set.
Further, in step S2, the neural network is a residual error network, the residual error network is provided with an LSTM long-term memory module, and the LSTM long-term memory module is provided with a forgetting gate.
Further, in step S2, the formula of the forgetting gate is:
ft＝σ(Wf·[ht-1，xt]+bf)
in the formula (f)tIs a forget gate function; sigma (#) is a sigmoid activation function; h ist-1Is the output of the previous time step (t-1); t is the time step indicator; x is the number oftInputting the current time step; bfIs a convolutional layer bias term; wfAre convolutional layer weights.
Further, in step S2, the output formula of the neural network is:
in the formula oijIs an attention weighted output feature; alpha is alphaijIs the normalized attention weight; i is an attention indicator; j is a one-way time step; n is the number of one-way time steps; h isjAs output for each time step.
Further, the specific step of step S2 is:
s2-1: dividing a mask face image data set into a training set and a testing set;
s2-2: training the neural network by using a training set to obtain an initial mask recognition model;
s2-3: and adding a confidence coefficient adaptation mechanism and a multi-frame clustering prediction mechanism into the initial mask recognition model, and optimizing the initial mask recognition model by using a test set to obtain and output an optimal mask recognition model.
Further, in step S2, the confidence level adaptation mechanism is: and adaptively adjusting the confidence threshold of the current model according to the imaging fuzziness of the face image data set input into the current model.
Further, the specific step of step S3 is:
s3-1: acquiring a live video in real time, carrying out human head detection on the live video, and acquiring the head characteristics of the current human head after the human head is detected;
s3-2: taking the head of a human body as an individual identifier, setting a head detection frame, continuously tracking the current individual in the field video according to the head characteristics and the position of the corresponding head detection frame, carrying out individual tracking matching to obtain the matching result of the individual, if the matching result is a new individual or an existing individual, acquiring a plurality of single-frame images of the current individual, and if not, ending the mask face recognition method;
s3-3: carrying out face detection on the current single-frame image, and carrying out front face detection on the current single-frame image by using a mask recognition model after a face is detected;
s3-4: after the front face is detected, acquiring the imaging fuzziness of the current single-frame image, adaptively adjusting a confidence threshold of a mask recognition model according to the imaging fuzziness, and performing face recognition on the current single-frame image by using the adjusted mask recognition model to obtain a face recognition result of the current frame image;
s3-5: updating the current single-frame image according to the plurality of single-frame images collected in the step S3-2, and returning to the step S3-3 until all the collected single-frame images are subjected to face recognition to obtain a plurality of face recognition results;
s3-6: and obtaining the final recognition result of the current individual based on a multi-frame clustering prediction mechanism according to the plurality of face recognition results.
Further, in step S3-3, face detection is performed, and the face detection result is: the current single-frame image is a mask face image or the current single-frame image is a non-mask face image.
Further, in step S3-3, performing face detection on the non-mask face image to obtain a non-mask face, and directly performing face recognition on the non-mask face to obtain a recognition result;
and (5) performing face correction detection on the face image of the mask to obtain a mask face, and entering the step S3-4 to obtain the imaging fuzziness of the current single-frame image according to the mask face.
The invention has the beneficial effects that:
1) the mask recognition accuracy under the monitoring scene is relatively high, the self-adaptive mask face recognition under most monitoring scenes can be met, and the scene influence is avoided;
2) the residual structure can effectively prevent the gradient from disappearing, so that the target feature learning capacity is very strong, the running speed is ideal, and the identification efficiency is improved;
3) the neural network is improved aiming at the special face scene of mask/face shield in the monitoring scene, and the enhanced bidirectional LSTM network is used, so that the robustness of the algorithm is greatly improved, the individual identification capability of the mask in the monitoring scene is enhanced, and the complex structure of the face identification neural network is avoided;
4) the post-processing logic of the multi-frame clustering prediction improves the accuracy of mask recognition prediction and improves the recognition accuracy.
Other advantageous effects of the present invention will be described in detail in the detailed description.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a mask face recognition method;
FIG. 2 is a non-masked face data image;
FIG. 3 is a mask face image dataset processing image;
fig. 4 is an image of the structure of the LSTM long-and-short memory module.
Detailed Description
The invention is further described with reference to the following figures and specific embodiments. It should be noted that the description of the embodiments is provided to help understanding of the present invention, but the present invention is not limited thereto. Functional details disclosed herein are merely illustrative of example embodiments of the invention. This invention may, however, be embodied in many alternate forms and should not be construed as limited to the embodiments set forth herein.
It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments of the invention. When the terms "comprises," "comprising," "includes," and/or "including" are used herein, they specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, elements, components, and/or groups thereof.
It should also be noted that, in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed substantially concurrently, or the figures may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
It should be understood that specific details are provided in the following description to facilitate a thorough understanding of example embodiments. However, it will be understood by those of ordinary skill in the art that the example embodiments may be practiced without these specific details. For example, systems may be shown in block diagrams in order not to obscure the examples in unnecessary detail. In other instances, well-known processes, structures and techniques may be shown without unnecessary detail in order to avoid obscuring example embodiments.
Example 1
As shown in fig. 1, the present embodiment provides a mask face recognition method, including the following steps:
s1: acquiring a mask face image data set according to the non-mask face image data;
processing the non-mask face image data by using a MaskGAN method for various and interactive face images to obtain a mask face image data set;
as shown in fig. 2, a second automatic affine transformation labeling is performed using a conventional 8-class 32-subclass face database, which is 200 ten thousand, and through 68 point pairs of faces and affine transformation, an automatic mask wearing is performed, and MaskGAN method is used to operate various and interactive face images to process non-mask face image data, as shown in fig. 3, the method includes:
1) the area shielded by the mask (5 percent is used as shielding proportion interval, and 16 different gears are divided by 5 to 80 percent);
2) in order to adapt to the influence of a multi-style mask on face recognition, N95 mask styles, common medical masks, common masks and facial tissues are adopted for classification and processing;
3) in order to adapt to the influence of different color masks/facial tissues on the face identification, N95 mask patterns of four colors of blue, gray, white and black, common medical masks, common masks and facial tissues are used for classified mixing treatment;
s2: based on a neural network, a confidence coefficient adaptation mechanism and a multi-frame clustering prediction mechanism, a mask face image data set is used for establishing a mask recognition model, and post-processing logic of multi-frame clustering prediction is adopted, so that the accuracy of mask recognition prediction is improved, and the recognition accuracy is improved;
the neural network is a residual error network, the residual error network is provided with an LSTM long-term memory module, the LSTM long-term memory module is provided with a forgetting gate, and a residual error structure can effectively prevent gradient disappearance, so that the learning target feature capacity is very strong, the operation speed is ideal, the recognition efficiency is improved, the neural network is improved aiming at a special face scene of mask/face shield shielding in a monitoring scene, the enhanced bidirectional LSTM network is used, the robustness of an algorithm is greatly improved, the individual recognition capability of a mask in the monitoring scene is enhanced, and the complexity of the face recognition neural network structure is avoided;
the formula for a forget gate is:
ft＝σ(Wf·[ht-1，xt]+bf)
in the formula (f)tIs a forget gate function; sigma (#) is a sigmoid activation function; h ist-1Is the output of the previous time step (t-1); t is the time step indicator; x is the number oftInputting the current time step; bfIs a convolutional layer bias term; wfIs the convolutional layer weight; the output of the previous time step and the input of the current time step are fused through the convolution layer in the whole calculation process, and then the excitation is carried out through a sigmoid function, the output is limited between 0 and 1, 0 represents that all the time is forgotten, and 1 represents that all the time is reserved;
the formula of the attention module of the neural network is:
eij＝tanh((hs·w)+b)*u
in the formula, eijAttention weight before normalization; tan h (—) is a hyperbolic tangent function; h issFor the output of each time step; w is the convolution weight; b is a convolution bias term; u is a scaling factor; i is an attention indicator; j is a one-way time step;
the formula for the attention weight of the neural network is:
in the formula, alphaijIs the normalized attention weight; e.g. of the typeijAttention weight before normalization; i is an attention indicator; j is a one-way time step; k is a time step indicator; n is the number of one-way time steps; the calculation is that activation of a normalization index softmax function is carried out, the output is limited to be 0-1, and attention distribution is obtained;
the formula of the attention weighted output characteristics of the neural network is:
in the formula oijIs an attention weighted output feature; alpha is alphaijIs the normalized attention weight; i is an attention indicator; j is a one-way time step; n is the number of one-way time steps; h isjFor the output of each time step;
the confidence adaptation mechanism is as follows: adaptively adjusting a confidence threshold of the current model according to the imaging fuzziness of the face image data set input into the current model, and turning down the confidence threshold to obtain a single recognition result when the imaging fuzziness is increased so as to avoid the problem of unrecognizable imaging;
the specific steps of step S2 are:
s2-1: dividing a mask face image data set into a training set and a testing set;
s2-2: training the neural network by using a training set to obtain an initial mask recognition model;
s2-3: adding a confidence coefficient adaptation mechanism and a multi-frame clustering prediction mechanism into an initial mask recognition model, and optimizing the initial mask recognition model by using a test set to obtain and output an optimal mask recognition model;
s3: the method comprises the following steps of acquiring a field video in real time, identifying by using a mask identification model according to the field video to obtain an identification result, wherein the mask identification accuracy under a monitoring scene is relatively high, the self-adaptive mask face identification under most monitoring scenes can be met, the scene influence is avoided, and the method comprises the following specific steps:
s3-1: acquiring a live video in real time, carrying out human head detection on the live video, and acquiring the head characteristics of the current human head after the human head is detected;
s3-2: taking the head of a human body as an individual identifier, setting a head detection frame, continuously tracking the current individual in the field video according to the head characteristics and the position of the corresponding head detection frame, carrying out individual tracking matching to obtain the matching result of the individual, if the matching result is a new individual or an existing individual, acquiring a plurality of single-frame images of the current individual, and if not, ending the mask face recognition method;
s3-3: carrying out face detection on the current single-frame image, and carrying out front face detection on the current single-frame image by using a mask recognition model after a face is detected;
and carrying out face detection, wherein the face detection result is as follows: the current single-frame image is a mask face image or the current single-frame image is a non-mask face image;
carrying out face detection on the non-mask face image to obtain a non-mask face, and directly carrying out face identification on the non-mask face to obtain an identification result;
carrying out face correction detection on the face image of the mask to obtain a mask face, entering a step S3-4, and acquiring the imaging fuzziness of the current single-frame image according to the mask face;
s3-4: after the front face is detected, acquiring the imaging fuzziness of the current single-frame image, adaptively adjusting a confidence threshold of a mask recognition model according to the imaging fuzziness, and performing face recognition on the current single-frame image by using the adjusted mask recognition model to obtain a face recognition result of the current frame image;
imaging ambiguity calculation rule:
1) pixel RGB to gray value calculation:
RGB is not less than Gray value Grayn＝R*0.3+G*0.59+B*0.11；
2) Calculating the average value of each pixel point of the gray picture:
μ＝∑Graynn, wherein μ is an average value of each pixel point of the gray-scale picture; graynIs the gray value of the gray picture; n is the total number of pixel points;
3) calculating the gray value variance of the whole gray image:
S2＝∑(Grayn-μ)2v (N-1), wherein S2The smaller S is, the more blurred the image is;
in this embodiment, S is greater than or equal to 10 and less than 20, the confidence coefficient weight is adjusted to 0.8, and then the current confidence coefficient threshold is 0.8 × the preset confidence coefficient threshold;
s is more than or equal to 20 and less than or equal to 30, the weight of the confidence coefficient is adjusted to be 0.9, and then the current confidence coefficient threshold value is 0.9 multiplied by the preset confidence coefficient threshold value;
s is not more than 30, the weight of the confidence coefficient is adjusted to be 1, then the current confidence coefficient threshold is 1 x the preset confidence coefficient threshold, and S is 30 which is the optimal critical value of the fuzzy detection;
s3-5: updating the current single-frame image according to the plurality of single-frame images acquired in the step S3-2, namely, taking the next single-frame image as the current single-frame image, and returning to the step S3-3 until all the acquired single-frame images are subjected to face recognition to obtain a plurality of face recognition results;
s3-6: according to a plurality of face recognition results, based on a multi-frame clustering prediction mechanism, obtaining identity information of a current individual, namely a final recognition result, continuously updating individual ID in continuous individual tracking and recognition, wherein in the initial tracking stage, the recognition result is likely to continuously change due to adverse factors such as mask shielding, long distance, insufficient light and the like, but after continuous tracking, the recognition result tends to be stable, accurate identity information is obtained, and the recognition success rate is greatly improved;
the multi-frame clustering prediction mechanism is as follows:
matched individual P0-PkThe confidence levels of (a) are respectively:
individual P0Contains m0 results:
individual P1Contains m1 results:
……
individual PkContains mk results:
the inter-frame enhancement similarity of the recognition results of each individual is as follows:
...
then R (F)0-Fn)＝Max(∑P0,∑P1,...,∑Pk) The corresponding individual P;
namely, a plurality of individuals are possibly matched in continuous tracking identification, the mechanism carries out comprehensive prediction according to the matching times and confidence degrees, the final confidence degrees of different individuals are calculated and compared, and the confidence degrees correspond to the maximum inter-frame enhancement similarity sigma PjThe individual P is the individual R (F) of the output resulti) Where j is 0,1, and k, i is 0,1, and n, the identification result becomes more stable with continuous tracking, and the identity information of the individual becomes more and more certain.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.
The embodiments described above are merely illustrative, and may or may not be physically separate, if referring to units illustrated as separate components; if reference is made to a component displayed as a unit, it may or may not be a physical unit, and may be located in one place or distributed over a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: modifications of the technical solutions described in the embodiments or equivalent replacements of some technical features may still be made. And such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
The present invention is not limited to the above-described alternative embodiments, and various other forms of products can be obtained by anyone in light of the present invention. The above detailed description should not be taken as limiting the scope of the invention, which is defined in the claims, and which the description is intended to be interpreted accordingly.
Claims (10)
1. A mask face recognition method is characterized in that: the method comprises the following steps:
s1: acquiring a mask face image data set according to the non-mask face image data;
s2: establishing a mask recognition model by using a mask face image data set based on a neural network, a confidence coefficient adaptation mechanism and a multi-frame clustering prediction mechanism;
s3: and acquiring a field video in real time, and identifying by using a mask identification model according to the field video to obtain an identification result.
2. The mask face recognition method according to claim 1, characterized in that: in step S1, the MaskGAN method of various and interactive facial image operations is used to process the non-mask facial image data, so as to obtain a mask facial image data set.
3. The mask face recognition method according to claim 1, characterized in that: in step S2, the neural network is a residual network, the residual network is provided with an LSTM long-and-short memory module, and the LSTM long-and-short memory module is provided with a forgetting gate.
4. A mask face recognition method according to claim 3, characterized in that: in step S2, the formula of the forgotten door is:
ft＝σ(Wf·[ht-1，xt]+bf)
in the formula (f)tIs a forget gate function; sigma (#) is a sigmoid activation function; h ist-1Is the output of the previous time step (t-1); t is the time step indicator; x is the number oftInputting the current time step; bfIs a convolutional layer bias term; wfAre convolutional layer weights.
5. The mask face recognition method according to claim 1, characterized in that: in step S2, the output formula of the neural network is:
in the formula oijIs an attention weighted output feature; alpha is alphaijIs the normalized attention weight; i is an attention indicator; j is a one-way time step; n is the number of one-way time steps; h isjAs output for each time step.
6. The mask face recognition method according to claim 1, characterized in that: the specific steps of step S2 are:
s2-1: dividing a mask face image data set into a training set and a testing set;
s2-2: training the neural network by using a training set to obtain an initial mask recognition model;
s2-3: and adding a confidence coefficient adaptation mechanism and a multi-frame clustering prediction mechanism into the initial mask recognition model, and optimizing the initial mask recognition model by using a test set to obtain and output an optimal mask recognition model.
7. The mask face recognition method according to claim 1, characterized in that: in step S2, the confidence level adaptation mechanism is: and adaptively adjusting the confidence threshold of the current model according to the imaging fuzziness of the face image data set input into the current model.
8. The mask face recognition method according to claim 1, characterized in that: the specific steps of step S3 are:
s3-1: acquiring a live video in real time, carrying out human head detection on the live video, and acquiring the head characteristics of the current human head after the human head is detected;
s3-2: taking the head of a human body as an individual identifier, setting a head detection frame, continuously tracking the current individual in the field video according to the head characteristics and the position of the corresponding head detection frame, carrying out individual tracking matching to obtain the matching result of the individual, if the matching result is a new individual or an existing individual, acquiring a plurality of single-frame images of the current individual, and if not, ending the mask face recognition method;
s3-3: carrying out face detection on the current single-frame image, and carrying out front face detection on the current single-frame image by using a mask recognition model after a face is detected;
s3-4: after the front face is detected, acquiring the imaging fuzziness of the current single-frame image, adaptively adjusting a confidence threshold of a mask recognition model according to the imaging fuzziness, and performing face recognition on the current single-frame image by using the adjusted mask recognition model to obtain a face recognition result of the current frame image;
s3-5: updating the current single-frame image according to the plurality of single-frame images collected in the step S3-2, and returning to the step S3-3 until all the collected single-frame images are subjected to face recognition to obtain a plurality of face recognition results;
s3-6: and obtaining the final recognition result of the current individual based on a multi-frame clustering prediction mechanism according to the plurality of face recognition results.
9. The mask face recognition method of claim 8, wherein: in step S3-3, face detection is performed, and the face detection result is: the current single-frame image is a mask face image or the current single-frame image is a non-mask face image.
10. The mask face recognition method of claim 9, wherein: in the step S3-3, performing front face detection on the non-mask face image to obtain a non-mask front face, and directly performing face recognition on the non-mask front face to obtain a recognition result;
and (5) performing face correction detection on the face image of the mask to obtain a mask face, and entering the step S3-4 to obtain the imaging fuzziness of the current single-frame image according to the mask face.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010773738.7A CN111860456B (en) | 2020-08-04 | 2020-08-04 | Face recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010773738.7A CN111860456B (en) | 2020-08-04 | 2020-08-04 | Face recognition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111860456A true CN111860456A (en) | 2020-10-30 |
| CN111860456B CN111860456B (en) | 2024-02-02 |
Family
ID=72953217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010773738.7A Active CN111860456B (en) | 2020-08-04 | 2020-08-04 | Face recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111860456B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101777114A (en) * | 2009-01-08 | 2010-07-14 | 北京中星微电子有限公司 | Intelligent analysis system and intelligent analysis method for video monitoring, and system and method for detecting and tracking head and shoulder |
| CN102622584A (en) * | 2012-03-02 | 2012-08-01 | 成都三泰电子实业股份有限公司 | Method for detecting mask faces in video monitor |
| CN104866843A (en) * | 2015-06-05 | 2015-08-26 | 中国人民解放军国防科学技术大学 | Monitoring-video-oriented masked face detection method |
| CN108875763A (en) * | 2017-05-17 | 2018-11-23 | 北京旷视科技有限公司 | Object detection method and object detecting device |
| CN109785363A (en) * | 2018-12-29 | 2019-05-21 | 中国电子科技集团公司第五十二研究所 | A kind of unmanned plane video motion Small object real-time detection and tracking |
| CN109919977A (en) * | 2019-02-26 | 2019-06-21 | 鹍骐科技（北京）股份有限公司 | A kind of video motion personage tracking and personal identification method based on temporal characteristics |
| CN110781784A (en) * | 2019-10-18 | 2020-02-11 | 高新兴科技集团股份有限公司 | Face recognition method, device and equipment based on double-path attention mechanism |
| CN110853033A (en) * | 2019-11-22 | 2020-02-28 | 腾讯科技（深圳）有限公司 | Video detection method and device based on inter-frame similarity |
| US20200086879A1 (en) * | 2018-09-14 | 2020-03-19 | Honda Motor Co., Ltd. | Scene classification prediction |
-
2020
- 2020-08-04 CN CN202010773738.7A patent/CN111860456B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101777114A (en) * | 2009-01-08 | 2010-07-14 | 北京中星微电子有限公司 | Intelligent analysis system and intelligent analysis method for video monitoring, and system and method for detecting and tracking head and shoulder |
| CN102622584A (en) * | 2012-03-02 | 2012-08-01 | 成都三泰电子实业股份有限公司 | Method for detecting mask faces in video monitor |
| CN104866843A (en) * | 2015-06-05 | 2015-08-26 | 中国人民解放军国防科学技术大学 | Monitoring-video-oriented masked face detection method |
| CN108875763A (en) * | 2017-05-17 | 2018-11-23 | 北京旷视科技有限公司 | Object detection method and object detecting device |
| US20200086879A1 (en) * | 2018-09-14 | 2020-03-19 | Honda Motor Co., Ltd. | Scene classification prediction |
| CN109785363A (en) * | 2018-12-29 | 2019-05-21 | 中国电子科技集团公司第五十二研究所 | A kind of unmanned plane video motion Small object real-time detection and tracking |
| CN109919977A (en) * | 2019-02-26 | 2019-06-21 | 鹍骐科技（北京）股份有限公司 | A kind of video motion personage tracking and personal identification method based on temporal characteristics |
| CN110781784A (en) * | 2019-10-18 | 2020-02-11 | 高新兴科技集团股份有限公司 | Face recognition method, device and equipment based on double-path attention mechanism |
| CN110853033A (en) * | 2019-11-22 | 2020-02-28 | 腾讯科技（深圳）有限公司 | Video detection method and device based on inter-frame similarity |
Non-Patent Citations (1)
| Title |
|---|
| 方明等: "结合残差网络及目标掩膜的人脸微表情识别", 《吉林大学学报》, vol. 51, no. 01, pages 303 - 313 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111860456B (en) | 2024-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Cross-domain face presentation attack detection via multi-domain disentangled representation learning | |
| CN109886121B (en) | Human face key point positioning method for shielding robustness | |
| Bai et al. | Infrared ship target segmentation based on spatial information improved FCM | |
| Ilbeygi et al. | A novel fuzzy facial expression recognition system based on facial feature extraction from color face images | |
| Iwama et al. | Gait verification system for criminal investigation | |
| Zhang et al. | Moving cast shadows detection using ratio edge | |
| Dass | Markov random field models for directional field and singularity extraction in fingerprint images | |
| Migdal et al. | Background subtraction using markov thresholds | |
| CN103035013B (en) | A kind of precise motion shadow detection method based on multi-feature fusion | |
| Cheng et al. | Scene analysis for object detection in advanced surveillance systems using Laplacian distribution model | |
| JP2003317101A (en) | Method for verifying face using method for automatically updating database and system therefor | |
| KR20140067604A (en) | Apparatus, method and computer readable recording medium for detecting, recognizing and tracking an object based on a situation recognition | |
| De Smet et al. | A generalized EM approach for 3D model based face recognition under occlusions | |
| Wang et al. | Data mapping by probabilistic modular networks and information-theoretic criteria | |
| Katircioglu et al. | Self-supervised human detection and segmentation via background inpainting | |
| Mohamed et al. | Automated face recogntion system: Multi-input databases | |
| Amato et al. | Robust real-time background subtraction based on local neighborhood patterns | |
| Côté et al. | Comparative study of adaptive segmentation techniques for gesture analysis in unconstrained environments | |
| Huang et al. | A people-counting system using a hybrid RBF neural network | |
| CN111104857A (en) | Identity recognition method and system based on gait energy diagram | |
| CN111860456B (en) | Face recognition method | |
| Méndez-Llanes et al. | On the use of local fixations and quality measures for deep face recognition | |
| Nasiri et al. | Video Surveillance Framework Based on Real-Time Face Mask Detection and Recognition | |
| Gritzman et al. | Threshold-based outer lip segmentation using support vector regression | |
| Jang et al. | Skin region segmentation using an image-adapted colour model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |