WO2019033572A1

WO2019033572A1 - Method for detecting whether face is blocked, device and storage medium

Info

Publication number: WO2019033572A1
Application number: PCT/CN2017/108751
Authority: WO
Inventors: 陈林; 张国辉
Original assignee: 平安科技（深圳）有限公司
Priority date: 2017-08-17
Filing date: 2017-10-31
Publication date: 2019-02-21
Also published as: CN107633204A; CN107633204B

Abstract

A method for detecting whether a face is blocked, an electronic device and a computer-readable storage medium. Said method comprising: acquiring a real-time image captured by a camera device, and extracting a real-time face image from the real-time image (S10); inputting the real-time face image into a face average model, and identifying the face feature points from the real-time face image (S20); determining, according to the position information of the face feature points, an eye region and a lip region, inputting the eye region and the lip region into a pre-trained eye classification model of the face and a lip classification model of the face (S30), determining the authenticity of the eye region and the lip region (S40), and determining, according to a determination result, whether the face in the real-time image is blocked (S50, S60). The method above can quickly determine whether the face in a face image is blocked.

Description

Face blocking detection method, device and storage medium

Priority claim

The present application is based on the priority of the Chinese Patent Application entitled "Face Blocking Detection Method, Apparatus and Storage Medium", filed on August 17, 2017, with the application number of CN 201710707944.6, the entire contents of the Chinese patent application. It is incorporated herein by reference.

Technical field

The present application relates to the field of computer vision processing technologies, and in particular, to a face occlusion detection method, apparatus, and computer readable storage medium.

Background technique

Face recognition is a biometric recognition technology based on human facial feature information for identity authentication. The detected face is matched and recognized by collecting an image or a video stream containing a face and detecting and tracking the face in the image. At present, the application field of face recognition is very extensive, and plays a very important role in many fields such as financial payment, access control, and identification, which brings great convenience to people's lives. However, it is important to ensure that no occlusion occurs on the face, so it is necessary to detect whether the face in the image is occluded before face recognition.

The general product in the industry judges that the face occlusion is determined by the deep learning training method, but the judgment method has high requirements on the sample size, and if the occlusion is predicted by the deep learning method, the calculation amount is large and the speed is relatively slow.

Summary of the invention

The present application provides a face occlusion detection method, apparatus, and computer readable storage medium, the main purpose of which is to quickly detect a face occlusion situation in a real-time facial image.

To achieve the above objective, the present application provides an electronic device, including: a memory, a processor, and an imaging device, wherein the memory includes a face occlusion detection program, and the face occlusion detection program is executed by the processor Implement the following steps:

Image acquisition step: acquiring a real-time image captured by the camera device, and extracting a real-time face image from the real-time image by using a face recognition algorithm;

a feature point recognition step: inputting the real-time facial image into a pre-trained facial average model, and using the facial average model to identify t facial feature points from the real-time facial image;

a feature region determining step: determining an eye region and a lip region according to position information of the t facial feature points, and inputting the eye region and the lip region into an eye part class model of a pre-trained face, a face The lip part type model determines the authenticity of the eye area and the lip area, and determines whether the face in the real-time image is occluded according to the judgment result.

Preferably, when the face occlusion detection program is executed by the processor, the following steps are further implemented:

a determining step: determining an eye part class model of the face, a lip part class model of the face Whether the judgment results of the eye area and the lip area are true.

When the eye part model of the face and the lip part model of the face are true to the eye area and the lip area, it is determined that the face in the real-time face image is not occluded; and

When the judgment result of the eye part class model of the face and the lip part class model of the face on the eye region and the lip region is not true, the face in the real-time face image is prompted to be occluded.

In addition, to achieve the above object, the present application further provides a method for detecting a face occlusion, the method comprising:

Preferably, the method further comprises:

The determining step is: determining whether the judgment result of the eye part class model and the lip part class model of the human face on the eye region and the lip region is true.

Preferably, the method further comprises:

In addition, in order to achieve the above object, the present application further provides a computer readable storage medium including a face occlusion detection program, where the face occlusion detection program is executed by a processor, as described above Any of the steps in the face occlusion detection method described.

The face occlusion detection method, the electronic device and the computer readable storage medium proposed by the present application identify the facial feature points in the real-time facial image by inputting the real-time facial image into the face averaging model, and use the eye part class of the human face The lip model of the model and the face determines the authenticity of the eye region and the lip region determined by the facial feature point, and determines whether the face in the real-time facial image occurs according to the authenticity of the eye region and the lip region. Occlusion.

DRAWINGS

1 is a schematic diagram of an application environment of a preferred embodiment of a method for detecting a face occlusion of an applicant;

2 is a block diagram of a face occlusion detection program of FIG. 1;

3 is a flow chart of a preferred embodiment of the applicant's face occlusion detection method.

The implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings.

Detailed ways

It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.

The present application provides a face occlusion detection method applied to an electronic device 1 . Referring to FIG. 1 , it is a schematic diagram of an application environment of a preferred embodiment of the applicant's face occlusion detection method.

In this embodiment, the electronic device 1 may be a terminal device having a computing function, such as a server, a smart phone, a tablet computer, a portable computer, or a desktop computer.

The electronic device 1 includes a processor 12, a memory 11, an imaging device 13, a network interface 14, and a communication bus 15. The camera device 13 is installed in a specific place, such as an office place and a monitoring area, and real-time images are taken in real time for the target entering the specific place, and the captured real-time image is transmitted to the processor 12 through the network. Network interface 14 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). Communication bus 15 is used to implement connection communication between these components.

The memory 11 includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card type memory, or the like. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 1, such as a hard disk of the electronic device 1. In other embodiments, the readable storage medium may also be an external memory of the electronic device 1, such as a plug-in hard disk equipped on the electronic device 1, a smart memory card (SMC), Secure Digital (SD) card, Flash Card, etc.

In this embodiment, the readable storage medium of the memory 11 is generally used to store a face occlusion detection program 10 installed on the electronic device 1, a face image sample library, a human eye sample library, and a human lip sample. The library, the facial average model of the face feature points constructed and trained, the eye part class model, and the lip part class model of the face. The memory 11 can also be used to temporarily store data that has been output or is about to be output.

The processor 12, in some embodiments, may be a Central Processing Unit (CPU), microprocessor or other data processing chip for running program code or processing data stored in the memory 11, such as performing face occlusion Test procedure 10, etc.

Figure 1 shows only the electronic device 1 with components 11-15, but it should be understood that not all illustrated components may be implemented, and more or fewer components may be implemented instead.

Optionally, the electronic device 1 may further include a user interface, and the user interface may include an input unit such as a keyboard, a voice input device such as a microphone, and the like. The device for voice recognition function, voice output device such as audio, earphone, etc., optionally the user interface may also include a standard wired interface, a wireless interface.

Optionally, the electronic device 1 may further include a display, which may also be appropriately referred to as a display screen or a display unit. In some embodiments, it may be an LED display, a liquid crystal display, a touch liquid crystal display, and an Organic Light-Emitting Diode (OLED) touch sensor. The display is used to display information processed in the electronic device 1 and a user interface for displaying visualizations.

Optionally, the electronic device 1 further comprises a touch sensor. The area provided by the touch sensor for the user to perform a touch operation is referred to as a touch area. Further, the touch sensor described herein may be a resistive touch sensor, a capacitive touch sensor, or the like. Moreover, the touch sensor includes not only a contact type touch sensor but also a proximity type touch sensor or the like. Furthermore, the touch sensor may be a single sensor or a plurality of sensors arranged, for example, in an array.

In addition, the area of the display of the electronic device 1 may be the same as or different from the area of the touch sensor. Optionally, a display is stacked with the touch sensor to form a touch display. The device detects a user-triggered touch operation based on a touch screen display.

Optionally, the electronic device 1 may further include a radio frequency (RF) circuit, a sensor, an audio circuit, and the like, and details are not described herein.

In the apparatus embodiment shown in FIG. 1, an operating system and a face occlusion detecting program 10 may be included in the memory 11 as a computer storage medium; when the processor 12 executes the face occlusion detecting program 10 stored in the memory 11, Implement the following steps:

Image acquisition step: acquiring a real-time image captured by the camera device 13, and extracting a real-time face image from the real-time image using a face recognition algorithm.

When the camera 13 captures a real-time image, the camera 13 transmits the real-time image to the processor 12. When the processor 12 receives the real-time image, it first acquires the size of the image to create a grayscale image of the same size. Converting the acquired color image into a grayscale image and creating a memory space; equalizing the grayscale image histogram, reducing the amount of grayscale image information, speeding up the detection speed, and then loading the training library to detect the person in the image Face, and return an object containing face information, obtain the data of the location of the face, and record the number; finally obtain the area of the avatar and save it, thus completing a real-time facial image extraction process.

Specifically, the face recognition algorithm for extracting the real-time facial image from the real-time image may be a geometric feature-based method, a local feature analysis method, a feature face method, an elastic model-based method, a neural network method, or the like.

Feature point identification step: input the real-time facial image into a pre-trained facial average model, and use the facial average model to identify t facial feature points from the real-time facial image.

Establishing a first sample library having n face images, and marking t facial feature points in each face image, the t facial feature points including: t ₁ eye feature points representing eye positions, t ₂ eye feature points and t ₃ lip feature points representing the position of the lips. In each face image in the first sample library, manually mark t ₁ eyelid feature point, t ₂ eyeball feature points, and t ₃ lip feature points, respectively in the face image (t ₁ +t ₂ + t ₃ ) The feature points constitute a shape feature vector S, and the n shape feature vectors S of the face are obtained.

The face feature recognition model is trained by using the t facial feature points to obtain a face average model. The face feature recognition model is an Ensemble of Regression Tress (ERT) algorithm. The ERT algorithm is expressed as follows:

Where t represents the cascading sequence number and τ _t (·, ·) represents the regression of the current stage. Each regression is composed of a number of regression trees, and the purpose of training is to obtain these regression trees.

Where S(t) is the shape estimate of the current model; each regression τ _t (·, ·) predicts an increment based on the input images I and S(t)

Add this increment to the current shape estimate to improve the current model. Each level of regression is based on feature points for prediction. The training data set is: (I1, S1), ..., (In, Sn) where I is the input sample image and S is the shape feature vector composed of the feature points in the sample image.

In the process of model training, the number of face images in the first sample library is t, assuming that each sample image has 34 feature points, the feature vector

x ₁ to x ₁₂ represent the abscissa of the eyelid feature point, x ₁₃ to x ₁₄ represent the abscissa of the eye feature point, and x ₁₅ to x ₃₄ represent the abscissa of the lip feature point. Take some feature points of all sample images (for example, randomly take 25 feature points out of 34 feature points of each sample picture) to train the first regression tree, and predict the first regression tree and the partial features. The residual of the true value of the point (the weighted average of the 25 feature points taken from each sample picture) is used to train the second tree... and so on until the predicted value of the Nth tree is trained and described The true value of some feature points is close to 0, and all the regression trees of the ERT algorithm are obtained. According to these regression trees, the face average model is obtained, and the model file and the sample library are saved in the memory. Since the sample of the training model marks 12 eyelid feature points, 2 eyeball feature points, and 20 lip feature points, the face average model of the trained face can be used to identify 12 eyelid feature points from the face image, 2 eye feature points and 20 lip feature points.

After obtaining the real-time facial image, the trained facial average model is called from the memory 11, the real-time facial image is aligned with the facial average model, and the feature extraction algorithm is used to search the real-time facial image with the facial average model. 12 eyelid feature points, 2 eyeball feature points, and 12 eyelid feature points, 2 eyeball feature points, and 20 lip feature points matched by 20 lip feature points. Wherein, the 20 lip feature points are evenly distributed on the lip.

a feature region determining step: determining an eye region and a lip region according to position information of the t facial feature points, the eye region and the eye region and the lip region according to the position information of the t facial feature points, The eye region and the lip region input an eye part class model of a pre-trained face, a lip part class model of the face, determine the authenticity of the eye region and the lip region, and determine the result according to the judgment result Whether the face in the live image is occluded.

A first number of human eye positive sample images and a second number of human eye negative sample images are collected, and local features of each human eye positive sample image and human eye negative sample image are extracted. The human eye positive sample image refers to an eye sample containing a human eye, and the binocular portion can be extracted from the face image sample library as an eye sample. The ocular negative eye sample image refers to an image of a broken eye region, and a plurality of human eye positive sample images and negative sample images form a second sample library.

A third number of lip positive sample images and a fourth number of lip negative sample images are collected, extracting local features of each lip positive sample image, lip negative sample image. The lip positive sample image refers to an image containing the human's lips, and the lip portion can be extracted from the face image sample library as a lip positive sample image. The lip negative sample image refers to an image of a person's lip region being defective, or the lip of the image is not the lip of a human (eg, an animal), and the plurality of lip positive sample images and the negative sample image form a third sample bank.

Specifically, the local feature is a Histogram of Oriented Gradient (HOG) feature, which is extracted from a human eye sample image and a lip sample image by a feature extraction algorithm. Since the color information in the sample image is not very effective, it is usually converted into a grayscale image, and the entire image is normalized, the gradient of the horizontal and vertical directions of the image is calculated, and the gradient direction of each pixel position is calculated accordingly. Values to capture outlines, silhouettes, and some texture information, and further weaken the effects of lighting. Then the whole image is divided into individual Cell cells (8*8 pixels), and a gradient direction histogram is constructed for each Cell cell to calculate the local image gradient information and quantize to obtain the feature description vector of the local image region. Then the cell cells are combined into a large block. Due to the change of local illumination and the change of the foreground-background contrast, the gradient intensity varies greatly. This requires normalization of the gradient intensity, further Light, shadow, and edges are compressed. Finally, all "block" HOG descriptors are combined to form the final HOG feature description vector.

The support vector machine (SVM) is trained by using the positive and negative sample images and the extracted HOG features in the second sample library and the third sample library to obtain the eye part class model of the face and The lip part class model of the face.

After identifying 12 eyelid feature points, 2 eyeball feature points, and 20 lip feature points from the real-time facial image, an eye region may be determined according to the 12 eyelid feature points and the two eyeball feature points, according to The 20 lip feature points define a lip region, and then the determined eye region and the lip region are input into the eye part class model of the trained face and the lip part class model of the face, and are judged according to the result obtained by the model. The determined authenticity of the eye region and the lip region, that is, the result of the model output may be all false or all true, and may include both true and false. When the result of the eye part class model of the face and the lip part class model of the face is false, it means that the eye area and the lip area are not the human eye area and the human lip area; Both the eye part class model of the face and the lip part class model output of the face are true, indicating that the eye region and the lip region are the human eye region and the human lip region.

The determining step is: determining whether the judgment result of the eye part class model and the lip part class model of the human face on the eye region and the lip region is true. When the eye part class model of the face and the lip part class model of the face output the result, whether the judgment result contains only true.

When the eye part type model of the face and the lip part type model of the face are true to the eye area and the lip area, it is determined that the face in the real-time face image is not occluded. That is to say, when the eye region and the lip region determined according to the facial feature points are both the human eye region or the human lip region, it is considered that the human face in the real-time facial image is not blocked.

When the judgment result of the eye part class model of the face and the lip part class model of the face on the eye region and the lip region is not true, the face in the real-time face image is prompted to be occluded. When any one of the eye region and the lip region determined according to the facial feature point is not the human eye region or the human lip region, the face in the real-time facial image is considered to be occluded, prompting the real-time The face in the face image is occluded.

Further, when the output result of the eye part class model of the face is false, the eye area in the image is considered to be occluded, and when the lip part class model output result of the face is false, the lip in the image is considered The area is occluded and prompted accordingly.

In other embodiments, if the face recognition is performed after detecting whether the face is occluded, when the face in the real-time face image is occluded, the face in the current face image is occluded and re-acquired. The real-time image captured by the imaging device 13 is subjected to subsequent steps.

The electronic device 1 of the embodiment extracts a real-time facial image from a real-time image, and uses a facial average model to identify a facial feature point in the real-time facial image, and uses an eye-part model of the human face and a lip of the human face. The classification model analyzes the eye region and the lip region determined by the facial feature points, and quickly determines whether the face in the current image is occluded according to the authenticity of the eye region and the lip region.

In other embodiments, the face occlusion detection program 10 can also be partitioned into one or more modules that are stored in the memory 11 and executed by the processor 12 to complete the application. A module as referred to in this application refers to a series of computer program instructions that are capable of performing a particular function. Referring to FIG. 2, it is a block diagram of the face occlusion detection program 10 of FIG.

In this embodiment, the face occlusion detection program 10 can be divided into: an acquisition module 110, an identification module 120, a determination module 130, and a prompting module 140. The functions or operational steps implemented by the modules 110-140 are similar to the above, and are not described in detail herein, by way of example, for example:

The acquiring module 110 is configured to acquire a real-time image captured by the camera device 13 and extract a real-time face image from the real-time image by using a face recognition algorithm;

The identification module 120 is configured to input the real-time facial image into a pre-trained facial average model, and use the facial average model to identify t facial feature points from the real-time facial image;

The determining module 130 is configured to determine an eye region and a lip region according to position information of the t facial feature points, and input the eye region and the lip region into an eye part class model and a face of the pre-trained face a lip part type model for judging the authenticity of the eye area and the lip area, and determining whether the face in the real-time image is occluded according to the judgment result;

The prompting module 140 is configured to prompt the real-time facial image when the eye part class model of the human face and the lip part class model of the human face include the unreality of the judgment result of the eye region and the lip region The face is occluded.

In addition, the present application also provides a face occlusion detection method. Referring to Figure 3, this application A flowchart of a first embodiment of a face occlusion detection method. The method can be performed by a device that can be implemented by software and/or hardware.

In this embodiment, the face occlusion detection method includes:

Step S10: Acquire a real-time image captured by the camera device, and extract a real-time face image from the real-time image by using a face recognition algorithm. When the camera captures a real-time image, the camera transmits the real-time image to the processor. When the processor receives the real-time image, first acquires the size of the image to create a grayscale image of the same size; Color image, converted into gray image, and create a memory space; equalize the gray image histogram, reduce the amount of gray image information, speed up the detection, then load the training library, detect the face in the picture, and return An object containing face information, obtains the data of the location of the face, and records the number; finally obtains the area of the avatar and saves it, thus completing a real-time facial image extraction process.

Step S20: input the real-time facial image into a pre-trained facial average model, and use the facial average model to identify t facial feature points from the real-time facial image.

The face feature recognition model is trained by using the t facial feature points to obtain a face average model. The face feature recognition model is an ERT algorithm. The ERT algorithm is expressed as follows:

After obtaining the real-time facial image, the trained facial average model is called from the memory, and the real-time facial image is aligned with the facial average model, and the feature extraction algorithm is used to search the real-time facial image with the facial average model. One eyelid feature point, two eyeball feature points, and twelve eyelid feature points, two eyeball feature points, and 20 lip feature points matched by 20 lip feature points. Wherein, the 20 lip feature points are evenly distributed on the lip.

Step S30, determining an eye region and a lip region according to position information of the t facial feature points, and inputting the eye region and the lip region into an eye part type model of a pre-trained face, and a lip of the face The classification model determines the authenticity of the eye region and the lip region, and determines whether the face in the real-time image is occluded according to the judgment result.

A first number of human eye positive sample images and a second number of human eye negative sample images are collected, and local features of each human eye positive sample image and human eye negative sample image are extracted. The human eye positive sample image refers to an eye sample containing a human eye, and the binocular portion can be extracted from the face image sample library as an eye sample, and the human eye negative eye sample image refers to an image of a broken eye region, and multiple human eye positive samples. The image and the negative sample image form a second sample library.

Specifically, the local feature is a direction gradient histogram (HOG) feature, which is extracted from a human eye sample image and a lip sample image by a feature extraction algorithm. Since the color information in the sample image is not very effective, it is usually converted into a grayscale image, and the entire image is normalized, the gradient of the horizontal and vertical directions of the image is calculated, and the gradient direction of each pixel position is calculated accordingly. Values to capture outlines, silhouettes, and some texture information, and further weaken the effects of lighting. Then the whole image is divided into individual Cell cells (8*8 pixels), and a gradient direction histogram is constructed for each Cell cell to calculate the local image gradient information and quantize to obtain the feature description vector of the local image region. Then the cell cells are combined into a large block. Due to the change of local illumination and the change of the foreground-background contrast, the gradient intensity varies greatly. This requires normalization of the gradient intensity, further Light, shadow, and edges are compressed. Finally, all "block" HOG descriptors are combined to form the final HOG feature description vector.

Using the positive and negative sample images and the extracted HOG features in the second sample library and the third sample library The support vector machine classifier is trained to obtain the eye part class model of the face and the lip part class model of the face.

In step S40, it is determined whether the judgment result of the eye part class model and the lip part class model of the human face on the eye area and the lip area is true. When the eye part class model of the face and the lip part class model of the face output the result, whether the judgment result contains only true.

Step S50, when the eye part class model of the face and the lip part class model of the face face the determination result of the eye area and the lip area are true, determining that the face in the real-time face image is not Occlusion occurs. That is to say, when the eye region and the lip region determined according to the facial feature points are both the human eye region or the human lip region, it is considered that the human face in the real-time facial image is not blocked.

Step S60, when the eye part class model of the face and the lip part class model of the face face the judgment result of the eye area and the lip area to be untrue, prompting the face in the real-time face image to occur Occlusion. When any one of the eye region and the lip region determined according to the facial feature point is not the human eye region or the human lip region, the face in the real-time facial image is considered to be occluded, prompting the real-time The face in the face image is occluded.

In other embodiments, if the face recognition is performed after the face is detected as being occluded, the smearing is performed on the face in the real-time face image, and the step S50 further includes:

Prompting that the face in the current facial image is occluded, reacquiring the real-time image captured by the camera, and performing the subsequent steps.

In the face occlusion detection method proposed by the embodiment, the facial facial feature model of the facial feature point is used to identify the key facial feature points in the real-time facial image, and the facial part class model of the human face and the lip partial class model of the human face are used. The eye area and the lip area determined by the point are analyzed, and according to the authenticity of the eye area and the lip area, whether the face in the current image is occluded is detected, and the occlusion of the face in the real-time face image is quickly detected.

In addition, the embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium The storage medium includes a face occlusion detection program, and when the face occlusion detection program is executed by the processor, the following operations are implemented:

Optionally, when the face occlusion detection program is executed by the processor, the following operations are also implemented:

Optionally, the training steps of the eye part class model and the lip part class model of the face include:

Collecting a first number of human eye positive sample images and a second number of human eye negative sample images, extracting local features of each human eye positive sample image and a human eye negative sample image;

Using the human eye positive sample image, the human eye negative sample image and its local features to train the support vector machine classifier to obtain the eye part class model of the face;

Collecting a third number of lip positive sample images and a fourth number of lip negative sample images, extracting local features of each lip positive sample image and lip negative sample image;

The support vector machine classifier is trained by using the lip positive sample image, the lip negative sample image and its local features to obtain the lip part class model of the face.

Optionally, the training step of the facial average model includes:

Establishing a first sample library having n face images, and marking t facial feature points in each face image, the t facial feature points including: t ₁ eye feature points representing eye positions, t ₂ eye feature points and t ₃ lip feature points representing the position of the lips; and

The face feature recognition model is trained by using the t facial feature points to obtain a face average model.

The specific implementation of the computer readable storage medium of the present application is substantially the same as the specific embodiment of the above-described face occlusion detection method, and details are not described herein again.

It should be noted that, herein, the terms "including", "comprising" or any other variant thereof are intended to mean The inclusion of non-exclusive inclusions, such that a process, apparatus, article, or method that comprises a plurality of elements includes not only those elements but also other elements that are not explicitly listed, or are included in the process, apparatus, item, or method. The inherent elements. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, the device, the item, or the method that comprises the element.

The serial numbers of the embodiments of the present application are merely for the description, and do not represent the advantages and disadvantages of the embodiments. Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, the technical solution of the present application, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM as described above). , a disk, an optical disk, including a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in the various embodiments of the present application.

The above is only a preferred embodiment of the present application, and is not intended to limit the scope of the patent application, and the equivalent structure or equivalent process transformations made by the specification and the drawings of the present application, or directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of this application.

Claims

An electronic device, comprising: a memory, a processor, and an imaging device, wherein the memory includes a face occlusion detection program, and the face occlusion detection program is executed by the processor to implement the following steps :

Image acquisition step: acquiring a real-time image captured by the camera device, and extracting a real-time face image from the real-time image by using a face recognition algorithm;

a feature point recognition step: inputting the real-time facial image into a pre-trained facial average model, and using the facial average model to identify t facial feature points from the real-time facial image;

a feature region determining step: determining an eye region and a lip region according to position information of the t facial feature points, and inputting the eye region and the lip region into an eye part class model of a pre-trained face, a face The lip part type model determines the authenticity of the eye area and the lip area, and determines whether the face in the real-time image is occluded according to the judgment result.
The electronic device according to claim 1, wherein when the face occlusion detection program is executed by the processor, the following steps are further implemented:

The determining step is: determining whether the judgment result of the eye part class model and the lip part class model of the human face on the eye region and the lip region is true.
The electronic device according to claim 2, wherein when the face occlusion detection program is executed by the processor, the following steps are further implemented:

When the eye part model of the face and the lip part model of the face are true to the eye area and the lip area, it is determined that the face in the real-time face image is not occluded; and

When the judgment result of the eye part class model of the face and the lip part class model of the face on the eye region and the lip region is not true, the face in the real-time face image is prompted to be occluded.
The electronic device according to claim 3, wherein the training steps of the eye part class model and the lip part class model of the face comprise:

Collecting a first number of human eye positive sample images and a second number of human eye negative sample images, extracting local features of each human eye positive sample image and a human eye negative sample image;

Using the human eye positive sample image, the human eye negative sample image and its local features to train the support vector machine classifier to obtain the eye part class model of the face;

Collecting a third number of lip positive sample images and a fourth number of lip negative sample images, extracting local features of each lip positive sample image and lip negative sample image;

The support vector machine classifier is trained by using the lip positive sample image, the lip negative sample image and its local features to obtain the lip part class model of the face.
The electronic device according to claim 1, wherein said facial average model Training steps include:

Establishing a first sample library having n face images, and marking t facial feature points in each face image, the t facial feature points including: t 1 eye feature points representing eye positions, t 2 eye feature points and t 3 lip feature points representing the position of the lips; and

The face feature recognition model is trained by using the t facial feature points to obtain a face average model.
The electronic device according to claim 5, wherein the face feature recognition model is an ERT algorithm, which is formulated as follows:

Where t represents the cascading sequence number, τ t (·, ·) represents the current class of the regression, S(t) is the shape estimate of the current model, and each regression τ t (·, ·) is based on the input current image I and S(t) to predict an increment
In the process of model training, some feature points of all sample images are trained to train the first regression tree, and the predicted value of the first regression tree and the residual of the true value of the partial feature points are used to train the second tree. The tree... and so on, until the predicted value of the Nth tree is trained and the true value of the partial feature points is close to 0, all the regression trees of the ERT algorithm are obtained, and the facial average model is obtained according to the regression trees.
The electronic device according to claim 1, wherein the face recognition algorithm comprises: a geometric feature based method, a local feature analyzing method, a feature face method, an elastic model based method, and a neural network method.
A method for detecting a face occlusion is applied to an electronic device, the method comprising:

Image acquisition step: acquiring a real-time image captured by the camera device, and extracting a real-time face image from the real-time image by using a face recognition algorithm;

a feature point recognition step: inputting the real-time facial image into a pre-trained facial average model, and using the facial average model to identify t facial feature points from the real-time facial image;

a feature region determining step: determining an eye region and a lip region according to position information of the t facial feature points, and inputting the eye region and the lip region into an eye part class model of a pre-trained face, a face The lip part type model determines the authenticity of the eye area and the lip area, and determines whether the face in the real-time image is occluded according to the judgment result.
The method for detecting a face occlusion according to claim 8, wherein the method further comprises:

The determining step is: determining whether the judgment result of the eye part class model and the lip part class model of the human face on the eye region and the lip region is true.
The method for detecting a face occlusion according to claim 9, wherein the method further include:

When the eye part model of the face and the lip part model of the face are true to the eye area and the lip area, it is determined that the face in the real-time face image is not occluded; and

When the judgment result of the eye part class model of the face and the lip part class model of the face on the eye region and the lip region is not true, the face in the real-time face image is prompted to be occluded.
The face occlusion detection method according to claim 10, wherein the training steps of the eye part class model and the lip part class model of the face comprise:

Collecting a first number of human eye positive sample images and a second number of human eye negative sample images, extracting local features of each human eye positive sample image and a human eye negative sample image;

Using the human eye positive sample image, the human eye negative sample image and its local features to train the support vector machine classifier to obtain the eye part class model of the face;

Collecting a third number of lip positive sample images and a fourth number of lip negative sample images, extracting local features of each lip positive sample image and lip negative sample image;

The support vector machine classifier is trained by using the lip positive sample image, the lip negative sample image and its local features to obtain the lip part class model of the face.
The face occlusion detection method according to claim 8, wherein the training step of the face averaging model comprises:

Establishing a first sample library having n face images, and marking t facial feature points in each face image, the t facial feature points including: t 1 eye feature points representing eye positions, t 2 eye feature points and t 3 lip feature points representing the position of the lips; and

The face feature recognition model is trained by using the t facial feature points to obtain a face average model.
The face occlusion detection method according to claim 12, wherein the face feature recognition model is an ERT algorithm, which is expressed by a formula as follows:

Where t represents the cascading sequence number, τ t (·, ·) represents the current class of the regression, S(t) is the shape estimate of the current model, and each regression τ t (·, ·) is based on the input current image I and S(t) to predict an increment
In the process of model training, some feature points of all sample images are trained to train the first regression tree, and the predicted value of the first regression tree and the residual of the true value of the partial feature points are used to train the second tree. The tree... and so on, until the predicted value of the Nth tree is trained and the true value of the partial feature points is close to 0, all the regression trees of the ERT algorithm are obtained, and the facial average model is obtained according to the regression trees.
The face occlusion detection method according to claim 8, wherein the face recognition algorithm comprises: a geometric feature based method, a local feature analysis method, a feature face method, and a base The method of elastic model, neural network method.
A computer readable storage medium, comprising: a face occlusion detection program, wherein the face occlusion detection program is executed by a processor to implement the following steps:

Image acquisition step: acquiring a real-time image captured by the camera device, and extracting a real-time face image from the real-time image by using a face recognition algorithm;

a feature point recognition step: inputting the real-time facial image into a pre-trained facial average model, and using the facial average model to identify t facial feature points from the real-time facial image;

a feature region determining step: determining an eye region and a lip region according to position information of the t facial feature points, and inputting the eye region and the lip region into an eye part class model of a pre-trained face, a face The lip part type model determines the authenticity of the eye area and the lip area, and determines whether the face in the real-time image is occluded according to the judgment result.
The computer readable storage medium according to claim 15, wherein when the face occlusion detection program is executed by the processor, the following steps are further implemented:

The determining step is: determining whether the judgment result of the eye part class model and the lip part class model of the human face on the eye region and the lip region is true.
The computer readable storage medium according to claim 16, wherein when the face occlusion detection program is executed by the processor, the following steps are further implemented:

When the eye part model of the face and the lip part model of the face are true to the eye area and the lip area, it is determined that the face in the real-time face image is not occluded; and

When the judgment result of the eye part class model of the face and the lip part class model of the face on the eye region and the lip region is not true, the face in the real-time face image is prompted to be occluded.
The computer readable storage medium according to claim 17, wherein the training steps of the eye part class model and the lip part class model of the face comprise:

Collecting a first number of human eye positive sample images and a second number of human eye negative sample images, extracting local features of each human eye positive sample image and a human eye negative sample image;

Using the human eye positive sample image, the human eye negative sample image and its local features to train the support vector machine classifier to obtain the eye part class model of the face;

Collecting a third number of lip positive sample images and a fourth number of lip negative sample images, extracting local features of each lip positive sample image and lip negative sample image;

The support vector machine classifier is trained by using the lip positive sample image, the lip negative sample image and its local features to obtain the lip part class model of the face.
A computer readable storage medium according to claim 15 wherein said face The training steps of the average model include:

Establishing a first sample library having n face images, and marking t facial feature points in each face image, the t facial feature points including: t 1 eye feature points representing eye positions, t 2 eye feature points and t 3 lip feature points representing the position of the lips; and

The face feature recognition model is trained by using the t facial feature points to obtain a face average model.
The computer readable storage medium according to claim 19, wherein the face feature recognition model is an ERT algorithm, which is formulated as follows:

Where t represents the cascading sequence number, τ t (·, ·) represents the current class of the regression, S(t) is the shape estimate of the current model, and each regression τ t (·, ·) is based on the input current image I and S(t) to predict an increment
In the process of model training, some feature points of all sample images are trained to train the first regression tree, and the predicted value of the first regression tree and the residual of the true value of the partial feature points are used to train the second tree. The tree... and so on, until the predicted value of the Nth tree is trained and the true value of the partial feature points is close to 0, all the regression trees of the ERT algorithm are obtained, and the facial average model is obtained according to the regression trees.