WO2021174819A1

WO2021174819A1 - Face occlusion detection method and system

Info

Publication number: WO2021174819A1
Application number: PCT/CN2020/118112
Authority: WO
Inventors: 戴栋根; 陆进; 陈斌; 宋晨
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-03-05
Filing date: 2020-09-27
Publication date: 2021-09-10
Also published as: CN111428581B; CN111428581A

Abstract

Embodiments of the present application relate to the field of artificial intelligence, and provide a face occlusion detection method, comprising: obtaining an image to be detected; obtaining a face region image from the image to be detected; recognizing the face region image by means of a face occlusion detection branch model, and performing pixel processing to generate a first occlusion result; when the first occlusion result indicates that the face region image is occluded, classifying the face region image by means of a face occlusion classification branch model and a preset occlusion tag to generate a classification result; and generating a final occlusion result according to the first occlusion result and the classification result. The embodiments of the present application further provide a face occlusion detection system. According to the embodiments of the present application, the recognition precision of face occlusion detection is improved, the classification type of an occlusion object is recognized, and intelligent application of face occlusion detection is facilitated.

Description

Human face occlusion detection method and system

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on March 5, 2020, the application number is 202010146004.6, and the invention title is "face occlusion detection method and system", the entire content of which is incorporated into this application by reference .

Technical field

The embodiments of the present application relate to the field of big data technology, and in particular to a method and system for detecting face occlusion.

Background technique

As a hot research issue in the field of pattern recognition, face recognition has received extensive attention. Face recognition technology has broad application prospects in identity verification in many fields. In the actual face image processing process, the occlusion of the face image will often appear, such as hair, mask, scarf, hat, sunglasses, etc., and the occlusion has a great impact on face registration and face recognition.

The inventor realizes that the existing face occlusion detection method usually recognizes the face image to identify whether each area of the face is occluded, and the recognition accuracy is low, which is not conducive to the intelligent application of the face occlusion detection method.

Summary of the invention

In view of this, the embodiments of the present application provide a face occlusion detection method, system, computer device, and computer readable storage medium, which are used to solve the problem of low recognition accuracy and low intelligence of the face occlusion detection method.

The embodiments of this application solve the above technical problems through the following technical solutions:

A face occlusion detection method includes:

Obtain the image to be detected;

Acquiring a face area image from the image to be detected;

Recognizing the face region image through a face occlusion detection branch model, and performing pixel processing to generate a first occlusion result;

When the first occlusion result indicates that the face region image is occluded, classify the face region image through a face occlusion classification branch model and a preset occlusion label to generate a classification result;

A final occlusion result is generated according to the first occlusion result and the classification result.

In order to achieve the foregoing objective, an embodiment of the present application also provides a face occlusion detection system, including:

The acquisition module is used to acquire the image to be detected;

An extraction module, which is used to obtain an image of a face area from the image to be detected;

A recognition module, configured to recognize the face region image through a face occlusion detection branch model, and perform pixel processing to generate a first occlusion result;

A classification module, configured to classify the face area image through a face occlusion classification branch model and a preset occlusion label when the first occlusion result indicates that the face region image is occluded, to generate a classification result;

The generating result module is configured to generate a final occlusion result according to the first occlusion result and the classification result.

In order to achieve the foregoing objective, an embodiment of the present application further provides a computer device. The computer device includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the Perform the following steps in the computer program:

Obtain the image to be detected;

Acquiring a face area image from the image to be detected;

In order to achieve the foregoing objective, the embodiments of the present application also provide a computer-readable storage medium having a computer program stored in the computer-readable storage medium, and the computer program may be executed by at least one processor, so that the at least A processor performs the following steps:

Obtain the image to be detected;

Acquiring a face area image from the image to be detected;

The face occlusion detection method, system, computer device, and computer-readable storage medium provided by the embodiments of the present application obtain the first occlusion result of the face occlusion area by recognizing the image of the face region, and use the preset occlusion tag to detect the face The regional image is classified to obtain the classification result of the classification type of the occlusion on the face, and the first occlusion result and the classification type of the occlusion are combined to obtain the final occlusion result, which improves the recognition accuracy of the face occlusion detection and recognizes the occlusion The classification type of objects facilitates the intelligent application of face occlusion detection.

The following describes the application in detail with reference to the accompanying drawings and specific embodiments, but it is not intended to limit the application.

Description of the drawings

FIG. 1 is a flow chart of the steps of the method for detecting face occlusion in the first embodiment of this application;

FIG. 2 is a schematic diagram of the process of obtaining a face region image in the method for detecting face occlusion according to the first embodiment of the application; FIG.

FIG. 3 is a schematic diagram of the flow of generating a first occlusion result in the face occlusion detection method according to Embodiment 1 of the application;

4 is a schematic diagram of the process of generating classification results of the face occlusion detection method according to the first embodiment of the application;

5 is a schematic diagram of the training process of the face occlusion detection branch model of the face occlusion detection method according to the first embodiment of the application;

6 is a schematic diagram of the training process of the face occlusion classification branch model of the face occlusion detection method according to the first embodiment of the application;

FIG. 7 is a schematic diagram of program modules of the face occlusion detection system according to the second embodiment of the application;

FIG. 8 is a schematic diagram of the hardware structure of the computer device according to the third embodiment of the application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of this application clearer and clearer, the following further describes the application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and are not used to limit the application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The technical solutions between the various embodiments can be combined with each other, but they must be based on what can be achieved by a person of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist. It is not within the scope of protection required by this application.

Example one

Please refer to FIG. 1, which shows a flowchart of steps of a method for detecting occlusion of a face according to an embodiment of the present application. It can be understood that the flowchart in this method embodiment is not used to limit the order of execution of the steps. The following is an exemplary description with computer equipment as the main body of execution, and the details are as follows:

In step S100, an image to be detected is acquired.

Exemplarily, the image to be detected is acquired in real time by the camera acquisition unit, and the image to be detected includes an image of a face area and a background area.

Step S200: Obtain a face area image from the image to be detected.

In an exemplary embodiment, referring to FIG. 2, the step S200 may further include:

Step S201: Obtain the coordinate positions of multiple facial feature points in the image to be detected.

Specifically, the multiple facial feature points are 68 facial feature points.

Step S202: Extract a face region image from the image to be detected according to the acquired coordinate positions of multiple facial feature points, the face region image including multiple facial features regions.

Specifically, the multiple facial features include the eyebrow area, the eye area, the nose area, the mouth area, the cheek area, the forehead area, the teeth area, and the like.

For example, in this embodiment of the application, the landmark algorithm (facial feature point extraction algorithm) is used to calibrate 68 facial feature points, that is, the landmark algorithm calls the predictor to obtain the coordinate positions of the 68 facial feature points. The coordinate position draws a circle at each facial feature point, and the serial number of the 68 facial feature points is marked according to the calibration sequence. The shape of the facial features is determined according to the coordinate positions of multiple facial feature points to obtain multiple facial features in the image to be detected.

Among them, each facial features area is marked by a rectangular frame.

In step S300, the face region image is recognized by the face occlusion detection branch model, and pixel processing is performed to generate a first occlusion result.

Specifically, in the embodiment of the present application, the face area image may be recognized by the face occlusion detection branch model of the face occlusion model to generate the first occlusion result.

The face occlusion detection branch model includes a plurality of first convolutional layers, second convolutional layers, first fully connected layers, and the like.

In an exemplary embodiment, referring to FIG. 3, the step S300 may further include:

Step S301: Perform convolution on the face region image through the face occlusion detection branch model to output multiple convolution feature maps, and the multiple convolution feature maps include multiple first convolution feature maps and second convolution feature maps. Convolution feature map.

Specifically, the image to be detected carrying the image of the face region is input into the face occlusion detection branch model, convolved through multiple first convolutional layers and second convolutional layers, and output multiple through the first fully connected layer. The first convolution feature map.

For example, input the to-be-detected image carrying the face region image into the face occlusion detection branch model, convolve through multiple first convolutional layers, and output multiple second convolution feature maps; then add multiple The second convolution feature map is input to the second convolution layer and the first fully connected layer for further convolution to output multiple first convolution feature maps. Wherein, the second convolution feature map includes occluder features, and the height and width of the second convolution feature map and the height and width of the first convolution feature map are consistent with the height and width of the image to be detected.

Specifically, the features of the shield include data such as the shape, position, and pixels of the shield.

Among them, the convolution features included in the first convolution feature map are thicker convolution features formed by feature fusion splicing.

Step S302: Combine and enlarge a plurality of first convolution feature maps, and adjust the pixel values of each facial feature region and the background region according to a preset rule to obtain a predicted face image. The predicted face image includes an occluded area.

Specifically, after combining a plurality of first convolutional feature maps, the edges are expanded and the pixel values of each facial features region and the background region are adjusted according to a preset rule to obtain a predicted face image. The preset rule is that the pixel value of the unoccluded part in the predicted face image is set to 0, and the pixel value of the occluded part and the background part in the predicted face image is set to 1, that is, the predicted person The unoccluded part of the face image is represented as white, and the background part and the occluded part in the predicted face image are both represented as black.

Step S303: Calculate the occlusion ratio of the occlusion area and the five sense organs area where the occlusion area is located.

Specifically, the occlusion area is represented as a black area in each rectangular frame, and the black area is a black part in each rectangular frame in the predicted face image obtained after pixel processing.

Specifically, the occlusion ratio of each occlusion area to the facial features of the rectangular frame in which the occlusion area is located is calculated, that is, the ratio of the black area in each rectangular frame to the entire rectangular frame is calculated to obtain the occlusion ratio of the corresponding occlusion area.

Step S304, comparing the occlusion ratio with a preset occlusion threshold corresponding to the facial features area to generate a first occlusion result.

In an exemplary embodiment, when the occlusion ratio is greater than the occlusion threshold, the first occlusion result indicates that the facial features area is occluded; when the occlusion ratio is less than the occlusion threshold, the first occlusion The occlusion result indicates that the facial features are not occluded.

Wherein, the preset occlusion threshold corresponding to each facial features area may be set to the same preset occlusion threshold, or may be set to different preset occlusion thresholds. Specifically, the setting of the preset occlusion threshold corresponding to each facial features area can be dynamically adjusted according to different scene requirements. For example, when the scene requirements are high and the ratio of the required occlusion is small, a smaller preset occlusion threshold can be set; when the scene requirements are low, a higher preset occlusion threshold can be set.

Further, the occlusion threshold can be manually set according to different scene requirements, for example, the occlusion threshold is set to 95%, 90%, 85%, and so on.

Step S400: When the first occlusion result indicates that the face region image is occluded, classify the face region image through the face occlusion classification branch model and a preset occlusion label to generate a classification result.

Specifically, the embodiment of the present application may classify the face region image by using the preset occlusion label in the face occlusion classification branch model of the face occlusion model to generate the classification result.

Among them, the preset occlusion tags include preset occlusion tags such as hats, bangs, sunglasses, eyes, beards, masks, and scarves.

In an exemplary embodiment, the face occlusion classification branch model includes several hidden layers, and the hidden layers are several third convolutional layers, second fully connected layers, and classification layers, and each layer is connected to each other.

In an exemplary embodiment, referring to FIG. 4, the step S400 may further include:

Step S401, extracting features of occlusion objects from multiple second convolution feature maps through the face occlusion classification branch model.

Specifically, a plurality of second convolution feature maps carrying features of the occluder is obtained from the last first convolution layer, and the features of the occluder are extracted from the plurality of second convolution feature maps.

Step S402, matching the features of the obstruction object with a preset obstruction label to generate a classification result.

In an exemplary embodiment, when the feature of the occluder matches any one of the preset occlusion labels, the classification result indicates that the feature of the occluder corresponds to the matching occlusion label; when the feature of the occluder matches the preset one When an occlusion label matches inconsistently, the occlusion feature is marked to obtain a new occlusion label, and the classification result indicates that the occlusion feature corresponds to the new occlusion label.

Further, the new obstruction label is saved in the preset database. In the subsequent model maintenance process, when a new occlusion label is detected, the new occlusion label corresponding to the new occlusion label is identified to generate a new occlusion label and save it in the preset database; and according to the new occlusion label generated The classification branch model of face occlusion is optimized. For example, extract features of the occluder from multiple second convolution feature maps, and match the features of the occluder with a preset occlusion label. When the feature of the occluder matches the hat, the generated classification result indicates the occlusion The object feature corresponds to the hat.

Step S500: Generate a final occlusion result according to the first occlusion result and the classification result.

Specifically, the final occlusion result includes the first occlusion result used to determine whether the face area is occluded and the classification result for determining the classification type of the occluder.

In the embodiment of the application, by dynamically adjusting the preset occlusion threshold of each facial features area and distinguishing the classification type of the occlusion feature at the same time, that is, at the same time distinguishing what is occluded by each facial features region, it will be misjudged for some needs to exclude beards, etc. For the occlusion factor, it is necessary to judge the occluded face occlusion detection scene at the same time, which can improve the accuracy of the face occlusion detection.

In an exemplary embodiment, the face occlusion detection method further includes: generating a feedback instruction according to the final occlusion result, the feedback instruction being used to indicate the occlusion area included in the user's face and the occlusion object located in the occlusion area .

For example, when the final occlusion result indicates that the user's eyes are blocked and the occluded object is sunglasses, the computer device generates a feedback instruction according to the final occlusion result. The feedback instruction is used to indicate that the user's eyes are blocked by the sunglasses and cannot be Face recognition, so that the user responds according to the feedback instruction, even if the user takes off the sunglasses according to the feedback instruction, so as to facilitate effective face recognition.

Before step S100, the embodiment of the present application also includes the training process of the face occlusion detection branch model of the face occlusion model. Please refer to FIG. 5, which is specifically as follows:

In step S600, a plurality of sample face images are preprocessed to obtain a comparison sample face image.

Specifically, multiple sample face images are acquired from a preset database, and the multiple sample face images include multiple occluded face sample images and unoccluded face sample images. Perform occlusion feature, unocclusion feature and background location recognition label for each sample face image to mark the location of occlusion feature, unocclusion feature and background feature, and set the pixels of occlusion feature, unocclusion feature and background feature respectively Value to output a comparison sample face image.

The pixel value of the part where the occlusion feature is determined is set to 1, and the pixel value of the part where the unoccluded feature and the background are determined is set to 0. At this time, the comparison sample face image is output. That is, in the face image of the comparative sample, the part of the unoccluded feature appears as white, and the part of the occluded feature and the background appear as black. When preprocessing the sample face image, the sample face image also includes an area that cannot be determined whether it is occluded, and the pixel value of the area is set to 225. The area that cannot be determined whether it is occluded is located and collected by the preset fuzzy area. In the process of calculating the subsequent intersection and ratio, in order to maintain the correct rate, the area that cannot be determined whether it is occluded can be excluded from the location acquisition.

Among them, the contrast face image is a black and white contrast face image.

Step S601, input a plurality of sample face images into the first convolutional layer, the second convolutional layer and the first fully connected layer of the deep neural network model to perform convolution to output a plurality of convolutional feature maps, so The multiple convolution feature maps include multiple first convolution feature maps and multiple second convolution feature maps.

In step S602, a plurality of first convolution feature maps are combined and enlarged to output a predicted sample face image.

Wherein, the prediction sample face image is presented as a black and white prediction sample face image.

Step S603: The prediction sample face image is matched with the comparison sample face image, and an intersection ratio is calculated, and the intersection ratio is the intersection and union of the prediction sample face image and the comparison sample face image. The ratio of the set.

Specifically, the intersection ratio refers to the ratio of the intersection and union of each corresponding occluded area and unoccluded area background area of the predicted sample face image and the comparison sample face image.

In step S604, when the intersection ratio is less than the preset contrast threshold, the deep neural network model is iterated through the first loss function to adjust the intersection ratio, so as to obtain an optimized face occlusion detection branch model.

Specifically, when the intersection ratio is smaller than the preset comparison threshold, and the intersection ratio is smaller, it means that the prediction sample face image is less similar to the comparison sample face image, and the model still needs further training. optimization.

In an exemplary embodiment, the first loss function is a U-net loss function. The U-net loss function can be a loss function with boundary weights. The purpose is to give higher weights to the pixels close to the boundary points in the sample face image.

Further, the embodiment of the present application also includes the training process of the face occlusion classification branch model of the face occlusion model. Please refer to FIG. 6, and the details are as follows:

Step S610: Input a plurality of second convolution feature maps into several hidden layers of the deep neural network model to extract the features of the sample occluder.

Specifically, the obtained multiple second convolution feature maps containing the features of the occluder are input to the third convolutional layer among the several hidden layers of the deep neural network model to perform a further convolution operation to extract the features of the sample occluder.

Step S611: Input the feature of the sample occluder into the classification layer to identify and classify, so as to generate a first classification conclusion.

Step S612: The first classification conclusion is compared with the sample classification conclusion of the sample face image to calculate a second loss value.

Step S613: Iterate the deep neural network model through the second loss value and the second loss function, reduce the loss value and update the model parameters of the face occlusion classification branch model to obtain an optimized face occlusion classification branch model.

Specifically, the second loss function may be a cross-entropy loss function.

The optimized face occlusion detection branch model and the face occlusion classification branch model are combined into an optimized face detection model.

In an exemplary embodiment, the preset occluder label of the face occlusion classification branch model may also be updated regularly according to the classification result output by the face occlusion classification branch model.

In an exemplary embodiment, it is also possible to match the corresponding weights for the face occlusion detection branch model and the face occlusion classification branch model, and the first occlusion result of the face occlusion detection branch model is the final occlusion result that affects the face detection model Important parameters.

In the embodiment of the present application, the first occlusion result of the occlusion area of the face is obtained by recognizing the face area image, and the face area image is classified by the preset occlusion label to obtain the classification result of the classification type of the occlusion on the face. The final occlusion result is obtained by combining the first occlusion result and the classification type of the occlusion object, which improves the recognition accuracy of the face occlusion detection, and recognizes the classification type of the occlusion object, which is convenient for the intelligent application of the face occlusion detection.

Example two

Please continue to refer to FIG. 7, which shows a schematic diagram of program modules of the applicant's face occlusion detection system. In this embodiment, the face occlusion detection system 20 may include or be divided into one or more program modules, and the one or more program modules are stored in a storage medium and executed by one or more processors to This application is completed, and the above-mentioned face occlusion detection method can be realized. The program module referred to in the embodiments of the present application refers to a series of computer program instruction segments capable of completing specific functions, and is more suitable for describing the execution process of the face occlusion detection system 20 in the storage medium than the program itself. The following description will specifically introduce the functions of each program module in this embodiment:

The acquisition module 700 is used to acquire an image to be detected.

The extraction module 710 is configured to obtain a face region image from the image to be detected.

Further, the extraction module 710 is further configured to: obtain the coordinate positions of multiple facial feature points in the image to be detected; extract a face region image from the image to be detected according to the coordinate positions of the multiple facial feature points obtained , The face region image includes a plurality of facial features regions.

In an exemplary embodiment, the multiple facial feature points are 68 facial feature points; the multiple facial features regions include eyebrows, eyes, nose, mouth, cheeks, forehead, teeth, and so on.

The recognition module 720 is configured to recognize the face region image to generate a first occlusion result.

Further, the recognition module 720 is further configured to: perform convolution on the face region image to output multiple convolution feature maps, and the multiple convolution feature maps include multiple first convolution feature maps; Combine and enlarge the multiple first convolution feature maps to obtain a predicted face image, where the predicted face image includes an occlusion area; calculate the occlusion ratio of the occlusion area and the five sense organs where the occlusion area is located; The occlusion ratio is compared with a preset occlusion threshold corresponding to the facial features area to generate a first occlusion result.

In an exemplary embodiment, after combining a plurality of first convolutional feature maps, the edge and background pixels are expanded to obtain a predicted face image, and the pixel value of the unoccluded part in the predicted face image is set to 0, The pixel values of the occluded part and the background part in the predicted face image are set to 1, that is, the unoccluded part in the predicted face image appears white, and the background part and the occluded part in the predicted face image are both Appears as black.

In an exemplary embodiment, the preset occlusion threshold corresponding to each facial features area may be set to the same preset occlusion threshold, or may be set to different preset occlusion thresholds. Specifically, the setting of the preset occlusion threshold corresponding to each facial features area can be dynamically adjusted according to different scene requirements. For example, when the scene requirements are high and the ratio of the required occlusion is small, a smaller preset occlusion threshold can be set; when the scene requirements are low, a higher preset occlusion threshold can be set.

The classification module 730 is configured to classify the face region image according to the preset occlusion label to generate a classification result.

Further, the classification module 730 is further configured to: extract features of occlusion objects from a plurality of second convolution feature maps; and match the features of the occlusion objects with a preset occlusion label to generate a classification result.

In an exemplary embodiment,

The generating result module 740 is configured to generate a final occlusion result according to the first occlusion result and the classification result.

Example three

Refer to FIG. 8, which is a schematic diagram of the hardware architecture of the computer device according to the third embodiment of the present application. In this embodiment, the computer device 2 is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. The computer device 2 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of multiple servers). As shown in FIG. 8, the computer device 2 at least includes, but is not limited to, a memory 21, a processor 22, a network interface 23, and a face occlusion detection system 20 that can communicate with each other through a system bus. in:

In this embodiment, the memory 21 includes at least one type of computer-readable storage medium. The readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory ( RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, the memory 21 may be an internal storage unit of the computer device 2, for example, the hard disk or memory of the computer device 2. In other embodiments, the memory 21 may also be an external storage device of the computer device 2, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) card, flash card (Flash Card), etc. Of course, the memory 21 may also include both the internal storage unit of the computer device 2 and its external storage device. In this embodiment, the memory 21 is generally used to store an operating system and various application software installed in the computer device 2, for example, the program code of the face occlusion detection system 20 in the second embodiment. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

The processor 22 may be a central processing unit (Central Processing Unit) in some embodiments. Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 22 is generally used to control the overall operation of the computer device 2. In this embodiment, the processor 22 is used to run the program code or process data stored in the memory 21, for example, to run the face occlusion detection system 20, so as to implement the face occlusion detection method in the embodiment of the present application.

The network interface 23 may include a wireless network interface or a wired network interface, and the network interface 23 is generally used to establish a communication connection between the computer device 2 and other electronic devices. For example, the network interface 23 is used to connect the computer device 2 with an external terminal through a network, and establish a data transmission channel and a communication connection between the computer device 2 and the external terminal. The network may be an intranet (Intranet), the Internet (Internet), a global system of mobile communication (Global System of Mobile) communication, GSM), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), 4G network, 5G network, Bluetooth (Bluetooth), Wi-Fi and other wireless or wired networks.

It should be pointed out that FIG. 8 only shows the computer device 2 with components 20-23, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

In this embodiment, the face occlusion detection system 20 stored in the memory 21 can also be divided into one or more program modules, and the one or more program modules are stored in the memory 21 and consist of one Or executed by multiple processors (in this embodiment, the processor 22) to complete the application.

For example, FIG. 7 shows a schematic diagram of the program modules of the second embodiment of the face occlusion detection system 20. In this embodiment, the face occlusion detection system 20 can be divided into a collection module 700, an extraction module 710, The recognition module 720, the classification module 730, and the generation result module 740. Among them, the program module referred to in the present application refers to a series of computer program instruction segments capable of completing specific functions, and is more suitable than a program to describe the execution process of the face occlusion detection system 20 in the computer device 2. The specific functions of the program modules 700-740 have been described in detail in the second embodiment, and will not be repeated here.

Example four

This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), only Read memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, servers, App application malls, etc., on which computer programs are stored, The corresponding function is realized when the program is executed by the processor. The computer-readable storage medium of this embodiment is used to store the face occlusion detection system 20, and when executed by a processor, the face occlusion detection method of the embodiment of the present application is implemented. The computer-readable storage medium may be non-volatile or volatile.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A face occlusion detection method, which includes:

Obtain the image to be detected;

Acquiring a face area image from the image to be detected;

Recognizing the face region image through a face occlusion detection branch model, and performing pixel processing to generate a first occlusion result;

When the first occlusion result indicates that the face region image is occluded, classify the face region image through a face occlusion classification branch model and a preset occlusion label to generate a classification result;

A final occlusion result is generated according to the first occlusion result and the classification result.
The method for detecting face occlusion according to claim 1, wherein said obtaining a face area image from the image to be detected comprises:

Acquiring coordinate positions of multiple facial feature points in the image to be detected;

Extracting a face region image from the image to be detected according to the acquired coordinate positions of the multiple facial feature points, the face region image including a plurality of facial features regions.
The method for detecting human face occlusion according to claim 2, wherein the image to be detected includes a background area;

The step of recognizing the face region image through the face occlusion detection branch model and performing pixel processing to generate the first occlusion result includes:

Performing convolution on the face region image by using a face occlusion detection branch model to output a plurality of convolution feature maps, the plurality of convolution feature maps including a plurality of first convolution feature maps;

Combining and magnifying the multiple first convolution feature maps, and adjusting the pixel values of each facial feature region and the background region according to preset rules to obtain a predicted face image, where the predicted face image includes an occluded area;

Calculating the occlusion ratio of the occlusion area and the facial features area where the occlusion area is located;

The occlusion ratio is compared with a preset occlusion threshold corresponding to the facial features area to generate a first occlusion result.
The method for detecting face occlusion according to claim 3, wherein the comparing the occlusion ratio with a preset occlusion threshold corresponding to the facial features area to generate a first occlusion result comprises:

When the occlusion ratio is greater than the occlusion threshold, the first occlusion result indicates that the facial features area is occluded;

When the occlusion ratio is less than the occlusion threshold, the first occlusion result indicates that the facial features are not occluded.
The face occlusion detection method according to claim 4, wherein the plurality of convolution feature maps include a plurality of second convolution feature maps, and the face occlusion classification branch model and the preset occlusion label are The face region image classification to generate classification results includes:

Extracting features of occlusion objects from multiple second convolution feature maps through the face occlusion classification branch model;

The feature of the occluder is matched with the preset occlusion label to generate a classification result.
The method for detecting occlusion of a human face according to claim 5, wherein the matching the feature of the occluder with a preset occlusion label to generate a classification result comprises:

When the feature of the occluder matches any one of the preset occlusion labels, the classification result indicates that the feature of the occluder corresponds to the matching occlusion label;

When the feature of the occluder is inconsistent with any preset occlusion label, mark the feature of the occluder to obtain a new occluder label, and the classification result indicates that the feature of the occluder corresponds to the new occluder label.
The face occlusion detection method according to claim 6, wherein the method further comprises: generating a feedback instruction according to the final occlusion result, the feedback instruction being used to indicate the occlusion area included in the user's face and the occlusion area Of occluders.
A face occlusion detection system, which includes:

The acquisition module is used to acquire the image to be detected;

An extraction module, which is used to obtain an image of a face area from the image to be detected;

A recognition module for recognizing the image of the face area to generate a first occlusion result;

The classification module is used to classify the face area image according to the preset occlusion tags to generate the classification result;

The generating result module is configured to generate a final occlusion result according to the first occlusion result and the classification result.
The face occlusion detection system according to claim 8, wherein the extraction module is further used for:

Acquiring coordinate positions of multiple facial feature points in the image to be detected;

Extracting a face region image from the image to be detected according to the acquired coordinate positions of the multiple facial feature points, the face region image including a plurality of facial features regions.
The face occlusion detection system according to claim 9, wherein the image to be detected includes a background area; and the recognition module is further used for:

Performing convolution on the face region image by using a face occlusion detection branch model to output a plurality of convolution feature maps, the plurality of convolution feature maps including a plurality of first convolution feature maps;

Combining and magnifying the multiple first convolution feature maps, and adjusting the pixel values of each facial feature region and the background region according to preset rules to obtain a predicted face image, where the predicted face image includes an occluded area;

Calculating the occlusion ratio of the occlusion area and the facial features area where the occlusion area is located;

The occlusion ratio is compared with a preset occlusion threshold corresponding to the facial features area to generate a first occlusion result.
The face occlusion detection system according to claim 10, wherein the recognition module is further used for:

When the occlusion ratio is greater than the occlusion threshold, the first occlusion result indicates that the facial features area is occluded;

When the occlusion ratio is less than the occlusion threshold, the first occlusion result indicates that the facial features are not occluded.
The face occlusion detection system according to claim 11, wherein the multiple convolution feature maps include multiple second convolution feature maps, and the classification module is further configured to:

Extracting features of occlusion objects from multiple second convolution feature maps through the face occlusion classification branch model;

The feature of the occluder is matched with the preset occlusion label to generate a classification result.
The face occlusion detection system according to claim 12, wherein the classification module is further used for:

When the feature of the occluder matches any one of the preset occlusion labels, the classification result indicates that the feature of the occluder corresponds to the matching occlusion label;

When the feature of the occluder is inconsistent with any preset occlusion label, mark the feature of the occluder to obtain a new occluder label, and the classification result indicates that the feature of the occluder corresponds to the new occluder label.
The human face occlusion detection system according to claim 13, wherein the system further comprises:

The feedback module is configured to generate a feedback instruction according to the final occlusion result, and the feedback instruction is used to indicate the occlusion area included in the user's face and the occluder located in the occlusion area.
A computer device includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor, wherein the processor executes the following steps when the computer program is executed:

Obtain the image to be detected;

Acquiring a face area image from the image to be detected;

Recognizing the face region image through a face occlusion detection branch model, and performing pixel processing to generate a first occlusion result;

When the first occlusion result indicates that the face region image is occluded, classify the face region image through a face occlusion classification branch model and a preset occlusion label to generate a classification result;

A final occlusion result is generated according to the first occlusion result and the classification result.
The computer device according to claim 15, wherein the processor executes the following steps when executing the computer program:

Acquiring coordinate positions of multiple facial feature points in the image to be detected;

Extracting a face region image from the image to be detected according to the acquired coordinate positions of the multiple facial feature points, the face region image including a plurality of facial features regions.
The computer device according to claim 16, wherein the image to be detected includes a background area; when the processor executes the computer program, the following steps are performed:

Performing convolution on the face region image by using a face occlusion detection branch model to output a plurality of convolution feature maps, the plurality of convolution feature maps including a plurality of first convolution feature maps;

Combining and magnifying the multiple first convolution feature maps, and adjusting the pixel values of each facial feature region and the background region according to preset rules to obtain a predicted face image, where the predicted face image includes an occluded area;

Calculating the occlusion ratio of the occlusion area and the facial features area where the occlusion area is located;

The occlusion ratio is compared with a preset occlusion threshold corresponding to the facial features area to generate a first occlusion result.
A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program can be executed by at least one processor, so that the at least one processor executes the following steps:

Obtain the database table to be audited in the database;

Obtain the image to be detected;

Acquiring a face area image from the image to be detected;

Recognizing the face region image through a face occlusion detection branch model, and performing pixel processing to generate a first occlusion result;

When the first occlusion result indicates that the face region image is occluded, classify the face region image through a face occlusion classification branch model and a preset occlusion label to generate a classification result;

A final occlusion result is generated according to the first occlusion result and the classification result.
The computer-readable storage medium according to claim 18, wherein the processor executes the following steps when executing the computer program:

Acquiring coordinate positions of multiple facial feature points in the image to be detected;

Extracting a face region image from the image to be detected according to the acquired coordinate positions of the multiple facial feature points, the face region image including a plurality of facial features regions.
The computer-readable storage medium according to claim 19, wherein the image to be detected includes a background area; the processor executes the following steps when executing the computer program:

Performing convolution on the face region image by using a face occlusion detection branch model to output a plurality of convolution feature maps, the plurality of convolution feature maps including a plurality of first convolution feature maps;

Combining and magnifying the multiple first convolution feature maps, and adjusting the pixel values of each facial feature region and the background region according to preset rules to obtain a predicted face image, where the predicted face image includes an occluded area;

Calculating the occlusion ratio of the occlusion area and the facial features area where the occlusion area is located;

The occlusion ratio is compared with a preset occlusion threshold corresponding to the facial features area to generate a first occlusion result.