WO2017036160A1

WO2017036160A1 - Glasses removal method for facial recognition

Info

Publication number: WO2017036160A1
Application number: PCT/CN2016/081152
Authority: WO
Inventors: 聂芸芸; 梁添才; 龚文川; 张永; 刘道余
Original assignee: 广州广电运通金融电子股份有限公司
Priority date: 2015-09-06
Filing date: 2016-05-05
Publication date: 2017-03-09
Also published as: CN105046250A; CN105046250B

Abstract

A glasses removal method for facial recognition, which is used for solving the problem that glasses lenses in a facial image reflect light. The method comprises: collecting a facial image, and performing grayscale processing on a plurality of obtained image frames to obtain a plurality of first images (101); performing face detection on the first images to acquire a face region (102); calculating the light reflectance of the face region (103); screening out the first image with a light reflectance not greater than a pre-set first light reflectance threshold value as a second image (104); locating a glasses frame region of a glasses frame on the second image (105); and restoring the glasses frame region on the second image to obtain a target image in which the glasses frame is removed (106).

Description

Face recognition method for eliminating glasses

The present application claims priority to the Chinese Patent Office, filed on Sep. 6, 2015, the entire disclosure of which is hereby incorporated by reference.

Technical field

The present invention relates to the field of face recognition technology, and in particular, to a method for eliminating glasses for face recognition.

Background technique

Light, posture and expression are the main factors affecting face recognition, and illumination changes have a greater impact on face recognition. Near-infrared images can effectively solve the influence of ambient light changes, but the near-infrared active light source illuminates on the glasses to cause reflection of the human eye. In addition, the occlusion of the frame of the glasses also affects the accurate positioning of the feature points and causes the recognition rate to decrease. .

The problem of reflective image of face image glasses is a long-standing technical problem in the field of face recognition. At present, there is no effective solution at the level of image processing.

Summary of the invention

Embodiments of the present invention provide a method for eliminating glasses for face recognition, which can solve the problem of reflection of a face image lens.

A method for eliminating glasses for face recognition according to an embodiment of the present invention includes:

Collecting a face image, and performing grayscale processing on the obtained plurality of image frames to obtain a plurality of first images;

Performing face detection on the first image to acquire a face region;

Calculating the illuminance of the face region;

And filtering the first image whose reflection degree is not greater than a preset first reflection threshold as a second image;

Positioning a frame area of the eyeglass frame on the second image;

Repairing the frame area on the second image to obtain a target image that eliminates the frame.

Optionally, performing face detection on the first image, and acquiring the face region specifically includes:

Obtaining a face region by performing face detection on the first image based on a Haar classifier;

Or, the face region is acquired by performing face detection on the first image based on the skin color detecting method.

Optionally, the data model of the Haar classifier is updated when the glasses elimination method is first executed:

A plurality of face images are expanded into the OpenCV library in the Haar classifier as original training samples, and the cascade training is re-derived to obtain a new data model.

Optionally, after performing face detection on the first image, acquiring a face region, and before calculating the shininess of the face region, the method further includes:

Obtaining an image of an eye region in the face region;

Performing binarization processing on the image of the eye region;

The calculating the illuminance of the face region is specifically:

The illuminance of the image of the eye region is calculated.

Optionally, the acquiring the image of the eye region in the face region specifically includes:

Obtaining two diagonal vertex coordinates of the face region;

An eye region image is obtained according to the coordinates of the two diagonal vertices and the relative position of the preset eye region.

Optionally, calculating the illuminance of the image of the eye region specifically includes:

Calculating a sum of the number of high-brightness pixels included in all connected domains on the image of the eye region, the high-brightness pixel being a pixel having a gray value of 1.

Optionally, positioning the frame area of the eyeglass frame on the second image is specifically:

The frame area of the eyeglass frame is positioned on the second image by the GVF-Snake method.

Optionally, repairing the frame area on the second image to obtain a target image of the elimination frame is specifically:

The frame region on the second image is interpolated and repaired by a weighted average difference method to obtain a target image from which the frame is eliminated.

Optionally, after calculating the illuminance of the face region and screening the first image whose glare is not greater than the preset first glare threshold as the second image, the method further includes:

Determining whether the glare of the face region on any of the first images is greater than a preset standard threshold, and if so, performing the screening of the first illuminance that is not greater than a preset first glare threshold The step of the image as the second image.

Optionally, the first reflection threshold is obtained by the following steps:

Collecting face images of glasses that are not less than a preset number of levels;

Extracting an image with accurate pupil positioning from the face image to form a standard image set;

Calculating the illuminance of all images in the standard image set;

Obtaining a maximum value of the shininess of all the images as the first reflection threshold.

It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:

In the embodiment of the present invention, first, a face image is collected, and the obtained plurality of image frames are grayed out to obtain a plurality of first images; then, the first image is subjected to face detection to obtain a face. a region; then, calculating a shininess of the face region; and filtering the first image whose glare is not greater than a preset first glare threshold as a second image; and further, on the second image Positioning the frame area of the eyeglass frame; finally, repairing the frame area on the second image to obtain a target image of the frame. In the embodiment of the present invention, the method for eliminating the face recognition glasses can solve the problem of reflection of the face image glasses, so that the human eye positioning no longer receives the influence of the glasses reflection, the feature points are accurately positioned and the recognition rate is improved.

DRAWINGS

1 is a flow chart of an embodiment of a method for eliminating glasses for face recognition according to an embodiment of the present invention;

2 is a flow chart of another embodiment of a method for eliminating glasses for face recognition according to an embodiment of the present invention;

FIG. 3 is an analytical diagram of weighted interpolation weight calculation proposed by the present invention.

detailed description

Embodiments of the present invention provide a method for eliminating glasses for face recognition, which is used to solve the problem of reflection of a face image lens.

In order to make the object, the features and the advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Referring to FIG. 1, an embodiment of a method for eliminating glasses for face recognition according to an embodiment of the present invention includes:

101. Collecting a face image, and performing grayscale processing on the obtained plurality of image frames to obtain a plurality of first images;

First, a face image can be acquired, and the obtained plurality of image frames are subjected to gradation processing to obtain a plurality of first images.

102. Perform face detection on the first image to obtain a face area.

After obtaining the plurality of first images, the first image may be subjected to face detection to acquire a face region.

103. Calculate a shininess of the face area;

After the face area is acquired, the illuminance of the face area can be calculated.

104. Filter the first image whose reflection degree is not greater than a preset first reflection threshold as the second image;

After calculating the illuminance of the face region, the first image whose illuminance is not greater than a preset first illuminating threshold may be selected as the second image.

105. Position a frame area of the eyeglass frame on the second image;

After the first image whose reflection degree is not greater than the preset first reflection threshold is selected as the second image, the frame region of the eyeglass frame may be positioned on the second image.

106. Repair the frame area on the second image to obtain a target image that eliminates the frame.

After positioning the frame area of the eyeglass frame on the second image, the frame area on the second image can be repaired to obtain a target image from which the frame is eliminated.

In this embodiment, first, a face image is collected, and the obtained plurality of image frames are subjected to grayscale processing to obtain a plurality of first images; then, the first image is subjected to face detection to obtain a face region; Then, calculating the illuminance of the face region; selecting the first image whose illuminance is not greater than the preset first illuminating threshold as the second image; and then positioning the frame region of the spectacles frame on the second image Finally, the frame area on the second image is repaired, and the target image of the frame is eliminated. In the embodiment, the method for eliminating the face recognition glasses can solve the problem of reflection of the face image glasses, so that the human eye positioning no longer receives the influence of the glasses reflection, the feature points are accurately positioned and the recognition rate is improved.

For ease of understanding, the following method for eliminating face recognition glasses in the embodiment of the present invention For a detailed description, referring to FIG. 2, another embodiment of a method for eliminating glasses for face recognition according to an embodiment of the present invention includes:

201, collecting a face image, and performing grayscale processing on the obtained plurality of image frames to obtain a plurality of first images;

First, a face image can be acquired, and the obtained plurality of image frames are subjected to gradation processing to obtain a plurality of first images. It should be noted that, in this embodiment, a near-infrared image of a human face can be collected, and the near-infrared camera can have the following characteristics: 1. Automatic rotation within a range of 45 degrees; 2. Band 850 nm; 3. Near the camera. Infrared fill light, the camera is activated, and the light source is turned on.

Therefore, it can be understood that the camera continuously photographs the human face at a fixed frequency or time interval to obtain a plurality of image frames, that is, multi-frame images. These images are then grayed out to obtain a corresponding plurality of first images.

202. Perform face detection on the first image to obtain a face region.

After obtaining the plurality of first images, the first image may be subjected to face detection to acquire a face region. Among them, there are two types of face detection methods: one is based on machine learning, such as the face detection method based on Haar classifier; the other is based on skin color detection method, or the fusion of the above two methods; Adaptive thresholds are more difficult to obtain due to the diversity of user skin tones. Therefore, in step 202, the face detection algorithm of the OpenCV library can be used for face detection, and the algorithm is implemented based on the Haar classifier. In addition, considering that the training samples of the OpenCV library are all image libraries in a visible light environment, the present invention can expand a plurality of face images (or a near-infrared image specifically for a face) into the OpenCV library in the Haar classifier in advance as The original training samples are re-conducted to obtain a new data model. For example, the original training samples are augmented with 70,000 NIR images, and cascading training is performed to obtain a new data model. The reconstructed data model can accurately locate the face region of the near-infrared face image.

203. Acquire an image of an eye region in the face region.

After the face area is acquired, an image of the eye area in the face area can be acquired. The acquiring the image of the eye region in the face region may specifically include:

A. Obtain coordinates of two diagonal vertices of the face region;

B. Obtain an eye region image according to the coordinates of the two diagonal vertices and the relative position of the preset eye region.

For example, according to the vertex coordinates of the face region, the rectangular region RI where the two eyes are located is calculated by the following formula (1), and the RI is binarized to obtain the binarized eye region image.

Let the coordinates of the top left corner of the face area be (FR _left , FR _top ), the coordinates of the bottom right corner (FR _right , FR _bottom ), and the width and height be recorded as FR _width and FR _{height respectively} . The rectangular area RI where the two eyes are located is calculated by the following formula:

RI _left =FR _left +FR _width /8

RI _top =FR _top +FR _width /4

RI _right =FR _left +FR _width -FR _width /8 (1)

RI _bottom =FR _top +FR _height /2

The initial threshold of binarization of RI is obtained by statistic of gray histogram, and then the threshold is adjusted according to the pixel distribution of the binary image, which can reduce the interference of other high-brightness noise in non-eye regions.

204. Perform binarization processing on the image of the eye region;

After acquiring the eye region image in the face region, the eye region image may be binarized as described in step 203.

205. Calculate a shininess of the image of the eye region;

After binarizing the image of the eye region, the illuminance of the image of the eye region can be calculated. It can be understood that, for the binarized image, the shininess is reflected by the number of high-brightness pixels (the gradation value is 1) on the image. Therefore, step 205 can be specifically as follows:

The sum of the number of high-brightness pixels included in all connected domains on the image of the eye region is calculated, and the high-brightness pixel is a pixel with a gray value of 1, wherein the higher the number of high-brightness pixels, the higher the spectacle of the glasses. The specific calculation formula (2) is as follows:

Where S _i is the number of pixels whose gradation value is 1 in the i-th connected domain, and m is the number of connected domains formed by high-brightness pixels.

206, determining whether there is any vertigo of the face area of the first image is greater than a preset standard threshold, and if so, executing step 207, if not, executing step 211;

After calculating the illuminance of the image of the eye region, it may be determined whether the glare of the face region on any of the first images is greater than a preset standard threshold, and if yes, step 207 is performed. If no, step 211 is performed.

It should be noted that the near-infrared light source is irradiated from a certain angle to the lens to cause strong reflection, forming a high-luminance connection region, and the illumination at any angle does not form a high-brightness connection region on the eye. Since the acquisition camera is rotated up and down and supplements near-infrared light, if glasses are worn, there must be multiple illumination angles such that the glare is above a certain threshold. According to this principle, the judgment criterion of the glasses detection is: multi-frame (for example, five frames) images sampled at intervals, if one frame of image glare is greater than a preset standard threshold, it is determined to wear glasses, and vice versa, no glasses are worn. ,

According to the description in the above step 205, if K ≥ K _δ , it is determined that glasses are worn, and vice versa, no glasses are worn. K _δ is the eyeglass detection threshold (ie, the standard threshold), which can be set experimentally.

207. Filter the first image whose reflection degree is not greater than a preset first reflection threshold as the second image.

If the illuminance of the face region on any of the first images is greater than a preset standard threshold, the person may be considered to have glasses on the face, and therefore, the glare may be filtered out to be greater than a preset first glare threshold. The first image is taken as the second image.

It should be noted that, in this step, the image with low reflectance and accurate positioning of the pupil is obtained by the saliency screening. Since the angle and intensity of the light source are constantly changing, the resulting image frame sequence has different shininess. The saliency screening is achieved by threshold control of the image shininess: if the illuminance of the image I is K ≤ δ, it is removed by screening to the next frame. The first reflective threshold may be calculated in advance, and specifically includes:

A. Collecting near-infrared images of faces of glasses that are not less than a predetermined number of glasses;

B. Screening the accurate image of the pupil from the near-infrared image of the face to form a standard image set;

C. Calculating the illuminance of all images in the standard image set;

D. Acquire the maximum value of the shininess of all the images as the first reflection threshold.

For example, the first reflection threshold δ is obtained through a data experiment: the collected 10,000 glasses images I ₁ , . . . , I _{10000 are} subjected to a face feature point localization algorithm test. The image set A with accurate pupil positioning is selected, and the reflectances K ₁ , . . . , K _N (N≤10000) of all the images in A are calculated by the formula of the reflectance, and the maximum value is the first reflection threshold δ.

208. Position a frame area of the eyeglass frame on the second image;

Filtering the first image whose glare is not greater than a preset first reflection threshold as a second After the image, the frame area of the eyeglass frame can be positioned on the second image.

It should be noted that the eyeglass frame positioning can adopt the GVF-Snake method, and the initial contour is gradually evolved to the target contour GI by the energy function. Since the method is based on the region edge information and does not depend on the image gray feature, the contour curve is highly resistant to noise and false boundaries during the evolution process, and the method is not sensitive to the initial contour. The initial contour of this step can be obtained by the Canny detection method: Gaussian filtering is performed on the image, and the eyeglass contour S _{0 is} extracted using the Canny operator in the eyeglass region RI.

209. Repair the frame area on the second image to obtain a target image that eliminates the frame.

After positioning the frame area of the eyeglass frame on the second image, the frame area on the second image can be repaired to obtain a target image of the frame. It can be understood that the common methods for eliminating the glasses frame area are PCA reconstruction, setting mask template processing, feature element compensation, weighted average interpolation, and the like. Among them, PCA reconstruction and feature element compensation methods require a large number of glasses face database and model database without glasses, but it is difficult to obtain such samples in practical applications. The template setting of the mask template processing method has a large dependence on the image itself, and the generalization performance of the method is not good.

The invention adopts a weighted average interpolation method to estimate the pixel point P ₀ (x ₀ , y ₀ ) in the target contour GI by using the gray value of the known pixel point.

N is the total number of P ₀ neighborhood pixels in the window, G(x _i , y _i ) is the gray value of the P ₀ neighborhood pixel in the window, and ω _i is the weight.

Wherein, the occlusion portion of the eyeglass frame is mostly skin or hair, and the degree of correlation with the pixels of the vertical neighborhood is higher than that of the horizontal neighborhood, so the window can adopt an m×n rectangular region centered on the point to be interpolated (where: m<n, such as : 3 × 7 windows). It is known that the farther the pixel point is from the point to be interpolated, the smaller the contribution of the point is, the lower the weight is; the more the pixel point deviates from the long axis direction of the window, the smaller the contribution of the point is, the lower the weight is. Consider the above two aspects, please refer to Figure 3, the weight calculation formula (3) is as follows:

d is the Euclidean distance from the interpolation point P0 to the neighborhood pixel point P _i :

θ is the angle between the line P ₀ P _i and the axis of the window.

210. Save the target image to a database;

After the target image of the frame is removed, the target image can be saved to a database for convenient use.

211. Save the first image as a target image to a database.

If the illuminance of the face region on any of the first images does not exceed the preset standard threshold, it can be considered that no glasses are worn on the face of the person, and the first image is the target image and is saved in the database.

In this embodiment, the face image of the wearing face can be accurately detected by using the face avatar acquisition, and the image with serious reflection of the lens is automatically filtered through a reasonable threshold, and the image with low reflection degree and not affecting the pupil positioning is further filtered. The frame is eliminated. The face image of the glasses after treatment by the device has no interference of the glasses reflection and the frame occlusion, and effectively solves the problem that the user recognition efficiency of the glasses in the near-infrared mode is low or even unrecognizable, and the algorithm has high efficiency and small memory consumption. The identification system user based on the collection device can complete the recognition with the same efficiency without removing the glasses, and greatly improves the user satisfaction.

A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated in In a unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the embodiments are modified, or the equivalents of the technical features are replaced by the equivalents of the technical solutions of the embodiments of the present invention.

Claims

A method for eliminating glasses for face recognition, comprising:

Collecting a face image, and performing grayscale processing on the obtained plurality of image frames to obtain a plurality of first images;

Performing face detection on the first image to acquire a face region;

Calculating the illuminance of the face region;

And filtering the first image whose reflection degree is not greater than a preset first reflection threshold as a second image;

Positioning a frame area of the eyeglass frame on the second image;

Repairing the frame area on the second image to obtain a target image that eliminates the frame.
The method for eliminating glasses according to claim 1, wherein performing face detection on the first image, and acquiring the face region specifically includes:

Obtaining a face region by performing face detection on the first image based on a Haar classifier;

Or, the face region is acquired by performing face detection on the first image based on the skin color detecting method.
The glasses eliminating method according to claim 2, wherein the data model of the Haar classifier is updated when the glasses eliminating method is first executed:

A plurality of face images are expanded into the OpenCV library in the Haar classifier as original training samples, and the cascade training is re-derived to obtain a new data model.
The method for eliminating glasses according to claim 1, wherein after performing face detection on the first image, acquiring a face region, and before calculating the illuminance of the face region, the method further comprises:

Obtaining an image of an eye region in the face region;

Performing binarization processing on the image of the eye region;

The calculating the illuminance of the face region is specifically:

The illuminance of the image of the eye region is calculated.
The method for eliminating the eyeglasses according to claim 4, wherein the acquiring the image of the eye region in the face region comprises:

Obtaining two diagonal vertex coordinates of the face region;

Obtaining an eye region according to the coordinates of the two diagonal vertices and the relative position of the preset eye region Domain image.
The method for eliminating glasses according to claim 4, wherein calculating the illuminance of the image of the eye region comprises:

Calculating a sum of the number of high-brightness pixels included in all connected domains on the image of the eye region, the high-brightness pixel being a pixel having a gray value of 1.
The method for eliminating glasses according to claim 1, wherein the frame area of the eyeglass frame positioned on the second image is specifically:

The frame area of the eyeglass frame is positioned on the second image by the GVF-Snake method.
The method for eliminating glasses according to claim 7, wherein the repairing the frame area on the second image to obtain the target image of the frame is specifically:

The frame region on the second image is interpolated and repaired by a weighted average difference method to obtain a target image from which the frame is eliminated.
The method for eliminating glasses according to claim 1, wherein after calculating the illuminance of the face region and filtering out the first image whose glare is not greater than a preset first glare threshold as a second The image also includes:

Determining whether the glare of the face region on any of the first images is greater than a preset standard threshold, and if so, performing the screening of the first illuminance that is not greater than a preset first glare threshold The step of the image as the second image.
The method for eliminating glasses according to any one of claims 1 to 9, wherein the first reflection threshold is obtained by the following steps:

Collecting face images of glasses that are not less than a preset number of levels;

Extracting an image with accurate pupil positioning from the face image to form a standard image set;

Calculating the illuminance of all images in the standard image set;

Obtaining a maximum value of the shininess of all the images as the first reflection threshold.