Disclosure of Invention
The invention provides an exposure self-adaptive adjusting method based on face brightness, which aims to solve the problem that the brightness of a face region cannot be ensured while the brightness of the whole picture is ensured by a camera, and is mainly applied to intelligent products needing face recognition.
The technical scheme adopted by the invention for solving the technical problems is as follows: an exposure self-adaptive adjusting method based on face brightness comprises the following steps:
setting an initial exposure E0;
Secondly, carrying out face detection on the video stream, obtaining a face area (x, y, w, h) through a face detection algorithm when a face appears in the picture, and calculating the weighted average brightness B of the face area1Wherein x and y respectively represent the abscissa and the ordinate of the upper left corner of the face rectangular region, and w and h respectively represent the width and the height of the face rectangular region;
third, given brightness threshold T1And T2(T1<T2) If the brightness B is1If the exposure parameter is within the threshold value range, saving the current exposure parameter, and performing the step IV; otherwise, adjusting the exposure degree and performing the step II;
fourthly, carrying out face recognition according to the face image to obtain a result;
fifthly, after the face disappears, the current exposure E is kept1And continuing to perform subsequent face detection.
The step of calculating the weighted average brightness of the face region comprises the following steps:
② -1 obtaining facial five sense organ points (x)i,yi) Wherein i is 1, 2, 3, 4, 5, (x)1,y1),(x2,y2) Is a pair of eyes, (x)3,y3) Is the tip of the nose, (x)4,y4),(x5,y5) Left and right mouth corners, satisfying the conditionsWherein W and H are the width and height of the whole picture;
2 the width w of the rectangular area of the face is the transverse distance l between the two eyes1Multiplied by a coefficient a (a > 1), where l1=x2-x1If w is equal to al1The initial x-x abscissa of the face rectangular region1-0.5(a-1)l1;
② -3 face rectangular regionThe height h of the field is the maximum value l of the longitudinal distance between the eye and the mouth2Multiplied by a coefficient b (b > 1), where l2=max(y4-y1,y4-y2.y5-y1,y5-y2) Then h is bl2The initial ordinate of the face rectangular region is y ═ min (y)1,y2)-0.5(b-1)l2;
② 4 face rectangle area (x, y, w, h) need to satisfy conditionWhere W and H are the width and height of the entire screen, x is 0 if x < 0 is obtained when x is calculated on the abscissa, y is 0 if y < 0 is obtained when y is calculated on the ordinate, W is decreased to W if x + W > W, and H is decreased to H if y + H > H.
Secondly-5, giving greater weight to the five sense organ region when calculating the average brightness of the face region;
secondly, 6, setting sampling step length, sampling the brightness once at every step pixel point to reduce the calculation burden, and finally calculating to obtain the weighted average brightness of the face rectangular region as B1。
The invention has the advantages that:
1) the optimization effect is obvious. At present, most of intelligent products based on human faces use wide dynamic cameras to ensure image brightness, but in practical application, human face images still have the conditions of backlight, yin and yang faces and the like, and the speed and the precision of recognition are influenced. The method has strong target, dynamically adjusts the exposure of the camera after calculating the brightness of the face area, ensures the brightness of the acquired face image to be in a proper range, and greatly improves the accuracy of face feature extraction;
2) the optimization method is simple. Most of common face recognition optimization algorithms involve modeling and training image data, and the calculation amount is huge; the method is simple in operation, only needs to calculate the brightness of the face image, is small in calculation amount, and is simple and clear in the adjustment process of the exposure of the camera;
3) the ductility is strong. Not only is the face track matching, but also the method is applicable to other object recognition and tracking applications, such as: video surveillance for vehicle travel, surveillance surveys for wildlife whereabouts, and so forth.
Detailed Description
The invention is described in further detail below with reference to the accompanying examples.
The invention discloses an exposure self-adaptive adjusting method based on face brightness, which is suitable for intelligent products based on face recognition. An initial exposure level is first set. When a face appears in the picture, the face area in the picture is positioned according to the result of the face detection, the average brightness of the area is calculated and compared with a set threshold value: if the image is within the threshold range, the human face image is directly used for recognition; otherwise, adjusting the exposure of the camera, and performing face detection again to obtain a face image. And when the face does not exist in the picture, the prior exposure is maintained to ensure the subsequent face detection and recognition precision. The embodiment runs on an Android 8.1 platform, the image format is YUV420 format, and the wide dynamic camera provides seven levels of exposure (-3, -2, -1, 0, 1, 2, 3).
The general flow diagram of the method is shown in fig. 1, and specifically includes the following steps:
setting an initial exposure E0In this embodiment, level 0 is taken;
secondly, the face detection is carried out on the video stream, when the face is detected in the picture, the facial feature points (x) are obtained1,y1),(x2,y2),(x3,y3),(x4,y4),(x5,y5) Wherein (x)1,y1),(x2,y2) Is a pair of eyes, (x)3,y3) Is the tip of the nose, (x)4,y4),(x5,y5) The left and right corners are calculated, and a face rectangular area (x, y, w, h) is calculated, wherein x and y respectively represent the upper left corner of the face rectangular areaThe horizontal coordinate and the vertical coordinate, w and h respectively represent the width and the height of a face rectangular region, and the weighted average brightness B of the face region image is calculated1Wherein the weight of the five sense organ region is q times of the other face region;
thirdly, obtaining proper brightness threshold value T through actual measurement1And T2(T1<T2) The image with the brightness in the interval can provide the optimal recognition effect, and the average brightness B of the rectangular area of the face is compared1Given a brightness threshold T1And T2(T1<T2) If the brightness B is1If the exposure parameter is within the threshold value range, saving the current exposure parameter, and performing the step IV; otherwise, adjusting the exposure degree and performing the step II;
fourthly, identifying the face image to obtain an identification result;
fifthly, after the face disappears, the current exposure E is kept1And continuing to perform subsequent face detection.
The detailed steps of calculating the face rectangular area in the step II are as follows:
② -1, obtaining facial five sense organ points (x)i,yi) Wherein i is 1, 2, 3, 4, 5, (x)1,y1),(x2,y2) Is a pair of eyes, (x)3,y3) Is the tip of the nose, (x)4,y4),(x5,y5) Left and right mouth corners, satisfying the conditionsWherein W and H are the width and height of the whole picture;
2, the width w of the rectangular area of the face is the transverse distance l between the two eyes1Multiplied by a coefficient a (a > 1), where l1=x2-x1If w is equal to al1The initial x-x abscissa of the face rectangular region1-0.5(a-1)l1. In this embodiment, if a is 2, the abscissa x is x1-0.5l1;
(ii) -3. the height h of the rectangular area of the face is the maximum value l of the longitudinal distance between the eyes and the mouth corner2Multiplied by a coefficient b (b > 1), where l2=max(y4-y1,y4-y2,y5-y1,y5-y2) Then h is bl2The initial ordinate of the face rectangular region is y ═ min (y)1,y2)-0.5(b-1)l2. In this embodiment, when b is 2, the ordinate y is min (y)1,y2)-0.5l2;
② 4. the face rectangle area (x, y, w, h) needs to satisfy the conditionWhere W and H are the width and height of the entire screen, x is 0 if x < 0 is obtained when x is calculated on the abscissa, y is 0 if y < 0 is obtained when y is calculated on the ordinate, W is decreased to W if x + W > W, and H is decreased to H if y + H > H.
In this embodiment, the rectangular region of the face in the step two has a size requirement, so as to avoid the waste of computing resources caused by the recognition of the face under the condition that the face is too small or too large.
The detailed steps of the face region weighted average brightness calculation in the step II are as follows:
secondly-5, obtaining the YUV420 format of the image format, and storing the data of a 2 x 2 YUV420 image in the format ofWherein Y is the image brightness, and the number of the pixel points is the number of the brightness elements, so for an image of W × H size, the first W × H elements of the source data are the brightness elements.
And 6, considering the importance of the five sense organ points to face recognition, the five sense organ regions are given greater weight when calculating the average brightness. From the second step, the area ratio of the five sense organ region to other face region is 1: 3, and the weight ratio is set to be 2: 1 in this embodiment;
and 7, setting a sampling step length, and sampling the brightness of the pixel point once every step to reduce the calculated amount. In this embodiment, the sampling step length is set to 10, and the final calculated weighted average brightness of the rectangular region of the face is B1。
In the embodiment, the step two is not required to be carried out every frame, and the face detection can be carried out once every several frames, so that the calculation load is reduced, and the exposure parameters of the camera are adjusted for a certain time to be buffered.
The brightness threshold value T in the step III1And T2From the measurements, T is taken in this example1=120,T2160. In the embodiment, the exposure of the camera has seven levels, and the average brightness B of the rectangular area of the face1Comparing with a brightness threshold value if B1<T1If so, increasing the first-level exposure, and performing face detection on the whole image again; if the brightness B is1>T2If so, reducing the first-stage exposure, and performing face detection on the whole image again; otherwise, the current exposure E is saved1And identifying the face image.
The fifth step of maintaining the exposure E1The situation that the human face cannot be detected due to uneven illumination is avoided, and the subsequent human face detection rate can be ensured.
To more strongly illustrate the feasibility and effectiveness of the method of the present invention, we compared the face recognition schemes before and after optimization under different lighting conditions, and the results are shown in table 1.
Table 1 statistical results of face recognition accuracy under different illumination conditions
|
Before optimization
|
After optimization
|
Normal lighting conditions
|
98.8%
|
99.1%
|
Back light
|
93.6%
|
98.9%
|
Backlight
|
95.7%
|
99.1%
|
Yin-yang face
|
91.1%
|
98.5% |
In the test data, the number of the human faces is 312, the test quantity is 5000 times under the normal illumination condition, and the test quantity is 5000 times in total for backlight, backlight and yin-yang face tests.
In addition, the data is the recognition accuracy after the face is detected, in an actual test, under the scenes of backlight, yin-yang face, the situation that the face cannot be detected often exists in the unoptimized scheme, and the situation basically does not exist in the optimized scheme, so the fifth step has an important meaning for improving the overall performance.