KR100871557B1 - A Method for Detecting and Determining Face for Image-based Security - Google Patents

Abstract

There is provided a face detection and discrimination method for image-based security that can detect a stable normal face and improve the reliability by accurately determining the wearing of sunglasses or a mask. A face detection and determination method for image-based security includes detecting a user's approach according to infrared sensing; Extracting a primary face candidate region using a space-time model and a background model; Extracting a second facial candidate region using a robust principal component analysis (RPCA) -based background model and a time-varying image of the extracted first facial candidate region; Extracting a facial contour using an active shape model (ASM) for the extracted second facial candidate region; Determining an eye region and a mouth region from the extracted face contour using histogram indexing; And determining the wearing of jewelry according to the determined eye area and mouth area.

ATM, Security, Face Determination, Contour Extraction, ASM, Histogram Indexing

Description

Face Detection and Determination for Image-Based Security {A Method for Detecting and Determining Face for Image-based Security}

1 is a schematic operation flowchart of a face detection and determination method for image-based security according to an embodiment of the present invention.

2 is a detailed operation flowchart of FIG. 1.

3 is a diagram illustrating a system usage environment according to an embodiment of the present invention.

4 is a diagram illustrating a time-varying image correction result according to an exemplary embodiment of the present invention.

5 is a diagram illustrating a process of detecting a primary face candidate area according to an embodiment of the present invention.

6 is a diagram illustrating a result of detecting face candidates and a torso region according to an exemplary embodiment of the present invention.

7 is a diagram illustrating a reconstructed image using RPCA according to an embodiment of the present invention.

8 is a view showing a change image of the input image and the reconstructed image according to an embodiment of the present invention.

9 is a diagram illustrating a second facial candidate area detection process according to an exemplary embodiment of the present invention.

10 is a diagram illustrating 21 landmark positions according to an embodiment of the present invention.

FIG. 11 is a view illustrating initial positions of five ASM models according to an embodiment of the present invention. FIG.

12 illustrates eye and mouth region extraction using registration landmarks in accordance with an embodiment of the present invention.

13A and 13B are diagrams showing results of comparing histograms of the eyes, sunglasses, mouth, and mask areas according to an embodiment of the present invention.

14A and 14B are views illustrating a result of moving a histogram for dark eyes and comparing the histograms of average eyes (left) and sunglasses (right) according to an embodiment of the present invention.

The present invention relates to a face detection and discrimination method, and more particularly, to a face detection and discrimination method for image-based automated teller machine (ATM) security.

Recently, a technique for determining a user by sensing physical characteristics of a body has been developed. As a method of finding and comparing characteristics distinguishing from others, they can select a security method using various biological characteristics such as fingerprint, iris, and face.

For example, in the case of iris recognition, the price of the terminal is expensive, and the processing time or method thereof is complicated. In the case of a face image-based security method, it is an appropriate method for reducing the rejection without limiting the convenience of the general user.

However, the problems of the existing face image-based security system can be summarized into two main categories. First, it was mostly a way of storing images assuming a face is seen in a situation. However, in this case, there is a problem in that an image without a face is also acquired.

Second, the system developed in the past did not reach the commercialization level because the reliability was low and it was difficult to secure stable performance by applying the initial face recognition technology or the face detection algorithm. As a result, it depends on the normal face detection, there is a problem that the reliability is lowered by simply assuming wearing sunglasses or mask when the face is covered or there is no face.

An object of the present invention for solving the above-mentioned problem is to provide a face detection and discrimination method for image-based security that can detect a stable normal face.

Another object of the present invention is to provide a face detection and determination method for image-based security that can accurately determine the wearing of sunglasses or mask to improve reliability.

According to an aspect of the present invention, there is provided a face detection and discrimination method for image-based security, comprising: detecting a user's approach according to infrared sensing; Extracting a primary face candidate region using a space-time model and a background model; Extracting a second face candidate region from the extracted first face candidate region using a RPCA-based background model and a time-varying image; Extracting a facial contour on the extracted second facial candidate region using an active shape model (ASM); Determining an eye region and a mouth region using histogram indexing from the extracted face contour; And determining the wearing of jewelry according to the determined eye area and mouth area.

The detecting of the user's approach may include: entering the operating state from the standby state upon detecting the user's approach, the waiting state obtaining a background image, updating the background model using the same, and the operating state of the face contour. It is characterized in that to detect the wearing and sunglasses and mask by estimating the eye and mouth area.

The detecting of the user's approach may include estimating a motion region of the user by measuring a motion amount of a difference image, which is a difference between a current frame image and a previous frame image.

In the extracting of the first face candidate region, binarization is performed by adding a temporal change image and a spatial change image corrected according to the amount of motion in units of pixels, and then vertically and horizontally projecting the respective binarized images so that the upper and upper body positions of the head are located. It is characterized by finding approximately.

In the extracting of the second face candidate region, a change image of an input image and a background image is used, and a second image using a Robust Principal Component Analysis (RPCA) -based background model is used. And extracting a facial candidate region.

In the extracting of the second face candidate region, the width of the face may be determined by performing vertical projection on the background change image of the input image and the reconstructed image to be limited to an area corresponding to the height of the face obtained by the first face region extraction. After estimating, the final face candidate area is estimated by performing horizontal projection only on the area corresponding to the estimated face width to estimate the height of the face again.

Here, the facial contour extraction step is characterized by selecting the optimal matching result by varying the initial position by a predetermined number of times.

In this case, the optimum matching determination may be performed using only coordinates corresponding to the left and right sides of the face according to the distance between the background change image and the vertical spatial change image and the coordinates of the matched model.

Here, the facial contour extraction step may be rotated to the front with respect to both eyes from the ASM registration result, and extracting the eye area and the mouth area using the landmark, respectively.

The determining of the eye region and the mouth region using the histogram indexing may include comparing the histogram of the input image in the direction of the reference histogram to be matched based on the largest binarization value bin. .

Hereinafter, a face detection and discrimination method for image-based security according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

In general, the appearance of the face is seriously changed by wearing a mask and sunglasses, so the existing appearance-based face detection technique cannot be applied as it is.

The embodiment of the present invention first extracts the contour of the face that is consistently maintained even when wearing sunglasses or a mask, and then determines whether to wear the eye and the mouth area based on the extracted contour. In addition, the embodiment of the present invention first detects the face candidate region by using a space-time change image and a periodically updateable background model-based change image to ensure performance of ASM-based face contour extraction. At this time, the infrared sensing result is used together with the time-varying image to determine the background modeling time. In addition, the embodiment of the present invention is to vary the initial position of the model to increase the matching accuracy of the ASM, to provide a matching accuracy measurement. Lastly, the embodiment of the present invention determines whether the sunglasses and the mask are worn by applying a histogram indexing technique that is robust against illumination for the eye and mouth regions.

1 is a schematic operation flowchart of a face detection and determination method for image-based security according to an embodiment of the present invention.

Referring to FIG. 1, in the face detection and determination method for image-based security according to an embodiment of the present invention, first, a user's approach is detected by infrared sensing (S110). Thereafter, the primary face candidate region is extracted using the space-time model and the background model (S120), and the second face candidate region is extracted using the Robust Principal Component Analysis (RPCA) model and the time-variable image (S130).

Next, a face contour is extracted using an active shape model (ASM) (S140), and an eye region and a mouth region are determined using histogram indexing from the extracted face contour (S150). Thereafter, wearing jewelry is determined according to the determined eye region and mouth region (S160).

2 is a flowchart illustrating a specific operation of FIG. 1.

Referring to FIG. 2, in the face detection and determination method for image-based security according to an embodiment of the present invention, infrared sensing is first performed using an infrared sensor (S201), and it is determined whether a user approaches (S202). ).

If the user does not approach, the background image is stored and the background modeling is continuously updated (S203).

If the user approaches, the camera captures an image using a camera and extracts an edge image (S204).

Thereafter, projection is performed on the difference image and the edge image, which are the difference between the current image and the previous image (S205), and the upper body region is extracted (S206).

After that, the first face candidate region is extracted using the space-time model and the background model (S207), and the second face candidate is applied to the extracted first face candidate region by applying a RPCA (Robust Principal Component Analysis) model and a time-varying image. The area is extracted (S208).

Next, after determining the optimal initial position of the model through the second step candidate detection method, various initial positions of the model are selected based on the face candidate area (S209), and ASM registration is performed (S210). The optimal matching result is calculated based on the ASM model matching result, and an eye region and a mouth region corresponding to the facial contour are extracted (S211). In other words, it uses the Active Shape Model (ASM), which has been widely used for face modeling. ASM is applied to Active Appearance Model (AAM), which requires individual modeling because it is an application for various people. In order to solve the problem that ASM is sensitive to the initial position selection of the model, the ASM registration is repeated while the initial position is changed to obtain the optimal registration.

Thereafter, face contour information is extracted from the extracted face contour using histogram indexing (S212), and subsequently, ornaments, that is, sunglasses according to the determined eye region and mouth region, are extracted. And mask wearing.

3 is a diagram illustrating an environment of using a face detection and discrimination system for image-based security according to an embodiment of the present invention.

Referring to FIG. 3, it is assumed that the predetermined interval used in the embodiment of the present invention is 19 to 45 cm, the predetermined time is 10 seconds, and the predetermined motion amount is 0.78. The position of the camera 330 allows the user to photograph the ceiling with a relatively simple background, and minimizes the case where the user 310 waiting immediately behind the user 310 can be photographed.

The face detection and discrimination system for image-based security according to an embodiment of the present invention comprises a standby state and an operating state. In the standby state, the background image is acquired and the background model is updated using the background image. In the operating state, the face contour is detected, and the eye and mouth regions are estimated to determine whether to wear sunglasses and a mask. The face detection and discrimination system for image-based security according to an embodiment of the present invention is deemed that the user is approaching the use of the device when the following two conditions are satisfied, and is switched from the standby state to the operating state. That is, when the distance between the user 310 and the ATM is within a predetermined distance and maintained for a predetermined time, the sensing-based distance by the infrared sensor 320 is measured. In addition, when more than a predetermined amount of motion is maintained for a predetermined time, the amount of motion in the time-varying image is measured.

Hereinafter, face contour extraction according to an embodiment of the present invention will be described with reference to FIGS. 4 to 10.

The user's motion region is estimated from a temporal gradient image that is a difference between the current frame image and the previous frame image. If the motion is slow, the estimated motion region is small because the change is small in the change image, and if the motion is fast, the estimated motion region is widened because the change is large in the change image due to motion blurring. have. In the embodiment of the present invention, the motion amount is defined as follows to maintain a constant level of change regardless of slow or fast movement.

는 i프레임의 k번째 픽셀 밝기 값을,

는 픽셀 값 차이가 0이상이면 1을 그렇지 않으면 0을 반환하는 이진함수를 나나내며, N은 영상의 픽셀 수를 나타낸다. 이 값이 하위 임계치 이하 또는 상위 임계치 이상인 경우, 각각 몰폴로지컬 다일레이션(Dilation)과 이로우전(Erosion)을 적용하여 모션량을 보정한다.here,

Is the kth pixel brightness value of the iframe,

Represents a binary function that returns 1 if the pixel value difference is greater than 0, or 0 otherwise, where N represents the number of pixels in the image. If this value is below the lower threshold or above the upper threshold, morphological dilation and effect are applied to correct the amount of motion, respectively.

4 is a diagram illustrating a time-varying image correction result according to an exemplary embodiment of the present invention, wherein 410 denotes a temporal change image and 420 denotes a modified temporal change image, respectively. That is, FIG. 4 is a diagram exemplifying correction of a motion amount by applying morphological dilation and rotation.

Here, the upper threshold and the lower threshold are determined experimentally as follows. The experimenter requires slow, normal, and fast movement to obtain a sequence of 150 frames of approximately 10 seconds. Obtain the time-varying image of the acquired sequence and obtain the motion amount. It divides into 3 parts between the minimum value and the maximum value of motion quantity, and divides into upper, middle, and lower. The average amount of motion from each of the 10 experimenters is obtained, and the centers of the middle, upper, and middle and lower averages are determined as upper and lower thresholds, respectively.

5 is a diagram illustrating a process of detecting a primary face candidate area according to an embodiment of the present invention.

Referring to FIG. 5, the primary face candidate area detection according to an embodiment of the present invention uses a temporal change image 503 and a spatial gradient image 501 and 502 corrected according to a motion amount. The temporal change image 503 and the vertical and horizontal spatial change images 501 and 502 are added by the adders 504 and 505 in units of pixels, respectively, and then binarized (506), and the respective binarized images 507 and 508. By vertical and horizontal projections 509 and 510 in the vertical and horizontal directions, the upper part 511 of the head and the face width 512 and the torso position can be roughly found. At this time, the information that the aspect ratio of the face is 1.2 can be used together.

In addition, when there is a fast movement or a slow movement, that is, when the movement region becomes larger or smaller, respectively, the position previously found from the position queue of the stored face candidate region is taken. But often it is out of the center of the face.

6 is a diagram illustrating a result of detecting face candidates and a torso region according to an exemplary embodiment of the present invention.

Referring to FIG. 6, reference numeral 610 denotes a case where a face candidate region is exactly located at the center of the face, reference numeral 620 denotes a case in which the face is included but is out of the center, and reference numeral 630 denotes that the face moves away from the face candidate region. Each case is shown. In order to apply ASM for face contour extraction, a result as shown by reference numeral 610 should be derived. In all three cases, however, the face may enter the upper and lower boundaries of the candidate area.

Next, face candidate region detection using RPCA-based background model will be described.

When the motion is too fast or slow, the change image of the input image and the background image are used to increase the accuracy of the face region detection based on the time-varying image. At this time, the background image is not fixed and is continuously updated to ensure performance. To this end, in the standby mode (when the user is unlikely to be in front of the ATM), the background model is updated using the continuously acquired image, and the background image is generated therefrom. In the embodiment of the present invention, RPCA (Robust Principal Component Analysis), which can use the entire image acquired in the standby state without using a purely background image, is introduced.

는

차원의 행렬이라고 하자. 여기서,

는 하나의 영상을, n은 학습영상의 수를, d 는 각 영상의 픽셀 수를 나타낸다.

에 대한 k 개의 주성분을

라고 하자. 그러면, 입력영상 f와 RPCA를 이용하여 얻어낸 복원영상

은 도 7과 같이 배경만으로 구성된다. 도 7은 본 발명의 실시예에 따른 RPCA를 이용한 복원 영상으로서, 왼쪽은 입력영상(710)을 나타내고, 오른쪽은 복원영상(720)을 각각 나타낸다.The process of detecting the face candidate area using the RPCA is as follows. first,

Is

Let's call it a matrix of dimensions. here,

Denotes one image, n denotes the number of learning images, and d denotes the number of pixels of each image.

K principal components for

Let's say Then, the restored image obtained by using the input image f and RPCA

7 is composed of only the background as shown in FIG. 7 is a reconstructed image using the RPCA according to an embodiment of the present invention, the left shows the input image 710, the right shows the reconstructed image 720, respectively.

Meanwhile, FIG. 8 is a diagram illustrating a change image of an input image and a reconstructed image according to an embodiment of the present invention, and shows input, reconstruction, and binarized background change images, respectively, from the left side.

9 is a view illustrating a second facial candidate area detection process according to an embodiment of the present invention.

Referring to FIG. 8, the second face candidate area detection process according to an embodiment of the present invention may be limited to an area corresponding to a height of a face obtained by extracting a first face region with respect to a background change image of an input image and a restored image. Perform a vertical projection and estimate the width of the face. The height of the face is estimated again by performing horizontal projection on the area corresponding to the estimated face width. In this way, the final face candidate area is estimated.

Referring to FIG. 9, reference numeral 910 denotes a primary face candidate area, reference numeral 920 denotes a binarized image of a change image of an input image and a reconstructed image, and reference numeral 930 denotes a final face candidate area, respectively.

Therefore, due to the limitation of the projection method, the face cannot be found accurately, but the center of the face is relatively coincident with the center of the face candidate area. This makes it possible to set the initial position of the ASM model at the center of the face, which is suitable for stabilizing ASM matching sensitivity sensitive to the initial position of the model.

Hereinafter, with reference to FIGS. 10 and 11, secondary face contour extraction using ASM according to an embodiment of the present invention will be described.

FIG. 10 is a view showing 21 landmark positions according to an embodiment of the present invention, and FIG. 11 is a view showing initial positions of five ASM models according to an embodiment of the present invention.

Referring to FIG. 10, in consideration of wearing a mask or sunglasses, an ASM is modeled using only 21 points (1010) mainly on the contour of the face. As a result, the matching performance of the ASM may be somewhat degraded.

Therefore, in order to increase the matching performance of the ASM which is particularly sensitive to the initial position of the model, five independent positions are performed by giving five initial positions 1110 and 1120 as shown in FIG. 11 around the face candidate region. Here, the selection of five initial positions 1110 and 1120 may be determined through experiments.

및 수직적 공간변화영상

와 정합된 모델의 좌표 간의 거리 s에 따른다. 이때, ASM 모델의 모든 모델 좌표를 사용하지는 않고, 얼굴의 좌우 측면에 해당하는 좌표만 사용한다.

및

을 각각 모델의 좌측 및 우측 좌표에 대한 벡터라고 하자. 이들 이외의 눈 및 입에 해당하는 좌표는 선글라스나 마스크 착용에 의해 안정된 변화영상을 얻을 수 없기 때문에 제외하였다. 이때, 거리는 다음 수학식 2와 같이 계산한다.11 shows the five ASM model initial positions 1110 and 1120 including the secondary face candidate region. The optimal matching result is selected from the five matching results by changing the initial position. At this time, the best match decision is based on the background change image

And vertical spatial change image

And the distance s between the coordinates of the matched model. At this time, not all model coordinates of the ASM model are used, only coordinates corresponding to the left and right sides of the face are used.

And

Let be the vector for the left and right coordinates of the model, respectively. Coordinates corresponding to eyes and mouths other than these are excluded because stable change images cannot be obtained by wearing sunglasses or a mask. At this time, the distance is calculated as in Equation 2 below.

는 대응되는 픽셀 값을 더하는 연산자이고,

는 임계치

에 따라 0 또는 1을 출력하는 이진화 함수이며,

는 좌표 (x, y)에서의 배경변화 영상이고,

는 수직적 공간변화 영상이다. 그리고 모델이 얼굴의 어느 한쪽 측면에만 잘 정합된 경우를 배제하기 위해

과

사이의 거리가 충분히 작은 경우에 정합 정확도가 양수가 되도록 다음과 같이 정의한다.here,

Is an operator that adds the corresponding pixel values.

Is the threshold

Is a binarization function that outputs 0 or 1 depending on

Is the background change image at coordinates (x, y),

Is a vertical spatial change image. And to exclude the case that the model is well matched to only one side of the face

and

If the distance between them is small enough, we define as follows so that the matching accuracy is positive.

Therefore, as described above, the facial contour may be extracted using the ASM.

Hereinafter, referring to FIGS. 12 to 14, determination of wearing sunglasses and a mask according to an embodiment of the present invention will be described.

12 illustrates eye and mouth region extraction using ASM registration landmarks in accordance with an embodiment of the present invention.

Referring to FIG. 12, the facial contour-based eye region and mouth region can be easily extracted from the ASM registration result. First, the image shown by reference numeral 1210 is rotated to the front with respect to both eyes, and the eye region and the mouth region are extracted using 0, 12, 2, 4, 8, and 10 landmarks, respectively, as shown by reference numeral 1220.

However, the accuracy of eye and mouth region extraction through ASM matching depends on ASM matching accuracy. Since the matching accuracy of ASM modeled with 21 contour centers is relatively low, the eye and mouth regions may not be extracted stably. Therefore, it is necessary to perform sunglasses and mask wearing discrimination using features independent of size, position, and rotation. To this end, a histogram indexing technique is introduced.

13A and 13B illustrate a histogram comparison of eyes, sunglasses, a mouth, and a mask area according to an embodiment of the present invention.

13A and 13B compare histograms in the case of wearing sunglasses and a mask with respect to the eye and mouth regions, and it can be seen that each histogram is relatively clearly distinguished.

At this time, the histogram varies depending on the lighting. To solve this problem, a preprocessing technique such as histogram smoothing cannot be used. This is because the extracted eye and region are composed of small and similar pixel values, so that histogram smoothing results in emphasizing miscellaneous effects. To solve this problem, a method of moving a histogram of an input image in the direction of a reference histogram to be matched based on a bin having the largest value, that is, a binarization value, is applied.

14A and 14B are views illustrating a result of moving a histogram for dark eyes according to an embodiment of the present invention and comparing the histograms of average eyes (left) and sunglasses (right).

Referring to FIGS. 14A and 14B, when the histogram for the dark eye area is converted by considering the histogram of the average eye and the sunglasses, the comparison result shows that the histogram is more similar to the average eye histogram than the average sunglasses histogram. Can be.

As a result, in an embodiment of the present invention, a fully automatic system for detecting a face wearing sunglasses and a mask for an image-based ATM security system and determining whether to wear them is provided. To this end, the system is divided into a standby mode and an operation mode. In the standby mode, the background model is updated, and in the operation mode, face area extraction, face contour extraction, eye and mouth area extraction, sunglasses and mask wearing discrimination are processed. In this process, camera installation angle, motion correction, background model automatic update, ASM landmark setting considering mask and sunglasses wearing, ASM optimal matching method, lighting histogram indexing method, etc. are applied. Although these methods are designed to be limited to ATM environments, they can be applied to hidden face detection in video.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and it is common in the art that various substitutions, modifications, and changes can be made without departing from the technical spirit of the present invention. It will be evident to those who have knowledge of.

According to the present invention, a stable normal face can be detected and reliability can be improved by accurately determining the wearing of sunglasses or a mask.

Claims (12)

Detecting user access according to infrared sensing; Extracting a primary face candidate region using a space-time model and a background model; Extracting a second face candidate region from the extracted first face candidate region using a robust principal component analysis (RPCA) -based background model and a time-varying image; Extracting a facial contour on the extracted second facial candidate region using an active shape model (ASM); Determining an eye region and a mouth region using histogram indexing from the extracted face contour; And Determining the wearing of jewelry according to the determined eye area and mouth area Face detection and identification method for image-based security comprising a. The method of claim 1, The detecting of the user's access may include entering the operating state from the standby state when the user's access is detected. The method of claim 2, And the standby state acquires a background image and updates the background model using the background image. The method of claim 2, The operation state is a face detection and detection method for image-based security, characterized in that for detecting the contour of the face, the eye and mouth area to determine whether to wear sunglasses and mask. The method of claim 1, And detecting the user's approach by estimating a motion region of the user by measuring a motion amount of a difference image, which is a difference between a current frame image and a previous frame image. The method of claim 1, The first facial candidate region extracting step binarizes the time-varying image and the spatial-changing image corrected according to the motion amount by pixel unit, and then vertically and horizontally projects each binarized image, thereby approximating the upper and upper body positions of the head. Face detection and discrimination method for image-based security, characterized in that finding. The method of claim 1, The second face candidate region extraction step uses a change image of the input image and the background image, and uses the RPCA (Robust Principal Component Analysis) -based background model that uses the entire image acquired in the standby state. A face detection and discrimination method for image-based security, comprising extracting candidate regions. The method of claim 1, In the extracting of the second face candidate region, the width of the face is estimated by performing vertical projection on the background change image of the input image and the reconstructed image, by performing a vertical projection on the region corresponding to the height of the face obtained from the first face region extraction. And then estimating the final face candidate area by performing horizontal projection only on the area corresponding to the estimated face width to estimate the height of the face again. The method of claim 1, In the facial contour extraction step, an optimal matching result is selected by varying an initial position by a predetermined number of times, and detecting and determining a face for image-based security. The method of claim 9, The optimum matching determination is based on the distance between the background change image and the vertical spatial change image and the coordinates of the matched model, using only the coordinates corresponding to the left and right sides of the face detection and face detection for image-based security How to determine. The method of claim 1, The facial contour extraction step rotates the front face with respect to both eyes from the ASM registration result, and extracts the eye region and the mouth region using landmarks, respectively. The method of claim 1, The determining of the eye region and the mouth region using the histogram indexing may include moving the histogram of the input image in the direction of the reference histogram to be matched based on the largest binarization value bin, and comparing the image. Face detection and discrimination method for security based.

Images