KR102317180B1 - Apparatus and method of face recognition verifying liveness based on 3d depth information and ir information - Google Patents

Abstract

Based on 3D depth information and infrared information, it is a face recognition device that checks the presence of a living body. RGB image (visible light) from a dual-aperture, single-lens, 4-color sensor camera through a single lens. and IR is receiving the same time the data of the image (IR), a value calculated from the depth information from the extraction unit, and the depth map comprising depth information processing chip for extracting and outputting a depth map the three-dimensional reference value or more, the three-dimensional reference a three-dimensional verification module for generating a pass signal; and a face matching module for generating a face recognition pass signal if the RGB image is compared with an RGB template of a pre-stored face template after receiving the three-dimensional reference pass signal, and matches within an error limit It is characterized in that it comprises a.

Description

A face recognition device and method for checking whether a person is alive based on three-dimensional depth information and infrared information {APPARATUS AND METHOD OF FACE RECOGNITION VERIFYING LIVENESS BASED ON 3D DEPTH INFORMATION AND IR INFORMATION}

The present invention relates to a face recognition apparatus and method for confirming whether a living body is present based on three-dimensional depth information and infrared information.

In general, a security system based on face recognition is known. This security system may serve to allow a person verified through face recognition to perform a specific operation, for example, entering a door, accessing an information processing device, and the like.

In the face recognition method, a security system including a camera for inputting an image and a control device for comparing the input image with a face template stored in advance is known. According to this, after making a face template in advance from images of people allowed to use the security system, at the time of actual security application, a real-time input image is captured with a camera for a person requesting use, and the stored face By comparing with the template and judging whether the feature points or the like match, it is determined whether the use is permitted or not.

As such, the conventional face recognition technology passes security authentication if only the pattern of the face image matches within the error limit. It is exposed to the security risk of making the error of false pass, which determines a pass for one face pattern.

That is, when generating an input image, there is an attempt to neutralize the security judgment by allowing the camera to take a picture of a person who is permitted to use the security. That is, since the input image obtained by capturing a sophisticated 2D photo may be extremely similar to the stored face template, the degree of matching of the feature points may be determined to be high. Therefore, there was a problem that the security system could make an erroneous decision.

In order to compensate for this, a technique for performing additional verification of information that is not a photograph is proposed. That is, it verifies that it is not a simple photograph by detecting the blink of an eye or movement of the face or hair.

However, the blinking of the eyes or the movement of the face can invalidate the security judgment by, for example, when the input image is generated, the camera captures a moving picture of the face of a person who is permitted to use the security. That is, since the moving image can blink eyes and move the face and hair, it can be determined that the input image by the moving image passes additional verification as a living body. Therefore, there was a problem that the security system could also make an erroneous decision.

On the other hand, in the following patent documents, 'a means for automatically finding a person's face in an image taken by a security system and finding an eye area after a normalization process, and a means for detecting an infrared reflection occurring in the pupil of the eye by infrared illumination And, a technology to improve the performance and usability of a security system related to face recognition by determining whether or not hacking is caused by a forged image' is disclosed.

Patent Publication No. 10-2010-0097866

In the technique of the patent document, a process of finding a face from an input image through image processing and finding eyes again is required. Therefore, hardware and software for high-speed processing required for processing are required.

delete

And in order to detect the infrared reflection of the pupil of the eye, a determination processing means for light emission of infrared illumination, light reception, and infrared reflection is required. These means are comprised of hardware and software separately provided in addition to the camera.

And for a person wearing glasses, sunglasses, or contact lenses, there are cases where infrared reflection does not occur even by infrared light emission. In this case, the technique is useless.

The present invention has been made to solve the problems of the prior art, and by grafting the conventional face recognition technology with a dual aperture single lens method 3D depth information and infrared information extraction device, complex hardware An object of the present invention is to provide a face recognition device and method for verifying a biometric so as to enhance security by mounting a two-step biometric verification function without equipment.

Another object of the present invention is to provide a face recognition apparatus and method for identifying a living body without the need to find a face from an input image through image processing and search for eyes again.

Another object of the present invention is to provide a face recognition apparatus and method for determining whether a person is a living body, which can minimize the size and cost of the device without the need for processing means for light emission of infrared illumination and infrared reflection.

Another object of the present invention is to provide a face recognition apparatus and method for confirming whether a living body exists, which can be applied to a person who wears glasses, sunglasses, or contact lenses.

The face recognition device for checking whether or not a living body of the present invention for achieving the above object is a face recognition device for checking whether or not a living body is present based on three-dimensional depth information and infrared information, a dual aperture single lens (Single Lens) An extraction device including a depth information processing chip that receives simultaneous data of RGB image (visible light) and IR image (infrared) through a single lens from a 4-color sensor camera, and extracts and outputs a depth map; , if the value calculated from the depth information from the depth map is equal to or greater than the stereoscopic reference value, a stereoscopic verification module for generating a stereoscopic reference passing signal; Compared with , it is characterized in that it includes a face matching module that generates a face recognition pass signal if it matches within the error limit.

Here, instead of the three-dimensional verification module or together with the three-dimensional verification module, if the intensity of the wavelength corresponding to the body temperature among the wavelengths of the IR image is greater than or equal to the body temperature reference value, a body temperature verification module for generating a body temperature standard pass signal is included. is characterized by

Here, the face matching module is preferably configured to further compare the IR image with a pre-stored IR template.

Alternatively, the face matching module is preferably configured to further compare the depth information from the depth map with a pre-stored depth information template.

delete

And the filter used for the double aperture of the barrel of the double aperture single lens 4-color sensor camera has a shallow depth of field of a relatively small first F value for visible light (R, G, B) having a wavelength of less than 650 nm. For infrared (IR) with a wavelength of 8 µm or more and less than 14 µm, it has a spectral characteristic implemented by a filter and is implemented by a deep-depth filter of a relatively large second F value, and the signal for each wavelength of the image signal According to this spectral characteristic, the color sensor for visible light is manufactured to receive a signal from the filter of the depth of the first F value, and the color sensor for infrared light is the second F value. It is preferable to be manufactured to receive a signal from a filter with a depth of .

On the other hand, the body temperature verification module, based on the RGB values (0 to 255) when the Bayer pattern is sprayed as a video stream, when the preset body temperature standard value is exceeded, the wavelength band It is preferable that the IR signal is strong and configured to determine that it is a living body.

On the other hand, the method of the present invention is a face recognition method for confirming the presence of a living body based on three-dimensional depth information and infrared information, and a single lens from a Dual Aperture Single Lens 4-color sensor camera An extraction step comprising a depth information processing process of receiving simultaneous data of an RGB image (visible light) and an IR image (infrared) through If it is equal to or greater than this stereoscopic reference value, a stereoscopic verification step of generating a stereoscopic reference pass signal; after receiving the stereoscopic reference passage signal, the RGB image is compared with the RGB template of a pre-stored face template; It is characterized by including a face matching step of generating a pass signal.

Here, instead of or together with the three-dimensional verification step, if the intensity of the wavelength corresponding to the body body temperature among the wavelengths of the IR image is greater than or equal to the body temperature reference value, a body temperature verification step of generating a body temperature reference pass signal is included. is characterized by

According to the present invention, by grafting the three-dimensional depth information and infrared information extraction device of the dual aperture single lens method to the conventional face recognition technology, a two-step biometric confirmation function without complicated hardware equipment A face recognition apparatus and method are provided for confirming the presence or absence of a biometric body to enhance security.

Further, there is provided a face recognition apparatus and method for determining whether a human body is a living body without the need to find a face from an input image through image processing and search for eyes again.

Further, there is provided a face recognition apparatus and method for confirming whether a living body exists, which can minimize the size and cost of the device without the need for processing means for light emission of infrared illumination and infrared reflection.

In addition, there is provided a face recognition device and method for confirming whether a living body, which can be applied to a person wearing glasses, sunglasses, or contact lenses.

1 is a block diagram of a face recognition apparatus for confirming whether a body is alive based on three-dimensional depth information and infrared information according to an embodiment of the present invention.
2 is a graph showing wavelengths of visible and infrared rays and infrared wavelengths corresponding to the body temperature.
3 is a block diagram of a face recognition apparatus for confirming whether a body is alive based on 3D depth information and infrared information according to another embodiment of the present invention.
4 is an exemplary photograph of a case in which body temperature determination is performed bipartitely according to a reference value.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, for elements having the same function due to the same structure, the same reference numerals are maintained even if the drawings are different, and thus detailed description thereof may be omitted. In addition, being connected includes the concept of passing through a medium in the middle. In addition, a certain functional block may be configured by being absorbed into another functional block, or may be implemented by being divided into a plurality of other blocks.

<Example 1 - Stereoscopic Verification>

The face recognition device for checking whether a living body is present according to the present invention is a face recognition device for checking whether or not a living body is present based on three-dimensional depth information and infrared information. 1 and 3, the face recognition device is characterized in that it comprises an extraction device 11 , a three-dimensional verification module 15, and face matching modules (13, 13a) .

The extraction device 11 includes an RGB image (visible light) 11a and an IR image (infrared light) 11b through a single lens from a Dual Aperture Single Lens 4-color sensor camera 10 . ) receives the same time data, the apparatus comprising a depth map (11c) depth information processing chip (SoC) and outputting the extracted. of

The dual-aperture single-lens 4-color sensor camera 10 separates the image received through one lens into visible light color light (RGB) and infrared black-and-white light (IR) while passing it through a double aperture, and these are captured It is a special camera that allows light to be received by a 4-color sensor composed of four light receiving units (R, G, B, IR) constituting each pixel unit of the device. Accordingly, according to this camera 10, simultaneously generated RGB images (color images) 11a and IR images (monochrome images) 11b having the same viewpoint passing through one lens are output. The RGB image 11a and the IR image 11b are transmitted to a depth information processing chip (SoC).

The depth information processing chip (SoC) is a semiconductor chip for extracting depth information of a double aperture single lens method, for example, eWBM Co., Ltd.'s model name DR1152, which is linked to the camera 10 to capture visible light, It is equipped with functions such as depth information extraction and infrared image extraction. As such, the technology for extracting depth with the double-aperture single lens 4-color sensor camera 10 can already be used by the depth information system in which the present applicant developed a system-on-a-chip (SoC) and solution.

From the depth information processing chip (SoC), as shown in FIGS. 1 and 3 , the RGB image 11a and the IR image 11b as well as the depth calculated from the distribution of depth information through a known processing process therefrom A map (Depth Map) 11c is output. The present invention enhances security processing by using these RGB images 11a, IR images 11b, and depth maps 11c. In particular, the IR image 11b may be simultaneously used for checking body temperature characteristics or capturing an image in low light.

The stereoscopic verification module 15 is a module for generating a stereoscopic reference passing signal when the value calculated from the depth information from the depth map 11c is equal to or greater than the stereoscopic reference value. As such, in the present invention, depth information, which is the output of the existing dual-aperture single lens depth information extraction solution technology, is utilized as three-dimensional (stereoscopic, 3D) object identification data.

The face matching module 13, 13a compares the RGB image 11a with the RGB template 12a of the face template 12 stored in advance after receiving the stereoscopic reference pass signal, and matches within the error limit. It is a module that generates a face recognition pass signal. Here, the face matching module 13 of FIG. 1 simply compares the captured RGB image 11a with a pre-stored RGB template 12a, a conventional basic RGB image-based two-dimensional (2D, flat) face recognition algorithm. am.

<Operation>

In the present invention, by controlling the operation of the simple 2D face matching module 13 to operate by triggering the stereoscopic reference passing signal of the stereoscopic verification module 15 as a trigger, errors in security judgment are prevented. In other words, face matching is performed only in the case of a three-dimensional shape, and security is passed when face matching is successful.

<Effect>

The present invention provides a dual-aperture single-lens camera having a 4-color sensor, a processing chip for extracting a depth map by receiving RGB and IR signals, a technology for determining whether stereoscopic from a depth map, and image comparison to determine identity technologies are integrated into one. As described above, if separate technologies are configured as a single module for extracting depth information and implemented as a face recognition device, while at the same time attempting to forge a 2D image (photo or video) or 3D model, this is detected and face authentication is rejected It is possible to implement an efficient system that can do this.

Through this technology, it is possible to implement a face recognition device with an overwhelmingly high security level with a small form factor and low cost.

<Embodiment 2 - Body Temperature Verification>

Here, instead of the three-dimensional verification module 15 or together with the three-dimensional verification module 15, if the intensity of the wavelength corresponding to the body temperature among the wavelengths of the IR image 11b is greater than or equal to the body temperature reference value, the body temperature standard pass signal It is preferable to include a body temperature verification module 14 to generate.

By using the infrared spectral characterization (waveform characterization) emitted from the human body, it can be confirmed whether body temperature is detected. In the characteristic of body temperature, the wavelength of infrared (IR) shows a local maximum waveform near 8 to 14 μm, particularly around 9.5 μm. In the present invention, using this, body temperature can be analyzed and detected only by analyzing the infrared image already emitted from the dual-aperture camera 10 module.

Body temperature verification can be implemented by mounting a body temperature detection algorithm through infrared analysis on an AP or an advanced depth information processing chip (SoC).

Of course, in order to use the existing technology for precise body temperature measurement, it is necessary to use an IR-FPA (Infrared Focal Plane Array) sensor or a thermal imaging camera. However, in the present invention, the purpose of the present invention is not to measure the precise body temperature, but only to confirm that it is the temperature of the living body. Therefore, when the intensity of the infrared image emitted from the dual-aperture camera using the 4-color sensor exceeds a certain intensity threshold (calibrated intensity threshold), it can be determined to be a living body. That is, instead of using a general IR-FPA sensor for body temperature detection, if the intensity of the IR image output from the 4-color sensor of FIG. 2 exceeds a certain level, body temperature may be read as the body temperature.

<Effect>

Body temperature, particularly human body temperature, appears in an infrared (IR) region with a wavelength of approximately 8 µm or more and less than 14 µm, with a peak value of about 9.5 µm as shown in FIG. 2 . And no matter how sophisticated it is, the body temperature having this pattern is not detected from a photograph, a video, or a moving mannequin model.

Of course, thermal energy can be detected from an electronic device that reproduces photos or videos, but the wavelength pattern of the thermal energy shows a different aspect from that of the body temperature. By verifying whether or not it is, it is possible to distinguish the electronic device from the living body. However, in order to avoid judgment errors caused by electronic devices or the like, a negative judgment routine may be added to suspend the body temperature verification when the intensity pattern in another wavelength region is stronger than the reference value.

Therefore, more reliable verification of the living body is possible by body temperature verification.

<Third embodiment - face matching further including IR matching>

The face matching module 13a may be configured to further compare the IR image 11b with a pre-stored IR template 12b. That is, in order to increase the accuracy, as in FIG. 3 , instead of the existing two-dimensional face recognition algorithm 13, not only the RGB data 11a but also the IR data 11b already provided in the double-aperture solution are used. , an algorithm 13a with an increased recognition rate may be used.

By using this, by comparing the IR image 11b, which is an all-focused image for all depths, it is possible to compare feature points that cannot be obtained only by comparing the RGB image 11a, so that the recognition rate can be dramatically improved. possible.

<Fourth embodiment - 3D matching by face matching further including depth information matching>

The face matching module 13a may be configured to further compare the depth information from the depth map 11c with a previously stored depth information template 12c. That is, in order to increase the accuracy, as shown in FIG. 3 , instead of the existing two-dimensional face recognition algorithm, an algorithm with an increased recognition rate may be used using depth information that is already provided in a double-aperture solution as well as RGB.

Using this, it is possible to compare three-dimensional feature points of a face, such as a person with a high nose and a person with protruding cheekbones, using depth information, so that the recognition rate can be dramatically improved.

<Configuration example of optimization of infrared region for body temperature verification>

The filter used for the double aperture of the barrel of the double aperture single lens 4-color sensor camera 10, for example, has a relatively small first F value for visible light (R, G, B) having a wavelength of less than 650 nm. It is implemented by a shallow depth filter, for example, for infrared (IR) having a wavelength of 8 μm or more and less than 14 μm, it can be made to have spectral characteristics achieved by being implemented by a deep depth filter having a relatively large second F value. .

In this case, the 4-color sensor for detecting signals for each wavelength of the image signal is manufactured to receive a signal from the filter of the depth of the first F value, and the color sensor for visible light is manufactured according to the spectral characteristics, and infrared rays The color sensor for use may be manufactured to receive a signal from a filter having a depth of the second F value.

Here, a general optical system for extracting double aperture depth information often senses near-infrared as an IR input when implementing a depth information extraction function, but has a side effect that the visible light image is contaminated with red due to near-infrared close to visible light. In the present invention, instead of near-infrared rays, it is possible to configure a dual aperture by realizing far-infrared rays in the region of 8-14 μm, which are typical infrared wavelengths emitted from the human body, as a filter with a large F value (deep depth). Accordingly, it is possible to check the body temperature of a person without degrading the original depth information extraction function, and by using far-infrared rays that do not border visible light, it is possible to have the effect of eliminating the side effect of red contamination of the visible light image. And it can be easily implemented without greatly increasing the form factor of hardware.

<Specific examples of body temperature verification>

The body temperature verification module 14, based on the RGB values (0 to 255) when the Bayer pattern is sprayed as a video stream, exceeds the preset body temperature reference value, the wavelength The IR signal of the band may be configured to be determined to be a living body because it is strong.

As a result, the body temperature can be determined by simply determining whether the intensity of the IR signal exceeds the reference value, so that the body temperature can be clearly and quickly determined, for example, as shown in FIG. 4 .

<Method>

The present invention can be embodied as an invention of a method, which corresponds to the invention of the device. The face recognition method for confirming the presence of a living body of the present invention is a face recognition method for confirming whether a living body is present based on three-dimensional depth information and infrared information, and includes an extraction step (11) and a three-dimensional verification step (15) and face matching steps (13, 13a) .

In the extraction step 11, an RGB image (visible light) 11a and an IR image (infrared light) 11b through a single lens from a Dual Aperture Single Lens 4-color sensor camera 10 ), and a depth information processing process of extracting and outputting the depth map 11c by receiving the simultaneous data.

The stereoscopic verification step 15 is a step of generating a stereoscopic reference pass signal when the value calculated from the depth information from the depth map 11c is equal to or greater than the stereoscopic reference value.

In the face matching steps 13 and 13a, after receiving the stereoscopic reference passing signal, the RGB image 11a is compared with the RGB template 12a of the face template 12 stored in advance, and if they match within the error limit, This is a step of generating a face recognition pass signal.

Here, instead of the three-dimensional verification step 15 or together with the three-dimensional verification step 15, if the intensity of the wavelength corresponding to the body temperature among the wavelengths of the IR image 11b is greater than or equal to the body temperature reference value, the body temperature standard pass signal It is preferable to include the temperature verification step 14 to generate.

In the above, the present invention has been described with reference to specific embodiments, but the present invention is not limited thereto. All modifications, improvements, and changes made by those skilled in the art within the scope described in the claims should be construed as belonging to the scope of the present invention.

For example, the IR image 11b may be used to obtain face image data in a dark environment with low illuminance or to perform face matching 13 and 13a, which is indicated by a dotted arrow in FIGS. 1 and 3 .

INDUSTRIAL APPLICABILITY The present invention can be used in the industry of a face recognition apparatus and method for confirming whether a living body is present based on three-dimensional depth information and infrared information.

10: Dual Aperture Single Lens 4-Color Sensor Camera
11: Extraction device including depth information processing chip (SoC)
11a: RGB image
11b: IR image
11c: Depth Map
12: Face Template
13: face matching module
14: body temperature verification module
15: three-dimensional verification module
16: reference strength
17: pass signal

Claims (6)

A face recognition device that checks whether a living body is present based on three-dimensional depth information and infrared information, comprising:
Dual Aperture Single Lens Depth information that extracts and outputs a depth map by receiving simultaneous data of RGB image (visible light) and IR image (infrared) through a single lens from a 4-color sensor camera an extraction device comprising a processing chip ;
If the value calculated from the depth information from the depth map is equal to or greater than the stereoscopic reference value, a stereoscopic verification module for generating a stereoscopic reference passing signal;
a face matching module that, after receiving the stereoscopic reference pass signal, compares the RGB image with an RGB template of a pre-stored face template, and generates a face recognition pass signal if they match within an error limit;
Body temperature verification module for checking whether the body temperature is detected from the subject of the 4-color sensor camera
is made, including
The body temperature verification module,
Among the wavelengths of the IR image, if the intensity of the wavelength corresponding to the body temperature is greater than or equal to the body temperature reference value, and the wavelength shows a local maximum waveform in the range of 8 to 14 μm, a body temperature standard passing signal is generated.
A face recognition device for confirming the presence of a living body, characterized in that. The method according to claim 1,
the face matching module is configured to further compare the IR image with a pre-stored IR template
A face recognition device for confirming the presence of a living body, characterized in that. The method according to claim 1 or 2,
the face matching module is configured to further compare the depth information from the depth map with a pre-stored depth information template
A face recognition device for confirming the presence of a living body, characterized in that. The method according to claim 1 or 2,
The filter used for the double aperture of the barrel of the double aperture single lens 4-color sensor camera is a shallow depth-of-field filter with a relatively small first F value for visible light (R, G, B) having a wavelength of less than 650 nm. For infrared (IR) with a wavelength of 8 μm or more and less than 14 μm, it has a spectral characteristic realized by a deep-depth filter of a relatively large second F value,
The 4-color sensor for detecting the signals for each wavelength of the RGB image and the IR image signal, according to this spectral characteristic, the color sensor for visible light is manufactured to receive a signal from the filter of the depth of the first F value, , the infrared color sensor is manufactured to receive the signal from the filter of the depth of the second F value
A face recognition device for confirming the presence of a living body, characterized in that. The method according to claim 1,
When the body temperature verification module exceeds the preset body temperature reference value based on the RGB values (0 to 255) when the Bayer pattern is applied as a video stream, the IR of the wavelength band The signal is strong and configured to determine that it is a living organism
A face recognition device for confirming the presence of a living body, characterized in that. A face recognition method for confirming whether a living body is present based on three-dimensional depth information and infrared information, comprising:
Dual Aperture Single Lens Depth information that extracts and outputs a depth map by receiving simultaneous data of RGB image (visible light) and IR image (infrared) through a single lens from a 4-color sensor camera an extraction step comprising a processing process ;
If the value calculated from the depth information from the depth map is equal to or greater than the stereoscopic reference value, a stereoscopic verification step of generating a stereoscopic reference passing signal;
a face matching step of comparing the RGB image with an RGB template of a pre-stored face template after receiving the stereoscopic reference pass signal, and generating a face recognition pass signal if they match within an error limit;
Body temperature verification step of confirming whether the body temperature is detected from the subject of the 4-color sensor camera
is made, including
The temperature verification step is
Among the wavelengths of the IR image, if the intensity of the wavelength corresponding to the body temperature is greater than or equal to the body temperature reference value, and the wavelength shows a local maximum waveform in the range of 8 to 14 μm, a body temperature standard passing signal is generated.
A face recognition method for confirming the presence of a living body, characterized in that

Images