KR102349424B1 - Personal identification apparatus using face recongintion - Google Patents

Abstract

INDUSTRIAL APPLICABILITY The present invention is applied to the field of identifying an individual using face recognition, and particularly relates to a technique for discriminating a fake face. An image frame is generated by synthesizing the odd field and the even field captured by the interlaced mode camera, and the image frame is compared with the image frame of the progressive mode camera photographing the same subject at the same moment, and based on the difference, a fake identify the face Additionally, in the images captured by the two cameras, the difference between the images according to the characteristics of the cameras may be corrected and normalized.

Description

Personal identification apparatus using face recongintion

The present invention is applied to the field of personal identification using face recognition, and particularly relates to a technique for discriminating a fake face.

A face spoofing attack usually refers to an attempt to deceive a facial recognition system by replacing a fake face, such as a photo, video recording, or 3D mask, with the real face of an authorized person. Liveness detection is performed to solve this face spoofing attack. That is, it recognizes whether the corresponding face is alive or a fraudulent copy.

In general, in the case of photos, fake faces can be easily recognized using technologies such as blink recognition, but when a face spoofing attack is attempted using video, blink recognition technology cannot be used and a thermal imager can be installed. The problem of cost arises.

Visage Technologies has developed a technology that illuminates a face with a light emitting diode and checks liveness based on reflectance. A fake face can be recognized by examining the way the screen's light is reflected by the camera. However, this method of inspecting the light reflection of light emitting diodes can be used in devices in which a camera is placed close to the screen, such as a smartphone or tablet, but does not work well in a security camera that shoots from a relatively long distance.

The proposed invention aims to discriminate a fake image based on a moving picture from an image taken from a relatively long distance. Furthermore, the proposed invention aims to discriminate a fake image by a moving image at a low cost. Furthermore, the proposed invention aims to discriminate a fake image by a moving picture in real time in a security camera.

An apparatus for personal identification using face recognition according to an aspect utilizes an interlaced mode camera to determine a fake image based on a moving picture. In the case of a moving picture, the frame frequency, line frequency, and/or pixel frequency do not match with the camera, and accordingly, there is a large difference in the image between the odd field and the even field compared to the progressive mode camera. According to an aspect, one image frame is generated by synthesizing the odd field and the even field photographed by the interlaced mode camera, and the image frame is compared with the image frame of the progressive mode camera photographing the same subject at the same moment. A fake face is identified based on the difference.

According to an additional aspect, in images captured by two cameras, a difference in images according to camera characteristics may be corrected and normalized. Accordingly, the reliability of fake face identification may be further improved.

According to an additional aspect, normalization of a camera characteristic or image discrepancy may be performed based on a histogram of the image.

Low-cost interlaced mode cameras used in the past alternately output even and odd fields. Accordingly, in the case of a fake face displayed on a tablet, smartphone, or computer screen, the pixel value of each pixel of the image frame output from the interlaced mode camera is obtained from the progressive mode camera used as a general main camera in the field of video security. Since it is different from the pixel value of the output image frame, it is possible to identify a fake face at a low cost.

Furthermore, according to an embodiment, the odd field and even field output from the interlaced mode camera are synthesized into one progressive frame, and the histogram characteristics of the synthesized first image frame and the histogram of the second image frame output from the progressive mode camera are synthesized. Normalize the characteristics. Therefore, the difference between the image frames between the case of the real face and the case of a fake face becomes larger, and the reliability of the discrimination can be increased.

1 is a block diagram illustrating a configuration of a personal identification apparatus according to an embodiment.
2 is a block diagram illustrating a configuration of a personal identification device according to another embodiment.
3 is a block diagram illustrating a configuration of a personal identification device according to another embodiment.
4 is a graph for explaining the operation of the image normalizer 310 according to an embodiment.
5 is a flowchart illustrating a configuration of a method for personal identification according to an embodiment.

The foregoing and additional aspects are embodied through the embodiments described with reference to the accompanying drawings. It is understood that various combinations of elements in the embodiments are possible within the embodiments as long as there is no contradiction between them or other mentions.

1 is a block diagram illustrating a configuration of a personal identification apparatus according to an embodiment. As shown, the personal identification apparatus according to an embodiment includes a progressive mode camera 110 , an interlaced mode camera 120 , an image comparison unit 300 , and a fake face determination unit 700 .

The interlaced mode camera 120 is a low-cost analog camera that has been widely used in the past. The interlaced mode camera alternately outputs an even field and an odd field. Here, the even field is composed of even lines among horizontal scan lines, and the odd field is composed of odd lines among horizontal scan lines. Since the scan frequency of a moving picture displayed on a smartphone screen or the like does not match that of the camera, it may be recorded in a state in which it is not displayed at all or only partially displayed in a specific scanning line or field among even-numbered fields and odd-numbered fields.

The progressive mode camera 110 is a camera generally used in the field of image security. The progressive image frame output by the progressive mode camera 110 is transmitted to the face recognition unit 500 of the personal identification device using face recognition. In an embodiment, the face recognition unit 500 extracts and encodes a face region from an input image frame, compares the encoded vector with reference vectors stored in a database, finds a vector that best matches, and the corresponding personal identification information to output Various types of these face recognition algorithms are known, and their reliability is rapidly improving thanks to the recent development of artificial intelligence technology.

The image comparison unit 300 calculates a difference between the image frame output from the interlaced mode camera 120 and the image frame output from the progressive mode camera 120 . For example, the image frame output from the interlaced mode camera 120 may be a frame generated by sequentially alternating horizontal scan lines of odd fields and even fields. For example, an interlaced mode image frame can be converted into a progressive mode image frame by writing an odd field in the odd line area and writing an even field in the even line area of the frame memory. Accordingly, the screen display unit (not shown) reads the image frame in the progressive mode stored in the frame memory and outputs it to the screen. A so-called de-interlacing technique itself of processing an input interlaced image frame, converting it into a progressive image frame, and displaying it on a screen is known.

In the case of a fake face displayed on the screen of a tablet, smartphone, computer, or the like, the screen displaying the fake face generally uses a progressive scan. Also, the display scan period of this screen is different from the scan period of the interlaced mode camera 120 . Also, in the interlaced mode camera 120 , there is a refresh delay between the even field and the odd field that does not exist on the screen displaying the fake face. Accordingly, the fake face image frame output from the interlaced mode camera 120 has relatively many artifacts compared to the fake face image frame output from the progressive mode camera 110 .

In order to calculate the difference between the real face image and the fake face image, in the proposed embodiment, the image comparison unit 200 includes an image frame output from the interlaced mode camera 120 and an image output from the progressive mode camera 110 . Calculates the magnitude of the difference between frames. For example, the image comparison unit 200 may perform a subtraction operation between corresponding pixel values in two corresponding image frames output from the respective cameras 110 and 120 . In the case of a real face, the difference between the image frames output by the image comparison unit 200, for example, a pixel value difference, is relatively small, and in the case of a fake face, the difference between the image frames output by the image comparison unit 200 For example, the pixel value difference is relatively large. The image difference may be calculated by calculating the difference for each pixel corresponding to the image frames of the two cameras converted to have the same resolution. As another example, the image difference may be calculated by calculating an average value of absolute values of differences for respective pixels of the image frames of the two cameras converted to have the same resolution. As another example, the image difference may be calculated by calculating the root mean square of the difference for each pixel of the image frames of the two cameras converted to have the same resolution.

The fake face determination unit 700 compares the output of the image comparison unit 300 with a reference value to determine whether a fake face is present. In an embodiment, the fake face determining unit 700 determines a fake face if the size of the output of the image comparison unit 200 is greater than or equal to a certain value, and determines a fake face if the size of the output of the image comparison unit 200 is less than or equal to a predetermined value. You can judge by the face. The reference value for judgment may be determined based on a required reliability by obtaining a statistical distribution of image differences for a plurality of images. The fake face determination unit 700 may stop the face recognition operation of the face recognition unit 500 and output a warning message when the input image is a fraudulent image.

2 is a block diagram illustrating a configuration of a personal identification device according to another embodiment. Components similar to the embodiment of FIG. 1 are referenced by the same reference numerals. As shown, the personal identification apparatus according to an embodiment includes a progressive mode camera 110 , an interlaced mode camera 120 , an image comparison unit 300 , and a fake face determination unit 700 . The progressive mode camera 110 , the interlaced mode camera 120 , and the face recognition unit 500 are similar to the corresponding components described with reference to FIG. 1 .

According to an additional aspect, the image comparing unit 300 may include a frame combining unit 330 , an image normalizing unit 310 , and a frame difference calculating unit 350 . The frame combining unit 330 synthesizes the odd field and the even field output from the interlaced mode camera 120 into a first image frame, which is a progressive frame. Odd scan lines of the generated progressive frame are from odd fields, and even scan lines are from even fields. Such an interlaced mode-progressive mode conversion technique is a technique widely known in analog television in the past. The image normalization unit 310 normalizes the image of the first image frame output from the frame combining unit 330 and the image of the second image frame output from the progressive mode camera 110 and outputs the normalized image. Since the images are taken from two different cameras, the images have different image characteristics such as luminance, brightness, and histogram. The image normalizer 310 may transform the two images so that the image characteristics of the two images match each other. As another example, one of the two images, in particular, the second image frame output from the progressive mode camera 110 may be converted to match the first image frame. When the image of the first image frame output from the frame combining unit 330 is converted, the difference between the odd field and the even field is reflected and an error may occur. Therefore, the second image frame output from the progressive mode camera 110 is converted. It is preferable to do The frame difference calculator 350 calculates a pixel-by-pixel difference between the normalized first image frame and the normalized second image frame. The image difference may be calculated by calculating the difference for each pixel corresponding to the image frames of the two cameras converted to have the same resolution. As another example, the image difference may be calculated by calculating an average value of absolute values of differences for each pixel of the image frames of the two cameras converted to have the same resolution. As another example, the image difference may be calculated by calculating the root mean square of the difference for each pixel of the image frames of the two cameras converted to have the same resolution. The fake face determination unit 700 compares the output of the image comparison unit 200 with a reference value to determine whether a fake face is present. The reference value may be determined based on a required reliability by obtaining a statistical distribution of image differences for a plurality of images.

3 is a block diagram illustrating a configuration of a personal identification apparatus according to another embodiment. Components similar to the embodiment of FIG. 2 are referenced by the same reference numerals. As shown, the personal identification device according to an embodiment includes a progressive mode camera 110 , an interlaced mode camera 120 , a face recognition unit 500 , a frame synthesis unit 330 , and an image normalization unit 310 , a frame difference calculating unit 350 , and a fake face determining unit 700 . The progressive mode camera 110, the interlaced mode camera 120, the face recognition unit 500, the frame combining unit 330, the frame difference calculating unit 350, and the fake face determining unit 700 are shown in Fig. It is similar to the corresponding component described with reference to 2 .

According to an aspect, the image normalization unit 310 may include a first image characteristic detection unit 315 , a second image characteristic detection unit 313 , and an image conversion unit 311 . In the illustrated embodiment, the first image characteristic detector 315 detects the image characteristic of the first image frame. The second image characteristic detector 313 detects an image characteristic of the second image frame. The image converter 311 converts the image characteristics of the first image frame and the second image frame to substantially match each other by using the image characteristics of the first image frame and the image characteristics of the second image frame.

The image characteristic may be, for example, an average and standard deviation of luminance values. As another example, the image characteristic may be a histogram of luminance values. In the image processing field, a histogram refers to a graph showing the number of pixels having a brightness level for each brightness level of an image. By looking at the histogram characteristics of the image output from the camera, the output characteristics of the camera can be known. Also, by looking at the histogram characteristics of the image output from the camera, it is possible to know the average brightness or the degree of dispersion of the captured image.

In an embodiment, the image converter 311 obtains the histogram characteristic of at least one of the first image frame and the second image frame by using the detected histogram characteristic of the first image frame and the histogram characteristic of the second image frame. By transforming the image frame so that the histogram characteristics of the two image frames coincide with each other, the characteristics of the two cameras 110 and 120 are matched as much as possible.

According to an embodiment, the image converter 311 converts the second image frame output from the progressive mode camera 110 to match the image characteristics of the first image frame output from the interlaced mode camera 120 . can be performed. Conversely, the image converter 311 may convert the first image frame output from the interlaced mode camera 120 to match the characteristics of the second image frame output from the progressive mode camera 110 . As another example, the image conversion unit 311 may perform image conversion on both the first image frame and the second image frame to match the image characteristics of the two cameras 110 and 120 as much as possible.

4 is a graph for explaining the operation of the image normalizer 310 according to an embodiment. In the drawing, a solid line is an exemplary histogram of the first image frame, and a dotted line is an exemplary histogram of the second image frame. Although the same subject is photographed, the first image frame and the second image frame show different histograms. The cause is mainly due to the difference in the characteristics of the two cameras, that is, the sensitivity of the image sensor, the difference in lighting, and the optical system. Histogram transformation is a known technique in image processing and corresponds to the mapping of pixel values.

In the illustrated embodiment, the first image characteristic detector 315 calculates a histogram by counting the distribution of luminance values of the first image frame. The second image characteristic detector 313 calculates a histogram by counting the distribution of luminance values of the third image frame. The first image characteristic detector 315 and the second image characteristic detector 313 do not necessarily calculate histograms for all luminance values. For example, only 16 of the 256 luminance values may be counted by counting pixels having a corresponding luminance value to calculate a rough characteristic of the histogram. The remaining luminance values may be interpolated and mapped based on these values. The image converter 311 matches the image characteristics of the two image frames, and in the illustrated embodiment, the histogram characteristics through luminance value mapping. This mapping can be performed at high speed by using the mapping table to address the mapping table with pixel values. As shown in FIG. 4 , a certain luminance value is amplified and a certain luminance value is attenuated to match the histogram characteristics of two image frames.

5 is a flowchart illustrating a configuration of a method for personal identification according to an embodiment. As shown, the personal identification method according to an embodiment includes a frame synthesis step 830 , an image normalization step 850 , a frame difference calculation step 870 , and a fake face identification step 890 . In the frame synthesizing step 830, the odd field and the even field output from the interlaced mode camera are received (step 810), and they are combined into a first image frame, which is a single progressive frame. In the image normalization step 850, the image of the first image frame and the image of the second image frame output from the progressive mode camera are normalized and output. In the frame difference calculation step 870 , a pixel-by-pixel difference between the normalized first image frame and the normalized second image frame is calculated. In the fake face determination step 890 , the difference between the image frames is compared with a reference value to determine whether a fake face is present. For these, since corresponding configurations have been described with reference to FIGS. 1 and 2 , a detailed description thereof will be omitted.

According to an additional aspect, the image normalization operation 850 may include a first image characteristic detection operation 851 , a second image characteristic detection operation 853 , and an image conversion operation 855 . In the first image characteristic detection step 851, the image characteristic of the first image frame is detected. In the second image characteristic detection step 853, the image characteristic of the second image frame is detected. In the image conversion operation 855 , the image characteristics of the first image frame and the image characteristics of the second image frame are converted to substantially match each other by using the image characteristics of the first image frame and the image characteristics of the second image frame. For these, since corresponding configurations have been described with reference to FIG. 3, detailed descriptions thereof will be omitted. According to an additional aspect, the image characteristic may include any one of an average of luminance values, a standard deviation, and a histogram.

Although the present invention has been described above with reference to the accompanying drawings, the present invention is not limited thereto, and it should be construed to encompass various modifications that can be apparent from those skilled in the art. The claims are intended to cover such modifications.

110: progressive mode camera 120: interlaced mode camera
310: image normalization unit 311: image conversion unit
313: second image characteristic detection unit 315: first image characteristic detection unit
330: frame synthesizing unit 350: frame difference calculating unit
500: face recognition unit 700: fake face discrimination unit

Claims (7)

an interlaced mode camera;
a progressive mode camera;
an image comparator for calculating a difference between an image frame output from the interlaced mode camera and an image frame output from the progressive mode camera; and
a fake face determination unit that compares the output of the image comparison unit with a reference value to determine whether a fake face is present;
including,
The image comparison unit:
a frame synthesizing unit for synthesizing the odd field and the even field output from the interlaced mode camera into a first image frame, which is one progressive frame;
an image normalizer for normalizing the image of the first image frame output from the frame combining unit and the image of the second image frame output from the progressive mode camera and outputting the normalized image; and
and a frame difference calculator for calculating a pixel-by-pixel difference between the normalized first image frame and the normalized second image frame;
The image normalization unit:
a first image characteristic detector for detecting image characteristics of the first image frame;
a second image characteristic detection unit for detecting image characteristics of the second image frame;
an image conversion unit;
The first image characteristic detector calculates a first histogram by counting the distribution of luminance values of the first image frame,
The second image characteristic detector calculates a second histogram by counting the distribution of luminance values of the second image frame,
wherein the image converter converts a luminance value of the second image frame so that the second histogram is the same as the first histogram;
A personal identification device that uses facial recognition.
delete delete delete a frame synthesizing step of synthesizing an odd field and an even field output from the interlaced mode camera into a first image frame, which is one progressive frame;
an image normalization step of normalizing the image of the first image frame and the image of the second image frame output from the progressive mode camera and outputting the normalized image; and
a frame difference calculating step of calculating a pixel-by-pixel difference between the normalized first image frame and the normalized second image frame; and
a fake face discrimination step of determining whether a face is a fake face by comparing the difference between the image frames with a reference value;
including,
The image normalization step is
calculating a first histogram by counting the distribution of luminance values of the first image frame;
calculating a second histogram by counting the distribution of luminance values of a second image frame;
converting the luminance value of the second image frame so that the second histogram is the same as the first histogram
containing,
A method of personal identification using facial recognition.
delete delete

Images