KR20200054643A - Method and apparatus for enhancing security of fingerprint recognition system - Google Patents

Abstract

The present invention is to prevent the theft of unique identification information to others by replacing a corresponding fingerprint image with a dummy image or by modulating the corresponding fingerprint image through image processing to prevent unique identification information of a person from being detected, when the fingerprint image of the person is included.

Description

System for improving security of fingerprint recognition system and fingerprint recognition method using the same {METHOD AND APPARATUS FOR ENHANCING SECURITY OF FINGERPRINT RECOGNITION SYSTEM}

According to the present invention, when a photograph of a person includes a fingerprint image, the fingerprint image is replaced with a dummy image or by modulating the fingerprint image through image processing, so that unique identification information of the person in the photo is not detected, and thus personal identification to others. This is to prevent the theft of information.

With the popularization of portable cameras and smartphones capable of taking pictures, photos of individuals taken in daily life have been widely distributed on the Internet. Meanwhile, with the improvement of image sensors used in portable cameras and smartphone cameras and the high performance of memory, the resolution of photographs that can be taken through portable cameras and smartphones is also increasing dramatically.

On the other hand, as a method of personal authentication using a mobile device and personal authentication such as a bank, personal authentication methods using biometric information such as fingerprints of pupils are gradually becoming common. Since the fingerprint of a human finger has a unique pattern for each individual, it is possible for the electronic device to memorize the pattern of the fingerprint of the user's finger and use it conveniently as a personal authentication means.

Fingerprint recognition is particularly popular because it is particularly convenient for users without having to remember their password. However, despite these advantages, the pattern of the fingerprint has a disadvantage that it cannot be used for a lifetime once exposed.

The use of a fingerprint recognition method as a personal authentication method is particularly vulnerable to the improvement of resolution of recent electronic devices. For example, as the resolution of a smartphone or a portable camera has recently increased, a photograph of a person taken with a general smartphone or portable camera is printed on a laser printer and reflected on the fingerprint recognition device, without using expensive special equipment. As a result, it may be possible to hack the fingerprint recognition device to recognize the fingerprint of the person. This problem is similar to the fingerprint pattern, and there is a possibility that the fingerprint pattern may be leaked by enlarging a photo after taking a finger with a camera having a high resolution.

Therefore, when the inventor photographs an image of a person, if the image includes a fingerprint image or a fingerprint image, the image is replaced with a dummy image that does not contain personal identification information or the image is modified or stored or distributed. Realizing that there is a need to do this, the present invention has been devised.

The present invention has been invented under the above-described spirit, and when a photograph of a person includes a fingerprint image, the fingerprint image is replaced with a dummy image, or the fingerprint image is modulated through image processing to obtain unique identification information of the person in the photo. By preventing detection, personal identification information is not stolen from others.

According to an aspect of the present invention for achieving the above object, a search unit for searching a fingerprint image area from a picture of a person; An image modification unit for modulating the fingerprint image or replacing it with a dummy image; An image replacement unit that replaces the fingerprint image deformed by the image modification unit in the fingerprint image area of the original portrait photograph; It characterized in that it comprises a.

In addition, according to another aspect of the present invention, a search step of searching a fingerprint image area in a picture of a person; An image modification step of modulating the fingerprint image or replacing it with a dummy image; There is provided an image capturing method comprising an image replacement step of replacing the fingerprint image modified by the image modification step with the fingerprint image area of the original portrait.

According to the present invention, when a photograph of a person includes a fingerprint image, the fingerprint image is replaced with a dummy image or by modulating the fingerprint image through image processing to prevent detection of unique identification information of the person in the photo to others It has the effect of preventing personally identifiable information from being stolen.

1 is a view for explaining a fingerprint recognition system system 100 according to the present invention.
FIG. 2 is a block diagram showing components related to the implementation of the present invention, among the internal components of an electronic device equipped with the fingerprint recognition system system 100 of the present invention, for example, a portable camera 110.
3 is a flowchart illustrating an operation of the fingerprint recognition system module 270 when an image of a person is photographed through a lens.
4 is a view for explaining a method of preventing exposure of personal information of a target person included in a fingerprint image by dividing the photographed fingerprint image into a plurality of image areas and randomly relocating the image.
FIG. 5 is a diagram illustrating a result after dividing the fingerprint image shown in FIG. 4 into a plurality of image regions 430-1 to 430 to 8 and relocating them randomly.
FIG. 6 is a diagram for explaining another method of preventing identification information of a photographing subject from being detected by converting a photographed fingerprint image.
7 is a view for explaining another embodiment of the fingerprint recognition system module.
FIG. 8 shows a screen of the smartphone 800 that determines that fingerprint information is included in the captured image as a result of executing the hacking prevention module 700 of FIG. 7 and displays a warning message (S810) to the user. will be.
FIG. 9 is a view for explaining an example of a method of replacing a fingerprint image with a dummy image stored in advance according to the method of FIG. 3.
10 is a flowchart of applying the concept of the hacking prevention module of the present invention to preventing fingerprint information hacking.

Hereinafter, the present invention will be described in detail so that those skilled in the art can easily understand and reproduce it through preferred embodiments described with reference to the accompanying drawings.

In the description of the present invention, when it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the subject matter of embodiments of the present invention, detailed descriptions thereof will be omitted.

Terms used throughout the specification of the present invention are terms defined in consideration of functions in the embodiments of the present invention, and may be sufficiently modified according to a user's or operator's intention, customs, etc. It should be made on the basis of the contents.

Fingerprint recognition System system

1 is a view for explaining a fingerprint recognition system system 100 according to the present invention. In FIG. 1, the portable camera 110 is a camera equipped with the present fingerprint recognition system system 100. When the portable camera 110 photographs the person 120, the fingerprint recognition system system 100 operates. In FIG. 1, although the portable camera 100 is illustrated as an object to which the fingerprint recognition system system 100 is implemented, all electronic devices having a function of taking a picture of a person such as a smart phone, a PDA, and a laptop are fingerprint recognition of the present invention. It is possible to have a system 100.

FIG. 2 is a block diagram showing components related to the implementation of the present invention, among the internal components of an electronic device equipped with the fingerprint recognition system system 100 of the present invention, for example, a portable camera 110.

First, a user of the portable camera 110 takes a picture of a person to be photographed through the lens 210. The image of the photographed person is recognized by the image sensor 220, converted into digital data, and processed by the microprocessor 230. The present invention presupposes that the photographed person's image is a still image, but the present invention is equally applicable to videos without departing from its basic spirit. The microprocessor 230 receives processing instructions from a nonvolatile memory 260 such as a read only memory (ROM) or flash memory, and processes and processes digital data transmitted from the image sensor 220 based on these instructions. After storing the digital data on the memory card 240, the user can later view the captured Mars through a display such as an LCD or move it to another electronic device such as a laptop, USB, desktop, or smartphone. . As shown in FIG. 2, the non-volatile memory 260 includes a fingerprint recognition system module 270 according to the present invention.

3 is a flowchart illustrating an operation of the fingerprint recognition system module 270 when an image of a person is photographed through a lens.

First, the user photographs an image of the subject recognized through the lens 210 (S310). Next, the microprocessor 230 determines whether the corresponding image includes an image of a person (S320). If it is determined that the image of the person is not included, the fingerprint recognition system module 270 proceeds to step S380 to end the module.

If it is determined that the image of the person is included, the module 270 proceeds to step S330 to determine whether the distance from the person is within a predetermined distance (S330). As a method of determining the distance from the person, various methods such as a method of determining the size of the person's face or using a focal length to the person can be used. If it is determined that the distance from the person is greater than a predetermined distance, the module 270 ends the corresponding process (S380).

If it is determined that the distance to the person is within a predetermined distance, the module 270 searches the fingerprint image area of the person in the captured image (S340). As a method of searching the fingerprint image area, a conventional method known to those skilled in the art may be used, such as searching a fingerprint image by image processing a finger image of a person and extracting feature values for a hand.

After searching for the fingerprint image, the fingerprint image is stored in the memory (S350). Next, the microprocessor 230 replaces the stored fingerprint image with a dummy image previously stored in the nonvolatile memory 260 or the memory card 240 (S360). The dummy image is an image that is disposed in the finger region of the person's image by replacing the fingerprint image of the person. The dummy image is an image having a pattern similar to a fingerprint and not including personal identification information or a virtual identification pattern.

After the above-described procedure, the stored fingerprint image is replaced with a dummy image, and the fingerprint recognition system module 270 is terminated (S370).

In FIG. 3, a method of preventing exposure of personal identification information by replacing a fingerprint image of a photographed person with a pre-stored dummy image has been described, but instead of pre-storing a dummy image to replace the fingerprint, the fingerprint of the person photographed by the user It is also possible to prevent exposure of the unique identification information of the person in the picture by modulating the pattern of the fingerprint by modulating the fingerprint image by properly processing the image. Hereinafter, a method of hiding a pattern of a fingerprint by modulating the fingerprint image will be described.

FIG. 4 shows a method of changing a fingerprint pattern included in a fingerprint image by dividing the photographed fingerprint image into a plurality of image regions and randomly relocating the fingerprint pattern, thereby preventing others from detecting a fingerprint pattern of a person in a photo using only the photographic image. It is a drawing for explaining. In FIG. 4, the person's hand 405 includes a finger 410 and a fingerprint 420. Since the information for identifying the individual is included in the fingerprint 420, in the present invention, a method of preventing detection of personal information by dividing the fingerprint into a plurality of image areas 430-1 to 430 to 8 and relocating them at random To suggest.

FIG. 5 is a diagram illustrating a result of dividing the fingerprint image shown in FIG. 4 into a plurality of image regions 430-1 to 430 to 8 and then randomly relocating them. 4 and 5 illustrate a method of dividing a fingerprint image into 8 image areas and rearranging them, but the image area exceeds 10 areas, for example, 16, 32, 64, or 100 areas Of course, it is also possible to divide and rearrange the data into a larger number of areas, for example, 128 or 256.

FIG. 6 is a diagram for explaining another method of preventing identification information of a photographing subject from being detected by converting a photographed fingerprint image. First, as shown in FIG. 6A, the user photographs a person who is a subject using a smartphone 601 (previously described as having a fingerprint recognition system module in a portable camera, but the present invention is illustrated in FIG. 6). As described above, it can be implemented not only in a portable camera, but also in any electronic device equipped with a smartphone or other photo-taking function). FIG. 6B is a view showing a finger portion extracted from the photographed subject. Next, the fingerprint recognition system module, as shown in FIG. 6C, removes the rest of the finger and extracts only the image 603 of the remaining fingerprint part. Next, as shown in the figure, the fingerprint image 603 is converted into two-dimensional data of the thickness W of the fingerprint and the length of 0 to 2 ð (FIG. 6D), and then Fourier transformed (FIG. 6E). A high frequency component is removed from the Fourier transformed data to obtain a transformed data set as shown in FIG. 6F. Next, the complex data thus obtained can be subjected to Fourier inverse transformation to obtain a fingerprint image from which high-frequency components have been removed, and then converted to circular data to obtain a fingerprint image from which detection of personal identification information is prevented. By doing in this way, it is possible to obtain a fingerprint image in which personal identification information is not detected, although there is no significant difference in appearance from the pupil image of the person who is the subject.

Fingerprint hacking warning system

7 is a view for explaining another embodiment of the fingerprint recognition system module. Smartphones that authenticate users through fingerprint recognition are already commercially available (for example, Galaxy Note 7, a smartphone manufactured by Samsung Electronics). On the other hand, with the trend of social network services such as Facebook, it is also common for smartphone users to take a picture of themselves and post them on the Internet.

The flowchart of the fingerprint recognition system module 700 shown in FIG. 7 is the same as the operation of the fingerprint recognition system module 300 shown in FIG. 3 until the step of searching for a fingerprint image area (S740).

After retrieving the fingerprint image, the fingerprint recognition system module 700 illustrated in FIG. 7 stores the corresponding fingerprint image in the memory (S750). Next, extraction of fingerprint identification information from the corresponding fingerprint image is attempted. If it is not possible to extract personal information through the fingerprint recognition extraction algorithm built into the smartphone, the module 700 ends (S790). If it is possible to extract personal information through the fingerprint recognition extraction algorithm embedded in the smartphone, the hacking prevention module 700 extracts personal information from the fingerprint image and compares it with personal identification information previously stored in the user's smartphone. do. If it is determined that the personal identification information stored in advance matches or is significantly related to (S770), the process proceeds to S780, whereby the photographed image includes fingerprint information of the subject, thereby reducing the risk of personal information leakage. Displays a warning screen indicating that there is, and allows the user to select whether to delete the corresponding image.

Next, the hacking prevention module 700 ends.

FIG. 8 shows a screen of the smartphone 800 that determines that fingerprint information is included in the captured image as a result of executing the hacking prevention module 700 of FIG. 7 and displays a warning message (S810) to the user. will be. When the user of the smartphone clicks the "delete fingerprint information" button 820 in response to the warning message, the hacking prevention module 700 replaces the stored fingerprint image of FIG. 3 with a dummy image (S360) or a secured fingerprint. By performing image processing on the image, it is possible to operate so that personal identification information included in the fingerprint image is not detected.

FIG. 9 is a view for explaining an example of a method of replacing a fingerprint image with a dummy image previously stored according to the method of FIG. 3. In FIG. 9A, if it is determined that the fingerprint image 901 is in a photograph of a person who is the subject, the hacking prevention module 270 extracts the donut-shaped fingerprint image 901 and the thickness W of the fingerprint, length 0-2 ð This is converted into the two-dimensional fingerprint data 902. Next, the hack prevention module 270 compares the fingerprint data 902 with various dummy fingerprint image data 903-1 to 903-4 stored in the nonvolatile memory 260 or the memory card 240, The dummy fingerprint data 903-4 closest to the fingerprint data 902 is selected. The dummy fingerprint image data 903-1 to 903-4 are fingerprint data to be used in place of the fingerprint data 902 of the person who is the subject. Next, a photograph of a person who is the subject by converting the fingerprint image data 903-4 (FIG. 9D) selected from the plurality of dummy fingerprint image data 903-1 to 903-4 into a donut-shaped fingerprint image 902 '. The process is terminated after replacement of the inner fingerprint image (FIG. 9E). The fingerprint data 902 'obtained as described above is similar to the person's original fingerprint image 901, but does not include personal identification information of the person. Although four dummy fingerprint image data 903-1 to 903-4 are described in this embodiment, for a natural image, a larger number of fingerprint image data can be provided. For example, the fingerprint image data may be provided in any number of 10 or more and 100 or less, for example, 16, 32, or 64. In addition, the fingerprint image data may be provided in any number of one hundred or more and one thousand or less.

Fingerprint hacking prevention system

10 is a flowchart of applying the concept of the hacking prevention module of the present invention to preventing fingerprint information hacking.

First, the user photographs an image of the subject recognized through the lens 210 (S1010). Next, the microprocessor 230 determines whether the corresponding image includes an image of a person (S1020). If it is determined that the image of the person is not included, the fingerprint recognition system module 270 proceeds to step S1080 to end the module.

If it is determined that the image of the person is included, the module 270 proceeds to step S1030 to determine whether the distance from the person is within a predetermined distance (S1030). As a method of determining the distance from the person, various methods such as a method of determining the size of the person's face or using a focal length to the person can be used. If it is determined that the distance from the person is greater than a predetermined distance, the module 270 ends the corresponding process (S1080).

If it is determined that the distance to the person is within a predetermined distance, the module 270 searches the fingerprint image area of the person in the captured image (S1040). As a method of searching for a fingerprint image area, a conventional method known to those skilled in the art, such as searching for a fingerprint image, by extracting feature values for parts such as a thumb, index finger, fingernail, and palm by image processing a hand image may be used.

After searching for the fingerprint image, the fingerprint image is stored in the memory (S1050). Next, the microprocessor 230 replaces the stored fingerprint image with a dummy image previously stored in the nonvolatile memory 260 or the memory card 240 (S1060). The dummy image is an image that is disposed in the fingerprint area of a finger by replacing the fingerprint image of a person, and is an image having a pattern similar to a fingerprint and not including personal identification information or a virtual identification pattern.

After the above procedure, the stored fingerprint image is replaced with a dummy image, and the hacking prevention module 270 is terminated (S1070).

In FIG. 3, a method of preventing detection of personal identification information by replacing a fingerprint image of a photographed person with a pre-stored dummy image has been described, but instead of storing a dummy image to be substituted in advance, the fingerprint of the person photographed by the user It is also possible to prevent detection of personal information of the target person by appropriately processing the image.

As described above, the present invention can improve the degree of protection of personal information by preventing the leakage of biometric information, for example, fingerprint information, etc., of a subject of a smartphone or portable camera, and thus the present invention presented above Can achieve its purpose.

Although the present invention has been described with reference to preferred embodiments referred to by the accompanying drawings, it is apparent from this description that various modifications are possible without departing from the scope of the present invention within the scope covered by the claims below. .

The present invention can be used industrially in the field of electronic devices with a photo-taking function, such as portable cameras, smart phones, personal computers, and application technologies thereof.

210: lens
220: image sensor
230: microprocessor
240: memory card
250: LCD / display
260: non-volatile memory
270: hack prevention module

Claims (2)

A search unit for searching a fingerprint image area among photographs of the person;
An image modification unit for modulating the fingerprint image area or replacing it with a dummy image;
An image replacement unit replacing the fingerprint image modified by the image modification unit with a fingerprint image area of an original portrait; Imaging apparatus comprising a.
A step of searching for a fingerprint image area among pictures of the person;
An image modification step of modulating the fingerprint image area or replacing it with a dummy image;
And an image replacement step of replacing the fingerprint image modified by the image modification unit with a fingerprint image area of an original portrait.

Images