KR102079952B1 - Method of managing access using face recognition and apparatus using the same - Google Patents

Abstract

The present invention relates to a face recognition access method, which comprises the steps of: scanning an identification card; capturing a face image included in the scanned identification card; photographing a user with a camera; comparing the captured image of the user with the captured face image; when the captured user image is identical to the captured face image, mapping and storing a personal identification number of the identification card and the captured user image; and verifying the identities of the faces of users photographed through a real-time imaging device by using the stored captured user image. The present invention compares the face image of the ID card of the entered user with the captured image to allow the user to enter and exit only when the two images are identical to each other, and maps the captured image with the personal identification number to easily confirm the identity of the user who entered a building by using the user face photographed through the real-time imaging device.

Description

Method of managing access using face recognition and apparatus using the same}

The present invention relates to a face recognition access method, and more specifically, it compares a face image and a shot image of a user accessing a store or airport, such as a PC room, a comic book room, or a convenience store, and compares the shot image to enter and exit. By mapping with a personal identification number, it relates to a face recognition access method and device that can easily identify a user's identity in a store or airport with a user's face photographed through a real-time imaging device.

Existing face recognition systems take a picture of the face of an ID card holder, compare whether the ID number entered by the holder is the same as the ID number on the ID card, and compare the photo of the ID card and the holder to determine whether they are the same person or not. If it is judged that it is the same, the photographed face and the social security number are mapped and stored.

However, considering the subjective judgment by the manager and the lack of time to judge when congested, etc., even if the ID card and the other person are different people, it is sometimes misrecognized as the same person. In this case, the face recognition system registers the face of the thief who has obtained the lost identification card and maps and stores the resident number on the identification card, and then the face recognition system may mistake the act of the thief as being the actual registrant of the resident number. do.

In the U.S., a lost ID card was used at an Apple store, and Apple's face recognition system misconducted the identity of the person and the person's face and was mistaken for a thief, and filed a trillion won lawsuit against Apple.

Therefore, in the case of confirming the identity by face recognition, it is necessary to objectively check whether the face included in the identification card matches the actual face. In addition, it is necessary to prevent leakage of biometric information, such as input fingerprints, by being encrypted.

The Korean Registered Patent (10-1480103) relates to an "access control device and method using face recognition", which relates to a method of allowing access by internal workers who have previously registered face images, or an identification card used when registering a user It is not disclosed to compare the photograph and the face of the photograph to register only the legitimate user.

Korean Patent Publication (10-2017-0123950) relates to a method and system for user authentication by comparing a feature point of a user's face photographed with a user's face feature point stored in a database. However, it is only about a method of comparing the captured image with the stored image, and it cannot be used to confirm that the ID card holder is a party holder.

Therefore, the first problem to be solved by the present invention is to compare the face image of the user who enters the store or the airport and the captured image, and only when the image matches and to access, and map the captured image with a personal identification number, thereby real-time imaging device. It is to provide a face recognition access method that makes it easy to confirm the identity of a user in a store or airport with the user's face taken through.

The second problem to be solved by the present invention is to identify a user by using fingerprint recognition in addition to face recognition, but does not store fingerprint information in the user terminal and the face recognition access device, and compares two encrypted fingerprint information in the server. By doing so, it is to provide a face recognition access device capable of more accurately identifying a user and reducing a possibility of leaking biometric information.

It is also to provide a recording medium readable by a computer on which a program for executing the above-described method is executed on a computer.

The present invention, to achieve the first task, scanning an identification card; Capturing a face image included in the scanned identification card; Photographing a user with a camera; Comparing the captured image of the user with the captured face image; Mapping and storing the personal identification number of the identification card and the photographed user photographed image when the comparison result is the same; And it provides a face recognition access method comprising the step of verifying the identity of the face of the user photographed through the real-time imaging device by using the stored user captured image.

According to an embodiment of the present invention, when it is confirmed that the user is not a valid ID card holder or a suspected identity card is forged, requesting fingerprint input from the user's terminal; Hash encrypting the fingerprint included in the identification card; Transmitting the encrypted ID fingerprint to a server; And receiving a fingerprint comparison result from the server, wherein the server receives a hash-encrypted input fingerprint from the user's terminal, and compares the received input fingerprint with the encrypted ID fingerprint.

In addition, the step of requesting a specific change of the face to the user; And it is preferable to further include the step of confirming whether the specific change of the requested face and the change of the face photographed by the camera match.

At this time, it is preferable that the identification information included in the scanned ID image is masked and stored.

According to another embodiment of the present invention, the captured face image further includes a step of transmitting to a server and storing the captured image, and the comparison of the captured image of the captured user with the captured face image is performed by the server. It is preferable that the comparison result performed by the server is transmitted to the face recognition access device.

According to another embodiment of the present invention, the method further includes comparing the photographed image of the user with the face image of a user who has already entered, and a face similar to a face in the user's image captured by the camera has already entered. If it is determined, it is preferable to create at least one of the user's glasses wearing, hairstyle, or clothes, and to distinguish using the additional information when distinguishing users who have already entered.

The present invention, in order to achieve the second problem, a scanning unit for scanning an identification card; A capture unit for capturing a face image included in the scanned identification card; A camera unit for photographing a user with a camera; A comparison unit that compares the captured user's captured image with the captured face image, and maps and stores the personal identification number of the identification card and the captured user's captured image in a storage unit when the comparison result is the same; And it provides a face recognition access device including a control unit for verifying the identity of the face of the user photographed through the real-time imaging device by using the stored user captured image.

According to an embodiment of the present invention, when it is confirmed that the controller is not a valid ID card holder or suspects that the ID card has been forged, a fingerprint input is requested to the user's terminal, and the fingerprint included in the ID card is hash encrypted, and the After transmitting the encrypted ID fingerprint to the server, the fingerprint comparison result is received from the server, wherein the server receives a hash-encrypted input fingerprint from the user's terminal, and the received input fingerprint and the encryption It is desirable to compare the fingerprints of the ID cards.

In addition, the control unit may request a specific change of the face from the user, and the comparison unit may check whether the specific change of the requested face and the change in the face captured by the camera match. At this time, it is preferable that the identification information included in the scanned ID image is masked and stored.

According to another embodiment of the present invention, the captured face image is transmitted to a server, the comparison of the captured user's captured image with the captured face image is performed by the server, and the comparison result performed by the server It is preferable to be transmitted to the face recognition access device.

According to another embodiment of the present invention, the comparator compares the photographed image of the user with the face image of a user who has already entered, and a face similar to a face in the user's image captured with the camera portion has already been entered. If it is determined, the face recognition unit generates additional information on at least one of whether or not the user wears glasses, a hairstyle, or clothes, and distinguishes using the additional information when the comparison unit distinguishes users who have already entered It is preferred.

In order to solve the above other technical problems, the present invention provides a computer-readable recording medium recording a program for executing the above-described face recognition access method on a computer.

According to the present invention, by comparing the face image of the user entering the store or the airport and the captured image, and entering and exiting only if they match, and mapping the captured image with a personal identification number, the store is stored as a user's face photographed through a real-time imaging device. It is easy to confirm the identity of a user at Ina Airport.

In addition, according to the present invention, user identification is performed by using fingerprint recognition in addition to face recognition, but by storing fingerprint information in the user terminal and the face recognition access device, the server compares the two encrypted fingerprint information, It is possible to more accurately identify the user and reduce the possibility of leaking biometric information.

Furthermore, according to the present invention, it is possible to extract only the images of the corresponding users by checking the face of the user in the video recorded by the real-time imaging device in the building for the purpose of secure access and unmanned stores.

1 is a block diagram of a face recognition access device according to a preferred embodiment of the present invention.
2 is a flowchart of a face recognition access method according to an exemplary embodiment of the present invention.
3 is a flowchart of a method of entering and exiting a store by receiving additional fingerprints using a user's terminal.

Hereinafter, with reference to the accompanying drawings will be described in detail preferred embodiments that can be easily carried out by the person of ordinary skill in the art. However, these examples are intended to illustrate the present invention in more detail, and it will be apparent to those skilled in the art that the scope of the present invention is not limited thereby.

The configuration of the invention for clarifying the solution to the problem to be solved by the present invention will be described in detail with reference to the accompanying drawings based on preferred embodiments of the present invention, but the same is used in assigning reference numbers to components of the drawings. For components, even if they are on different drawings, the same reference number is assigned, and it is revealed in advance that components of other drawings may be cited when necessary for the description of the drawings. In addition, in the detailed description of the operating principle for the preferred embodiment of the present invention, if it is determined that the detailed description of the known functions or configurations related to the present invention and all other matters may unnecessarily obscure the subject matter of the present invention, The detailed description is omitted.

In addition, in the entire specification, when a part is said to be 'connected' with another part, it is not only 'directly connected', but also 'indirectly connected' with another element in between. Includes. In this specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, "comprises" or "comprising" excludes the presence or addition of one or more other components, steps, operations, or elements other than the components, steps, operations, or elements mentioned. I never do that.

1 is a block diagram of a face recognition access device according to a preferred embodiment of the present invention.

Referring to FIG. 1, the face recognition access device according to the present embodiment includes a scan unit 110, a capture unit 120, a camera unit 130, a face recognition unit 140, a comparison unit 150, and a storage unit ( 160), a control unit 170, a display unit 180, an information notification unit 190, and a real-time imaging device 195.

The scan unit 110 scans the user's identification card. The ID card can be a resident registration card, driver's license, or passport, or a student ID for minors.

The capture unit 120 captures the face image included in the scanned ID. In addition to the face image, the capture unit 120 may capture a fingerprint or personal identification number on the identification card.

The camera unit 130 photographs the entire body, upper body, or face of the user.

The face recognition unit 140 recognizes a face area from an image taken by the camera unit 130.

The comparison unit 150 compares the face image captured by the capture unit 120 with the face region recognized by the face recognition unit 140.

If the comparison result of the comparison unit 150 determines that the face image captured by the capture unit 120 and the face area recognized by the face recognition unit 140 do not match, the comparison result is transmitted to the control unit 170 It outputs that the user does not match the ID through the display unit 180.

When the comparison result of the comparison unit 150 determines that the face image captured by the capture unit 120 and the face area recognized by the face recognition unit 140 are the same person, the storage unit 160 personally identifies the scanned ID. The number and the captured user-photographed image are mapped and stored. When the photographed image is a whole body or an upper body, only the face region may be recognized separately and mapped and stored with the personal identification number.

The comparison unit 150 may extract a plurality of feature points from the face image captured by the capture unit 120 and the face region recognized by the face recognition unit 140, and compare the extracted feature points to determine whether the same face is the same. . The characteristic value of the face refers to a measurement that can characterize the face, such as the size of the ears, the length of the nose, and the distance between both eyes.

It is preferable that the control unit 170 controls the number of feature points extracted by the comparison unit 150 to control the difficulty level for determining whether the face is the same. If the number of feature points is increased, the time to determine whether it is the same is increased, but the accuracy is increased, and when the number of feature points is decreased, the time to determine whether it is the same is reduced, but the accuracy is lowered. It is desirable for the user to adjust the number of feature points according to the situation in the store or the airport.

For example, it would be desirable to increase the number of user feature points for people entering the airport in an epidemic or suspicious country.

The control unit 170 receives the comparison result of the comparison unit 150 and outputs the comparison result through the display unit 180.

The control unit 170 may verify the identity of the faces of users in the store photographed through the real-time imaging device 195 installed in the store. The real-time imaging device 195 is preferably a closed circuit television (CCTV).

By retrieving the same captured image as the user's face taken through the real-time imaging device 195 among the captured images stored in the storage 160, mapping the personal identification number and the user's face taken through the real-time imaging device 195 to identify Can be confirmed.

Through the control of the control unit 170, it is possible to separately manage the minors by separating adults from the minors, such as restricting access for minors after 10 pm and outputting a warning message when purchasing alcohol and tobacco at convenience stores.

The comparison unit 150 and the storage unit 160 may be included in the face recognition access device according to an embodiment of the present invention, or may be included in a separate server 200.

When the comparison unit 150 and the storage unit 160 are included in separate servers 200, sensitive personal information such as a captured image, an ID card captured image, and a personal identification number is not the face recognition access device 100 of each store. Since the server 200 is stored and processed, the possibility of personal information leakage can be reduced.

Under the control of the controller 170, the display unit 180 outputs a request to scan a user ID card, a photographing request, a comparison result of the ID card face image and the photographed image, and a request for a specific change of the face.

The information notification unit 190 notifies the user of the access is restricted if there is a minor in the store after 10 pm under the control of the control unit 170.

By photographing the faces of the users in the store through the real-time imaging device 195, and comparing the captured face images stored in the storage unit 160 with the faces captured by the comparison unit 150, a personal identification number is confirmed, and a minor in the store You can see that there is.

2 is a flowchart of a face recognition access method according to an exemplary embodiment of the present invention.

Referring to FIG. 2, the face recognition access method according to the present embodiment includes steps that are processed in time series in the face recognition access device illustrated in FIG. 1. Therefore, even if omitted below, the contents described above with respect to the face recognition access device illustrated in FIG. 1 also apply to the face recognition access method according to the present embodiment.

In step 200, the face recognition access device 100 scans the user's identification card. It is preferable that the face image included in the scanned ID is captured.

In step 210, the face recognition access device 100 photographs the user with a camera to generate a captured image.

In step 220, the face recognition access device 100 determines whether the captured image and the captured face image are the same person. If it is determined that the result of the above determination is not the same person, the process proceeds to step 230 and refuses access. However, if it is determined that the result is the same person, the process proceeds to step 240. As another embodiment, the process may proceed to step 260 without going through steps 240 and 250.

In step 230, the face recognition access device 100 outputs a message for rejecting access because the captured image and the captured face image are not the same person.

In step 240, the face recognition access device 100 requests a specific change of the face from the user.

In step 250, the face recognition access device 100 determines whether a specific change in the requested face and a change in the face captured by the camera match. If the check result does not match, the process proceeds to step 230 to output a message for refusing access.

However, if it is determined that the result of the above is consistent, the process may proceed to step 260.

In step 260, the face recognition access device 100 maps and stores the personal identification number of the identification card and the captured user photographed image. It is preferable that the captured image taken through the camera or the image taken through the real-time imaging device is stored in connection with the personal identification number. Thereafter, a specific user's captured image, captured image, and real-time imaging device image can be classified using a personal identification number.

In step 270, the face recognition access device 100 uses the stored user photographed image to confirm the identity of the faces of users photographed through a real-time imaging device installed in a store or airport.

3 is a flowchart of a method of entering and exiting a store by receiving additional fingerprints using a user's terminal.

The face recognition access device 100 other than the user terminal 300 may be provided with a fingerprint input unit to receive fingerprints from a user, but in this case, the user's biological information may be leaked to the outside. It is preferable to transmit the fingerprint entered through the server 200. Also, when the input fingerprint is transmitted to the server 200, the encrypted input fingerprint is transmitted using an encryption method that cannot be decrypted, so there is no possibility of leakage of biometric information.

In step 300, the face recognition access device 100 checks whether there is a suspected situation such as that the scanned identification card is forged or the photographed user is judged not to be a party holder of the scanned identification card.

Whether the scanned ID is forged or not can be checked by checking the hologram and light transmittance included in the ID. That is, it is preferable to check the presence or absence of a hologram inserted in the identification card, or to determine whether the identification card is a fake ID by measuring the transmittance through which light is transmitted. Or, by detecting an image that appears when irradiating ultraviolet rays, it can be confirmed whether or not it is forged.

As another embodiment, if the user's personal information is consented and verified online through personal information such as a family relationship certificate, whether there are siblings having a birthday on the same date through personal identification information of the identification card, it is judged as a doubt 310 You can proceed to the step. Through this, if it is judged to be the case of identical twins, it is preferable to request fingerprint input since the appearance is difficult to distinguish in the same way.

In step 310, the face recognition access device 100 requests a fingerprint input to the user's terminal 300. The user terminal 300 is preferably a personal smart phone capable of receiving a user's fingerprint by installing a separate application.

In step 320, the user terminal 300 receives the fingerprint and encrypts the input fingerprint using a hash function.

The hash function refers to a function that maps arbitrary data having an arbitrary length to data of a fixed length. A fixed length value obtained by applying this hash function to the input fingerprint becomes an encrypted fingerprint. Since the hash function is an irreversible one-way function, it cannot decrypt the encrypted fingerprint and reproduce the original fingerprint.

In step 330, the face recognition access device 100 encrypts the fingerprint included in the scanned ID using a hash function.

In step 340, the fingerprint encrypted by the user terminal 300 is transmitted to the server 200, and in step 350, the ID fingerprint encrypted by the face recognition access device 100 is also transmitted to the server 200. After the ID card fingerprint encrypted by the face recognition access device 100 is transmitted to the server 200, it is preferable to delete the fingerprint information of the scanned ID card.

Since the fingerprint information transmitted by the user terminal 300 and the face recognition access device 100 to the server 200 is encrypted by a hash function, it is impossible to decrypt even if it is leaked in the transmission process.

In step 360, the server 200 compares the encrypted input fingerprint transmitted by the user terminal 300 and the encrypted identification card fingerprint transmitted by the face recognition access device 100 to determine whether it is the same.

In step 370, the face recognition access device 100 receives a fingerprint comparison result from the server 200.

Embodiments of the present invention are implemented in the form of program instructions that can be executed through various computer means and can be recorded on computer readable media. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

The term '~ unit' used in this embodiment means a software or hardware component such as a field-programmable gate array (FPGA) or an ASIC, and the '~ unit' performs certain roles. However, '~ wealth' is not limited to software or hardware. The '~ unit' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, '~ unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, database, data structures, tables, arrays, and variables. The functions provided within components and '~ units' may be combined into a smaller number of components and '~ units', or further separated into additional components and '~ units'. In addition, the components and '~ unit' may be implemented to play one or more CPUs in the device or secure multimedia card.

All of the functions described above may be performed by a processor such as a microprocessor, controller, microcontroller, or Application Specific Integrated Circuit (ASIC) according to software or program code coded to perform the function. The design, development and implementation of the code will be apparent to those skilled in the art based on the description of the present invention.

Although the present invention has been described above with reference to examples, those skilled in the art can practice various modifications and changes of the present invention without departing from the spirit and scope of the present invention. You will understand. Therefore, it is not limited to the above-described embodiment, the present invention will be said to include all embodiments within the scope of the following claims.

Claims (13)

delete Scanning an identification card;
Capturing a face image included in the scanned identification card;
Photographing a user with a camera;
Comparing the captured image of the user with the captured face image;
Mapping and storing the personal identification number of the identification card and the photographed user photographed image when the comparison result is the same; And
And verifying the identity of the faces of the users photographed through the real-time imaging device using the stored user photographed images,
Requesting fingerprint input to the user's terminal when it is confirmed that the user does not have a valid identification card or suspects that the identification card has been forged;
Hash encrypting the fingerprint included in the identification card;
Transmitting the encrypted ID fingerprint to a server; And
Further comprising the step of receiving a fingerprint comparison result from the server,
The server receives a hash-encrypted input fingerprint from the user's terminal, and compares the received input fingerprint with the encrypted identification fingerprint. Scanning an identification card;
Capturing a face image included in the scanned identification card;
Photographing a user with a camera;
Comparing the captured image of the user with the captured face image;
Mapping and storing the personal identification number of the identification card and the photographed user photographed image when the comparison result is the same; And
And verifying the identity of the faces of the users photographed through the real-time imaging device using the stored user photographed images,
Requesting a specific change of the face from the user; And
And confirming whether the specific change of the requested face and the change of the face captured by the camera match. delete delete Scanning an identification card;
Capturing a face image included in the scanned identification card;
Photographing a user with a camera;
Comparing the captured image of the user with the captured face image;
Mapping and storing the personal identification number of the identification card and the photographed user photographed image when the comparison result is the same; And
And verifying the identity of the faces of the users photographed through the real-time imaging device using the stored user photographed images,
Further comprising the step of comparing the captured image of the user with the face image of the user has already entered,
If it is determined that a face similar to a face in the user's captured image photographed by the camera has already been entered, additional information of at least one of the user's glasses wearing, hairstyle, or clothes is generated,
When distinguishing users who have already entered, the face recognition access method is characterized in that it is distinguished by using the additional information. delete A scanning unit that scans an identification card;
A capture unit for capturing a face image included in the scanned identification card;
A camera unit for photographing a user with a camera;
A comparison unit that compares the captured user's captured image with the captured face image, and if the comparison results are identical, maps the personal identification number of the identification card to the captured user's captured image and stores it in a storage unit; And
It includes a control unit for verifying the identity of the face of the user photographed through the real-time imaging device using the stored user captured image,
When it is confirmed that the controller is not a valid ID card holder or suspects that the ID card has been forged, the user requests fingerprint input to the user's terminal, hash-encrypts the fingerprint included in the ID card, and transmits the encrypted ID card to the server. Then, characterized in that for receiving the fingerprint comparison result from the server,
The server receives a hash-encrypted input fingerprint from the user's terminal, and compares the received input fingerprint with the encrypted identification fingerprint. A scanning unit that scans an identification card;
A capture unit for capturing a face image included in the scanned identification card;
A camera unit for photographing a user with a camera;
A comparison unit that compares the captured user's captured image with the captured face image, and if the comparison results are identical, maps the personal identification number of the identification card to the captured user's captured image and stores it in a storage unit; And
It includes a control unit for verifying the identity of the face of the user photographed through the real-time imaging device using the stored user captured image,
The controller requests a specific change of the face from the user, and the comparator checks whether the specific change of the requested face matches the change of the face captured by the camera. delete delete A scanning unit that scans an identification card;
A capture unit for capturing a face image included in the scanned identification card;
A camera unit for photographing a user with a camera;
A comparison unit that compares the captured user's captured image with the captured face image, and if the comparison results are identical, maps the personal identification number of the identification card to the captured user's captured image and stores it in a storage unit;
A control unit for verifying the identity of the faces of users photographed through the real-time imaging device using the stored user photographed images; And
And a face recognition unit generating additional information on at least one of the user's glasses wearing, hairstyle, or clothes,
When the comparison unit compares the photographed image of the captured user with a face image of a user who has already entered, and determines that a face similar to a face in the photographed image of the user photographed by the camera unit is already entered,
A face recognition access device, characterized in that when the comparison unit distinguishes users who have already entered, it is distinguished using the additional information. A computer-readable recording medium recording a program for executing the method of any one of claims 2, 3, or 6 on a computer.

Images