KR20220115452A - Liveness detection method based on appearance of mouth when pronouncing, and computing apparatus for performing the same - Google Patents

Abstract

Provided is a liveness detection method based on the mouth shape at the time of pronunciation, which includes the steps of: receiving an image of a user pronouncing at least one authentication word when the user attempts biometric authentication; and determining whether the user is a pre-registered user based on the received image and pre-stored biometric information corresponding to the user.

Description

A method for real-life detection based on the shape of a mouth during pronunciation and a computing device for performing the same

Embodiments disclosed in the present specification relate to facial recognition that performs authentication by recognizing a user's face shape among biometric technologies, and in detail, a real object for blocking face theft using pre-photographed photos or videos. It relates to a liveness detection method.

Biometric authentication technology through face recognition is largely a technology that uses 3D information included in an image captured by an IR camera or a depth camera, and 2D information (RGB information) included in an image captured by a general camera. It can be divided into techniques using

However, in the case of a technology using two-dimensional information, there is a problem in that it may be exposed to a face theft attack using a pre-photographed photo or video.

An object of the present invention is to provide a real detection method for effectively preventing face theft attacks in performing biometric authentication using a facial recognition method, and an apparatus for performing the same.

A liveness detection method based on a mouth shape when pronouncing according to an embodiment includes the steps of, when a user attempts biometric authentication, receiving an image of the user pronouncing at least one authentication word; The method may include determining whether the user is a pre-registered user based on the received image and biometric information stored in advance to correspond to the user.

In the embodiments disclosed herein, it is possible to effectively prevent face theft attacks by performing real-life detection based on the characteristics of the user's mouth shape when pronouncing a specific word.

Effects obtainable in the disclosed embodiments are not limited to the above-mentioned effects, and other effects not mentioned are clear to those of ordinary skill in the art to which the embodiments disclosed from the description below belong. will be able to be understood

1 is a diagram illustrating a system for performing an object sensing method according to an embodiment.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly describe the characteristics of the embodiments, detailed descriptions of matters widely known to those of ordinary skill in the art to which the following embodiments belong are omitted. And, in the drawings, parts not related to the description of the embodiments are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a component is said to be "connected" with another component, it includes not only the case where it is 'directly connected' but also the case where it is 'connected with another component in between'. In addition, when a component "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

In the case of a technology that performs biometric authentication based on RGB information (two-dimensional information) included in an image of a user's face using a general camera, the following face theft attack may be applied.

First, authentication is attempted using a pre-photographed user's photo, which is called a print attack.

Second, authentication is attempted using a pre-recorded user's video, which is called a replay attack.

The following liveness detection technology can be used to defend against the face theft attack.

In order to prevent print attack, when authentication is attempted, it is first checked whether the face of the user (authentication attempter) matches that of the pre-registered user, and then secondarily, the user who can confirm that the photo is not pre-taken. It can be determined that authentication is successful only when motion (e.g. blinking eyes) is detected.

In addition, in order to prevent replay attack, when authentication is attempted, the user first checks whether the user's face matches the pre-registered user's face, and then allows the user to perform a specific action ( e.g. to make a gesture by placing a hand around the face), and it can be determined that authentication is successful only when the user performs the corresponding action.

In the present specification, embodiments in which real sensing is performed through an image of a user pronouncing a specific word will be described. That is, in the embodiments disclosed herein, it is possible to effectively prevent a face theft attack by performing real-life detection based on a feature of a user's mouth shape when pronouncing a specific word.

The embodiment will be described in detail with reference to the drawings as follows.

1 is a diagram illustrating a system for performing an object sensing method according to an embodiment.

Referring to FIG. 1 , when a user 1 attempts biometric authentication through the mobile terminal 200 , the mobile terminal 200 records the user 1 pronouncing the authentication word and sends it to the authentication server 100 . can be transmitted At this time, the 'authentication word' means a word used for real sensing, one or a plurality of authentication words may be set, and selection of the authentication word may be performed by the user 1 or the authentication server 100. have. In detail, when the user 1 attempts biometric authentication through the mobile terminal 200, a UI instructing the mobile terminal 200 to perform an authentication operation (e.g., pronounce the authentication word while photographing a face with a camera). When the screen is displayed and the user 1 pronounces the authentication word while photographing his or her face according to the instruction, the mobile terminal 200 takes a picture of the user 1 pronouncing the authentication word and authenticates the captured image It can be transmitted to the server 100 .

The authentication server 100 may recognize a change in the user's mouth shape from the received captured image, and determine whether the user is a true user based on this. In detail, the authentication server 100 extracts key points from the captured image using a pre-learned artificial neural network model, and stores the extracted key points to correspond to the user 1 in advance of biometric information (when the user registers biometric information, the authentication server By comparing with the feature points extracted from the image stored in the .

In order to perform real sensing in this way, the feature points extracted from the image of the user 1 pronouncing the authentication word are stored in advance as biometric information corresponding to the user 1 in the authentication server 100. can For example, when the user 1 takes a picture of himself pronouncing an authentication word through the mobile terminal 200 and requests biometric information registration while transmitting the captured image to the authentication server 100, the authentication server ( 100) extracts a feature point from a captured image using an artificial neural network model, stores the extracted feature point as biometric information corresponding to the user 1 , and then uses it when the user 1 attempts biometric authentication later.

Even when the same word is pronounced, since each person has a different mouth shape, biometric authentication can be performed in this way.

Hereinafter, various deformable embodiments will be described.

1. First Example

When registering biometric information, an image of pronouncing one authentication word is captured and transmitted to the authentication server 100 . In this case, the authentication word may be arbitrarily selected by the user 1 or may be selected and presented by the authentication server 100 .

When the user 1 attempts authentication in the mobile terminal 200 , information on the authentication word is not displayed on the UI screen of the mobile terminal 200 . That is, the UI screen of the mobile terminal 200 does not display a guide on which word to pronounce for real detection, and the user 1 remembers the authentication word he used when registering biometric information, and then You should try biometric authentication while recording the pronunciation of the word.

Since the user 1 knows the authentication word used when registering biometric information, the user 1 can try to authenticate while recording the pronunciation of the word with the mobile terminal 200, but the third party cannot know what the authentication word is. Therefore, you can expect a defensive effect against theft attacks.

The mobile terminal 200 transmits the captured image to the authentication server 100, and the authentication server 100 extracts a feature point from the received captured image using an artificial neural network, and stores the extracted feature point in advance to correspond to the user. By comparing the information (feature points extracted from the captured image when registering the user's biometric information and stored), it is determined whether the user matches the registered user, and if it is determined that the user matches the registered user, it can be determined that authentication is successful.

2. Second Example

When registering biometric information, an image of pronouncing one authentication word is captured and transmitted to the authentication server 100 . In this case, the authentication word may be arbitrarily selected by the user 1 or may be selected and presented by the authentication server 100 .

When the user 1 attempts authentication in the mobile terminal 200, the UI screen of the mobile terminal 200 displays a part of the authentication word (e.g. at least one syllable or the initial consonant of at least one syllable - for example, the authentication word is "airplane") If ", only "Rain" or "B ㅎ ㄱ" may be displayed) or all of it may be displayed.

The user 1 recalls an authentication word through the UI screen of the mobile terminal 200 and attempts biometric authentication while photographing the pronunciation of the authentication word with the mobile terminal 200 . Since the user 1 can obtain a hint about the authentication word through the UI screen when attempting authentication, the burden on remembering the authentication word is reduced, thereby increasing user convenience.

The mobile terminal 200 transmits the captured image to the authentication server 100, and the authentication server 100 extracts a feature point from the received captured image using an artificial neural network, and stores the extracted feature point in advance to correspond to the user. By comparing the information (feature points extracted and stored from the captured image at the time of user registration), it is determined whether the user matches the registered user, and when it is determined that the user matches the registered user, it can be determined that authentication is successful.

3. Third Example

When registering biometric information, an image of pronouncing a plurality of predetermined authentication words is captured and transmitted to the authentication server 100 . In this case, the authentication word may be arbitrarily selected by the user 1 or may be selected and presented by the authentication server 100 .

When the user 1 attempts biometric authentication in the mobile terminal 200 , at least one of a plurality of authentication words may be randomly displayed on the UI screen of the mobile terminal 200 . At this time, at least one authentication word randomly selected from among the plurality of authentication words may be completely displayed on the UI screen, or a part of at least one authentication word randomly selected (e.g. at least one syllable or the initial consonant of at least one syllable) - For example, if the authentication word is "airplane", only "rain" may be displayed, or "bha" may be displayed) may be displayed.

The user 1 may attempt authentication while photographing the pronunciation of at least one authentication word displayed on the UI screen of the mobile terminal 200 with the mobile terminal 200 . The user 1 can check the authentication word to be pronounced through the UI screen or obtain a hint for the authentication word through the UI screen when attempting authentication, thereby reducing the burden on remembering the authentication word and increasing user convenience.

The mobile terminal 200 transmits the captured image to the authentication server 100, and the authentication server 100 extracts a feature point from the received captured image using an artificial neural network, and stores the extracted feature point in advance to correspond to the user. By comparing the information (feature points extracted and stored from the captured image at the time of user registration), it is determined whether the user matches the registered user, and when it is determined that the user matches the registered user, it can be determined that authentication is successful.

4. Additional Examples

Although the above embodiments have described a method of performing real sensing using an image of pronouncing a specific word, it is also possible to perform real sensing using sentences other than words.

Also, words that can be combined with words pronounced by the user 1 when registering biometric information may be used when attempting biometric authentication. For example, when registering biometric information, an image of the user 1 pronouncing the numbers 1 to 10 is used, and when biometric authentication is attempted, at least one of the numbers 1 to 99 is used for the user 1 It can also be made to pronounce randomly.

5. Additional Features - Related to Selection of Verification Words

As such, when a word is recognized by extracting feature points from an image captured by the user 1 pronouncing a word, there may be a difference in recognition accuracy for each word.

Accordingly, the authentication server 100 may select a word with high recognition accuracy as the authentication word. For example, the authentication server 100 may calculate recognition accuracy for each authentication word used in the process of continuously performing authentication, and depending on the result, words having a recognition accuracy of less than or equal to a certain standard may be excluded from the authentication word pool.

The authentication server 100 may select an authentication word from among the words included in the authentication word pool when directly selecting the authentication word, and when the user 1 selects the authentication word, the user 1 is not included in the authentication word pool. When biometric information registration is attempted by selecting an authentication word, it may be suggested to the user 1 to use another authentication word.

The term '~ unit' used in the above embodiments means software or hardware components such as field programmable gate array (FPGA) or ASIC, and '~ unit' performs certain roles. However, '-part' is not limited to software or hardware. '~unit' may be configured to reside in an addressable storage medium or may be configured to refresh one or more processors. Thus, as an example, '~' denotes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

Functions provided in components and '~ units' may be combined into a smaller number of components and '~ units' or separated from additional components and '~ units'.

In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

The method for detecting an object based on the shape of a mouth during pronunciation according to the above-described embodiment may be implemented in the form of a computer-readable medium for storing instructions and data executable by a computer. In this case, the instructions and data may be stored in the form of program codes, and when executed by the processor, a predetermined program module may be generated to perform a predetermined operation. In addition, computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may be a computer recording medium, which is a volatile and non-volatile and non-volatile embodied in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. It may include both volatile, removable and non-removable media. For example, the computer recording medium may be a magnetic storage medium such as HDD and SSD, an optical recording medium such as CD, DVD, and Blu-ray disc, or a memory included in a server accessible through a network.

In addition, the method for detecting an object based on the shape of a mouth during pronunciation according to the above-described embodiment may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor, and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language. . In addition, the computer program may be recorded in a tangible computer-readable recording medium (eg, a memory, a hard disk, a magnetic/optical medium, or a solid-state drive (SSD), etc.).

Accordingly, the method for detecting an object based on the shape of a mouth during pronunciation according to the above-described embodiment may be implemented by executing the above-described computer program by a computing device. The computing device may include at least a portion of a processor, a memory, a storage device, a high-speed interface connected to the memory and the high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using various buses, and may be mounted on a common motherboard or mounted in any other suitable manner.

Here, the processor may process a command within the computing device, such as for displaying graphic information for providing a Graphical User Interface (GUI) on an external input or output device, such as a display connected to a high-speed interface, for example. For example, instructions stored in memory or a storage device. In other embodiments, multiple processors and/or multiple buses may be used with multiple memories and types of memory as appropriate. In addition, the processor may be implemented as a chipset formed by chips including a plurality of independent analog and/or digital processors.

Memory also stores information within the computing device. As an example, the memory may be configured as a volatile memory unit or a set thereof. As another example, the memory may be configured as a non-volatile memory unit or a set thereof. The memory may also be another form of computer readable medium such as, for example, a magnetic or optical disk.

In addition, the storage device may provide a large-capacity storage space to the computing device. The storage device may be a computer-readable medium or a component comprising such a medium, and may include, for example, devices or other components within a storage area network (SAN), a floppy disk device, a hard disk device, an optical disk device, Alternatively, it may be a tape device, a flash memory, or other semiconductor memory device or device array similar thereto.

The above-described embodiments are for illustration, and those of ordinary skill in the art to which the above-described embodiments pertain can easily transform into other specific forms without changing the technical idea or essential features of the above-described embodiments. You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may also be implemented in a combined form.

The scope to be protected through this specification is indicated by the claims described below rather than the above detailed description, and should be construed to include all changes or modifications derived from the meaning and scope of the claims and their equivalents. .

1: User 100: Authentication Server
200: mobile terminal

Claims (1)

In the liveness detection method based on the shape of the mouth during pronunciation,
receiving an image captured by the user pronouncing at least one authentication word when the user attempts biometric authentication; and
and determining whether the user is a pre-registered user based on the received image and biometric information stored in advance to correspond to the user.

Images