KR20240054466A - Method of certifying identity using face video and apparatus using the same - Google Patents

Abstract

The present invention is a method of providing a user interface for identity authentication using a face image, the method comprising: receiving a user's identity authentication request from a user device; , in response to a request for identity authentication of the user, receiving an image of the user from the user device, determining the identity of the user based on the facial muscles of the received image, and determining the identified identity of the user. It is configured to include the step of providing to the user device.

Description

Identity authentication method using face image and device using same {METHOD OF CERTIFYING IDENTITY USING FACE VIDEO AND APPARATUS USING THE SAME}

The present invention relates to an identity authentication method using a face image and a device using the same.

Identity authentication methods include passwords, patterns, fingerprint recognition, face recognition, and voice recognition. At this time, in the case of face recognition, it cannot be changed and the risk of hacking such as password and pattern exposure is low, so it is used in financial transactions such as banking and payment authentication.

However, due to the recent emergence of technologies that imitate other users' faces, such as deepfake, problems are being raised about the security of simple facial recognition authentication.

The technology behind the invention has been written to facilitate easier understanding of the invention. It should not be understood as an admission that matters described in the technology underlying the invention exist as prior art.

There is a problem with the security stability of facial recognition being reduced due to technological advancements.

Accordingly, a method is required to identify the user by detecting muscle movement near the lips when the user says the password.

As a result, the inventors of the present invention sought to develop an identity authentication method and a device using the same using facial images based on muscle movements when the user speaks.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

In order to solve the problems described above, a method for providing a user interface according to an embodiment of the present invention is provided. The method includes receiving a user's identity authentication request from a user device; In response to the user's identity authentication request, receiving the user's image from the user device; determining the user's identity based on facial muscles in the received image; It is configured to include; providing the determined identity of the user to the user device.

Specific details of other embodiments are included in the detailed description and drawings.

The present invention can increase the accuracy and security of face recognition by using changes in muscles that move when a person pronounces pronunciation as a criterion.

Additionally, the present invention can increase processing speed by using only the areas with high change values instead of processing changes in all muscle areas of the face when the user pronounces pronunciation.

The effects according to the present invention are not limited to the details exemplified above, and further various effects are included within the present invention.

1 is a schematic diagram of an identity authentication system using a face image according to an embodiment of the present invention.
Figure 2 is a block diagram showing the configuration of a user device according to an embodiment of the present invention.
Figure 3 is a block diagram showing the configuration of a user interface providing server according to an embodiment of the present invention.
Figure 4 is a flowchart of an identity authentication server using a face image according to an embodiment of the present invention.
5 and 6 are exemplary diagrams of a method of providing a user interface for user identity authentication according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various different forms. The present embodiments only serve to ensure that the disclosure of the present invention is complete and are within the scope of common knowledge in the technical field to which the present invention pertains. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, similar reference numbers may be used for similar components.

In this document, expressions such as “have,” “may have,” “includes,” or “may include” refer to the existence of the corresponding feature (e.g., a numerical value, function, operation, or component such as a part). , and does not rule out the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, Or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in this document can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, a first component may be renamed a second component without departing from the scope of rights described in this document, and similarly, the second component may also be renamed to the first component.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as being “connected to,” it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component). On the other hand, when a component (e.g., a first component) is said to be “directly connected” or “directly connected” to another component (e.g., a second component), It may be understood that no other component (e.g., a third component) exists between other components.

As used in this document, the expression “configured to” depends on the situation, for example, “suitable for,” “having the capacity to.” ," can be used interchangeably with "designed to," "adapted to," "made to," or "capable of." The term “configured (or set to)” may not necessarily mean “specifically designed to” in hardware. Instead, in some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase "processor configured (or set) to perform A, B, and C" refers to a processor dedicated to performing the operations (e.g., an embedded processor), or by executing one or more software programs stored on a memory device. , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

Terms used in this document are merely used to describe specific embodiments and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by a person of ordinary skill in the technical field described in this document. Among the terms used in this document, terms defined in general dictionaries may be interpreted to have the same or similar meaning as the meaning they have in the context of related technology, and unless clearly defined in this document, have an ideal or excessively formal meaning. It is not interpreted as In some cases, even terms defined in this document cannot be interpreted to exclude embodiments of this document.

Each feature of the various embodiments of the present invention can be partially or fully combined or combined with each other, and as can be fully understood by those skilled in the art, various technical interconnections and operations are possible, and each embodiment may be implemented independently of each other. It may be possible to conduct them together due to a related relationship.

For clarity of interpretation of this specification, terms used in this specification will be defined below.

1 is a schematic diagram of an identity authentication system using a face image according to an embodiment of the present invention.

Referring to FIG. 1, the user identity authentication system 1000 includes a user device 100 and a user interface providing server 200 (hereinafter, user identity authentication server 200).

The user identity authentication system 1000 is capable of determining the user's identity in response to an identity authentication request for the user. At this time, the user identity authentication system 1000 may receive an image of the user. Specifically, the user identity authentication server 200 of the user identity authentication system 1000 may provide a user image input interface to the user device 200 possessed by the user, and the user's identity determination result may be performed by the user device 200. A provision interface can be provided. Here, the user device 100 is an electronic device capable of capturing and outputting images, and may include a smartphone, tablet PC, PC, laptop, etc.

Figure 2 is a block diagram showing the configuration of a user device according to an embodiment of the present invention.

Referring to FIG. 2 , the user device 100 may include a memory interface 110, one or more processors 120, and a peripheral interface 130. Various components within user device 100 may be connected by one or more communication buses or signal lines.

The memory interface 110 is connected to the memory 150 and can transmit various data to the processor 120. Here, the memory 150 is a flash memory type, hard disk type, multimedia card micro type, card type memory (e.g. SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage, and cloud. , It may include at least one type of storage medium among the blockchain database.

In various embodiments, memory 150 includes operating system 151, communications module 152, graphical user interface module (GUI) 153, sensor processing module 154, telephony module 155, and application module 156. ) can be stored. Specifically, the operating system 151 may include instructions for processing basic system services and instructions for performing hardware tasks. The communication module 152 may communicate with at least one of one or more other devices, computers, and servers. The graphical user interface module (GUI) 153 can process a graphical user interface. Sensor processing module 154 may process sensor-related functions (eg, processing voice input received using one or more microphones 192). The phone module 155 can process phone-related functions. Application module 156 may perform various functions of a user application, such as electronic messaging, web browsing, media processing, navigation, imaging, and other processing functions. In addition, the user device 100 may store one or more software applications 156-1 and 156-2 associated with one type of service in the memory 150. At this time, the application 156-1 may provide the user device 100 with the determined user's identity result. Here, the application 156-1 can determine the user's identity based on the muscles of the user's face image.

In various embodiments, memory 150 may store a digital assistant client module 157 (hereinafter referred to as DA client module), thereby storing various user data 158 and instructions for performing client-side functions of the digital assistant. (e.g. user-customized vocabulary data, preference data, and other data such as the user's electronic address book, to-do list, shopping list, etc.).

Meanwhile, the DA client module 157 receives the user's voice input, text input, touch input, and/or gesture input through various user interfaces (e.g., I/O subsystem 140) provided in the user device 100. It can be obtained.

Additionally, the DA client module 157 can output data in audiovisual and tactile forms. For example, the DA client module 157 may output data consisting of a combination of at least two or more of voice, sound, notification, text message, menu, graphics, video, animation, and vibration. In addition, the DA client module 157 can communicate with a digital assistant server (not shown) using the communication subsystem 180.

In various embodiments, DA client module 157 may collect additional information about the surrounding environment of user device 100 from various sensors, subsystems, and peripheral devices to construct a context associated with user input. . For example, the DA client module 157 may infer the user's intention by providing context information along with user input to the digital assistant server. Here, context information that may accompany the user input may include sensor information, for example, lighting, ambient noise, ambient temperature, images of the surrounding environment, video, etc. As another example, the context information may include the physical state of the user device 100 (e.g., device orientation, device location, device temperature, power level, speed, acceleration, motion pattern, cellular signal strength, etc.). As another example, context information may include information related to the software status of the user device 100 (e.g., processes running on the user device 100, installed programs, past and present network activity, background services, error logs, resource usage). etc.) may be included.

In various embodiments, the memory 150 may include added or deleted instructions, and further, the user device 100 may also include additional components other than those shown in FIG. 2 or exclude some components.

The processor 120 can control the overall operation of the user device 100 and execute various commands to implement an interface for providing future financial status by running an application or program stored in the memory 150.

The processor 120 may correspond to a computing device such as a Central Processing Unit (CPU) or an Application Processor (AP). Additionally, the processor 120 may be implemented in the form of an integrated chip (IC) such as a system on chip (SoC) in which various computing devices such as a neural processing unit (NPU) are integrated.

The peripheral interface 130 is connected to various sensors, subsystems, and peripheral devices and can provide data so that the user device 100 can perform various functions. Here, the user device 100 performing a certain function may be understood as being performed by the processor 120.

The peripheral interface 130 may receive data from the motion sensor 160, the light sensor (light sensor) 161, and the proximity sensor 162, through which the user device 100 can determine orientation, light, and proximity. It can perform detection functions, etc. For another example, the peripheral interface 130 may receive data from other sensors 163 (positioning system-GPS receiver, temperature sensor, biometric sensor), through which the user device 100 may use other sensors. Functions related to (163) can be performed.

In various embodiments, the user device 100 may include a camera subsystem 170 connected to the peripheral interface 130 and an optical sensor 171 connected thereto, through which the user device 100 can take photos and video. You can perform various shooting functions such as clip recording. At this time, the user device 100 may transmit the captured video clip to the user identity authentication server 200.

In various embodiments, user device 100 may include a communication subsystem 180 coupled with a peripheral interface 130. The communication subsystem 180 consists of one or more wired/wireless networks and may include various communication ports, radio frequency transceivers, and optical transceivers.

In various embodiments, user device 100 includes an audio subsystem 190 coupled to a peripheral interface 130, which includes one or more speakers 191 and one or more microphones 192. By doing so, user device 100 can perform voice-activated functions, such as voice recognition, voice duplication, digital recording, and telephony functions.

In various embodiments, user device 100 may include an I/O subsystem 140 coupled with a peripheral interface 130. For example, the I/O subsystem 140 may control the touch screen 143 included in the user device 100 through the touch screen controller 141. As an example, the touch screen controller 141 uses any one of a plurality of touch sensing technologies such as capacitive, resistive, infrared, surface acoustic wave technology, proximity sensor array, etc. to detect the user's touch and movement or contact. and cessation of movement can be detected. As another example, the I/O subsystem 140 may control other input/control devices 144 included in the user device 100 through other input controller(s) 142. As an example, other input controller(s) 142 may control one or more buttons, rocker switches, thumb-wheels, infrared ports, USB ports, and pointer devices such as a stylus.

Figure 3 is a block diagram showing the configuration of a user interface providing server according to an embodiment of the present invention.

Referring to FIG. 3, the user identity authentication server 200 may include a communication interface 210, a memory 220, an I/O interface 230, and a processor 240, and each component may include one or more communication buses. Alternatively, they can communicate with each other through signal lines.

The communication interface 210 can be connected to the user device 100 through a wired/wireless communication network to exchange data. For example, the communication interface 210 may receive the user's image from the user device 100. At this time, the user's video may be a video of the user pronouncing the password.

Meanwhile, the communication interface 210 that enables transmission and reception of such data includes a communication pod 211 and a wireless circuit 212, where the wired communication port 211 is connected to one or more wired interfaces, for example, Ethernet, May include Universal Serial Bus (USB), Firewire, etc. Additionally, the wireless circuit 212 can transmit and receive data with an external device through RF signals or optical signals. Additionally, wireless communications may use at least one of a plurality of communication standards, protocols and technologies, such as GSM, EDGE, CDMA, TDMA, Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol.

The memory 220 may store various data used in the user identity authentication server 200. For example, the memory 220 may store data about the user, including an image of the user pronouncing the password when registering authentication.

In various embodiments, memory 220 may include volatile or non-volatile recording media capable of storing various data, instructions, and information. For example, the memory 220 may be a flash memory type, hard disk type, multimedia card micro type, card type memory (e.g. SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage storage. , cloud, or blockchain database may include at least one type of storage medium.

In various embodiments, memory 220 may store configuration of at least one of operating system 221, communication module 222, user interface module 223, and one or more applications 224.

Operating system 221 (e.g., embedded operating system such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) is a variety of software to control and manage common system tasks (e.g., memory management, storage device control, power management, etc.) It may contain components and drivers and may support communication between various hardware, firmware, and software components.

The communication module 223 may support communication with other devices through the communication interface 210. The communication module 220 may include various software components for processing data received by the wired communication port 211 or the wireless circuit 212 of the communication interface 210.

The user interface module 223 may receive a user's request or input from a keyboard, touch screen, microphone, etc. through the I/O interface 230 and provide a user interface on the display.

Applications 224 may include programs or modules configured to be executed by one or more processors 240 . Here, an application for providing recommended items may be implemented on a server farm.

The I/O interface 230 may connect at least one of input/output devices (not shown) of the future financial status providing server 200, such as a display, keyboard, touch screen, and microphone, to the user interface module 223. The I/O interface 230 may receive user input (eg, voice input, keyboard input, touch input, etc.) together with the user interface module 223 and process commands according to the received input.

The processor 240 is connected to the communication interface 210, memory 220, and I/O interface 230 to control the overall operation of the future financial status providing server 200, and the application stored in the memory 220. Alternatively, various commands to provide recommended items can be executed through the program.

The processor 240 may correspond to a computing device such as a Central Processing Unit (CPU) or an Application Processor (AP). Additionally, the processor 240 may be implemented in the form of an integrated chip (IC) such as a system on chip (SoC) in which various computing devices are integrated. Alternatively, the processor 240 may include a module for calculating an artificial neural network model, such as a Neural Processing Unit (NPU).

In various embodiments, processor 240 may receive an identity authentication request for a user from user device 100 . At this time, the processor 240 may receive data about the user, including an image of the user's pronunciation when registering his or her identity, from the memory 220.

The processor 240 may receive an image of at least one user pronouncing a password from the user device 100. At this time, the password pronounced by the user may be the same as the password pronounced when the user registered his/her identity.

The processor 240 may determine whether the user's identity is the same by comparing the muscle movements of the image of the user pronouncing the user's identity when registering the user's identity with the muscle movements of the image of the person pronouncing the received password.

In more detail, the processor 240 can derive landmarks for each muscle of the face for each image. At this time, the processor 240 may derive a landmark by considering the characteristics of the muscle. Here, the processor 240 may determine the amount of change in distance between a plurality of landmarks. At this time, the processor 240 can only use movements limited to areas where the amount of change in distance between landmarks is large. For example, the processor 240 uses only the movements of the small zygomaticus muscle located across the lip and cheekbones, which move greatly when the user pronounces in the image, and the masseter muscle located at the tip of the nose and the jaw joint, which moves less during pronunciation. movements can be excluded. Additionally, the processor 240 can use only the movement of landmarks on one side of the face in consideration of facial symmetry.

Here, the processor 240 can determine whether the identity is the same as the registered identity by analyzing the amount of change compared to time in the video pronouncing the password when registering the identity and the amount of change compared to time in the video pronouncing the password when requesting identity authentication.

The processor 240 may provide the determined user's identity result to the user device 100.

So far, the configuration of the future financial status providing server 200 according to an embodiment of the present invention has been described. Hereinafter, a method of providing a user interface for providing future financial status through the future financial status providing server 200 described above will be described.

Figure 4 is a flow chart of a method of providing a user interface for user identity authentication according to an embodiment of the present invention, and Figures 5 and 6 are exemplary diagrams of a method of providing a user interface for user identity authentication according to an embodiment of the present invention.

Referring to FIG. 4, the user identity authentication server 200 may receive an identity authentication request for the user from the user device 100. At this time, the future financial status providing server 200 may receive data about the user, including an image of the user's pronunciation when registering his or her identity, from the memory 220.

The future financial status providing server 200 may receive an image of at least one user pronouncing a password from the user device 100 . At this time, the password pronounced by the user may be the same as the password pronounced when the user registered his/her identity.

The future financial status providing server 200 may determine whether the user's identity is the same by comparing the muscle movements of the image of the user pronouncing the user's identity when registering the user's identity with the muscle movements of the video of the user's voice pronouncing the received password.

In more detail, referring to (a) of FIG. 5, the future financial status providing server 200 may derive landmarks for each muscle of the face for each image. At this time, the future financial status providing server 200 may derive a landmark by considering the characteristics of the muscle. Here, the future financial status providing server 200 may determine the amount of change in distance between a plurality of landmarks. At this time, the future financial status providing server 200 can be used only in areas where the amount of change in distance between landmarks is large. For example, referring to (b) of FIG. 5, the future financial status providing server 200 uses only the movements of the small zygomaticus muscle across the tip of the lips and cheekbones, which move greatly when the user pronounces in the video, Movement of the masseter muscle located at the tip of the nose and jaw joint, which moves less during pronunciation, can be excluded. In addition, the future financial status providing server 200 can use only the movement of landmarks on one side of the face in consideration of facial symmetry.

Here, the future financial status provision server 200 analyzes the change over time in the video pronouncing the password when registering an identity and the change over time in the video pronouncing the password when requesting identity authentication to determine whether the identity is the same as the registered identity. can do. For example, referring to Figure 6, L1 is the amount of change compared to time in the video of the user pronouncing the password when registering his/her identity, L2 is the amount of change compared to time of the video of the user pronouncing the password when requesting his/her identity, and L3 is the random amount of change over time. This is the amount of change over time in the video of the user pronouncing the same password. At this time, the flows of L1 and L2 proceed similarly, but the flow of L3 proceeds dissimilarly. Therefore, the future financial status providing server 200 may determine that L1 and L2 are the same user, and that L1 and L3 are different users.

Referring again to FIG. 4 , the future financial status providing server 200 may provide the determined user's identity result to the user device 100 .

Although embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and may be modified and implemented in various ways without departing from the technical spirit of the present invention. there is. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

100: User device
110: memory interface 120: processor
130: peripheral interface 140: I/O subsystem
141: Touch screen controller 142: Other input controller
143: touch screen
144: Other input control devices
150: Memory 151: Operating system
152: Communication module 153: GUI module
154: sensor processing module 155: phone module
156: Applications
156-1, 156-2: Application
157: Digital assistant client module
158: User data
160: motion sensor 161: light sensor
162: Proximity sensor 163: Other sensors
170: Camera subsystem 171: Optical sensor
180: Communication subsystem
190: Audio subsystem
191: Speaker 192: Microphone
200: Server providing recommended items
210: communication interface
211: wired communication port 212: wireless circuit
220: memory
221: operating system 222: communication module
223: User interface module 224: Application
230: I/O interface 240: processor

Claims (1)

An identity authentication method using a facial image implemented by a processor,
Receiving a user's identity authentication request from the user device;
In response to the user's identity authentication request, receiving the user's image from the user device;
determining the user's identity based on facial muscles in the received image;
providing the determined identity of the user to the user device; An identity authentication method using a face image, including.

Images