KR102627254B1 - An Electronic apparatus, Face Recognition system and Method for preventing spoofing thereof - Google Patents

Abstract

Electronic devices and facial recognition systems, and methods for preventing spoofing thereof are provided. The spoofing determination method of this electronic device includes receiving an image obtained from a camera, detecting the user's face area included in the input image, recognizing the user based on the detected user's face area, and collecting the image. Analyzing and determining whether the image is a spoofing image, Providing a message requesting a task to be performed by the user, Determining whether the image is spoofed by determining whether the user performs the task and authenticating the user based on a result of determining whether the image is a spoofed image and a result of determining whether the image is spoofed.

Description

Electronic apparatus, Face Recognition system and Method for preventing spoofing thereof}

The present invention relates to an electronic device, a facial recognition system, and a method for preventing spoofing thereof. More specifically, the present invention relates to a facial recognition system that uses a photo or other device to prevent spoofing, in which a user pretends to actually be in front of a camera in front of a camera. It relates to an electronic device and facial recognition system for preventing spoofing, and a method for preventing spoofing thereof.

Recently, various systems have been developed to authenticate users by photographing their faces. In these systems, technology is essential to prevent spoofing, which intentionally disguises another person's identity using photos or other devices.

In order to prevent spoofing, technology used to analyze images captured using general RGB cameras and information obtained from special sensors (e.g. Depth sensor, Stereo camera, infrared, heat sensor, etc.) in addition to general RGB cameras were used. The technology existed.

When using only an RGB camera, the discrimination performance against spoofing attacks is inevitably lower than when using a special sensor. Specifically, in the case of an RGB camera, the image coming through the camera sensor is in the form of a 2D image. When comparing the case where an actual person is standing in front of the camera and the case where a face photo is shown in front of the camera, the image body coming through the sensor is the same in the form of a 2D image. Therefore, in order to distinguish them, more complex calculations are required for image analysis.

When using a special sensor, the performance to prevent spoofing can be higher than when using an RGB camera, but there are limitations such as the additional special sensor required being expensive or installation restrictions.

The present invention was created to solve the above-mentioned problems, and the purpose of the present invention is to provide an electronic device and facial recognition system that can authenticate users more accurately and safely by applying a plurality of anti-spoofing technologies, and a method to prevent spoofing thereof. It is provided.

According to an embodiment of the present disclosure, a method for determining spoofing of an electronic device includes receiving an image acquired from a camera; detecting a user's face area included in the input image; Recognizing the user based on the detected user face area; Analyzing the image to determine whether the image is a spoofing image; Providing a message requesting a task to be performed by the user; determining spoofing by determining whether the user performs the task; and authenticating the user based on a result of determining whether the image is a spoofed image and a result of determining whether the image is spoofed.

In the step of determining whether the image is a spoofing image, if the input image is a single frame still image, the user's face included in the image can be predicted in 3D form to determine whether the image is a spoofing image. .

In addition, the step of determining whether the input image is a spoofed image includes estimating information about rPPG (remote photoplethysmography) from a plurality of frames included in the image when the input image is a video image including a plurality of frames. It is possible to determine whether is a spoofing image.

And, the step of determining whether or not there is spoofing may include obtaining an image taken of the user from the camera while the user performs the task; and analyzing the image to determine whether the task is performed and determining spoofing.

In addition, the providing step provides a message requesting a task of uttering a preset word to the user, and the step of determining whether the user is spoofing includes obtaining an image taken of the user while the user is speaking. step; detecting the user's lips included in the image captured while the user speaks; Obtaining text uttered by the user based on the detected lip movement; and comparing the obtained text with the preset word to determine whether there is spoofing.

In addition, the providing step provides a message requesting a task for performing a preset gesture to the user, and the step of determining whether spoofing occurs involves photographing the user through the camera after providing the message. acquiring an image; It may include the step of analyzing an image taken of the user after providing the message and determining whether the preset gesture has been performed to determine whether there is spoofing.

In addition, the providing step provides a message requesting the user to perform a task of uttering a preset word, and the step of determining whether spoofing occurs includes providing a message for the voice uttered by the user while the user is speaking. acquiring features; and comparing the acquired voice characteristics with the user's pre-stored voice characteristics to determine whether there is spoofing.

Meanwhile, according to an embodiment of the present disclosure, an electronic device includes a communication unit; a memory for controlling the electronic device; And receiving an image acquired from a camera provided in an external device through the communication unit, detecting the user's face area included in the input image, and recognizing the user based on the detected user's face area, Analyzes the image to determine whether the image is a spoofing image, controls the communication unit to transmit a message requesting a task to be performed by the user to the external device, and responds to information received from the external device. and a processor that determines whether the user performs the task based on whether the image is spoofed and authenticates the user based on the result of determining whether the image is a spoofed image and the result of determining whether the image is spoofed.

Additionally, if the input image is a single-frame still image, the processor may determine whether the image is a spoofing image by predicting the user's face included in the image in 3D form.

In addition, when the input image is a video image including a plurality of frames, the processor determines whether the image is a spoofing image by estimating information about remote photoplethysmography (rPPG) from the plurality of frames included in the image. can do.

And, while the user is performing the task, the processor receives an image taken of the user from the camera of the external device through the communication unit, analyzes the image, and determines whether the user is performing the task, thereby performing spoofing. You can judge whether or not.

In addition, the processor controls the communication unit to transmit a message requesting the user to perform a task of uttering a preset word to the external device, and sends an image of the user while the user is speaking from the external device. Received through the communication unit, detecting the user's lips included in the image captured while the user spoke, obtaining text uttered by the user based on the detected movement of the lips, and obtaining the obtained text Spoofing can be determined by comparing the preset words.

And, the processor controls the communication unit to transmit a message requesting the user to perform a preset gesture to the external device, and after transmitting the message from the external device, the user is sent through the camera. The captured image is received through the communication unit, and after sending the message, the image captured of the user is analyzed to determine whether the preset gesture has been performed to determine spoofing.

In addition, the processor controls the communication unit to transmit a message requesting the user to perform a task of uttering a preset word to the external device, and records voice characteristics of the voice uttered by the user while the user speaks. spoofing can be determined by comparing the acquired voice characteristics with the user's previously stored voice characteristics.

Meanwhile, a facial recognition system according to an embodiment of the present disclosure includes an external device including a camera and a display, and an electronic device that authenticates a user by communicating with the external device, where the electronic device includes, Receiving an image acquired from a camera provided in the external device from the external device, detecting the user's face area included in the input image, and recognizing the user based on the detected user's face area, Analyzes the image to determine whether the image is a spoofing image, sends a message requesting a task to be performed by the user to the external device, and based on the information received from the external device, the user The user can be authenticated based on the result of determining whether the image is spoofed and the result of determining whether the image is spoofed by determining whether the task is performed.

As described above, by determining additional spoofing based on whether the user performs a task in addition to the existing spoofing prevention method, it is possible to prevent external malicious spoofing attacks and perform user authentication more accurately and safely.

1 is a diagram illustrating a face authentication system according to an embodiment of the present disclosure;
2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure;
3 is a flowchart illustrating a method for preventing anti-spoofing of an electronic device according to an embodiment of the present disclosure;
4A and 4B are diagrams illustrating messages for guiding tasks to be performed by a user according to various embodiments of the present disclosure;
5 is a flowchart illustrating a method for preventing anti-spoofing of an electronic device according to another embodiment of the present disclosure;
Figure 6 is a block diagram showing the configuration of an electronic device according to another embodiment of the present disclosure.

Since these embodiments can be modified in various ways and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives to the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numbers may be used for similar components.

In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted.

In addition, the following examples may be modified into various other forms, and the scope of the technical idea of the present disclosure is not limited to the following examples. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to completely convey the technical idea of the present disclosure to those skilled in the art.

The terms used in this disclosure are merely used to describe specific embodiments and are not intended to limit the scope of rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present disclosure, expressions such as “have,” “may have,” “includes,” or “may include” refer to the presence of the corresponding feature (e.g., component such as numerical value, function, operation, or part). , and does not rule out the existence of additional features.

In the present disclosure, expressions such as “A or B,” “at least one of A or/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” includes (1) at least one A, (2) at least one B, or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in the present disclosure can modify various components regardless of order and/or importance, and can refer to one component. It is only used to distinguish from other components and does not limit the components.

A component (e.g., a first component) is “(operatively or communicatively) coupled with/to” another component (e.g., a second component). When referred to as being “connected to,” it should be understood that any component may be directly connected to the other component or may be connected through another component (e.g., a third component).

On the other hand, when a component (e.g., a first component) is said to be “directly connected” or “directly connected” to another component (e.g., a second component), It may be understood that no other component (e.g., a third component) exists between other components.

The expression “configured to” used in the present disclosure may mean, for example, “suitable for,” “having the capacity to,” depending on the situation. ," can be used interchangeably with "designed to," "adapted to," "made to," or "capable of." The term “configured (or set to)” may not necessarily mean “specifically designed to” in hardware.

Instead, in some contexts, the expression “a device configured to” may mean that the device is “capable of” working with other devices or components. For example, the phrase "processor configured (or set) to perform A, B, and C" refers to a processor dedicated to performing the operations (e.g., an embedded processor), or by executing one or more software programs stored on a memory device. , may refer to a general-purpose processor (e.g., CPU or application processor) capable of performing the corresponding operations.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented as at least one server, except for 'modules' or 'units' that need to be implemented with specific hardware.

Hereinafter, the face authentication system will be described with reference to FIG. 1. The face authentication system 10 may include an electronic device 100 and an external device 200, as shown in FIG. 1 . At this time, the electronic device 100 may be a server, but this is only an example and may be implemented as a user terminal such as a smart phone, personal PC, tablet PC, etc. The external device 200 may be a device that includes a camera and a display.

When a user inputs a user command to perform user authentication through the external device 200, the external device 200 may acquire an image by photographing the user. At this time, the external device 200 can photograph the user using an RGB camera.

The external device 200 may transmit the acquired image to the electronic device 100.

The electronic device 100 may detect the user's face from the acquired image. In addition, the electronic device 100 may recognize the user based on the detected user face area, and determine whether the received image is a spoofed image by analyzing the detected user face area, and the user may use the electronic device ( Spoofing can be determined by determining whether the task requested by 100) has been performed.

The electronic device 100 may perform user authentication based on a user recognition result, a determination result of whether the image is a spoofing image, and a determination result of whether the image is spoofing.

The electronic device 100 may transmit the user authentication result to the external device 200.

In particular, as shown in FIG. 1, when spoofing is attempted by using a photo displayed on a device other than the user to pretend that there is a real user in front of the camera, the electronic device 100 detects the user's face from the acquired image. By detecting and determining whether the acquired image is a spoofed image, it is possible to prevent spoofing as shown in FIG. 1 by requesting the user to perform a task of uttering a specific word or performing a specific gesture.

FIG. 2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 2, the electronic device 100 may include a communication unit 110, a memory 120, and a processor 130.

The communication unit 110 includes a circuit and can perform communication with the external device 200. Specifically, the processor 130 can receive various data or information from the external device 200 connected through the communication unit 110, and can also transmit various data or information to the external device. For example, the communication unit 110 may receive an image including a user from the external device 200 and may transmit information about the user authentication result to the external device 200.

The communication unit 110 may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, and an NFC module. Specifically, the WiFi module and the Bluetooth module can each communicate using WiFi and Bluetooth methods. When using a WiFi module or Bluetooth module, various connection information such as SSID is first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this.

Additionally, the wireless communication module can perform communication according to various communication standards such as IEEE, Zigbee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and 5th Generation (5G).

Additionally, the communication unit 110 may communicate with the external device 200 through various wired communication modules such as a USB module and an HDMI module.

The memory 120 may store various instructions for controlling the overall operation of the electronic device 100. In particular, the memory 120 may store data necessary for a module to authenticate a user using a plurality of anti-spoofing methods to perform various operations. At this time, the modules for authenticating the user using multiple spoofing prevention methods include an image input module 131, a face detection module 132, a user recognition module 133, a spoofing image determination module 134, and a spoofing determination module ( 135), and may include a user authentication module 136.

Meanwhile, the memory 120 may include a non-volatile memory that can maintain stored information even when power supply is interrupted, and a volatile memory that requires a continuous power supply to maintain the stored information. Data for performing an operation to authenticate a user using a plurality of anti-spoofing methods may be stored in a non-volatile memory.

Additionally, the memory 120 may store information (eg, face information, voice print information) about at least one registered user in order to perform user authentication.

Additionally, the memory 120 may include at least one buffer that temporarily stores image frames acquired through the camera of the external device 200.

The processor 130 is electrically connected to the memory 120 and can control the overall functions and operations of the electronic device 100.

When an operation for user authentication is initiated, the processor 130 loads data for performing various operations on a module for authenticating the user using a plurality of spoofing prevention methods stored in the non-volatile memory into the volatile memory. )can do. Here, loading refers to an operation of loading and storing data stored in non-volatile memory in volatile memory so that the processor 130 can access it.

Specifically, the processor 130 may authenticate the user by performing an operation to prevent spoofing using a plurality of modules 131 to 136.

Specifically, the image input module 131 may acquire an image from the external device 200 through the communication unit 110. At this time, the image acquired from the external device 200 is an image captured through an RGB camera included in the external device 200, and may be an image including the user (particularly, the face).

The face detection module 132 may detect the user's face area from the acquired image. Specifically, the face detection module 132 detects a feature area (or landmark area) included in the face (e.g., eyes, nose, mouth, ears, etc.), and detects the user's face based on the detected feature area. The area can be detected.

The user recognition module 133 can recognize the user based on the detected face area. Specifically, the user recognition module 133 may recognize the user by comparing user information (eg, face image) stored in the memory 132 with the detected face area. When a user is recognized, the user recognition module 133 may obtain information about the user (eg, user name, gender, age, etc.). If the user is not recognized, the user recognition module 133 may provide a message to register the user.

The spoofing image determination module 134 may analyze the image and determine whether the image is a spoofing image. In particular, the spoofing image determination module 134 can analyze the face area included in the image to determine whether it is a spoofing image. For example, when the input image is a single-frame still image, the spoofing image determination module 134 may determine whether the acquired image is a spoofing image by predicting the user's face included in the image in 3D form. As another example, when the input image is a video image including a plurality of frames, the spoofing image determination module 134 estimates information about remote photoplethysmography (rPPG) from the plurality of frames included in the image and It is possible to determine whether is a spoofing image.

The spoofing determination module 135 may determine whether it is spoofing based on whether the user performed the task. Specifically, the spoofing determination module 135 may control the communication unit 110 to transmit a message requesting a task to be performed by the user to the external device 200. At this time, the task may be one of a task of uttering a word preset to the user or a task of performing a gesture preset to the user.

In one embodiment of the present disclosure, the spoofing determination module 135 may acquire an image captured by the camera of the external device 200 while the user performs a task. Additionally, the spoofing determination module 135 may analyze the image to determine whether a task is being performed to determine whether there is spoofing. For example, when the task is to utter a word preset to the user, the spoofing determination module 135 acquires an image captured of the user while the user speaks, and the user included in the image captured while the user speaks The lips are detected, the text uttered by the user is obtained based on the detected lip movements, and spoofing can be determined by comparing the obtained text with preset words. As another example, if the task is to perform a gesture preset to the user, the spoofing determination module 135 may obtain an image captured through the camera of the external device 200 after providing a message. Additionally, the spoofing determination module 135 may analyze an image taken of a user after providing a message and determine whether a preset gesture has been performed to determine whether there is spoofing.

In another embodiment of the present disclosure, the spoofing determination module 135 may determine whether it is spoofing based on the user's voice characteristics (eg, voiceprint). For example, if the task is to utter a word preset to the user, the spoofing determination module 135 may acquire voice characteristics for the voice uttered by the user while the user is speaking. Additionally, the spoofing determination module 135 may determine whether there is spoofing by comparing the acquired voice characteristics with the user's pre-stored voice characteristics. That is, if the acquired voice feature and the recognized user's pre-stored voice feature match, the spoofing determination module 135 may determine that it is not spoofing, and the acquired voice feature and the recognized user's pre-stored voice feature do not match. If not, the spoofing determination module 135 may determine that it is spoofing.

The user authentication module 136 may authenticate the user based on the determination results of the user recognition module 133, the spoofing image determination module 134, and the spoofing determination module 135. For example, if the user is not recognized, the image received from the external device 200 is determined to be a spoofed image, or is determined to be spoofed based on whether the user performed the task, the user authentication module 136 fails to authenticate the user. and request re-authentication of the user. However, if the user is recognized and the image received from the external device 200 is not a spoofed image and is determined not to be spoofed based on whether the user performed the task, the user authentication module 134 may authenticate the user. At this time, the user authentication module 134 may transmit the user authentication result to the external device 200.

Hereinafter, the spoofing prevention method of the present invention will be described in more detail with reference to FIGS. 3 to 5.

FIG. 3 is a flowchart illustrating a method for preventing anti-spoofing of an electronic device, according to an embodiment of the present disclosure.

The electronic device 100 can receive an image (S310). At this time, the electronic device 100 can receive an image including the user and store the input image in a buffer.

The electronic device 100 may detect a face area from the acquired image (S320). At this time, the electronic device 100 may extract the feature area of the face from the acquired image and detect the user's face area from the extracted feature area. At this time, algorithms for detecting facial feature areas include AAM, DeepFace, and Face++.

The electronic device 100 may perform user recognition based on the acquired facial area (S330). Specifically, the electronic device 100 may recognize the user included in the acquired image by comparing the acquired facial area with a previously stored facial image. At this time, the electronic device 100 may recognize the user using the entire area of the user's face, but this is only an example, and may recognize the user using a partial area of the user's face (for example, iris). You can.

At this time, if the user's face is not recognized, the electronic device 100 may distinguish the user included in the image as an unregistered user. Additionally, the electronic device 100 may transmit a message requesting user registration to the external device 200 in order to inquire whether the user wants to register.

Additionally, the electronic device 100 may determine whether the input image is a spoofed image (S340). Specifically, the electronic device 100 may analyze the input image and detect whether the input image is a spoofed image. That is, the electronic device 100 can analyze the input image and determine whether spoofing, such as an attacker playing a photo or video of a registered user, has occurred in the input image.

For example, when the image input from the external device 200 is a single image frame, the electronic device 100 predicts a 3D face from the detected face area and determines whether the input image is a normal image or a spoofed image. can do. As another example, when an image input from the external device 200 includes a plurality of image frames, the electronic device 100 analyzes the plurality of image frames to estimate rPPG and based on information about the estimated rPPG. It is possible to determine whether the input image is a normal image or a spoofed image. However, as described above, determining a spoofed image is only an example, and a spoofed image may be determined through other methods.

Additionally, the electronic device 100 may provide a message requesting a task to check whether the user is a real user (S350). Specifically, the electronic device 100 may transmit a message to the external device 200 to request the user to perform a task of uttering a preset word or a task of performing a preset gesture. In one embodiment, as shown in FIG. 4A, the external device 200 utters the preset word “Please utter “xxx”” on the display based on the message received from the electronic device 100. A request message 410 may be provided. As another example, as shown in FIG. 4B, the external device 200 displays a message requesting performance of a preset gesture such as “Please wave your hand twice” on the display based on the message received from the electronic autonomy 100. (420) can be provided. However, as shown in Figure 4b, the gesture "Please wave your hand twice" is only an example, and various gestures such as "Please wink twice with your right eye" or "Please make a V-shape with your hand" are just one example. You can request it randomly, every time or every day with a random gesture.

In addition, although it is explained in FIGS. 4A and 4B that a visual message is provided on the display, this is only an example, and an audio message can be provided through a speaker.

Additionally, the task may be a task already registered by the user. For example, in the electronic device 100, a user may pre-register his or her face and simultaneously register at least one of his or her own words or gestures. When a task is registered by a user, the electronic device 100 may immediately determine whether the user has performed the registered task without providing a message requesting the registered task.

The electronic device 100 may determine spoofing based on whether the task is performed (S360). In one embodiment of the present disclosure, the electronic device 100 may acquire an image captured while the user performs a task from the camera of the external device 200. Additionally, the electronic device 100 may analyze the acquired image to determine whether the user has performed the task and determine whether there is spoofing. For example, when the task is a task in which the user utters "XXX", the electronic device 100 may detect the lip area among the user's face area and obtain information about the movement of the lips from the detected input area. . In addition, the electronic device 100 can obtain the text uttered by the user based on information about the movement of the lips, and compare the obtained text with “XXX” to determine whether the user has performed the task. there is. As another example, when the task is a task in which the user winks his right eye twice, the electronic device 100 detects the right eye area among the user's face areas, determines whether the detected right eye area blinked twice, and determines whether the user blinked twice. It can be determined whether the task has been performed.

In another embodiment of the present disclosure, the electronic device 100 may detect a voice characteristic of a voice uttered by a user and compare it with a recognized user's pre-registered voice characteristic. Additionally, the electronic device 100 may determine whether there is spoofing by comparing the detected voice characteristics with the user's pre-registered voice characteristics.

Meanwhile, steps S330, S340, and S350 (including step S360) shown in FIG. 3 may be performed in parallel.

The electronic device 100 may authenticate the user based on the user recognition result in step S330, the spoofing image determination result in step S340, and the spoofing determination result in step S360. Specifically, if the user is recognized, the input image is not a spoofed image, and it is determined not to be spoofing based on whether the user performed the task, the electronic device 100 authenticates the user and sends the authentication result to the external device 200. It can be sent to . However, if the user is not recognized, the input image is a spoof image, or it is determined to be spoofing based on whether the user performed the task, the electronic device 100 fails user authentication, and the authentication result (or user authentication re-request (including) can be transmitted to the external device 200.

Meanwhile, in the above-described embodiment, it has been described that user recognition, spoofing image determination, and spoofing determination are performed in parallel. However, this is only an example, and user recognition, spoofing image determination, and spoofing determination may be performed sequentially. . This will be explained with reference to FIG. 5 .

First, the electronic device 100 can acquire an image (S510). At this time, the image may be an image acquired from the camera of the external device 200.

Then, the electronic device 100 can detect the face area from the acquired image (S520).

The electronic device 100 may recognize the user based on the detected face area (S530).

If the user is not recognized (S530-N), the electronic device 100 may perform a user registration process (S540). When the user is recognized (S530-Y), the electronic device 100 may determine whether the acquired image is a spoofed image (S550).

If it is determined that the acquired image is not a spoofed image (S550-N), the electronic device 100 may provide a message requesting a task (S560). That is, the electronic device 100 may transmit a message requesting a task to an external device.

Then, the electronic device 100 may determine spoofing based on whether the task is performed (S570). Specifically, the electronic device 100 may determine spoofing based on whether the user performs the task, as described above with reference to FIGS. 2 to 4 .

If it is determined that it is not spoofing based on whether the user performed the task (S570-N), the electronic device 100 may authenticate the user (S580).

However, if the acquired image is a spoofed image (S550-Y) or is determined to be spoofed based on whether the user performed the task (S570-Y), the electronic device 100 may fail user authentication (S590).

As shown in FIG. 3, when user recognition, spoofing image determination, and spoofing determination are performed in parallel, the electronic device 100 can perform user authentication more quickly, and as shown in FIG. 5, When user recognition, spoofing image determination, and spoofing determination are performed sequentially, the electronic device 100 can prevent unnecessary judgment operations.

Meanwhile, in the above-described embodiment, it was explained that the facial recognition system 10 authenticates the user by recognizing the user's face through the electronic device 100 and the external device 200, but this is only an embodiment and The user can be authenticated by recognizing the user's face through an electronic device. This will be explained with reference to FIG. 6.

Figure 6 is a block diagram showing the configuration of an electronic device according to another embodiment of the present disclosure. As shown in FIG. 6, the electronic device 600 includes a communication unit 610, a display 620, a microphone 630, a memory 640, a user input unit 650, a camera 660, and a processor 670. It can be included. At this time, the electronic device 100 is a terminal device for user authentication and may be implemented as a smart phone, tablet PC, laptop PC, etc., but this is only an example and may be implemented as various devices such as a front door, a car, etc. You can. Meanwhile, since the description of the communication unit 610 and memory 640 overlaps with the communication unit 110 and memory 120 described in FIG. 2, detailed descriptions are omitted.

The display 620 can display an image captured through the camera 660. Additionally, the display 620 may display a message requesting the user to perform a task. Meanwhile, the display 620 may be implemented as a Liquid Crystal Display Panel (LCD), Organic Light Emitting Diodes (OLED), etc., and in some cases, the display 620 may also be implemented as a flexible display, transparent display, etc. . However, the display 620 according to the present disclosure is not limited to a specific type.

The microphone 630 can acquire the user's voice. Specifically, the microphone 630 can acquire the user's voice uttering a preset word. At this time, the microphone 630 may be provided inside the electronic device 600, but this is only an example and may be electrically connected to the electronic device 600.

The user input unit 650 includes a circuit, and the processor 670 can receive user commands for controlling the operation of the electronic device 600 through the user input unit 650. Specifically, the user input unit 650 may be implemented as a touch screen included in the display 620, but this is only an example and may be configured such as buttons and a remote control signal receiver (not shown). there is.

In particular, in various embodiments according to the present disclosure, the user input unit 650 may receive input of various user commands, such as a user command for starting user authentication and a user command for registering a user.

The camera 660 can capture images. In particular, the camera 660 can capture an image including the user. At this time, the image may be a still image including one image frame or a moving image including a plurality of image frames.

Additionally, the camera 660 may include a plurality of different lenses. Here, the fact that the plurality of lenses are different from each other may include a case where the field of view (FOV) of each of the plurality of lenses is different from each other and a case where each of the plurality of lenses is disposed is different.

In addition, the electronic device 600 may further include a speaker for providing an auditory message, a sensor for detecting the user or user proximity, etc.

The processor 670 acquires an image from the camera 660, detects the user's face area included in the acquired image, recognizes the user based on the detected user face area, and analyzes the image so that the image is a spoofed image. (spoofing image), a message requesting a task to be performed by the user is displayed through the display 620, and the user performs the task based on information acquired through the camera 660 or microphone 630. The user can be authenticated based on the results of determining whether or not the image is spoofed and the results of determining whether or not the image is spoofed.

According to one embodiment, methods according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products are commodities and can be traded between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g. compact disc read only memory (CD-ROM)) or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (e.g. downloaded or uploaded) directly between smartphones) or online. In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is stored on a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server. It can be temporarily stored or created temporarily.

Each component (e.g., module or program) according to various embodiments of the present disclosure as described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted. Alternatively, other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration.

Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage media that can be read by a machine (e.g., a computer). The device calls the stored instructions from the storage medium. and, as a device capable of operating according to the called instruction, it may include an electronic device (e.g., electronic device 100) according to the disclosed embodiments. When the instruction is executed by a processor, the processor may directly or the processor The function corresponding to the command may be performed using other components under the control of. The command may include code generated or executed by a compiler or interpreter.

In the above, preferred embodiments of the present disclosure have been shown and described, but the present disclosure is not limited to the specific embodiments described above, and may be used in the technical field to which the disclosure pertains without departing from the gist of the disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical ideas or perspectives of the present disclosure.

100: Electronic device 110: Communication department
120: memory 130: processor

Claims (15)

In a method for determining spoofing of an electronic device,
Receiving an image obtained from a camera;
detecting a user's face area included in the input image;
Recognizing the user based on the detected user face area;
Analyzing the image to determine whether the image is a spoofing image;
Providing a message requesting a task to be performed by the user;
determining spoofing by determining whether the user performs the task; and
A step of authenticating the user based on a result of determining whether the image is a spoofed image and a result of determining whether the image is spoofed,
The step of determining whether or not there is spoofing is,
acquiring an image of the user from the camera while the user performs the task; and
Comprising: analyzing the image to determine whether the task is performed and determining spoofing;
The steps provided above are:
Providing a message requesting the user to perform a task of uttering a preset word,
The step of determining whether or not there is spoofing is,
Obtaining an image of the user while the user is speaking;
detecting the user's lips included in the image captured while the user speaks;
Obtaining text uttered by the user based on the detected lip movement; and
A spoofing determination method including; comparing the obtained text with the preset word to determine whether there is spoofing. According to paragraph 1,
The step of determining whether the image is a spoofing image is:
A spoofing determination method that determines whether the image is a spoofing image by predicting the user's face included in the image in 3D form when the input image is a single-frame still image. According to paragraph 1,
The step of determining whether the image is a spoofing image is:
When the input image is a video image including a plurality of frames, a spoofing determination method for determining whether the image is a spoofing image by estimating information about rPPG (remote photoplethysmography) from the plurality of frames included in the image. delete delete According to paragraph 1,
The steps provided above are:
Provides a message requesting a task for performing a preset gesture to the user,
The step of determining whether or not there is spoofing is,
Obtaining an image of the user through the camera after providing the message;
After providing the message, analyzing an image taken of the user to determine whether the preset gesture has been performed to determine whether there is spoofing. According to paragraph 1,
The steps provided above are:
Provides a message requesting the user to perform a task of uttering a preset word,
The step of determining whether or not there is spoofing is,
Obtaining voice characteristics for a voice uttered by the user while the user is speaking; and
A spoofing determination method including; comparing the acquired voice characteristics with the user's pre-stored voice characteristics to determine whether there is spoofing. In electronic devices,
Ministry of Communications;
a memory for controlling the electronic device; and
Receive images acquired from a camera provided in an external device through the communication unit,
Detect the user's face area included in the image,
Recognize the user based on the detected user face area,
Analyzing the image to determine whether the image is a spoofing image,
Controlling the communication unit to transmit a message requesting a task to be performed by the user to the external device,
Based on the information received from the external device, determine whether the user performs the task and determine whether there is spoofing,
A processor authenticating the user based on a result of determining whether the image is a spoofed image and a result of determining whether the image is spoofed,
The processor is
Receiving an image taken of the user from a camera of the external device while the user is performing the task through the communication unit,
Analyzing the image to determine whether the task is performed to determine whether it is spoofing,
The processor,
Controlling the communication unit to transmit to the external device a message requesting the user to perform a task of uttering a preset word,
Receiving an image taken of the user while the user speaks from the external device through the communication unit,
Detecting the user's lips included in the image captured while the user speaks,
Obtain the text uttered by the user based on the detected lip movements,
Comparing the obtained text with the preset word to determine whether there is spoofing. According to clause 8,
The processor,
An electronic device that determines whether the image is a spoofed image by predicting the user's face included in the image in 3D form when the image is a single-frame still image. According to clause 8,
The processor,
When the image is a moving image including a plurality of frames, an electronic device that determines whether the image is a spoofing image by estimating information about remote photoplethysmography (rPPG) from the plurality of frames included in the image. delete delete According to clause 8,
The processor,
Controlling the communication unit to transmit a message requesting the user to perform a preset gesture to the external device,
After transmitting the message from the external device, an image taken of the user through the camera is received through the communication unit,
An electronic device that determines spoofing by analyzing an image taken of the user after transmitting the message and determining whether the preset gesture has been performed. According to clause 8,
The processor,
Controlling the communication unit to transmit a message to the external device requesting the user to perform a task of uttering a preset word,
Obtaining voice characteristics for the voice uttered by the user while the user is speaking,
An electronic device that determines spoofing by comparing the acquired voice characteristics with the user's previously stored voice characteristics. In the face recognition system,
External devices, including cameras and displays, and
It includes an electronic device that authenticates the user by communicating with the external device,
The electronic device is,
Receive from the external device an image acquired from a camera provided in the external device,
Detect the user's face area included in the image,
Recognize the user based on the detected user face area,
Analyzing the image to determine whether the image is a spoofing image,
Transmits a message requesting a task to be performed by the user to the external device,
Based on the information received from the external device, determine whether the user performs the task and determine whether there is spoofing,
Authenticating the user based on a result of determining whether the image is a spoofed image and a result of determining whether the image is spoofed,
The electronic device is,
Receiving an image taken of the user from a camera of the external device while the user is performing the task,
It includes determining whether the image is spoofed by analyzing the image to determine whether the task is performed,
The electronic device is,
Transmitting a message to the external device requesting the user to perform a task of uttering a preset word,
Receiving an image taken of the user while the user is speaking from the external device,
Detecting the user's lips included in the image captured while the user speaks,
Obtain the text uttered by the user based on the detected lip movements,
A face recognition system comprising comparing the obtained text with the preset word to determine whether there is spoofing.

Images