KR100725771B1 - Apparatus and method of face recognition and authentication for a mobile terminal - Google Patents

Abstract

The present invention relates to an apparatus and method for face recognition and authentication for a portable terminal, and in particular, requires a specific behavior (specific facial expression) after the first authentication for a user of the portable terminal and secondly authenticates the response without transmitting information to the server. A face recognition and authentication apparatus and method for a portable terminal capable of accurate face recognition and authentication.

Handheld terminal, facial recognition, facial expression request, 2nd authentication

Description

APPARATUS AND METHOD OF FACE RECOGNITION AND AUTHENTICATION FOR A MOBILE TERMINAL}

1 is a diagram illustrating a configuration of a portable terminal for implementing user recognition and authentication according to the present invention;

2 is a diagram illustrating a configuration of a portable terminal for implementing primary recognition and authentication for a user according to the present invention;

3 is a diagram illustrating a configuration of a portable terminal implementing secondary recognition and authentication for a user according to the present invention;

4 is a flowchart illustrating an operation process of a portable terminal for implementing primary and secondary recognition and authentication for a user according to the present invention;

5 is a diagram of a portable terminal showing that a message for requesting a user to move in a specific direction is displayed on a display unit according to the present invention;

6 is a diagram of a portable terminal showing that an authentication success and failure message is displayed on a display unit in response to requesting a specific facial expression from a user according to the present invention;

The present invention relates to an apparatus and method for face recognition and authentication for a portable terminal, and in particular, requests for a specific behavior (such as making a specific facial expression) after the first authentication for a portable terminal user (image recognition subject) and a response accordingly. The present invention relates to a face recognition and authentication apparatus and method for a portable terminal.

Recently, a portable terminal such as a mobile communication terminal or a personal digital assistant (hereinafter referred to as a PDA) has been widely used. Such portable terminals are being developed as tools for realizing more than just means of communication, and the emergence of a new paradigm called ubiquitous requires a technology that enables users to receive and enjoy desired services anytime, anywhere. This implies personalization of services beyond the wide range of services offered to an unspecified majority.

In order to meet these demands, user awareness has emerged as an essential element. As a user recognition method, a biometric method has recently attracted the most attention. Unlike conventional ID (IDENTIFICATION) or password input, the biometric method has many advantages in that it can be more conveniently used by the user and less likely to be stolen. There are various recognition methods such as face, fingerprint, and iris, but among them, the recognition of face has high user convenience, and the recognition system can be implemented by cameras that are widely used instead of expensive sensors. The research is being actively conducted.

In particular, the face has the advantage of expressing the user's emotions at the same time as the user's unique biometric information, which is applied to the ubiquitous environment and is the most suitable method for satisfying the mobile terminal infrastructure. In other words, it is possible to provide various services by acquiring unique information of a face through a digital camera rather than a specific expensive sensor and acquiring a facial expression of a user beyond a range of user recognition.

The conventional invention includes "personal face-based personal identity verification method and device for the terminal {application number: 2001-0003492, filing date: 2001.01.20, Applicant: Watch Vision Co., Ltd. Injeongsu} This invention is a user input through a camera The present invention relates to a method and apparatus for verifying an identity of a person based on a technology of automatically comparing and verifying a face image of a face image and a face image registered in advance, wherein the face information and first verification of the user are pre-stored in the terminal memory. Verify the identity, and secondly encrypt the user information and face information verified in the terminal and transmit it to the remote server through the communication network, the remote server receives the data transmitted through the decryption process, the user stored in the remote server Facial information, and other various information of the user to perform the second verification.

In another conventional invention, "a method for setting and releasing a lock function of an image data communication terminal" {application number 2000-0014775) obtains a user face image photographed through a camera and extracts user face information data through a face recognition processor. The extracted user face information data is stored in the memory unit, and the image data communication terminal is switched to a lock mode that cannot be used except for a registered user. The extracted user face information data is compared with the user face information data stored in the memory unit. The present invention relates to a method of releasing a lock function of a terminal and switching to a call mode by checking whether a user is a registered user.

However, these methods require a lot of time for verification as the user recognizes and authenticates the second time in the information exchange terminal with the remote server itself, and there is a problem that cannot provide solutions for various lighting changes in the face recognition process. .

Therefore, there is a need for a method in which a mobile terminal can accurately recognize and authenticate a face under various lighting environments.

In addition, there is a need for a method capable of accurately recognizing and authenticating a face by verifying the mobile terminal device several times without transmitting information with a server.

An object of the present invention is to provide a method and apparatus capable of accurately recognizing and authenticating a face under a variety of lighting environments.

Another object of the present invention is to provide a method and apparatus capable of accurately recognizing and authenticating a face by verifying the mobile terminal device several times without transmitting information with a server.

In order to achieve the above object, the present invention has a function of operating the flash to the camera or instructing the user's position movement on the display unit according to the brightness of the user's input face image captured by the camera, the user's input The face recognition unit which performs the first authentication of the user by comparing the face image with the face image of the face database, receives notification that the first authentication is completed from the face recognition unit, and inputs the input expression image photographed by the camera in the expression database. A control unit for controlling the expression recognition unit performing a second authentication compared to the facial expression image, the first recognition result of the face recognition unit to notify the facial expression recognition unit, and the control to display the secondary authentication results of the facial expression recognition unit on the display unit And a face recognition and authentication device including a display unit to display the second authentication result. Provide a portable terminal.

According to an aspect of the present invention, there is provided a light processing step of operating a flash to the camera according to the brightness of an input face image of a user photographed by a camera or instructing a user to move a position of a display unit; A first authentication step of performing first authentication of the user by comparing the input face image of the user with a face image of a face database, a notification step of notifying the facial recognition unit of the first authentication result of the face recognition unit, the face recognition Secondary to be notified that the primary authentication is completed, requesting a specific facial expression from the user and performing the second authentication by comparing the input facial expression image of the specific facial expression taken by the camera with the facial expression image of the facial expression database It provides a face authentication method of a portable terminal including an authentication step and a display step of displaying the secondary authentication results of the facial expression recognition unit on the display unit.

The present invention requires a specific behavior (specific facial expression) after the first authentication for the user of the portable terminal, and thereby secondly authenticates the response, thereby enabling accurate face recognition and authentication without transmitting information to the server. The present invention relates to a face recognition and authentication device and method for a portable terminal capable of coping with a change in lighting by recognizing an ambient lighting environment and inducing a user to an optimal recognition environment to perform face recognition.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

Hereinafter, a description will be given with reference to FIGS. 1 to 6.

Preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of a portable terminal for implementing user recognition and authentication according to the present invention. Referring to FIG. 1, the portable terminal includes a control unit 100, a camera 102, a keypad 108, a display unit 104, a memory 106, a baseband processor 110, an RF module 112, and an antenna 114. ), A codec 116, a microphone 120, a speaker 118, a face recognition unit 115, and an expression recognition unit 117.

The controller 100 performs a function of controlling the overall operation of the portable terminal. For example, it performs processing and control for voice call and data communication. In the following description, description of the processing and control of the conventional control unit 100 is omitted.

The camera 102 takes image of the user and provides image data to the face recognition unit and the expression recognition unit 116 for face recognition of the user.

Key pad 108 includes 0 to 9 numeric keys, menu key (menu), cancel key (clear), confirm key, call key (TALK), end key (END), internet access key, navigation keys. It includes a plurality of function keys such as (▲ / ▼ / ◀ / ▶) and provides key input data corresponding to a key pressed by the user to the controller 100.

The display unit 104 displays the terminal (or indicator), limited numbers and characters, moving pictures, still pictures, and the like. The display unit 104 may use a color liquid crystal display (LCD).

The memory 106 temporarily stores a program for controlling the overall operation of the mobile communication terminal, data generated during the operation, storage data (telephone number, SMS message, image data, etc.), and a call log.

The RF module 112 reduces the frequency of the radio frequency signal received through the antenna 114 to the baseband processor 110 and raises the baseband signal of the baseband processor 110 by increasing the frequency. Transmit via antenna 114. The baseband processor 110 processes a baseband signal transmitted and received between the RF module 112 and the controller 100. For example, in the case of transmission, channel coding and spreading of data to be transmitted are performed, and in the case of reception, the function of despreading and channel decoding of a received signal is performed. .

An audio codec (Coder-Decoder) 116 connected to the controller 100, a microphone 120 and a speaker 118 connected to the codec 116 are voice input / output blocks used for a voice call. The codec 116 converts the PCM data provided from the control unit 100 into an analog voice signal and transmits the same through the speaker 118. The codec 116 converts the voice signal received through the microphone 120 into PCM data and controls the control unit ( 100).

The face recognition unit 115 and the expression recognition unit 117 will be described in detail below.

2 is a diagram illustrating a configuration of a portable terminal for implementing primary recognition and authentication for a user according to the present invention. Specifically, the face recognition unit 115 that recognizes a user's face for user recognition and authentication in the portable terminal is shown. The face recognition unit 115 receives an input image from the camera 102 in the portable terminal 110, and when the input image determines that the illumination of the surrounding environment needs to be changed according to the brightness of the camera 102. A lighting processor 220 for operating a flash or instructing a user's position movement to a display unit, that is, lighting processing of an input image, an image normalizing unit 230 for normalizing an input image for face detection, and a face. A face detector 240 for detecting an image, a face register 150 for registering a detected face image, a face authentication unit 260 for primary authentication by comparing the detected face image with a face image in a face database, Face database 270 for storing face information, the display unit 280 to inform the user of the first authentication result. The display unit may be replaced with the display unit 104 of FIG. 1.

3 is a diagram illustrating a configuration of a portable terminal for implementing secondary recognition and authentication for a user according to the present invention. That is, the facial recognition unit 117 recognizes the facial expression for the second recognition light authentication of the user in the portable terminal. The facial expression recognition unit 117 is the input image unit 310 that receives an image from the camera 102, such as the face recognition unit 115, and when the input image determines that the illumination of the surrounding environment is required according to the brightness, the camera The lighting processor 320 for operating the flash or instructing the display unit to move the position of the user to the display, that is, the lighting processor 320 to perform lighting processing of the input image, and the image normalization unit 330 for normalizing the image for detecting the expression. The facial expression detection unit 340 for detecting the facial expression image, the facial expression imager 350 for performing secondary authentication by comparing the facial expression image information with the detected input facial image, and the facial expression database for storing the facial expression image information ( 360, a display unit 370 for notifying the user of the second authentication result, and a terminal controller 380 for controlling the operation of the terminal according to the verification result. The display unit 370 may be replaced with the display unit 104 of FIG. 1.

4 is a flowchart illustrating an operation process of a portable terminal for implementing primary and secondary recognition and authentication for a user according to the present invention. First, the face image captured by the camera 102 is input (410). Next, in operation 420, the lighting processor 220 of the face recognition unit 115 determines whether an illumination change of the surrounding environment is necessary according to the brightness of the input face image. If it is determined that the input face image is dark, the camera 102 operates the flash, and if it is determined to be too bright, the display unit issues a movement command for instructing the user to move the position (to a slightly dark place). (440). That is, the overall brightness of the input image is measured, and the light is bright when the image is dark overall or when face detection is difficult due to polarization. On the contrary, when the image is too bright, the display part is slightly dark to the user. Instruct to move the place. Thereafter, the same environmental suitability determination step is performed (450). Next, the face recognition unit performs a first authentication step in which the face image detected through the face detection 430 and the image normalization 460 process is first compared with the face image in the face database (470). When the first authentication is completed, the face recognition unit notifies the facial recognition unit that the first authentication process is completed through the control unit. Next, a second authentication step of authenticating the facial expression is performed through the same process of the same face authentication process (480). That is, the facial expression recognition unit requests an arbitrary facial expression from the user, and the camera photographs the facial expression. Next, the facial expression image is normalized through the same process as the above facial authentication process, and the second facial verification 490 is performed by comparing the input facial expression image with a previously registered facial expression database. Of course, if the first authentication and the second authentication fails in the authentication failure (480,491) is not allowed to use the user. If both the first authentication and the second authentication pass, the authentication is completed and the user is given permission to use or access the portable terminal (343).

5 is a diagram of a portable terminal showing that a message for requesting a user to move in a specific direction is displayed on a display unit according to the present invention. In other words, when the image is input, if the surroundings are too dark and difficult to detect and recognize depending on the brightness of the surrounding environment, the lighting module attached to the camera operates to adjust the brightness or move the display slightly brighter to the user. On the other hand, when the surroundings are too bright to detect and recognize, the display unit asks the user to move to a slightly darker place.

6 is a diagram of a portable terminal showing that an authentication success and failure message is displayed on a display unit in response to a request for a specific facial expression from a user according to the present invention. In other words, according to the present invention, it is a diagram of a portable terminal showing that an authentication success and failure message is displayed on a display unit in response to requesting a specific facial expression from a user. When the first authentication is completed, a method of finally authenticating the user is shown when the user makes a facial expression as requested after requesting a certain facial expression from the user for the second authentication of the user.

As described above, the present invention relates to a method and apparatus for verifying a user's identity through user recognition and authentication in a mobile terminal device without transmitting information to a separate server, and a portable terminal optimally recognizes an ambient lighting environment by itself. It creates a recognition environment of, and compares the input face with a face already registered, undergoes first recognition and authentication, and then requests a random expression from the user to enable second verification.

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the scope of the following claims, but also by the equivalents of the claims.

The present invention requires a specific behavior (specific facial expression) after the first authentication for the user of the portable terminal, thereby authenticating the response accordingly, enabling accurate face recognition and authentication without transmitting information to the server, and in addition, the ambient lighting environment. By providing a portable terminal capable of recognizing and authenticating a face according to a change in lighting by recognizing the present invention, a method and apparatus for preventing unauthorized use of a portable terminal of another person, and also providing a function of preventing theft of another person using a photograph, are provided.

Claims (5)

A face database for storing face images; According to the brightness of the user's input face image captured by the camera has a function to operate the flash or instruct the user to move the position of the display unit, and compares the input face image with the face image of the face database of the user 1 A face recognition unit performing difference authentication; An expression database for storing a plurality of expression images; Upon receiving notification that the first authentication is completed from the face recognition unit, the user requests an arbitrary one specific expression among the plurality of expression images and expresses an input expression image for the specific expression photographed by the camera in the expression database. Expression recognition unit for performing the second authentication compared to the image; A control unit which controls to notify the facial recognition unit of the first authentication result of the face recognition unit and displays the secondary authentication result of the facial recognition unit on a display unit; A portable terminal comprising a face recognition and authentication device including a display unit for displaying the second authentication result. The method of claim 1, The face recognition unit is an input image unit for receiving a face image from the camera, If it is determined that the input image is darker than the reference face operation, and if it is determined that the input image is brighter than the reference face recognition, characterized in that it comprises a lighting processor for instructing the user's position movement and A portable terminal comprising an authentication device. The method of claim 1, The facial expression recognition unit input image unit for receiving a face image from the camera, If it is determined that the input image is darker than the reference face operation, and if it is determined that the input image is brighter than the reference face recognition, characterized in that it comprises a lighting processor for instructing the user's position movement and A portable terminal comprising an authentication device. An illumination processing step of causing the camera to operate the flash according to the brightness of the input face image of the user photographed by the camera or instructing the user to move the position on the display unit; A first authentication step of performing first authentication of the user by comparing the input face image of the user with a face image of a face database; A notification step of notifying the facial recognition unit of the first authentication result of the face recognition unit; Upon receiving notification from the facial recognition unit that the first authentication is completed, the user requests an arbitrary one specific facial expression among a plurality of stored facial expression images and expresses an input facial expression image of the specific facial expression photographed by the camera in the facial expression database. A face authentication method of a portable terminal comprising a second authentication step of performing a second authentication compared to the image. The method of claim 4, wherein And a display step of displaying the second authentication result of the facial expression recognition unit on the display unit after the second authentication step.

Images