KR20010074059A - Personal Verification Method and Apparatus Based on Facial Images for Mobile Systems - Google Patents

Abstract

PURPOSE: A method and device for verifying a personal identity based on a face for a mobile terminal is provided to verify a person's identity based on a face image inputted through a camera for removing a user's rejection in an identity verification system. CONSTITUTION: The elements of a terminal verifying unit(100) are described as follows. A face input unit(110) receives a face image from a camera being attached to a mobile terminal. A display unit(120) displays the inputted face image(120). A processor(130) registers and verifies a face through all sorts of image processing processes. A database(140) stores face information. A terminal control unit(150) controls an operation of the terminal in accordance with a verifying result. A communication control unit(160) performs an encryption for a security on a network and transmits data through a communication network(300). The elements of a server verifying unit(200) are described as follows. A communication control unit(260) decodes data inputted through the communication network(300) and performs an encryption for transmitting the data. A display unit(220) displays working contents(220). A processor(230) processes a user verification and data to be transmitted to a terminal server. A database(240) stores user information and other services.

Description

Personal Verification Method and Apparatus Based on Facial Images for Mobile Systems}

The present invention relates to a face-based personal identity verification method and apparatus for a mobile terminal, based on a technology for automatically comparing and verifying a face image of a user input through a camera attached to the mobile terminal and a pre-registered face image. By first verifying the identity of an individual, it prevents others from using the mobile terminal without permission, transmits face information through a communication network, and then performs e-commerce through the Internet through the second verification of the user's face stored in the server. It is a new design to provide the ability to control the security of information and the use of ineligible persons on the network, such as financial transactions.

In order to prevent unauthorized use of a mobile terminal or to authenticate and secure information on a network such as electronic commerce and financial transactions over the Internet, simple password input and ID card such as credit card and password input have been common methods. Increasingly, biometric technologies such as fingerprint recognition, vein recognition, iris recognition, voice recognition, and facial recognition are increasingly applied.

In the case of face recognition, for example, just five years ago, face recognition and verification of an individual was too expensive to introduce and was considered complex and could not be applied to real life. Technologists in the industry have since worked to reduce the number of data needed to define an individual's face, and the computer has become faster than five years ago and can retrieve millions of facial data stored in a computer database in less than a minute. It became possible.

These developments in facial recognition and verification technology have resulted from a growing interest in integrating biometric technologies such as iris, vein, fingerprint, and voice recognition systems into access control and identity verification. Biometrics is emerging as an alternative to identity verification and access control because of the widespread distrust that passwords and personal passwords can be easily exposed or stolen. Among these, face verification technology has no strange rejection when a finger is placed in a fingerprint recognition system or a hand in a vein recognizer or a fear of being close to the iris recognition system. Way.

However, various solutions using cameras that can be easily attached to various mobile terminals such as laptops, PDAs, and IMT-2000 terminals have been developed. However, most of them focus on transmission rather than processing images, and are rapidly increasing. Adaptation to a variety of uses, such as security solutions through the process of personal identification required for e-commerce and e-business solutions, is very rare.

Accordingly, the present invention is a method for eliminating the discomfort felt by the user when using the identity verification system using the iris recognition, vein recognition, fingerprint recognition, etc., is already attached or easily attached to the rapidly increasing mobile terminal system An object of the present invention is to provide a method and apparatus for verifying an individual's identity based on a face image input through an attachable camera.

In order to achieve the above object, the face-based personal identity verification method for a mobile terminal of the present invention is included in the terminal use and the terminal itself of the ineligible due to the password exposure by the first verification through the terminal verification unit 100 Prevents data retrieval, encrypts terminal information, primary verification results, and other necessary data to transmit to remote servers through communication network 300, and maintains thorough security through secondary verification in server verification unit 200. It provides a method and apparatus to provide.

1 is a schematic block diagram of a face-based personal identity verification device for a mobile terminal according to an embodiment of the present invention

FIG. 2 is a flowchart illustrating an operation process through a preferred embodiment of the face-based personal identity verification device shown in FIG. 1;

3 is a flowchart for explaining a user registration process in the face-based personal identity verification device for a mobile terminal shown in FIG.

FIG. 4 is a diagram illustrating an example of facial feature extraction capable of expressing a face feature of an individual only with a minimum feature dimension in a detailed operation process of the face-based personal identity verification device illustrated in FIG. 3.

5 illustrates an example of locations of a camera and a display unit for various terminals.

<Description of the code | symbol about the principal part of drawing>

100: terminal verification unit 200: server verification unit

300: communication network 110: face input unit

120: display unit 130: a processor of the terminal

140: terminal database 150: terminal control unit

160: communication unit of the terminal 220: display unit of the server

230: server processor 240: server database

260: communication unit of the server

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a schematic block diagram of a face image based personal identity verification system according to an exemplary embodiment of the present invention, which includes a terminal verification unit 100, a server verification unit 200, and a communication network 300. The terminal verification unit 100 registers or verifies a face through a face input unit 110 for receiving a face image from a camera attached to the mobile terminal, a display unit 120 for displaying the input face image, and various image processing processes. The processor 130, the database 140 for storing face information, the terminal controller 150 for controlling the operation of the terminal according to the verification result, the encryption for data security on the network, and through the communication network 300 Communication control unit 160 for transmitting data, the server verification unit 200 is a communication control unit 260 performing a process of decrypting the data coming through the communication network 300 and encrypting for transmission, work history Display unit 220 for displaying the user, processor 230 for processing user authentication and data to be transmitted to the terminal server, and stores the user information and other service information It is composed of a database 240.

Based on the block diagram of the present invention shown in FIG. 1, the personal identity verification process operating in the terminal and the server is shown in FIG.

First, when the operation is started in the terminal verification unit 100, the terminal is initially set to the user registration mode (S101), and basically set the terminal lock state (S102) to prevent unauthorized use of the terminal by others. Next, check whether the current lock mode is the face mode (S103), if not in the face mode, input the password (S104) from the user, check whether it is the correct password (S105), and release the terminal lock state if it is the correct password (S114). ) And give the user permission to use the terminal. If not, the terminal stays locked (S102).

When the user wants to use the terminal again, if the terminal lock mode is the face mode in step S103, the image input from the camera is displayed on the display unit 120, and the processor 130 extracts the features of the face (S107). . Next, it is checked whether the current mode is the user registration mode (S108). If the user registration mode is registered, face information is registered in the database 140 (S109), and then the user verification mode is set (S110). Release (S114) and grant the terminal use rights. In step S108, if the user registration mode is not the user verification mode, the user information stored in the database 140 and the currently input face information are compared (S111) to verify whether the user is a registered user (S112). If the user is not a registered user, the terminal lock state setting is maintained (S102). If the registered user updates the database 140 to reflect face information of a user that changes with time (S113), and releases the terminal lock state ( After using S114), a user right is granted to operate the terminal (S117).

When the user who used the terminal wants to search for and modify his information in the server through the network, or to perform a task requiring server authentication such as financial transaction or electronic commerce (S115), the communication control unit 160 stores the information in the terminal. The personal information and the current face information are applied to the server through the communication network 300 by applying a commercially available encryption technique (S116).

The server verification unit 200 decrypts the encrypted terminal user information and the face information transmitted through the communication network in the communication control unit 260 and compares the information stored in the server database 240 with the transmitted information to identify the user. Verify (S2O2). If the user is authenticated through the verification (S203), the communication control unit 260 of the server encrypts the use permission information (S206), and transmits it to the terminal through the communication network 300. If the user is not authenticated in step S203, a use rejection message is prepared (S207) and transmitted to the terminal through the communication network 300.

Next, the terminal verification unit 100 receives the authentication information transmitted from the server, and grants the authority to perform operations such as searching and modifying information on the server, financial transactions, and electronic commerce according to the authentication status. Will receive.

In the above embodiment, the verification performance of the personal identity verification system using the face greatly depends on the performance of the facial feature extraction method of step S107, the face registration method of step S109, and the face verification method of step S111. Particularly, in the present invention, when the user registered in step S112 succeeds in verifying the terminal in consideration of the fact that the user's face is deformed as time changes, the step of updating the face database is added by adding the step (S113). Improvement was aimed at.

3 shows an embodiment of a face registration process of an initial user. In particular, the step (S107) and the registration step (S109) of facial feature extraction affecting the verification performance among the steps shown in FIG. 2 are displayed in more detail. The camera used as the input device for the face image in the mobile terminal of the present invention is usually a low resolution camera. As described above, in order to extract relatively stable facial features from the low resolution input image, a preprocessing step (S107a), a face region extraction step (S107b), and a face region normalization step (S107a) to help extract a face region by improving the quality of the input face image ( S107c), a detailed process of the facial feature extraction step S107d that can reflect the characteristics of the individual is necessary.

In the preprocessing step (S107a), since the image input from the low resolution camera generally has a lot of noise and varies according to the lighting and the surrounding environment, it is difficult to use the input image itself as it is for extracting the face region. Therefore, it is necessary to preprocess the histogram equalization, noise reduction, and contour enhancement to facilitate face region extraction. In the present invention, a method suitable for the use environment can be selectively applied.

The face region extraction step S107b is a technique for separating a background from a face from an input image and is a core technology for accurate face verification and face image search. However, the face is severely distorted by facial expressions, lighting, viewpoint changes, and the like, and large changes are caused by the shape of the hair, makeup, and glasses. The method of extracting face region from 2D image has been studied in various ways such as using topographic features of face, statistical analysis method, using neural network, and methods based on features. However, the study of extracting faces from complex backgrounds in general photographic images began in earnest in the 1990s, and the study of real-time face region extraction began in the mid-90s. The face region extraction method may include a face region extraction using a mosaic image, a method using principal component analysis of a face image, a multilayer perceptron method using a mosaic image, and a face region extraction method using an isolated region. In addition, a method of using the feature amount by focusing on the shape and geometrical arrangement of the components of the face (eyes, nose, mouth, and ears) is introduced, and the method using color information, which is another feature of the face, is also introduced. Is being studied. In the present invention, a complex background of a general input image is targeted, and a method of extracting a face region using a geometrical arrangement of main components of a face in order to extract the final face region from the extracted candidate region is a main feature. The method can be modified to suit the terminal environment.

The face region normalization step (S107c) may be performed by changing the brightness of an image due to illumination recognized as a big problem in the face-based identity verification system, changing the size of the face by the distance from the camera, and complicated and diverse backgrounds of the background or the camera. This is an application process to solve the problem of deterioration of verification performance due to the inclination of image due to unconscious habit. Normalization technology applied in the present invention is the brightness correction, tilt correction, face size correction, etc. of the image, this process can also be modified to suit the terminal environment.

The final facial feature extraction step (S107d) is a process of obtaining feature values that can distinguish each individual regardless of facial expressions and viewpoint changes, and may be a core technology of face recognition and face image search and verification techniques. These facial features include the shape and geometrical arrangement of major components such as eyes, nose and mouth, the orientation of the nose, the ratio of the length and nose length between the pupils and the ratio of the nose width. To find feature points, use templates, log-polar transformations, mathematical transformations such as PCA (Principal Component Analysis) or LFA (Local Feature Analysis), deformable models In addition to searching for facial components and features by mapping them, the three-dimensional structure of the face, such as vertical cross-sectional gray scale distribution and isodensity lines, which are three-dimensional shape features of the face, may be analyzed two-dimensionally and used as features. In addition, Gabor wavelet-based feature extraction using biological models of the human visual system has been studied in developed countries. In the present invention, in order to efficiently store a face image of a user, a feature extraction method capable of accurately distinguishing each individual with only minimal feature information may be used as an example. Examples of such feature extraction are shown in FIGS. 4A and 4B. In FIG. 4A, a face of a person may be expressed using only coordinates and texture information represented by 31 black dots. In FIG. 4B, a total of 31 stars may be extracted through an embodiment.

Now, a process of registering a user's face in the terminal based on the extracted facial feature information (S109) will be described. In order to improve the verification performance, the registration step S109 is completed by extracting a feature from a plurality of face images and storing the face information of the user in the database 140 through a learning process. In addition, when the extracted facial feature is used for verification, it is a matter of additionally considering a value that can efficiently represent the extracted feature. Although the extracted facial features may be represented by one value, one facial feature may be represented by dozens of values. For example, when the feature extracted from the face is a pattern for a face shape, the number of pixels constituting the pattern becomes a feature value. When used directly in a search, the efficiency of the search may be reduced due to too many feature values. Therefore, the extracted feature values are set to an appropriate number of representative values. In this case, the average value or the variance value may be representative values. The performance of the verification system can be improved by determining the representative value among the extracted various facial features.

Finally, in the face verification step (S111), when index information of facial features having various shapes is mapped to a single point in a multidimensional space, and the distance difference between them is used as a search criterion, the correlation or distribution diagram between the features Similarity measurement technique is applied to solve the problem of limiting accuracy by evaluating different independent dimensions by simple distance measure. Representative methods of similarity measurement that can be applied in the present invention include SVM (Support Vector Machine) and PCA (Principal Component Analysis) methods, and have the flexibility to select, modify and apply the method to each mobile environment. .

The face registration and verification process on the terminal described through the preferred embodiment as described above can be applied in the same context on the server, and the detailed functions can be appropriately selected and applied according to the characteristics of the performance of the server and the goal of the verification system. Has the characteristics.

As described through an embodiment of the present invention, according to the present invention, by receiving the user's face image from the camera that is basically attached or easily attached to the mobile terminal, performs the first verification in the terminal, and performs a task that requires server authentication In order to provide a dual security method and device that encrypts the user's face information obtained from the terminal through the communication network to the server and verifies the user's face in more detail in the server, exposing the password of the password input method. By controlling the use of ineligible people due to the use of the biometric method, such as fingerprint recognition, iris recognition, vein recognition, etc. There is an advantage that can provide a thorough security through a convenient interface.

In addition, the present invention may obtain different application examples and effects according to the type of the terminal to be applied, the CPU performance, the memory limit, the position of the camera, the resolution of the camera, the position of the display.

First, when the present invention is applied to a notebook terminal, unlike the PDA and IMT-2000 terminal, CPU performance and memory performance is almost similar to that of a general desktop computer, so the high-performance face verification method is used in the first verification process of the terminal verification unit of the present invention. Can be used. However, as the position of the camera and the position of the display unit are slightly different, it is difficult for the user to move his or her face to a position where the face and the eyes can easily extract and recognize the face while looking at the camera. Therefore, the processor of the terminal needs to be able to verify the face in consideration of this point. Validation on a notebook can be used as an alternative to password entry when the notebook is first run and when the screen saver is running in between.

Next, the PDA has a limited processor and memory due to the intermediate performance of the IMT-2000 terminal capable of video communication with the notebook terminal. The position of the camera and the display is also appropriate to some extent so that the user can easily verify the position of the face through the display and start the verification process. However, due to the limitations of the processor and memory, it is necessary to adopt a relatively simple and low face verification method. In the case of such a PDA terminal, it can be used as an alternative means of password input when the terminal is first operated.

Finally, the IMT-2000 terminal capable of video communication has a very basic processor and memory for video communication and some terminal operations due to limitations in the size and weight of the terminal. Therefore, when the present invention is applied to the IMT-2000 terminal, face verification in the terminal itself is not possible at the current level of the terminal, and can be applied by transmitting a face image input through the camera of the terminal to the server and verifying it. . In case of IMT-2000 terminal, it can be used as an alternative means of password input when the terminal is operated for the first time and when the terminal is to be used in the middle.

Claims (6)

In order to achieve the above object, the face-based personal identity verification method for a mobile terminal of the present invention comprises a terminal verification unit 100, server verification unit 200, communication network 300 for Face based personal identity verification method and apparatus. The apparatus of claim 1, wherein the terminal verification unit 100 performs a face input unit 110 receiving a face image from a camera attached to a mobile terminal, a display unit 120 displaying an input face image, and various image processing processes. A processor 130 for registering or verifying a face, a database 140 for storing face information, a terminal controller 150 for controlling the operation of the terminal according to the verification result, and performing encryption for data security on the network, And a communication control unit 160 for transmitting data through the communication network 300, wherein the server verification unit 200 performs a process of decrypting data received through the communication network 300 and encrypting the data for transmission. 260, a display unit 220 for displaying the work history, a processor 230 for processing user authentication and data to be transmitted to the terminal server, user information and other documents Face based personal identity verification method and apparatus for a mobile terminal, characterized in that consisting of a database 240 for storing the scan information. 3. The face-based personal identity of a mobile terminal according to claim 2, wherein the identity verification of the terminal itself and the face image transmitted through the communication network are verified twice so that the electronic image server, the financial server, or a separate server can be verified. Verification method and apparatus. [5] The method and apparatus of claim 3, wherein the terminal itself prevents unauthorized use of the terminal by verifying the user through face recognition. [5] The method of claim 3, wherein when transmitting a face image through a communication network, the face image itself is encrypted as a means for preventing unauthorized duplication of another person, or the feature is extracted and the result is encrypted and transmitted. Face based personal identity verification method and device for a terminal. The face-based personal identity verification method of claim 2, wherein the face data of the user is updated from time to time for each of the verified user's faces because the user's face changes with time. And a device.