WO2010126190A1

WO2010126190A1 - Portable terminal mounted with camera and security method using the same

Info

Publication number: WO2010126190A1
Application number: PCT/KR2009/002323
Authority: WO
Inventors: 김광태; 김형표
Original assignee: 경북대학교 산학협력단
Priority date: 2009-04-30
Filing date: 2009-05-01
Publication date: 2010-11-04
Also published as: KR20100119390A; KR101053197B1

Abstract

The present invention relates to fingerprint recognition using a portable terminal equipped with a camera and a security method using the terminal. To this end, the present invention provides a portable terminal equipped with a camera, comprising: a fingerprint recognition unit for recognizing the fingerprints of a user's fingers; the camera for capturing fingerprint images of the fingerprint recognition unit; and an image matching unit for performing an image matching process between the fingerprint images captured through the camera with previously registered fingerprint images, wherein an authentication function for use of the portable terminal or an unlocking function is carried out if the images are matched in the image matching unit, and a user locking function is set if the images are not matched.

Description

[Correction 26.01.2010 by Rule 26] Mobile terminal equipped with camera and security method using same

The present invention relates to a portable terminal such as a mobile phone equipped with a camera, and more particularly, a portable device that recognizes a fingerprint without having a separate fingerprint recognition device by using the mounted camera and thereby enables authentication or use of the mobile phone. It relates to a terminal and a security method using the same.

Among the various functions of mobile phones, domestic users' love for cameras is exceptional. The most common feature of these phones is calling, texting, and so on, but camera functions are no longer available. As described above, a mobile phone equipped with a camera (hereinafter referred to as a camera phone) is a technology that expresses individual personality well by voice and text message while converging with the needs of young people who want to image through photography often. It can be seen as the most successful type of trend.

Recently, domestic handset makers released 5 million pixel camera phone in 2004 to reflect this trend, and next year, it released 7 million pixel camera phone and then 10 million pixel camera phone in 2006. The situation is changing.

As a result, not only are they replacing digital cameras in everyday recording and daily life photography, but the functions of mobile phones are almost becoming PCs with the improvement of memory performance and the development of high-speed Internet technology, making them the foes of personal information media.

Mobile phones, which have great advantages in portability and convenience, contain the most personal information according to the above-mentioned various functions, and financial services using mobile phones such as mobile banking are also provided, which increases the risk of personal information leakage due to loss and theft. The importance of security is increasing day by day.

According to the present invention and the importance of the security of the mobile phone as described above using a camera mounted on the mobile phone without a separate fingerprint recognition device for fingerprint recognition and through which a mobile terminal and a mobile terminal to enable the authentication or use thereof An object of the present invention is to provide a security method.

In order to solve the above problems, the present invention provides a portable terminal equipped with a camera, a fingerprint recognition unit for recognizing a user's finger fingerprint, the camera for capturing a fingerprint image of the fingerprint recognition unit, and through the camera An image matching section for performing image matching with the turned on fingerprint image and the already registered fingerprint image; When the image is matched in the image matching unit, the portable terminal is used for authentication or unlocking of the portable terminal, and otherwise, a user lock function is provided.

In addition, the fingerprint recognition unit is formed on the upper portion of the camera at a predetermined interval to place the user's finger, and the fingerprint recognition window open in the normal camera shooting mode, the fingerprint recognition mode in the fingerprint recognition mode Closing the recognition window is characterized in that it comprises a fingerprint recognition window opening and closing portion for controlling the opening and closing of the fingerprint recognition window.

In addition, the fingerprint recognition window is characterized in that the tempered glass.

In addition, the fingerprint recognition window is characterized in that the optical film is formed to prevent light from being dispersed and improve the light condensation.

The hologram layer may be formed on a lower surface of the fingerprint recognition window facing the camera.

In addition, a flash is formed on the camera.

In addition, a second memory unit for temporarily storing the fingerprint image captured by the camera to display on the display window, and a first memory for converting the fingerprint image captured by the camera into a compression format and compressing and storing the fingerprint image at a predetermined ratio. And a third memory unit for storing a fingerprint image of a person having a legitimate authority for the use of the portable terminal.

In addition, the fingerprint image stored in the third memory unit may be deleted or added.

The feature extraction of the stored fingerprint image of the first memory unit may be a method using a ridge or a method using a feature point.

According to another embodiment of the present invention, in a portable terminal equipped with a camera, a user selects a fingerprint recognition mode, capturing a fingerprint image of the user through the camera, and a fingerprint image turned on through the camera. Performing image matching with an already registered fingerprint image; unlocking the portable terminal if the image is matched in the matching step; otherwise, setting the portable terminal lock function. to provide.

In addition, when the fingerprint recognition mode is selected, a fingerprint recognition window installed on the top of the camera is closed and a user's finger is placed on the closed fingerprint recognition window to enable fingerprint recognition.

The method may further include converting a fingerprint image captured by the camera through a first memory unit into a compressed format, compressing the fingerprint image at a predetermined ratio, and storing the fingerprint image captured by the camera through the second memory unit before the storage. Temporarily storing for display.

The method may further include extracting a feature of the captured fingerprint image, wherein the fingerprint image feature extraction is characterized by using any one of a method using a ridge of a fingerprint or a method using a feature point of a fingerprint.

The method may further include deleting the already registered fingerprint image or adding a new fingerprint image.

The image matching step may include comparing and determining a fingerprint image registered in a one-to-one form and a captured fingerprint image, and comparing and determining a plurality of registered fingerprint images and the captured fingerprint image in a one-to-many form. It is characterized by.

The mobile phone locking function may be individually set according to each function according to a user selection.

In addition, the user authentication process may be replaced with the fingerprint matching process without inputting the ID and password.

Since the portable terminal according to the present invention can perform fingerprint recognition using a mounted camera without having a separate fingerprint recognition device, the portable terminal can be provided with an inexpensive fingerprint recognition portable terminal without additional cost.

In addition, there is an advantage that can double the security and safety because it is based on fingerprint recognition, not ID or password.

1 is a view showing a fingerprint recognition camera phone according to a preferred embodiment of the present invention.

2 and 3 are views illustrating a fingerprint recognition unit in the camera phone of FIG. 1.

4 is a flowchart illustrating a fingerprint recognition process using the fingerprint recognition camera phone of FIG. 1.

5 is a diagram showing the morphological features of the fingerprint.

6A to 6F illustrate a process of extracting features of a fingerprint image using ridges.

7 is a diagram illustrating a process of extracting features of a fingerprint image using feature points.

8 is a diagram illustrating a fingerprint matching process characterized by a flow of wires.

9 is a diagram illustrating a fingerprint matching process characterized by feature points.

10 illustrates a fingerprint matching score distribution.

Hereinafter, a mobile phone according to a preferred embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a view showing a fingerprint recognition camera phone 170 according to a preferred embodiment of the present invention.

Referring to FIG. 1, the camera phone 170 according to the present invention includes a fingerprint recognition unit 160 for recognizing a user's fingerprint, and a camera for capturing a user fingerprint image of the fingerprint recognition unit 160. The mobile phone unit 102 for transmitting and receiving the voice call or the fingerprint image unique to the mobile phone, the display device 106 for displaying the fingerprint image and the text, and the fingerprint image on the display unit 106. And an interface unit 104 for displaying characters.

The camera unit 100 includes a lens unit 108, an image sensor 110, an analog digital converter (ADC) 112, a digital signal processor (DSP) 114, an image compressor 116, and a first memory unit ( 118), and a camera control unit 120.

The fingerprint image of the user is provided to the image sensor 110 through the lens unit 108. The lens unit 108 may include a zoom lens, a focus lens, an iris, and the like. have.

The image sensor 110 converts and outputs an optical signal of a fingerprint image passing through the lens unit 108 into an electrical signal (imaging image signal), and has functions such as exposure, gamma, gain adjustment, white balance, and color matrix of the camera. Do this. The image sensor 110 may be a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) sensor, etc., and uses a high-pixel sensor of more than 8 million pixels to increase the accuracy of fingerprint recognition It is preferable.

The camera controller 120 is implemented as a central processing unit (CPU) and controls the overall operation of the camera unit 100.

The second memory unit 124 is a plurality of buffers for temporarily storing a fingerprint image input through the lens unit 108 and may be generally implemented as a dynamic random access memory (DRAM).

In a situation where the camera shutter is not pressed in the fingerprint recognition mode through the key input unit 132 of the user, the image signal output from the DSP 114 is output to the second memory unit 124 through the camera controller 120 for a predetermined time. The fingerprint image is temporarily stored as an image frame having an interval, and then the display unit 150 passes through a path of the camera controller 120, the DSP 114, the multiplexer (MUX) 144, and the driver 148 of the display device 106. Displayed in turn on the image. At this time, the fingerprint image entering the lens unit 108 through the display unit 150 can be confirmed by the user, and the non-explained portions 146, 138, and 142 are COMP terminals, OSD (D / A), and encoders, respectively.

The ADC 112 converts the gain-adjusted fingerprint image signal from the image sensor 110 into a digital signal and outputs the digital signal to the DSP 114.

The DSP 114 enables high-speed processing of the digitally converted fingerprint image signal, and sends the image signal to the image compression unit 116 under the control of the mobile phone controller 128.

When the camera shutter is pressed in the fingerprint recognition mode through the key input unit 132 of the user, the image compressor 116 compresses the digitally converted fingerprint image signal provided from the DSP 114 under the control of the camera controller 120. (JPEG format, etc.), the data is compressed at a predetermined rate and stored in the first memory unit 118.

The first memory unit 118 is implemented as a flash memory as a memory for storing the compressed image output from the image compression unit 116, and the amount of the captured image is adjusted according to the memory capacity.

Then, the compressed fingerprint image stored in the first memory unit 118 is decompressed and provided to the DSP 114 under the control of the camera controller 120.

In addition, under the control of the mobile phone control unit 128 provided in the mobile phone unit 102, signals corresponding to various keys input from the key input unit 132 of the mobile phone unit 102 are transmitted under the control of the mobile phone control unit 128. If so, various operations are performed accordingly. Here, the

non-explained portions

126, 130, and 136 are RF units, audio units, and interfaces, respectively.

On the other hand, while the camera control unit 120 and the mobile phone control unit 128 is shown as being configured separately, it can be implemented as a single chip using a single CPU.

The third memory unit 134 is not only a program for performing various operations for implementing a mobile phone function, but also a person who has a legitimate authority for using a mobile phone (for example, a mobile phone owner or a mobile phone owner is authorized to use a mobile phone). I) to store fingerprint information. Here, the fingerprint information of the owner of the mobile phone is stored in the basic format, but the fingerprint information of the person authorized to use the mobile phone from the owner of the mobile phone can be deleted, changed, and inputted so that the information can be changed.

2 and 3 are views illustrating the fingerprint recognition unit 160 in the camera phone 170 of FIG. 1.

2 and 3, the fingerprint recognition unit 160 is formed on the upper portion of the camera unit 170 at a predetermined interval to recognize the user's fingerprint is a fingerprint recognition window for placing the user's thumb The fingerprint recognition window 161 controls opening and closing of the fingerprint recognition window 161 by opening the fingerprint recognition window 161 in the normal camera photographing mode and closing the fingerprint recognition window 161 in the fingerprint recognition mode. ) Opening and closing portion 162 is provided. In this case, opening and closing of the fingerprint recognition window 161 may be implemented by an electric drive method or a mechanical drive method using a small server motor (not shown) through the battery 138 of the mobile phone. In addition, the fingerprint recognition window 161 is a transparent glass that can reflect light is preferably tempered glass in consideration of durability. Unexplained part 111 is a flash.

An optical film (not shown) may be applied or deposited on the inner surface of the fingerprint recognition window facing the camera unit 170 to prevent the light from being dispersed and improve light condensation. In this case, a polarizing film or the like may be used as the optical film, and when the fingerprint is recognized through the camera unit 170 using the reflection of light, the fingerprint recognition rate may be improved without departing from the influence of the external environment such as external lighting.

2 is a view showing a case where a fingerprint recognition window 161 is opened for a conventional camera shooting, Figure 3 is a fingerprint recognition window 161 is closed and the user's thumb placed on the closed fingerprint recognition window 161 fingerprint recognition It is a figure which shows the state which enabled this.

4 is a flowchart illustrating a fingerprint recognition process using the fingerprint recognition camera phone 170 of FIG. 1.

Referring to FIG. 4, when the user selects the fingerprint recognition mode through the key input unit 132, the fingerprint recognition window 161 is closed (S201), and the user places a finger fingerprint on the fingerprint recognition window 161 to display a fingerprint image. It is input (S202).

Thereafter, the fingerprint image input through the lens unit 108 is temporarily stored in the second memory unit 124 (S203), and the user sees the temporarily stored fingerprint image displayed on the display unit LCD 150 (S205). When the image is exactly in the square window is pressed the camera shutter (S204).

Subsequently, when the camera shutter is pressed, the fingerprint image is stored in the first memory unit 118, and a fingerprint image acquisition process is performed (S206).

As such, the camera unit 100 acquires an image reflected from the fingerprint of the finger placed on the fingerprint recognition window 161 through the lens unit 108. In this case, a hologram is formed below the fingerprint recognition window 161 to obtain an image by projecting the image reflected from the fingerprint of the finger placed on the fingerprint recognition window 161 onto the hologram layer. It can be implemented in a manner. Alternatively, the fingerprint recognition window 161 may serve as a prism, and the fingerprint recognition window 161 may be implemented using a prism method that acquires a fingerprint by using total reflection of light incident on the prism. Here, the light source may use an external light source such as sunlight or an internal light source such as an LED may be added to the inside as necessary. An internal light source such as the LED may use the flash 111.

The camera controller 120 extracts a feature of the stored fingerprint image of the first memory unit 124. At this time, the fingerprint image feature extraction may be a method using a ridge and a method using a feature point. In the recognition method using the fusion, the flow degree according to the direction of the ridge is numerically used, and in the recognition method using the feature point, the end point and the branch point are used as the feature. This will be described with reference to FIGS. 5 to 7.

5 is a diagram showing the morphological features of the fingerprint.

Referring to FIG. 5, the flow of various types of lines appearing in the fingerprint is called a ridge, and the space between the ridges and the ridges is called valleys, furrows. The various shapes between the ridges form various feature points (Minutiae), in which the ridge flows smoothly and breaks, called the ending point. It is called).

First, when the fingerprint image as shown in FIG. 6A is input, the point with the largest curvature is found in the image. This point is referred to as the center point and is represented as shown in FIG. 6B. Based on the displayed center point, a large area for calculating the flow of the ridge is set as shown in FIG. 6D, and the large area is divided into small areas as shown in FIG. 6C by dividing the large area by a predetermined distance and a direction interval. For each area, the flow rate of the ridge is calculated in several predetermined directions as shown in FIG. 6E, and the size is recorded for each small area in the form as shown in FIG. 6F and used as a feature.

7 is a diagram illustrating a process of extracting features of a fingerprint image using feature points. Referring to FIG. 7, the process of improving the quality of the input fingerprint image suitable for ridge extraction, the process of finding the direction component of each ridge, and distinguishing the ridges and the valleys to extract the evolving process and finally the feature points For this purpose, the thickness of each ridge is divided into a process of thinning it into a line having a pixel thickness of one pixel.

After extracting the features of the fingerprint image as described above, the fingerprint image is subjected to a fingerprint matching process of measuring the similarity between the fingerprint image of the person having the authority to use the mobile phone databased in the third memory unit and the two fingerprints. The fingerprint matching process will be described with reference to FIGS. 8 to 10 (S208).

FIG. 8 is a diagram illustrating a fingerprint matching process characterized by a stream of wires. In matching with a flow of ridges, the fingerprint feature image information previously stored in the third memory unit 134 and the first memory unit 134 may be used. The difference of the fingerprint feature image information extracted from the stored fingerprint image is compared and represented as a matching score.

9 is a diagram illustrating a fingerprint matching process characterized by feature points, and extracted from fingerprint feature image information (here, stored feature point information) stored in the third memory unit 134 and the fingerprint image stored in the first memory unit 134. The degree of coincidence of each feature point of the fingerprint feature image information (herein, extracted feature point information) is calculated as a score.

Representative methods for evaluating system performance in fingerprint matching include False Acceptance Ratio (FAR) and False Rejection Ratio (FRR). Here, acceptance of another person is wrongly determined as the fingerprint of the same user even though a fingerprint of a user different from the registered user's fingerprint is input. It is wrong to judge others.

10 illustrates a fingerprint matching score distribution.

Referring to FIG. 10, there is a part that does not properly distinguish between the same person and others, such as the fingerprint matching score distribution, and when one threshold value is used, these parts become the other person's acceptance and the person's rejection, thereby deteriorating the overall fingerprint recognition performance. Let's go. In this case, the fingerprint recognition performance can be improved by rejecting the determination of this range.

Here, a person who has a legitimate authority for using a mobile phone registered in the third memory unit, for example, as compared with a mobile phone owner, compares the registered fingerprint information with the input fingerprint information in a one-to-one form. The case of judgment was explained.

In addition, there may be several persons with legitimate authority over the use of mobile phones. Therefore, in this case, the fingerprint information of a person who has a legitimate authority for the use of the mobile phone in the third memory unit 134, for example, a person authorized to use the mobile phone by the mobile phone owner, is newly stored or an existing delegation relationship is established. When it is finished, it can be deleted or changed, and when there are a plurality of authorized persons, fingerprint information with a plurality of legitimate authorities is stored.

In this case, the fingerprint information of the user is input through the fingerprint recognition unit 160 and the registered fingerprint information of a plurality of legitimate persons stored in the third memory unit 134 is compared with one-to-many. This may be implemented in a form in which a plurality of one-to-one comparisons are sequentially made.

According to the fingerprint matching process, if the fingerprint matching score distribution is within the same range, the mobile phone lock function is released and enabled (S209), otherwise, a lock function is set to the important use function (S210).

Here, the mobile phone lock function may be individually set for each function, such as personal information that the user considers important, text message information, mobile banking related authentication information, call history information, and the like.

In addition, the fingerprint matching process may be replaced with the fingerprint matching process without going through the user authentication process through the ID and password as well as the mobile phone lock function.

In addition, although described with reference to a mobile phone, the present invention is not limited thereto and may be widely applied to a portable terminal device such as a PDA and a game machine equipped with a high pixel camera capable of fingerprint recognition.

While the invention has been shown and described in connection with specific embodiments thereof, it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention by the claims. Anyone can grow up easily.

Since the present invention can be very widely used as a security device for a portable terminal with a camera, it is considered that the present invention can be widely used as a security device for an expensive portable terminal such as a game machine and a digital camera as well as a camera phone which is widely used.

Claims

In a portable terminal equipped with a camera,

A fingerprint recognition unit for recognizing a user's finger fingerprint,

The camera for capturing a fingerprint image of the fingerprint recognition unit;

An image matching unit configured to perform image matching with a fingerprint image turned on through the camera and a fingerprint image already registered;

And if the image is matched in the image matching unit, authenticate or unlock the portable terminal; otherwise, set up a user lock function.
The method of claim 1,

The fingerprint recognition unit is formed on an upper portion of the camera at predetermined intervals so that the user's finger can be placed thereon, and the fingerprint recognition window is opened in the normal camera shooting mode and the fingerprint recognition window in the fingerprint recognition mode. A portable terminal comprising a fingerprint recognition window opening and closing to control the opening and closing of the fingerprint recognition window by closing the.
The method of claim 2,

The fingerprint recognition window is a portable terminal, characterized in that the tempered glass.
The method of claim 2,

A portable terminal, characterized in that the optical film for preventing light from being dispersed in the fingerprint recognition window and condensing light is formed.
The method of claim 2,

And a hologram layer formed on a lower surface of the fingerprint recognition window facing the camera.
The method of claim 1,

Portable terminal, characterized in that for forming a flash on the camera.
The method of claim 1,

A second memory unit which temporarily stores a fingerprint image captured by the camera to be displayed on a display window;

A first memory unit converting the fingerprint image captured by the camera into a compression format and compressing and storing the fingerprint image at a predetermined ratio;

And a third memory unit for storing a fingerprint image of a person who has a legitimate authority for the use of the portable terminal.
The method of claim 7, wherein

The fingerprint image stored in the third memory unit can be deleted or added.
The method of claim 7, wherein

The feature extraction of the stored fingerprint image of the first memory unit may be a method using a ridge or a method using a feature point.
On portable terminals with cameras,

Selecting a fingerprint recognition mode by the user,

Capturing a fingerprint image of a user through the camera;

Performing image matching with a fingerprint image turned on through the camera and a fingerprint image already registered;

If the image is matched in the matching step, unlocking the portable terminal; otherwise, setting the portable terminal locking function.
The method of claim 10,

When the fingerprint recognition mode is selected, the fingerprint recognition window installed on the top of the camera is closed, and the user's finger is placed on the closed fingerprint recognition window to enable fingerprint recognition.
The method of claim 10,

Converting the fingerprint image captured by the camera through a first memory into a compression format and compressing and storing the fingerprint image at a predetermined ratio;

Temporarily storing a fingerprint image captured by the camera through a second memory unit prior to the storing for display on a display window

Security method of a portable terminal comprising a.
The method of claim 10,

And extracting a feature of the captured fingerprint image, wherein the fingerprint image feature extraction uses any one of a method using a ridge of a fingerprint or a method using a feature point of a fingerprint.
The method of claim 10,

And deleting the registered fingerprint image or adding a new fingerprint image.
The method of claim 14,

The image matching step may include comparing and determining a fingerprint image registered in a one-to-one form and a captured fingerprint image;

And comparing the plurality of registered fingerprint images with the captured fingerprint images in a one-to-many form.
The method of claim 10,

The mobile phone locking function is set separately according to each function according to the user's selection.
The method of claim 10,

And a fingerprint matching process without replacing the user authentication process with the ID and password.