KR20040048114A - Authentication method and device using multi biological identification in portable radiotelephone - Google Patents

Abstract

PURPOSE: A device and a method for certification through the multi-biometric recognition on a portable terminal are provided to perform the certification with one minutia information obtained from a recognition result through the biometric recognition on the portable terminal and extract the minutia information from a fingerprint image obtained through a fingerprint recognition sensor. CONSTITUTION: An image input part(217) captures an iris, a face, a vein, and a hand shape, converts an optical signal inputted through a lens into digital data, and outputs the digital data to a controller(211). A fingerprint detecting sensor(219) senses/outputs the fingerprint image to the controller. A voice processor(213) converts a voice signal inputted through a microphone(231) into the digital data and outputs the digital data to the controller. A touch screen panel(215) outputs the coordinate data generated by touching a panel to the controller. The controller preprocesses the biometric recognition data inputted from each part, extracts the minutia information from the preprocessed data, and judges the certification by matching the extracted minutia information with the registered minutia information.

Description

AUTHENTICATION METHOD AND DEVICE USING MULTI BIOLOGICAL IDENTIFICATION IN PORTABLE RADIOTELEPHONE}

The present invention relates to a portable terminal, and more particularly, to an apparatus and method for authentication through multiple biometrics in a portable terminal.

In recent years, desktop PCs have been miniaturized into notebooks, laptops, and handheld PCs, and the popularity of personal digital assistants including electronic notebooks, mobile communication terminals, and personal digital assistants (PDAs) is rapidly increasing. This is to meet the needs of consumers who want to do anything (anytime, anywhere, anything) anytime, anywhere. For this reason, the necessity of identity verification (user verification) using a personal portable terminal in a mobile environment has emerged.

As a solution of this, many information protection technologies have been developed, and biometrics using the characteristics of the body as one of the methods has also been developed into a very important technology field. The human body has many features, but the face, vulva, iris, retina, foot print, palm, fingerprint, signature, DNA, etc. can be used for biometrics.

With these physical features, if you represent your own unique password, your own physical feature is your password, which means that you can remove all the tools that have been used to indicate that you are yourself. it means.

However, the biometric schemes proposed in the related art have limitations in terms of performance and reliability because authentication is performed by recognizing one physical feature. For example, about 3% of people have no fingerprints or are not good enough to use for authentication, iris recognition is difficult to focus, facial recognition is sensitive to the surrounding lighting environment, Recognition is sensitive to ambient noise. Therefore, fingerprint recognition is useless for a person with poor fingerprint status, face recognition is impossible in a poor lighting condition, and voice recognition is difficult in a place with a lot of noise. However, if a multi-biometric system is developed instead of biometrics limited to one body feature, it will be a way to improve the performance and reliability of personal authentication.

Accordingly, an object of the present invention is to provide a terminal capable of multiple biometrics and a signal processing method thereof.

Another object of the present invention is to provide an apparatus and method for performing authentication in at least one biometric identification and at least one feature information obtained as a result of the recognition in a portable terminal.

Still another object of the present invention is to provide an apparatus and method for extracting feature information from a fingerprint image acquired through a fingerprint sensor.

According to a first aspect of the present invention for achieving the above objects, the mobile communication terminal device is capable of photographing at least one or more of the iris, face, vein and hand shape, converts the optical signal incident to the lens into an electrical signal And an image input unit for converting the electrical signal into digital data and outputting the digital signal to the controller, an illumination unit for emitting the illumination for photographing, a fingerprint sensor for sensing a fingerprint image and outputting it to the controller, and a voice input from a microphone. A voice processing unit for converting a signal into digital data and outputting the signal to the controller, a touch screen panel for outputting coordinate data generated by contacting an upper surface of the panel to the controller, the image input unit, the fingerprint sensor, and the authentication mode. Biometric data input from at least one of a voice processing unit and the touch screen panel. And a control unit configured to preprocess each of the biometric types, extract biometric feature information from the preprocessed data, and determine whether to authenticate by matching the extracted feature information with pre-registered feature information. It is characterized by.

According to the second aspect of the present invention, the authentication method through biometrics in a mobile communication terminal capable of multiple biometrics, in the biometric information registration mode, feature information from a plurality of biometric information received from a plurality of biometric modules Extracting and registering the information in the memory, checking whether the lock mode is set when the terminal access is checked, and biometric types of the registered feature information when the lock mode is set. Reading the message, sequentially displaying a message requesting a user action for each of the read biometric types, and feature information from the plurality of biometric information received from the plurality of biometric modules. Extracting the extracted features; matching the extracted feature information with the registered feature information; And displaying the authentication success message and releasing the lock mode.

According to a third aspect of the present invention, a method for extracting feature information from a fingerprint image sensed by a fingerprint recognition sensor includes a process of removing a background from the fingerprint image and calculating a variance of pixel values of the image from which the background is removed. Performing a histogram equalization based on the calculated variance, normalizing the pixel value mean and standard deviation of the image to a predetermined value, and performing a 16 on the smoothed image. The standard deviation of the x16 blocks is determined as a reference value, the blocks below the reference value are set as the background, and the segmentation above the reference value is performed in the foreground, and the segmented image is morphed into a window of 3x3 blocks. morphology, and smoothing the morphologically processed image to remove noise. Dwelling, gabor filtering the smoothed image based on the direction of the fingerprint and the frequency of the ridge, converting the gabor filtered image into a binary image, and smoothing the binary image. Performing noise reduction to remove noise, thinning the noise-removed binary image such that the ridge is represented by only one pixel, and extracting feature points from the thinned image. It is characterized by.

1 is a view showing the external configuration of a mobile communication terminal according to an embodiment of the present invention.

2 is a block diagram of a mobile communication terminal according to an embodiment of the present invention;

3 is a diagram illustrating an authentication procedure through multiple biometrics according to an embodiment of the present invention.

4 is a diagram showing a detailed configuration of the fingerprint recognition sensor described in FIG.

5 is a diagram illustrating a procedure for processing image data (image) according to fingerprint recognition according to an embodiment of the present invention.

6 shows a fingerprint image with the background removed.

7 shows a fingerprint image after segmentation processing.

8 shows directional data of a fingerprint image before smoothing.

9 shows directional data of a fingerprint image after a smoothing process.

10 is a diagram showing a fingerprint image after Gabor filtering process.

<Brief description of symbols for the main parts of the drawings>

211 control unit 212 memory

213: voice processing unit 214: display unit

215: touch screen panel 216: key input unit

217: image input unit 218: lighting unit

219: fingerprint sensor 220: channel coding unit

221: diffusion modulation unit 222: transmitting RF unit

223: channel coding unit 224: despreading demodulation unit

225: receiving RF unit 226: transmitting unit

227: receiver 228: duplexer

229: antenna 230: speaker

231: microphone

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, if it is determined that the detailed description of the related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

The present invention described below relates to a technique for performing authentication through multiple biometrics in a portable terminal. As mentioned above, the portable terminal may be a laptop, a laptop, a handheld PC, an electronic notebook, a mobile communication terminal, a personal digital assistant (PDA), and the like. I will explain.

Looking at the biometrics that can be applied to the present invention. However, it is noted that the present invention is not limited to the following types of biometrics and other biometrics can be used.

First, referring to the fingerprint recognition, the fingerprint recognition system refers to a system that checks the user's identity by reading the user's finger electronically and comparing it with pre-input data. Although there is an inconvenience in that an accurate image is not recognized due to a problem in a user's hand or a problem of receiving a scanner, it is excellent in convenience and safety in a manner commonly known in the biometric technology field. There are optical scanners and non-optical systems for receiving fingerprint images. Non-optical scanners include semiconductor, CCD, CMOS, Tactile, and ultrasonic sensors. have. Fingerprint recognition technology can be divided into three element technologies. Technology to acquire fingerprint image through input device, technology to extract feature of fingerprint from acquired fingerprint image, matching by comparing feature with user's feature information pre-stored in database It can be divided into the technology to judge.

Next, looking at the iris recognition, the iris recognition system is a system for authenticating a user using a donut-shaped iris pattern that exists between the central black pupil and the sclera in the human eye. In general, a person will have his or her own iris within three years of birth. In order to widely use the iris, which is known to have a higher discrimination ability among individuals than any other biometric features, as a biometric means, a technique for effectively expressing the characteristics of the living body and a technology for providing a more convenient environment to the user are required. Personal identification using iris consists of iris image acquisition part and iris recognition part. The image acquisition part performs the function of acquiring the eye image of the quality suitable for iris recognition, and the iris recognition part is the acquired eye image and performs functions such as iris region extraction, iris feature extraction, iris code generation, and feature matching. Perform.

Input devices required for iris image acquisition include a CCD camera, a video camera, illumination to obtain a clear iris pattern, and a capture board or frame grabber for converting an analog image into a digital image. Such an input device is related to the performance of the whole system in terms of the system, and is one of the important components of the iris identification system in terms of user convenience. In the iris recognition part, iris region extraction is a step of separating the iris region from the eye image acquired through the input device, which plays a very important role for consistent iris feature extraction. Iris feature extraction is a step of extracting the feature of the pattern pattern appearing in the iris region, it is expressed as a value reflecting the change in the gray value of the iris pattern pattern. The iris code generation step encodes the extracted iris characteristics in a predetermined form for efficient storage and comparison of the iris code. The identification of an individual using the iris uses the similarity of the two iris codes to be compared. Is done.

Next, when looking at the vein recognition, the vein recognition system is a system that recognizes the shape of the blood vessels of the palm or wrist, it has the advantage of reducing the user's rejection and can be used by people without fingerprints or fingers. . Vein patterns are also invisible, but they have their own unique patterns like other biometrics. The process of extracting the vein pattern uses infrared light and a filter to maximize the contrast of blood vessels to see the vessels, and then extracts the distribution of veins and stores them in a database to recognize outliers and to compare overall vessel shapes. Check. Here, the infrared light is obtained as a digital image through a CCD camera, and the image captured by the CCD camera can acquire excellent features for use in personal identification.

Next, looking at face recognition, the face recognition system refers to a system for identifying the identity using a given face database of one or more faces present in a still image or video. Unlike other biometric technologies, face recognition technology does not have to directly contact a part of its body with a recognition device, and thus has no feature of rejection and discomfort that can be given to users due to less coerciveness in the method of obtaining biometric information. The method of extracting the singularity of the facial image uses the nodes of the forehead and upper lip, which do not change in facial expression and hairstyle. In the case of a still image, a scanner is used as an input device, and a CCD camera installed in a bank is used for a moving image. When face detection is successful, compensation for illumination or compensation for various deformations is made, and the face image which has undergone this normalization process is determined by whose recognition algorithm.

Next, looking at the speech recognition, the speech recognition system is a system that identifies the identity as a feature of the voice that appears during speech is a system that does not require any special equipment other than the microphone. This method has no merit of loss and there is an advantage that user authentication can be performed through the communication network without any special equipment in the remote place. The recognition principle is to acquire the voice data, extract the features using the LPC-Cepstrum coefficients, store them in a database by vector quantization, and compare the LPC coefficients of the input voice signals with the stored database. In addition, neural networks have been widely used in recent years.

Next, with regard to handwriting (or signature) recognition, handwriting recognition systems identify people by their unique handwriting. The signature, which is uniquely created by the pressure and the movement, direction, and shape of the pen, is carried out repeatedly through large and small deformations over a long time, and is fixed in a unique form. Personal identification using signatures can be classified into on-line and off-line methods. In the offline method, there is an optical method using a camera and a scanner. Thus, after the signature is first created, the signature is entered and signature verification is performed. The main feature used to distinguish signatures in the offline method is the appearance of the signature. On the other hand, in the online signature verification method, a signature is input by using an electronic pen and a tablet, and a signature is created and verification processing is performed. In addition to the appearance of the signature, the main features used in the online method include the abscissa, ordinate and velocity, pressure, and pen tilt information over time.

The present invention described below is based on the biometric methods described above, and in particular, the present invention includes at least one device (or module) capable of at least one biometric recognition of a portable terminal and at least one recognized from the devices. Characterization is characterized by performing the biometric information of.

1 shows an appearance configuration of a mobile communication terminal 10 according to an embodiment of the present invention.

As shown, the terminal 10 according to the present invention is composed of a main body 20 and a folder 30 as a sub-body.

On the main body 20, a keypad assembly 21 composed of a plurality of key buttons is installed. The keypad assembly 21 may include a plurality of numeric key buttons at the bottom and a predetermined navigation key button at the top. Under the keypad assembly 21, a microphone device 22, which is a talker for inputting voice by a terminal during a call, is installed. The microphone device 22 is used as a tool for recognizing a user's voice.

The display device 31 is installed in the folder 30. The display device 31 is combined with the LCD (LCD) module and the touch screen module. Accordingly, as shown in the drawing, various types of information may be input using a predetermined stylus pen 80. According to the present invention, the touch screen module and the stylus pen are used as a tool for recognizing a user's handwriting. In the upper portion of the display device 31, a speakerphone device 32, which is a receiver for outputting the other party's voice, is installed.

The folder 30 and the main body 20 may be installed to be opened and closed by rotating the axis A of FIG. 1 by a predetermined hinge device 50, the opening and closing angle may be a convenient angle for the user to talk, It preferably has an opening and closing angle range of 135 degrees to 145 degrees.

In addition, a lens assembly 60 for using the terminal 10 as an optical photographing device may be built in the hinge device 50, and the lens assembly 60 may include a knob installed at one end of the hinge device 50. 61 is installed rotatably in conjunction with 360). This is to match a unique shooting angle or position for each subject. The lens assembly 60 is used as a tool for recognizing the shape of the user's iris, face, hand, and vein. In addition, a predetermined lighting device 33 is installed in place of the display device 31. The lighting device 33 is for emitting light (for example, infrared light) corresponding to a unique photographing characteristic for each subject.

In addition, a fingerprint recognition assembly 70 that is electrically removable on one side of the terminal is installed. The fingerprint recognition assembly 70 has a jack as a terminal and is inserted into a jack hole formed in the main body 20 to perform electrical connection with the terminal. However, the present invention is not limited thereto, and the fingerprint recognition assembly 70 may use other electrical connection means, such as flexible connection means such as fusible printed circuits.

2 is a block diagram of a mobile communication terminal according to an embodiment of the present invention.

As illustrated, the mobile communication terminal includes a control unit 211, a memory 212, a voice processing unit 213, a display unit 214, a touch screen panel 215, a key input unit 216, an image input unit 217, Illumination unit 218, fingerprint detection sensor 219, channel coding unit 220, diffusion modulation unit 221 and transmission RF unit 222, channel decoding unit 223, despreading demodulation unit 224, reception RF The unit 225 includes a duplexer 228, an antenna 229, a speaker 230, and a microphone 231.

Referring to FIG. 2, first, the controller 211 controls the overall operation of the mobile communication terminal. In particular, in an embodiment of the present invention, the control unit 211 performs user authentication with biometric information received from at least one or more biometric devices. The control unit 211 is typically a one chip microprocessor, and may use, for example, a mobile station modem (MSM) chip manufactured by Qualcomm.

The memory 212 includes a volatile memory (eg, RAM) and a nonvolatile memory (eg, flash memory, EEPROM). The memory 212 is generated according to program data, initial service data, and performance of an operation for controlling the overall operation of the mobile communication terminal. Store temporary data. In particular, the memory 212 stores programs for extracting biometric information by recognizing a fingerprint, an iris, a vein, a face, a handwriting, etc. and a plurality of biometric information previously registered by a user according to the present invention. For example, the biometric information may be fingerprint recognition information, iris recognition information, vein recognition information, face recognition information, handwriting recognition information, and the like as described above.

The voice processor 213 converts the PCM data from the controller 211 into an analog voice signal and outputs the same to the microphone 230, and converts the analog voice signal from the microphone 231 into PCM data to the controller 211. Output Here, in the normal mode, the PCM data from the control unit 211 is the other party's voice data received from the base station (BTS), and the PCM data output to the control unit 211 is the voice data transmitted to the base station (BTS). In the voice recognition mode, the controller 211 extracts biometric information (feature information) from the PCM data input from the voice processor 213 and stores the biometric information (feature information) in a specific area of the memory 212 (register biometric information). Mode), authentication is performed by comparing (or matching) biometric information registered in advance (authentication mode).

The display unit 214 displays an overall state of the mobile communication terminal and input user input information and the like on a display window (for example, a liquid crystal display (LCD)) under the control of the controller 211. The touch screen panel 215 provides the controller 211 with coordinate data generated when the user touches the upper surface of the panel using a hand, a writing instrument (stylus pen), or the like. The memory 212 includes a conversion table for converting the coordinate data into a character code, and the controller 211 outputs a character code corresponding to the received coordinate data to the display unit 214 to display a predetermined area of the display window. Display on. Here, the liquid crystal display window and the panel are integrally formed. That is, the user writes letters on the upper surface of the display window using a writing instrument or the like. In the handwriting recognition mode, the controller 211 extracts biometric information (feature information) from the coordinate data from the touch screen panel 215 and stores it in the memory 212 (biometric information registration mode), or pre-registers it. Authentication is performed by comparing the biometric information (authentication mode).

The key input unit 216 includes a plurality of numeric key buttons, navigation key buttons, and the like, and transmits a key signal corresponding to a key selected by the user to the controller 211.

The image input unit 217 converts the optical signal incident to the lens into an electrical signal, converts the electrical signal into digital data (image data), and outputs the converted digital signal to the controller 211. For example, the image input unit 217 may use a charge-coupled device (CCD) camera. The image input unit 217 is used for video communication in the normal mode, and is used as a tool for recognizing the iris, vein, hand shape and face in the biometric mode. The controller 211 extracts biometric information (feature information) from the image data (iris imaging data, vein imaging data, hand imaging data, and facial imaging data) from the image input unit 217 to determine the memory 212. Store in the area (biometric information registration mode) or perform authentication by comparing biometric information registered in advance (authentication mode).

The lighting unit 218 emits light (eg, infrared light) to help photograph the subject of the image input device 217. Here, the lighting unit 218 may emit different lights according to a subject (face, iris, vein) under the control of the controller 211.

The fingerprint sensor 219 converts the electrical signal according to the fingerprint detection into digital data (image data) and outputs it to the controller 211. As mentioned above, the fingerprint sensor 219 is largely classified into an optical scanner method and a non-optical scanner method. In the present invention, it is assumed that a relatively small volume non-optical sensor (heat sensor, semiconductor sensor, LCD sensor, ultrasonic sensor, capacitor sensor, etc.) is used. For example, an ATMEL thermal sensor 'FCD4B14' may be used. The fingerprint sensor 219 will be described later in detail with reference to FIG. 4. The controller 211 extracts biometric information (feature information) from the image data from the fingerprint sensor 219 and stores the biometric information (feature information) in a predetermined region of the memory 212 (biometric information registration mode), or preregistered biometric information. Authentication is performed in comparison with (Authentication mode).

The channel coding unit 220 performs channel coding and interleaving on the transmission message from the control unit 211, converts the transmission message into an analog baseband signal, and provides the spreading modulator 221. The spreading modulator 221 spreads and modulates the baseband signal from the channel coding unit 220 through orthogonal spreading and PN spreading. The transmitting RF unit 222 converts the spread modulated signal from the spread modulator 221 into a high frequency (Radio frequency) signal and outputs the signal to the antenna 229 through the duplexer 228. The antenna 229 radiates the change of the electrical signal on the lead to electromagnetic waves in the air. The channel coding unit 220, the spreading modulator 221, and the transmitting RF unit 222 will be referred to as a 'transmitter'.

The duplexer 228 separates a transmission signal and a reception signal. In other words. The transmission signal from the transmission RF unit 222 is transmitted to the antenna 229, and the signal received through the antenna 229 is transmitted to the reception RF unit 225. The receiving RF unit 225 converts a high frequency (RF) signal received from an adjacent base station into a baseband analog signal by adjusting the frequency down and outputs the baseband analog signal. The despread demodulator 224 despreads and demodulates the output signal from the reception RF unit 225 through PN despreading and orthogonal spreading. The channel decoding unit 223 restores the despread demodulated signal from the despread demodulator 224 to an original information message through deinterleaving and channel decoding, and restores the restored information message. The control unit 211 is provided. The channel decoding unit 223, the despread demodulator 224, and the receiving RF unit 225 will be referred to as a 'receiver'.

If user authentication is required on a system such as financial settlement, the extracted biometric information is packetized according to a predetermined rule and transmitted to the base station (system) via the transmitter 226. Although not shown, the mobile communication terminal may include a wireless LAN card for accessing the Internet.

3 illustrates a procedure for performing authentication through at least one biometric in a mobile communication terminal according to an embodiment of the present invention.

Referring to FIG. 3, the controller 211 first checks whether there is terminal access in step 301. For example, it checks whether a folder opening is detected or if there is a key button input. When it is determined that there is access to the terminal, the controller 211 checks whether the current mode is the "lock mode" in which the terminal access is prohibited in step 303. If it is not the lock mode, the controller 211 proceeds to step 328 and performs a corresponding function according to a user's key input. In the lock mode, the control unit 211 proceeds to step 305 to read the biometric types necessary for authentication from the memory 212. That is, the biometric types of the biometric information registered in the memory are read. As mentioned above, the biometric type may be fingerprint recognition, iris recognition, vein recognition, face recognition, handwriting recognition, or the like.

In operation 307, the control unit 211 displays a message requesting a user action according to a specific biometric in a predetermined order among the read biometric types. For example, assuming that the necessary biometrics are fingerprint recognition and vein recognition, and the predetermined order is fingerprint recognition-> vein recognition, the control unit 211 first displays the message "Please enter the fingerprint of the right thumb". Display on. In this case, the user contacts the thumb with the fingerprint recognition assembly 70 of FIG. 1. Then, the fingerprint recognition assembly 70 outputs the image data (biometric information) according to the fingerprint of the thumb to the controller 211.

That is, the controller 211 determines whether biometric information is received from the biometric module in step 309. When the biometric information is received, the controller 211 performs preprocessing on the biometric information according to a corresponding algorithm in step 311, extracts feature information from the preprocessed biometric information in step 313, and stores the memory. It stores in the predetermined area of 212. In operation 315, the controller 211 determines whether the last biometric identification of the required biometric types is completed according to the predetermined order. If the last biometric is not completed, the flow returns to step 307 to request the user for the next biometric.

Otherwise, if all necessary biometrics have been performed, the controller 211 proceeds to step 317 to read at least one biometric characteristic information stored in the memory 212. In operation 319, the controller 211 matches the at least one piece of biometric characteristic information with a template (biometric characteristic information) registered in advance. For example, feature information corresponding to each other may be matched, or acquired feature information is combined by a predetermined rule (eg, exclusively added by a predetermined rule) to create a new data string, and the new data string and biometrics are recognized. You can also match data columns created in the same way in registration mode. This is to generate a stronger authentication key (authentication data) by processing (encrypting) a plurality of biometric characteristic information once again.

On the other hand, the control unit 211 checks in step 321 whether the currently acquired feature information (authentication key) and the pre-registered feature information (authentication key) match within a given allowable value. Here, the feature information corresponding to each other is matched to check whether there is any match. If any match is found, authentication success processing is performed. As another example, authentication success may be performed when all are matched.

If it is determined that the match is successful, the control unit 211 proceeds to step 323 and displays an authentication success message on the display window. In step 325, the controller 2 releases the lock mode to allow terminal access and ends the program. If it is determined that the matching fails, the control unit 211 proceeds to step 327 and displays the authentication failure message on the display window and ends the present program.

Looking at the fingerprint recognition of the above-described biometric types in more detail as follows.

4 illustrates a detailed configuration of the fingerprint recognition sensor 219 described with reference to FIG. 3.

As shown, the fingerprint sensor 219 is a column selector 401, a sensor array 402, a line selector 403, an amplifier 404, the first ADC 405, the second ADC 406, the latch ( 407, a chip temperature sensor 408 is configured.

Referring to FIG. 4, the sensor array 402 is composed of 281 columns including dummy columns to indicate the start position of the sensor array, and each column is composed of 8 pixels. Accordingly, the sensor array 402 outputs an electrical signal for a total of 2248 pixels, and the electrical signal changes according to the pressure and thermal image of a finger placed on the contact surface.

The column selector 401 generates a selection signal for selecting a column of the sensor array 402. The sensor array 402 outputs electrical signals for the pixels of the column according to the selection signal from the column selector 401. In this case, since each column includes eight pixels, a total of eight electrical signals are output. The line selector 403 outputs a line selection signal to the amplifier 404 for sequentially selecting electrical signals from the sensor array 402. The amplifier 404 sequentially amplifies and outputs electrical signals from the sensor array 402 according to the line selection signal. The amplifier 404 outputs an odd number signal (pixel) to the first ADC 405 and an even number signal (pixel) to the second ADC 406.

The first ADC 405 and the second ADC 406 convert and output the electrical analog signal from the amplifier 404 into 4-bit digital data, respectively. The 8-bit data generated as described above is provided to the controller 211 through a latch 407, and the controller 211 controls 8-bit 256 data of each 4-bit data obtained through the first ADC 405 and the second ADC 406. The gray value is converted to preprocessing for feature point extraction. On the other hand, the chip temperature sensor 408 detects a temperature difference before reading information from the sensor array 402 and a temperature difference after reading the information. The detected temperature difference is used to correct an error of the sensing data. When sensing as described above, one frame includes 2240 pixels of information, and one pixel is represented by four bits. The controller 211 processes image data (image) input in units of frames from the latch 407 to extract fingerprint feature information.

5 illustrates a procedure for processing image data (image) based on fingerprint recognition according to an embodiment of the present invention.

Referring to FIG. 5, first, in step 501, the control unit 211 determines whether image data (fingerprint image) according to fingerprint recognition is received from the fingerprint recognition sensor 219. If the fingerprint image is received, the controller 211 removes a background from the image and selects only the fingerprint region in step 503 to perform normalization on the fingerprint region. For example, divide the whole image into 5 × 5 size blocks to find the difference between the four vertices of each block, and determine the area where the difference is large as the border line that changes from background to foreground (fingerprint area) or from foreground to background. Removes a background from the received fingerprint image according to the determined boundary line. 6 shows the image with the background removed.

In operation 505, the controller 211 calculates a variance of the image from which the background is removed. First, an average of all pixels of the image from which the background has been removed is calculated, and the difference between the average and the mean is calculated from each pixel value, and then the average is obtained to determine the variance value. The equation for obtaining the variance value σ ² is as follows.

σ ² = E [{XE (X)} ² ]

In operation 507, the controller 211 performs histogram equalization on the image from which the background is removed, and an average of all pixels is 150 (for each pixel). When the brightness value is 0 to 255 (256 levels), normalization is performed so that the standard deviation is 70. That is, the normalization is a function of converting an image having a general brightness for normal image processing when an image too bright or too dark comes in.

After performing the smoothing, the controller 211 performs a segmentation process on the smoothed image in step 509. For example, the average pixel value of the 16 × 16 block and the standard deviation of the smoothed image are obtained, and the standard deviation is determined as the reference value. If the pixel value of the 16 × 16 block is less than the reference value, the image is processed as a background (all the blocks are processed as black data). If the pixel value is higher than the reference value, the fingerprint region of good quality is suitable for extracting minutia information of the fingerprint. Decide on That is, the smoothed image is segmented into a fingerprint area, which is a background and a foreground. 7 shows an image after the segmentation process.

After performing the segmentation, the controller 211 calculates the direction of the ridge θ for the 8 × 8 block size in step 511. This calculated direction (θ) is used as the feature point data. In operation 513, the control unit 211 morphology the image segmented into the 16 × 16 block size into a window of 3 × 3 blocks. The accompanying FIG. 8 shows the directionality before processing directional smoothing for morphologically imaged windows with 3 × 3 blocks. In operation 515, the controller 211 smoothly processes a sudden change in directionality caused by the influence of noise on the morphological image. 9 shows the directional image after the smoothing process.

In operation 517, the controller 211 clarifies the flow of the ridge by Gabor filtering the smoothed image based on the direction of the fingerprint and the frequency of the ridge. 10 shows an image after Gabor filtering process. Meanwhile, the smoothing process may be performed after the Gabor filtering process. After the Gabor filtering process, the controller 211 determines binary pixel data by determining an 8-bit pixel value as black (0) or white (255) in step 519. The Gabor-filtered pixel value is determined to be a lump (0) if it is large, and a bag (255) if it is small compared with a value of a reference level for masking black / white. In operation 521, the control unit 211 performs smoothing on the binary image data to remove noise, and in operation 523, a thinning process of converting a fingerprint ridge into a ridge of 1 pixel unit is performed. .

In operation 525, the controller 211 extracts minutia from the thinned image data. The feature point may be a bifurcation at which the ridge splits and a ending point at which the ridge breaks. That is, the controller 211 extracts coordinates (x, y) and an angle (angle) for the branch point and the shortcoming from the image data. In addition, the controller 211 determines whether the number of feature points extracted in operation 527 is sufficient for matching, and if the number is sufficient for matching, stores the data of the feature points in a memory.

Thereafter, as described above, the controller 211 determines whether to authenticate by matching the data of the stored feature points with data of the registered feature points. 5 described above is performed in the authentication mode and the biometric information registration mode in the same manner. However, in the biometric information registration mode, data of the extracted feature points is registered in the memory, and in the authentication mode, authentication is performed by matching the data of the extracted feature points with data of the feature points registered in the memory.

For example, the data of the feature point is configured as shown in Table 1 below.

Angle X coordinate Y coordinate Feature Point Type (Branch Point, Disadvantage)

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the scope of the following claims, but also by those equivalent to the scope of the claims.

As described above, the present invention can perform authentication through multiple biometrics in the mobile communication terminal, thereby preventing personal information leakage and theft, which are problems caused by the existing authentication method. That is, the terminal according to the present invention can be applied to an existing electronic payment settlement system using a credit card, an unmanned electronic payment system using an automated device of a bank, an electronic payment system using the Internet, and the like. And accidents due to leaking password or card loss can be prevented in advance. In addition, it is possible to improve the performance and reliability of biometrics by proposing multiple biometrics.

Claims (11)

In the mobile communication terminal device, An image input unit capable of photographing at least one of an iris, a face, a vein, and a hand shape, converting an optical signal incident to a lens into an electrical signal, converting the electrical signal into digital data, and outputting the digital signal to a controller; Lighting unit for emitting the light for the shooting, A fingerprint sensor for sensing a fingerprint image and outputting the fingerprint image to the controller; A voice processor for converting a voice signal input from a microphone into digital data and outputting the digital signal to the controller; A touch screen panel for outputting coordinate data generated by contacting an upper surface of the panel to the controller; In the authentication mode, each of the biometric data input from at least one of the image input unit, the fingerprint sensor, the audio processor, and the touch screen panel is preprocessed according to a biometric type, and biometric recognition is performed from the preprocessed data. And a control unit which extracts information and determines whether to authenticate by matching the extracted feature information with pre-registered feature information. The method of claim 1, And the image input unit is a charge-coupled device (CCD) camera. The method of claim 1, And the fingerprint sensor is a heat sensor. The method of claim 1, wherein the fingerprint sensor, Outputs electrical signals for a predetermined number of pixels, the electrical signals being varied by a pressure and thermal image of a finger placed on a contact surface; An amplifier for amplifying an electrical signal from the sensor array; A first ADC for converting and outputting electrical signals for odd-numbered pixels of the electrical signals output from the amplifier into digital data; A second ADC for converting and outputting electrical signals for even pixels among the electrical signals output from the amplifier into digital data; And a latch outputting the digital data from the first and second ADCs to the controller in units of 2 pixels. In the biometric authentication method in a mobile communication terminal capable of multiple biometrics, In the biometric information registration mode, extracting feature information from a plurality of biometric information received from a plurality of biometric modules and registering the feature information in a memory; When the terminal access is checked, checking whether the lock mode is set; Reading the biometric types for the registered feature information when the lock mode is set; Sequentially displaying a message requesting a user's action for each of the read biometric types; Extracting feature information from a plurality of biometric information received from the plurality of biometric modules; Matching the extracted feature information with the registered feature information; And upon successful matching, displaying the authentication success message and releasing the lock mode. The method of claim 5, Determining that the matching is successful when at least one of the feature information matches. The method of claim 5, And if all of the feature information match, determining that the matching is successful. The method of claim 5, The biometric types may be at least one of fingerprint recognition, iris recognition, face recognition, vein recognition, hand shape recognition, and handwriting recognition. In the biometric authentication method in a mobile communication terminal capable of multiple biometrics, In the biometric information registration mode, feature information is extracted from a plurality of biometric information received from a plurality of biometric modules, the feature information is combined by a predetermined rule to generate first authentication data, and the first authentication. Registering data in memory, When the terminal access is checked, checking whether the lock mode is set; Reading the biometric types for the registered feature information when the lock mode is set; Sequentially displaying a message requesting a user's action for each of the read biometric types; Extracting feature information from a plurality of biometric information received from the plurality of biometric modules, and combining the feature information by the predetermined rule to generate second authentication data; Matching the first authentication data with the second authentication data; And upon successful matching, displaying the authentication success message and releasing the lock mode. The method of claim 9, The biometric types may include at least one of fingerprint recognition, iris recognition, face recognition, vein recognition, hand shape recognition, and handwriting recognition. In the method for extracting feature information from the fingerprint image sensed by the fingerprint recognition sensor, Removing a background from the fingerprint image; Calculating a pixel value variance of the image from which the background is removed; Performing histogram equalization based on the calculated variance, normalizing the pixel value average and standard deviation of the image to a predetermined value; Obtaining a standard deviation of a 16 × 16 block from the smoothed image as a reference value, segmenting the block below the reference value in the background, and segmenting the reference value above the foreground to the foreground; Morphology the segmented image into a window of 3 × 3 blocks; Smoothing the morphologically processed image to remove noise; Gabor filtering the smoothed image based on the direction of the fingerprint and the frequency of the ridges; Converting the Gabor filtered image into a binary image; Smoothing the binary image to remove noise; Thinning the noise-free binary image so that the ridge is represented by only one pixel, And extracting feature points from the thinned image.