JP2007004823A - Biological information obtaining apparatus and authentication apparatus using biological information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize high-quality characteristic information verification by obtaining undistorted images in high speed using a small amount of memory capacity, in an apparatus which obtains information needed for personal authentication from a plurality of partial images that are successively captured by, for example, a sweep-type fingerprint sensor with a fingertip moved on the sensor surface. <P>SOLUTION: The apparatus includes a relative position detecting means 30 which detects mutual relative positions among two or more partial images based on characteristics contained in a region where the two or more partial images out of the captured plurality of partial images mutually overlap, and correcting means 40 which calculates the amount of distortion based on the detected relative positions and the positions of the characteristics in each partial image, and which corrects the positions of the characteristics in each partial image based on the calculated distortion amount to obtain the relative position of the characteristics. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an apparatus for acquiring biological information such as fingerprints, palm prints, blood vessel patterns, and the like, and more specifically, for example, while moving a finger with respect to the sensor surface (collecting surface) in a sweep type fingerprint sensor. The present invention relates to a biometric information acquisition apparatus for acquiring information necessary for personal authentication or the like from a plurality of partial images (fingerprint images) collected continuously. The present invention also relates to an authentication device that performs personal authentication using biometric information such as a fingerprint, a palm print, and a blood vessel pattern.

Small information devices such as mobile phones and PDAs (Personal Digital Assistants) have become capable of being connected to a network or storing a large amount of personal information, etc. with the recent advancement in functionality. The demand for improved security performance is extremely high.
In order to ensure security in such a device, it is conceivable to employ personal authentication using a password or an ID (IDentification) card that has been widely used. However, since passwords and ID cards have a high risk of being stolen, it is strongly desired to realize more reliable personal authentication (authentication that the user of the device is a pre-registered user). Yes. In response to such a demand, personal authentication based on biometric information (biometric information) is considered to be highly reliable and meet the above demand. In particular, when fingerprints are used as biometric information, the convenience is high.

  When performing personal authentication using fingerprints as biometric information, a capacitive fingerprint sensor or optical fingerprint sensor can be used to place fingerprints from the finger of the person to be authenticated on the same surface as the ridge that can contact the surface of the fingerprint sensor. A pattern consisting of a valley line that does not touch) is collected as image information. Then, feature information (for example, position information of a branch point or an end point) is extracted from the foreground (for example, ridge image) of the fingerprint image, and the extracted feature information and registered feature information of the authenticated person registered in advance are obtained. By collating, it is determined whether or not the person to be authenticated is the person, that is, personal authentication is performed.

  Incidentally, a general fingerprint sensor that collects a fingerprint image from a person to be authenticated usually has a sensor surface (collection surface) larger than the size of the finger. However, in recent years, in order to mount a fingerprint sensor on a small information device such as a mobile phone or a PDA, the size of the sensor surface is made smaller than the size of the finger, and a plurality of partial images continuously taken through the sensor surface are collected. Integration is performed to obtain an image of the entire fingerprint.

  As a fingerprint sensor corresponding to such a situation, there is a sweep type sensor. This sweep type fingerprint sensor is sufficiently shorter than the length of a finger and has a rectangular sampling surface (sensor surface / imaging surface) with a small area. Then, while moving the finger with respect to the collection surface or moving the collection surface (fingerprint sensor) with respect to the finger, a plurality of partial images of the finger fingerprint are continuously collected by the fingerprint sensor. Attempts have been made to reconstruct the entire fingerprint image of a finger from a plurality of collected partial images. Note that the relative movement of the finger with respect to the sampling surface as described above is referred to as “sweep”. Further, information of feature points (ridge branch points and end points) is extracted and generated from the reconstructed fingerprint image, and the personal authentication is performed based on the information.

  Here, as a conventional technique for reconstructing the entire fingerprint image of a finger from a plurality of partial images, for example, there are techniques disclosed in Japanese Patent Laid-Open Nos. 8-263631 and 2001-155137. The techniques disclosed in these publications acquire the relative position between the finger and the sensor in a direction orthogonal to the sensor longitudinal direction, and reconstruct an image of the entire finger. Other conventional techniques include those disclosed in US Pat. Nos. 6,289,114 and 6,317,508, for example. The techniques disclosed in these US patents measure the overlap of successively captured images and reconstruct the image of the entire finger based on the overlap.

However, the image collected and reconstructed by the sweep type fingerprint sensor as described above contains distortion due to various factors, and the fingerprint data obtained by using the fingerprint sensor and the registered fingerprint data registered in advance. It may be difficult to collate with.
For example, in a fingerprint sensor that collects a fingerprint image by moving a finger while touching the sensor surface, as described later with reference to FIGS. 5 to 7 and FIGS. The finger itself is deformed when the finger is pressed against a flat sensor surface, or when the finger contact part is caught on the sensor surface due to friction between the sensor surface and the finger, and the fingerprint image contains a large distortion. It will be. The method of deformation varies greatly depending on the finger sweep direction.

  As another case, as will be described later with reference to FIGS. 19 to 26, when there is a detection delay in an imaging device that captures a fingerprint image or the like, the image is distorted regardless of the deformation of the finger itself. . For example, in the case of a sensor having a structure in which information is detected by an adjacent scanning line after a certain delay time after information is detected for each scanning line (see FIG. 25), the relative position between the finger and the sensor changes with time. If it changes greatly, the aspect ratio of the fingerprint image to be acquired changes, so that stretching distortion occurs (see FIGS. 21 to 24). In the case of a sensor having a detection delay time in cell units (see FIG. 26), skew distortion occurs (see FIGS. 25 and 26).

  Degradation of reconstructed image reproducibility due to such distortion is a major problem, and by combining multiple partial images with different distortion methods, fingerprint patterns that are not present on the actual finger (secreted from the finger) The appearance of a sebum or sweat image or an image caused by a flaw on the sensor surface or a defect of the detection element) is also a problem. That is, as described above, when a distortion occurs or a fingerprint pattern that does not exist on an actual finger appears, the feature information obtained from the person to be authenticated using the fingerprint sensor and the feature information registered in advance are Even if the person to be authenticated is the person himself / herself, there is a possibility that the person who is to be authenticated cannot be authenticated, and the verification performance is deteriorated, that is, the reliability of the personal authentication is lowered.

In addition, the techniques disclosed in the above publications and the above U.S. patents not only cause problems such as image distortion as described above, but also reconstruct the entire fingerprint image from a plurality of partial images. Since a memory having a capacity capable of storing an image is required and processing time is required, there is a problem that it is not suitable for a small information device.
The present invention has been devised in view of such problems, and by correcting the feature information extracted from each partial image, feature information without distortion can be obtained at a high speed with a small memory capacity. In this way, it is possible to perform collation using high-quality feature information, and to improve collation performance, that is, improve the reliability and convenience of personal authentication.

  In order to achieve the above object, the biological information acquisition apparatus of the present invention images biological information and continuously collects a plurality of partial images of the biological information, and is collected by the image collecting means. From each of the plurality of partial images, a feature extraction unit that extracts a feature and a position of the feature in each partial image, and two or more partial images of the plurality of partial images collected by the image collection unit Relative position detection means for detecting a relative position between the two or more partial images based on the features existing in the mutually overlapping region, the relative position detected by the relative position detection means, and the relative position in each partial image A correction method for calculating a distortion amount of the feature based on the position of the feature and correcting the position of the feature in each partial image based on the calculated distortion amount to obtain a relative position of the feature It is characterized in (first correction means) that is configured to include a.

  In addition, the biological information acquisition apparatus as a related technique of the present invention includes the same image acquisition means, feature extraction means, and relative position detection means as those described above, and the collection time interval of each partial image by the image acquisition means and the image Based on the sampling delay time of each partial image by the sampling means and the relative position detected by the relative position detecting means, the distortion in each partial image due to the sampling delay of the image sampling means is eliminated. It is characterized by comprising correction means (second correction means) for correcting the position of the feature and obtaining the relative position of the feature.

Furthermore, the biological information acquisition apparatus as a related technique of the present invention includes the same image collection means, feature extraction means, and relative position detection means as those described above, and functions of both the first correction means and the second correction means. And a correcting means (third correcting means) having the above.
On the other hand, the biometric information authentication apparatus according to the present invention and the related art of the present invention is any one of the same image collection means, feature extraction means and relative position detection means as those described above, and the first to third correction means. And at least one of registration data and verification data for authenticating the person to be authenticated using the feature extracted by the feature extraction unit and the relative position of the feature obtained by the correction unit. A generating unit configured to generate and a verification unit that executes verification processing to authenticate the person to be authenticated by using at least one of registration data and verification data generated by the generating unit; It is characterized by.

Then, the following configurations (1-1) to (1-15) may be further added to the above-described biometric information acquisition device or biometric information authentication device.
(1-1) The image collecting means continuously collects the plurality of partial images from a subject (a subject to be authenticated) moving relative to the image collecting means. In addition, before performing feature extraction by the feature extraction unit, there is further provided a removal unit that removes images of non-moving patterns included in the plurality of partial images collected by the image collection unit.

(1-2) In the above (1-1), the removing means calculates a weighted average image of the partial images collected by the image collecting means until immediately before, and based on the calculated weighted average image Thus, the moving pattern and the non-moving pattern are separated, and the image of the non-moving pattern is removed.
(1-3) The feature extraction means extracts the foreground and the foreground edge in each partial image as the features.

(1-4) The feature extraction means extracts the end points and branch points in the pattern obtained by thinning the foreground in each partial image as the features.
(1-5) The relative position detection means uses the relative position of the two or more partial images continuously acquired by the image acquisition means as a reference based on one or more relative positions detected immediately before. To detect.

(1-6) The relative position detecting means estimates a relative position between the two or more partial images continuously collected by the image collecting means based on one or more relative positions detected immediately before. The relative position to be detected next is detected as a reference.
(1-7) The relative position detection means handles the two or more partial images continuously collected by the image collection means by dividing them into two or more partial areas each having an overlapping area, A relative position between two or more partial images is detected for each of the two or more partial regions.

  (1-8) In the authentication apparatus having the second correction unit, the relative position detection unit has a region where the two or more partial images continuously collected by the image collection unit overlap each other. The two or more partial areas are handled separately, the relative positions of the two or more partial images are detected for each of the two or more partial areas, and the correction means The position of the feature is corrected so as to eliminate the distortion of each partial image accompanying the sampling delay of the image sampling means.

(1-9) The image collection means continuously collects the plurality of partial images from the subject (subject of the subject to be authenticated) moving relative to the image collection means. And a moving object detecting means for detecting the presence or absence of a moving object moving with respect to the image collecting means based on the plurality of partial images collected by the image collecting means.
(1-10) In the above (1-9), the moving object detection means calculates a weighted average image of the partial images collected by the image collection means until immediately before, and the calculated weighted average image Based on the above, the presence or absence of the moving object is detected.

(1-11) In the above (1-10), when the moving object detection means has a difference value between the latest partial image collected by the image collection means and the weighted average image exceeds a predetermined threshold The predetermined threshold value is set to be larger than the fluctuation value due to noise.
(1-12) In the authentication apparatus having the configurations of (1-9) to (1-11) above, while the presence of the moving object is not detected by the moving object detection unit, the verification by the verification unit Do not execute processing.

(1-13) In the authentication device having the configuration of (1-9) to (1-11), the collating unit includes the feature obtained from the partial image whose imaging time is early by the image sampling unit and the feature The collation process is executed using the relative position preferentially.
(1-14) In the above (1-13), the collation process ends when the collation means determines the collation result of the person to be authenticated.

(1-15) The plurality of partial images are continuously collected from the subject (the subject of the person to be authenticated) who is moving relatively while the image collecting unit is in contact with the image collecting unit. And a frictional force reducing means for reducing the frictional force generated with the contact movement of the subject.
In addition, a biological information acquisition apparatus as a related technique of the present invention includes an image sampling unit that visualizes biological information of a subject that moves while being in contact with a sampling surface, and continuously acquires a plurality of partial images of the biological information. And a frictional force reducing means for reducing the frictional force generated with the contact movement of the subject.

According to the above-described present invention and the biometric information acquisition apparatus and the biometric information authentication apparatus as related techniques, the following effects or advantages (2-1) to (2-16) can be obtained.
(2-1) Detecting the relative position between two or more partial images based on the features existing in the region where two or more partial images overlap each other (overlapping region), and detecting the relative position and the feature of each partial image Based on the amount of distortion estimated from the position, the position of the feature in each partial image is corrected, and the relative position of the feature is obtained. Are removed from the feature information. Thereby, characteristic information without distortion can be obtained, and collation with high-quality characteristic information can be performed at the time of personal authentication, so that it is possible to improve collation performance, that is, reliability of personal authentication.

  (2-2) The relative position between two or more partial images is detected based on a feature existing in a region where two or more partial images overlap each other (overlapping region), and the relative position and each portion obtained by the image capturing means Based on the image sampling time interval and the sampling delay time of each partial image by the image sampling means, the position of the feature in each partial image is corrected so as to eliminate the distortion of each partial image caused by the sampling delay of the image sampling means. By obtaining the relative position of the feature, for example, image distortion (stretching distortion or skew distortion) caused by the detection delay of the sweep type fingerprint sensor is removed from the feature information. Thereby, characteristic information without distortion can be obtained, and collation with high-quality characteristic information can be performed at the time of personal authentication, so that it is possible to improve collation performance, that is, reliability of personal authentication.

  (2-3) By performing both of the corrections (2-1) and (2-2) above, for example, distortion caused by the finger itself deforming when taking a partial image of a fingerprint, for example, a sweep type Both image distortion (stretching distortion and skew distortion) caused by the detection delay of the fingerprint sensor is removed from the feature information, and collation with higher quality feature information becomes possible at the time of personal authentication, further improving the collation performance. Improvement, that is, further improvement of the reliability of personal authentication can be realized.

(2-4) Since the feature information extracted from each partial image is corrected, there is no need to store each partial image as it is, and feature information without distortion can be obtained at high speed with a small memory capacity. it can.
(2-5) By performing control to sequentially perform collation processing in units of partial images in the order in which feature position correction is completed, highly accurate biometric information collation can be performed using a small sensor, and extremely convenient A personal authentication system can be realized.

(2-6) It is not necessary to acquire the entire distribution of biometric information required for verification at once, and personal authentication using biometric information can be easily performed by a small sensor, so small information such as mobile phones and PDAs It is possible to improve the security performance by mounting a biological information sensor even for a device such as a device that does not have sufficient space for incorporating the sensor.
(2-7) When the image sampling means is, for example, a sweep-type biometric information sensor, by removing images of non-moving patterns in each partial image, for example, sebum, sweat, water vapor, etc. adhering to the sensor surface The image, the image of the scratch on the sensor surface, and the image due to the defect of the detection element forming the sensor are removed from the partial image as a non-moving pattern (background). As a result, it is possible to reliably prevent a pattern that does not move as described above from appearing on a partial image as a fingerprint pattern that does not exist on an actual finger, so that feature information unnecessary for collation is not extracted, Verification performance can be further improved.

  (2-8) Extract the foreground (for example, the luminance value of the ridge image) and the foreground edge (for example, the value of the luminance gradient) in each partial image as features, and use these both foreground and edge as the relative positions. Since the relative position between the partial images can be accurately detected even if the partial images have a stripe pattern having a similar shape, the relative position detection performance can be improved.

  (2-9) Extracting the end points and branch points in the pattern obtained by thinning the foreground (for example, ridge image) in each partial image as features, and detecting the relative position using these end points and branch points This eliminates the need to extract feature points from the entire image such as a fingerprint when collating feature information (endpoints or branch points in the case of fingerprints) to perform personal authentication, and can thus greatly reduce processing time. Further, the partial image after the feature information extraction does not need to be stored in the memory, and since the partial image can be discarded, the memory usage can be greatly reduced.

  (2-10) By detecting the relative position between two or more partial images based on one or more relative positions detected immediately before, the number of computations when detecting the relative position can be reduced. it can. That is, the range (search range) for calculating the correlation value to detect the relative position is limited to, for example, the range along the sweep direction, and the processing time for detecting the relative position can be shortened. At this time, if the partial image acquisition interval is sufficiently short, the sweep direction cannot be suddenly reversed. Therefore, sufficient relative position detection performance can be obtained even if the search range is limited.

  (2-11) By detecting the relative position between two or more partial images based on the relative position to be detected next, estimated based on one or more relative positions detected immediately before However, as in (2-10) above, the number of computations when detecting the relative position can be reduced. In other words, if the partial image acquisition interval is sufficiently short, it can be assumed that the sweep direction and sweep speed cannot change drastically, so the next search range should be limited based on the relative positions detected immediately before. Can do. Therefore, the processing time for detecting the relative position can be shortened, and sufficient relative position detection performance can be obtained even if the search range is limited.

(2-12) Handle two or more partial images by dividing them into two or more partial areas each having an overlapping area, and detect the relative positions of the two or more partial images for each of the two or more partial areas. By doing so, for example, even when a finger is greatly deformed when a partial image of a fingerprint is collected, the relative position of each part can be detected with high accuracy.
(2-13) For example, when a plurality of partial images are continuously collected using a sweep-type biological information sensor, movement with respect to the biological information sensor is performed based on the plurality of partial images obtained by the biological information sensor. By detecting the presence or absence of a moving object (for example, a finger), it is not necessary to prepare a separate sensor for detecting the moving object, and it is possible to improve the practicality by detecting the moving object without incurring costs. it can. At that time, the weighted average image of the partial images collected immediately before is calculated, and the presence of the moving object is detected when the difference value between the weighted average image and the latest partial image exceeds a predetermined threshold value. Can be determined. At this time, if the weighted average image calculated when removing the non-moving pattern (background) is used as it is, it is possible to detect the moving object without increasing the amount of calculation. Moreover, by setting the predetermined threshold value to be larger than the fluctuation value due to noise, it is possible to reliably prevent noise from being erroneously detected as a moving object, and improve the detection performance of the moving object.

  (2-14) At least information other than moving objects does not need to be collated, so collation processing is not executed while the presence of a moving object is not detected, that is, collation processing is performed while no moving object is detected. By configuring so that the collation process is performed only while the presence of the moving object is detected without starting the operation, the time and load required for the calculation can be reduced. Further, since it is possible to avoid the feature information extracted from the noise being used for the matching process, the matching performance is not deteriorated.

  (2-15) Execute collation processing preferentially using features obtained from partial images with early imaging time and their relative positions, and end collation processing when the verification result of the person to be authenticated is confirmed With this configuration, features are sequentially generated from the partial image group input earlier by the sweep operation, so that the matching process can be started before obtaining the entire feature, while the matching result It becomes possible to abort the collation process at an early point when is confirmed.

  (2-16) Collecting multiple partial images continuously from a subject (subject's subject) moving relatively while contacting the sensor surface, such as a sweep-type biological information sensor When the sensor is used, the friction force reducing means for reducing the frictional force generated by the contact movement of the subject can be provided, so that the sweep operation of the subject can be easily performed and the convenience of the subject is improved. Deformation can be reduced, biometric information images with high reproducibility can be obtained, and collation performance can be improved.

Embodiments of the present invention will be described below with reference to the drawings.
[1] Configuration of Authentication Device of this Embodiment FIGS. 1 and 2 each show an authentication device (biological information acquisition device) using biometric information as one embodiment of the present invention. FIG. FIG. 2 is a block diagram showing a specific configuration. FIG. 2 is a block diagram showing the specific configuration. In these FIG. 1 and FIG. 2, the same code | symbol is attached | subjected to the same part.

As shown in FIG. 1, the authentication apparatus of this embodiment includes a biometric information input means (image collection means) 10, a feature extraction means 20, a relative position detection means 30, a correction means 40, and a moving object detection means (removal means). 50, functions as a generation means 60 and a collation means 70 are provided.
Actually, as shown in FIG. 2, the authentication apparatus according to the present embodiment is, for example, a general personal computer having a real-time clock 80, a volatile memory unit 90, a nonvolatile memory unit 91, and a CPU (Central Processing Unit) 100. Further, it is realized by attaching a capacitance type sweep type fingerprint sensor 10 as biometric information input means (image collection means). At that time, a feature extraction unit (feature extraction unit) 20, a relative position detection unit (relative position detection unit) 30, a correction unit (correction unit) 40, a moving object detection unit (moving object detection unit / removal unit) 50, which will be described later. , Functions as a registration / collation data generation unit (generation unit) 60 and a collation unit (collation unit) 70 are realized by the CPU 100 executing a predetermined program.

  The electrostatic capacitance type swept fingerprint sensor (image collecting means) 10 visualizes the biometric information of the person to be authenticated and continuously collects a plurality of partial images of the biometric information. While moving the authenticator's finger (subject; reference numeral 200 in FIGS. 3 to 7, 32, and 33) relative to the sampling surface (sensor surface) 11, a partial image of the fingerprint of the finger is displayed. It is collected continuously.

  The fingerprint is formed on the skin (finger; subject) of the person to be authenticated, and a ridge (contact portion) that can contact the sensor surface 11 and a valley line (non-contact portion / void portion) that does not contact the sensor surface 11 ). The fingerprint sensor 10 takes a fingerprint partial image as a multi-valued image by utilizing the fact that the detection sensitivity differs between a ridge portion that contacts the sensor surface 11 and a valley portion that does not contact the sensor surface 11. ing. In multi-valued images, the brightness varies depending on the distance from the sensor. Normally, the ridge portion that is close to the sensor is displayed with low brightness, and the valley portion that is relatively far from the sensor is displayed with high brightness. Is displayed.

  At the time of authentication by fingerprint, the person to be authenticated touches the sensor surface 11 of the fingerprint sensor 10 with his / her finger in any direction such as the finger base side to the fingertip side, the fingertip side to the finger base side, and the finger right side to the left side. Move your finger. However, when a mechanism for moving the fingerprint sensor 10 side with respect to the finger is provided, the person to be authenticated does not need to move the finger. Hereinafter, in the present embodiment, as will be described later with reference to FIGS. 3, 4, 6, and 7, a case where the person to be authenticated sweeps the finger 200 from the root side toward the fingertip side will be described. The detailed configuration of the fingerprint sensor 10 will be described later with reference to FIGS. 19 and 20.

The real time clock 80 is used to add a time stamp to each partial image continuously collected by the fingerprint sensor 10.
The volatile memory unit 90 realizes partial images continuously collected by the fingerprint sensor 10, features obtained by the function of the CPU 100, relative positions, correction results, and the function as the authentication device of the present embodiment by the CPU 100. Various parameters (for example, correction parameters necessary for processing by the correction unit 40) and the like necessary for the storage are stored.

  The nonvolatile memory unit 91 holds fingerprint data registered in advance for the person to be authenticated. The fingerprint data held in the nonvolatile memory unit 91 may be extracted from a fingerprint image collected by a general fingerprint sensor having a sensor surface larger than the size of the finger. The registration data generated by the registration / collation data generation unit 60 may be used.

  The feature extraction unit (feature extraction means) 20 extracts a feature in each partial image and the position of the feature from each of a plurality of partial images collected by the fingerprint sensor 10. Here, as features, both the foreground in each partial image (in this embodiment, the luminance value of the ridge image; see FIG. 13) and the edge of that foreground (in this embodiment, the value of the luminance gradient; see FIG. 14). Or, as described later with reference to FIG. 15, the end points and branch points in the pattern obtained by thinning the foreground (ridge image in the present embodiment) in each partial image may be extracted. Good.

  The relative position detector (relative position detector) 30 is a feature (extracted by the feature extractor 20) that exists in a region where two consecutive partial images of the plurality of partial images collected by the fingerprint sensor 10 overlap each other. The relative position between these two partial images is detected based on the feature). At this time, the relative position detection unit 30 may detect the relative position on the basis of one or more relative positions detected immediately before, or as described later with reference to FIG. May be detected based on a relative position to be detected next, which is estimated based on one or more relative positions detected immediately before. In addition, as will be described later with reference to FIGS. 17 and 18, the relative position detection unit 30 divides each partial image continuously collected by the fingerprint sensor 10 into two or more partial regions having overlapping regions. The relative position may be detected for each of two or more partial areas.

  The correction unit (correction unit) 40 is for correcting image distortion and distortion caused by finger deformation caused by the detection delay of the fingerprint sensor 10. The correction unit 40 has the following two types of correction functions. As will be described later with reference to FIGS. 3 to 7 and FIGS. 27 to 31, the first correction function is characterized based on the relative position detected by the relative position detection unit 30 and the position of the feature in each partial image. This is a function of calculating the amount of distortion (distortion amount due to finger deformation) and correcting the position of the feature in each partial image based on the calculated amount of distortion. The second correction function is based on the sampling time interval of each partial image by the fingerprint sensor 10, the sampling delay time of each partial image by the fingerprint sensor 10, and the relative position detected by the relative position detection unit 30. The position of the feature in each partial image is corrected so as to eliminate the distortion of each partial image (stretching distortion and skew distortion; described later with reference to FIGS. 19 to 26) due to the sampling delay of the fingerprint sensor 10. This is a function for obtaining the relative position.

  The relative position detection unit 30 handles each partial image continuously collected by the fingerprint sensor 10 by dividing it into two or more partial areas having overlapping areas, and handles the relative position of two or more partial areas. When detecting each, the correction | amendment part 40 correct | amends the position of the characteristic about each partial area | region so that distortion of each partial image accompanying the collection delay of the fingerprint sensor 10 may be eliminated.

  The moving object detection unit (moving object detection means / removal means) 50 is based on a plurality of partial images collected by the fingerprint sensor 10 and is moving with respect to the fingerprint sensor 10 (here, the person to be authenticated). A weighted average image of partial images collected by the fingerprint sensor 10 until immediately before, and detecting the presence of a moving object based on the calculated weighted average image It is supposed to be. More specifically, the moving object detection unit 50 detects the presence of a moving object when the difference value between the latest partial image collected by the fingerprint sensor 10 and the calculated weighted average image exceeds a predetermined threshold. The predetermined threshold is set to be larger than the fluctuation value due to noise. In the present embodiment, the collating process by the collating unit 70 described later is not executed while the moving object detecting unit 50 does not detect the presence of the moving object.

  In addition, the moving object detection unit 50 according to the present embodiment removes an image of a non-moving pattern included in a plurality of partial images collected by the fingerprint sensor 10 before performing feature extraction by the feature extraction unit 20. It also has a function as a means. According to the function as the removing means, as will be described later with reference to FIGS. 8 to 12, the weighted average image of the partial images collected by the fingerprint sensor 10 until immediately before is calculated, and the calculated weighted average is calculated. Based on the image, the moving pattern and the non-moving pattern are separated, and the non-moving pattern image is removed.

  The registration / collation data generation unit (generation unit) 60 uses the feature extracted by the feature extraction unit 20 and the relative position of the feature obtained by the correction unit 40 to authenticate the person to be authenticated. Fingerprint data (registration data and collation data) is generated. When registering fingerprint data, fingerprint data (known information such as ridge branch points, end point positions and patterns) generated by the registration / collation data generation unit 60 is registered in the nonvolatile memory unit 91 as registration data.・ Retained. At the time of authenticating the person to be authenticated, fingerprint data (information on ridge branch points and end points) generated by the registration / verification data generation unit 60 is sent to the verification unit 70 as verification data.

  The collation unit (collation unit) 70 compares the collation data generated by the registration / collation data generation unit 60 with the registration data of the person to be authenticated held in the non-volatile memory unit 91 and performs collation processing. This is executed to authenticate the person to be authenticated. The collation unit 70 executes the collation process by using the feature obtained from the partial image whose time of imaging by the fingerprint sensor 10 (time stamp time by the real-time clock 80) and the relative position thereof are preferentially used, The collation process is completed when the collation result of the person to be authenticated is confirmed.

Further, as will be described later with reference to FIGS. 32 to 35, the frictional force generated with the contact movement of the finger (subject) 200 around the sensor surface 11 of the fingerprint sensor 10 in the authentication device of the present embodiment. Friction force reducing means 15 to 18 for reducing the above are provided.
[2] Detailed Configuration, Operation, and Effect of Authentication Device of Present Embodiment Next, a more detailed configuration, operation, and effect of the authentication device of the present embodiment will be described with reference to FIGS. To do.

[2-1] About Sweep Fingerprint Sensor and Authentication Device of the Present Embodiment In the authentication device of the present embodiment, as will be described later, an image of the entire biometric information (hereinafter referred to as a plurality of partial images obtained by the sweep fingerprint sensor 10). , Which is referred to as “whole image”), the function of acquiring the distribution of features of biometric information (fingerprint feature point distribution) necessary for collation is realized.

In this embodiment, as described above, fingerprints are used as biometric information. With reference to FIG. 3 and FIG. 4, a fingerprint image collecting operation (sweep operation) with the sweep fingerprint sensor 10 and the sweep fingerprint sensor 10 continuously. A partial image collected automatically will be described.
In order to obtain information on the entire fingerprint image by gathering a part of the fingerprint image (hereinafter referred to as “partial image”) obtained by the fingerprint sensor 10 having the sensor surface 11 sufficiently smaller than the size of the finger 200. Obviously, it is necessary to obtain at least the relative position between the partial images.

  FIGS. 3 and 4 show a state in which partial images are collected with time from the base side of the finger 200 to the fingertip side with respect to the fingerprint sensor 10. For example, if the partial image captured at time T and the partial image captured at time T + 1 have a sufficiently large overlapping area, almost the same pattern is present in the overlapping area of the two partial images. Appear. That is, a wider range of fingerprint images can be acquired by connecting partial images so that substantially the same pattern overlaps. At this time, it is necessary to detect a relative position between partial images collected at a later time with a partial image collected at a certain time as a reference. Hereinafter, as shown in FIGS. 3 and 4, the operation of shifting the collection position of the partial image in a substantially constant direction is referred to as “sweep”.

  In a plurality of partial images obtained by the fingerprint sensor 10, extremely similar patterns (for example, fingerprint stripe patterns) repeatedly appear. For this reason, in order to detect the relative position between partial images with high accuracy, it is necessary to extract a more characteristic part as a feature from each partial image. The characteristics include a luminance gradient, a foreground part obtained by binarization, or a contour line of the foreground part, as will be described later with reference to FIGS. Further, when attention is paid to a pattern peculiar to fingerprints, the end points and branch points of ridges are also mentioned as the above features.

  In the present embodiment, such a feature is extracted by the feature extraction unit 20, and the relative position between the partial images is detected by the relative position detection unit 30 using the extracted feature. At this time, if the features required for matching (end points and branch points of ridges in the case of fingerprints) are extracted from the partial image, registration data and matching data can be easily generated without reconstructing the entire image. It becomes possible to do.

A known method is used as the relative position detection method in the relative position detection unit 30. For example, a value representing the correlation is calculated while sequentially changing the overlapping area, and a position where the value is optimal is detected as a relative position.
By the way, as described in the background art section, the partial image includes distortion due to the delay characteristic of the fingerprint sensor 10 and the plasticity of the finger 200. Further, when reproducibility cannot be obtained in the direction in which the finger 200 or the fingerprint sensor 10 is displaced (hereinafter referred to as the sweep direction) or the speed (hereinafter referred to as the sweep speed), the fingerprint data can be obtained during registration and verification. It is inevitable that the feature distribution will be different.

When a general fingerprint sensor having a sensor surface (collection surface) larger than the size of the finger is used, almost the same fingerprint image (for example, an image as shown in FIG. 5) can always be collected from the same finger. it can.
On the other hand, when the sweep type fingerprint sensor 10 is used, the finger 200 is deformed by a frictional force generated in the direction opposite to the sweep direction generated between the finger 200 and the sensor surface 11. When the sweep operation is realized by moving the finger 200 with respect to the fixed sensor surface 11, for example, as shown in FIGS. 6 and 7, the sweep direction side skin is stretched to widen the ridge interval, and the sweep The epidermis opposite to the direction is shrunk and the ridge spacing is narrowed.

  Therefore, for example, as shown in FIG. 6, when fingerprint data is registered, the fingerprint data obtained by moving the finger 200 in the lower right direction with respect to the fingerprint sensor 10 is compared with the fingerprint data shown in FIG. The fingerprint data obtained by moving the finger 200 in the lower left direction with respect to the fingerprint sensor 10 has a greatly different feature distribution even if it is obtained from the same person to be authenticated, The feature positions do not necessarily match at the time of matching. For this reason, the deformation of the finger 200 at the time of collecting fingerprint data becomes a factor of deterioration of collation performance.

In the present embodiment, the distortion due to the deformation as described above is corrected and removed by the correction unit 40 as described later with reference to FIGS. 27 to 31 to improve the matching performance.
Further, in the present embodiment, distortion due to the delay characteristic of the fingerprint sensor 10 that occurs as will be described later with reference to FIGS. 19 to 26 is corrected and removed by the correction unit 40 to improve collation performance.

  As described above, in the authentication device according to the present embodiment, the relative position detection unit 20 detects the relative position between the partial images based on the features existing in the overlapping region of the two consecutive partial images, and will be described later. As described above, the correction unit 40 corrects the position of the feature in each partial image so as to remove distortion due to finger deformation and distortion due to the delay characteristic of the fingerprint sensor 10. The fingerprint data generated from the corrected feature information by the registration / collation data generation unit 60 is registered in the nonvolatile memory unit 91 as registration data, or is collated in the collation unit 70 as collation data. Used. As a result, fingerprint data with reduced distortion can be obtained, and collation with high-quality fingerprint data can be performed at the time of personal authentication, so it is possible to improve collation performance, that is, improve the reliability of personal authentication. .

  At this time, since the feature extraction unit 20 corrects the feature extracted from each partial image, it is not necessary to store each partial image as it is, and fingerprint data without distortion is obtained at a high speed with a small memory capacity. be able to. Further, by using the relative positions of the extracted features without using the partial image or the entire image as they are, it is possible to perform highly reliable verification while greatly reducing the memory usage.

Furthermore, in the authentication apparatus according to the present embodiment, the collation processing is sequentially performed in units of partial images by the collation unit 70 in order from the partial image for which the correction of the feature position by the correction unit 40 has been completed. It is possible to perform highly accurate fingerprint collation while using the sweep type fingerprint sensor 10, and to realize a highly convenient personal authentication system.
In addition, it is not necessary to acquire the entire distribution of biometric information necessary for collation (in this embodiment, information on the feature points of fingerprints) at once, and personal authentication using fingerprints can be easily performed by the small sweep fingerprint sensor 10. As a result, it is possible to improve the security performance by mounting the fingerprint sensor 10 even on a device that does not have a sufficient space for incorporating the sensor, such as a small information device such as a mobile phone or a PDA. .

[2-2] Background Removal Function Next, the function (removal function) of the moving object detection unit 50 as the removal unit will be described with reference to FIGS. 8 is a diagram illustrating an example of a partial image collected in a state where the sweep operation is not performed (a state in which the finger 200 is stopped), and FIG. 9 is a diagram illustrating an example of a partial image collected during the sweep operation. 10 is a diagram showing an example of a partial image collected when the sensor surface 11 of the sweep type fingerprint sensor 10 has a defect or dirt, and FIG. 11 shows a part collected when there is a defect or dirt as shown in FIG. FIG. 12 is a diagram illustrating an average image, and FIG. 12 is a diagram illustrating an example of a partial image from which a non-moving pattern (defect or dirt image) has been removed.

In the authentication apparatus according to the present embodiment, a pattern that is not a moving object is removed in advance using the removal function, so that features are not extracted from image information that does not reflect biological information.
For example, a large number of sweat glands are opened on the surface of the finger 200 (on the ridge), and secretions such as sweat, sebum, and water vapor are always secreted from these sweat glands. For this reason, when the finger 200 is moved in contact with the sensor surface 11 of the fingerprint sensor 10 in order to collect a fingerprint image, secretions from the sweat glands adhere to the sensor surface 11, and the fingerprint sensor 10 is moved to the sensor surface 11. In some cases, the secretions adhering to the image are visualized in the same manner as the ridge portion and collected as a low-luminance pattern in the partial image, as shown in FIG.

  In a state where the sweep operation is not performed (a state where the finger 200 is stopped), sebum and sweat adhere to a portion where the ridge and the sensor are in contact as shown in FIG. On the other hand, when the sweep operation is performed, as shown in FIG. 9, the sweat is dragged in the sweep direction and is imaged together with the ridge. Since such a sweat image has features similar to the ridge image, feature information that does not actually exist can be extracted from the partial image as shown in FIG. There is sex. In addition, as shown in FIG. 9, the sweat adhering to the sensor surface 11 gradually fades with the sweep operation of the finger 200.

In addition to the sweat image as described above, the sebum or water vapor image between the ridges (that is, the valleys) reduces the contrast between the fingerprint ridges and the valleys, thereby reducing the matching performance. It is a factor to invite.
Furthermore, when the sensor surface 11 is scratched, when the detection element (cell) forming the fingerprint sensor 10 has a local defect, or when the sensor surface 11 is dirty, depending on the characteristics of the fingerprint sensor 10, As shown in FIG. 10, these scratches, defects, and stains may have the same luminance as the foreground (ridge image). In such a case, the scratch, defect, and stain images deteriorate the relative position detection accuracy. As a result, the matching performance is degraded.

Therefore, it is possible to improve the matching performance by removing patterns that are not moving objects before the feature extraction unit 20 extracts features from the partial image.
In the moving object detection unit 50 in the authentication device of the present embodiment, a weighted average image is used to remove a pattern that is not a moving object as a background.
As shown in FIG. 10, since the moving object (finger 200; actually the ridge image) changes its position every moment, the ridge image may appear as the same pattern at the same position on a plurality of partial images. Is remarkably low. For this reason, in the weighted average image (superimposed image) obtained from a plurality of partial images, the image corresponding to the ridge is averaged, and the image information (image intensity) of the ridge is thinned.

  On the other hand, a background image that is a pattern that does not move appears as the same pattern at the same position on a plurality of partial images. For example, when a pattern such as sebum, sweat, or water vapor adheres to the sensor surface 11 when the finger 200 is swept with respect to the sensor surface 11 as shown in FIG. 10, the pattern is wiped off with the finger 200 along with the sweep operation. Or until it evaporates from the sensor surface 11, it remains appearing in the same position of each partial image. Therefore, for example, as shown in FIG. 11, in the weighted average image (superimposed image) obtained from a plurality of partial images, the image corresponding to the pattern that does not move is emphasized compared to the surroundings.

  Therefore, the moving object detection unit 50 according to the present embodiment has a pattern that is not a moving image such as a ridge, a pattern in which sweat or the like secreted from the living body (finger 200) is attached to the sensor surface 11, or dirt, A weighted average image of a plurality of partial images is calculated in order to detect as a defect or a flaw, and a fingerprint image (ridge image) in a plurality of fingerprint partial images captured by the fingerprint sensor 10 using the weighted average image The image of the pattern that is not a moving object is separated, and the image of the pattern that does not move is removed from the partial image, for example, as shown in FIG.

  However, as described above, sweat, sebum, and the like adhering to the sensor surface 11 are wiped off with the finger 200 along with the sweep operation as time passes, or are not visualized by evaporating from the sensor surface 11. Old image information is meaningless. Therefore, in order to calculate the weighted average image G (i, j, K) that reflects the pattern that does not move by always using a new partial image, for example, the following equation (1) is used.

G (i, j, K) = wF (i, j, Kk) + (1-w) G (i, j, K-1) (1)
Where 0 <w <1
Using such an equation (1), the weighted average image G (i, j, K) of n partial images is obtained by multiplying the input image F (i, j, Kk) by the weight w (k). Is calculated as a non-moving pattern (that is, a background image). Here, n may be defined as the number of images acquired within a predetermined time, or may be defined as the number of images acquired within a predetermined distance. Further, the weight w (k) is set to be larger as the new partial image (that is, as k is smaller).

At this time, in order to reduce the calculation time and the amount of memory used, the weighted average image G (i, j, K) may be calculated using, for example, the following equation (2) instead of the above equation (1). Good.
G (i, j, K) = Σ [w (k) · F (i, j, K-1) + G (i, j, K-1)] (2)
Note that the weighted average image G (i, j, K) may be calculated by performing an operation according to the above formulas (1) and (2). It is clear that the pattern by is always emphasized as background.

In order to remove the non-moving pattern as the background, for example, the maximum quantization level is M, and the image F ′ (i, j) from which the background image is removed may be calculated using the following equation (3). .
F ′ (i, j) = F (i, j) + w (MG (i, j)),
Or F ′ (i, j) = F (i, j) + (Mw · G (i, j)) (3)
Where 0 <w <1
The background image may be removed by performing an operation according to the equation (3). Even if a partial image as shown in FIG. 10 is obtained by such an operation, an average image as shown in FIG. 11 is obtained. By removing the pattern (background image) that does not move by using, it is possible to obtain a partial image including only the ridge pattern of the moving object, that is, the fingerprint, as shown in FIG.

Another way of thinking is to use a predetermined threshold Th
F ′ (i, j) = F (i, j) (G (i, j) <Th)
N (N: luminance value considered as background) (G (i, j) ≧ Th) (4)
However, ridges are detected with low brightness and valleys are detected with high brightness.
For such sensors.
As shown in the above equation (4), the partial image F ′ from which the background image is removed may be calculated using G (i, j) as a mask, or an operation based on this may be performed.

  By removing the image (background image) of the non-moving pattern in each partial image before the feature extraction as described above, for example, an image of sebum, sweat, water vapor, etc. attached to the sensor surface 11 or a sensor The scratch image on the surface 11 and the image due to the defect of the detection element (cell) constituting the fingerprint sensor 10 are removed from the partial image. As a result, it is possible to reliably prevent a pattern that does not move as described above from appearing on a partial image as a fingerprint pattern that does not exist on the actual finger 200, so that feature information unnecessary for collation is not extracted. The collation performance can be further improved.

[2-3] Features Extracted by Feature Extraction Unit Next, features extracted by the feature extraction unit 20 of the present embodiment will be described with reference to FIGS.
The feature extraction unit 20 may extract both the foreground and its edges as features used for detecting the relative positions between the partial images, or feature points (end points, branch points) of the ridge that is the foreground. ) May be extracted.

First, the case where the feature extraction unit 20 extracts both the foreground and its edges as features will be described.
Here, the foreground refers to a region of the partial image that is detected as biological information, for example, a region that is regarded as a ridge in the case of a fingerprint. As the foreground, the luminance value of the partial image may be used as it is, or a foreground after binarization may be used. For the foreground edge, a value of a luminance gradient may be used, or a foreground edge given a label may be used.

  Since the partial image of the fingerprint generally has a stripe pattern having a similar shape, when the relative position between the partial images is detected using only the input partial image or only the foreground, the relative position is shown in FIG. As shown, the position shifted by one ridge may be detected erroneously, and the relative position detection performance deteriorates. More specifically, in FIG. 13, f (T) is a partial image collected at time T, f (T + 1) is a partial image collected at time T + 1, and these partial images f (T) and When detecting the relative position with respect to f (T + 1), if only the foreground information is used, it is possible to correctly determine whether the relative position is a position shifted by one ridge or a position shifted by two ridges. Can not.

Further, when the relative position is detected using only the edge, as shown in FIG. 14, the position where the ridge and the valley line are alternately detected may be erroneously detected, and the relative position detection performance is deteriorated. . In FIG. 14, f (T) is a partial image sampled at time T, and f (T + 1) is a partial image sampled at time T + 1.
Therefore, if the foreground has a higher feature than the background and the edge has a higher feature than the foreground, the partial images are similar by detecting the relative position using at least these two features (foreground and edge). Even if the stripe pattern has a shape, the relative position between the partial images can be accurately detected, so that the relative position detection performance can be improved. In addition, by using a known sharpness enhancement filter, in principle, it is possible to obtain the same effect as that in which the relative position is detected using both the foreground and the edge.

  On the other hand, when the feature extraction unit 20 extracts, as the features, end points and branch points in the pattern obtained by thinning the foreground (ridge image) of the biological information image in each partial image, for example, as shown in FIG. Since the features (end points and branch points) represent local shapes, it is not always necessary to reconstruct the entire fingerprint image from a plurality of partial images. Since the input partial image itself does not need to be stored in the volatile memory unit 90 after the feature extraction, the memory of the volatile memory unit 90 is discarded by discarding the partial image after the feature extraction. The amount used can be greatly reduced.

  Also, as described above, end points and branch points are extracted as features, and feature information (end points and branch points in the case of fingerprints) for personal authentication is detected by using these end points and branch points to detect relative positions. Since it is no longer necessary to extract feature points from the entire image such as a fingerprint when collating, the processing time can be greatly reduced. Note that features such as end points and branch points can be detected if there is an image having the size of the thick frame in FIG. 15, for example.

[2-4] Relative Position Detection Method Next, a relative position detection method by the relative position detection unit 30 of the present embodiment will be described with reference to FIGS. 16 to 18.
When the relative position between the partial images is detected by the relative position detection unit 30, the number of computations can be reduced by detecting the relative position based on one or more relative positions detected immediately before. In other words, on the assumption that the sweep direction does not fluctuate extremely, the range for calculating the value representing the correlation to detect the relative position (hereinafter referred to as the search range) is limited to, for example, the range along the sweep direction. The number of times can be reduced, and the processing time for detecting the relative position can be shortened.

  At this time, if the interval at which the partial image is acquired by the fingerprint sensor 10 is sufficiently short, it can be assumed that the sweep direction cannot be suddenly reversed. This is obvious because all objects do not move against the laws of inertia. Therefore, a sufficient relative position detection performance can be obtained only by setting the search range along the sweep direction. In addition, when the image acquisition interval is sufficiently short, the number of calculations can be further reduced by considering the search range as one ridge or less, and the risk of erroneous detection of the relative position can be reduced.

  On the other hand, when the relative position detection unit 30 detects the relative position between the partial images, the next position estimated based on one or more relative positions detected immediately before is determined to limit the search range in the relative position detection. By detecting the relative position based on the relative position to be detected at the same time, the number of operations can be reduced as in the above detection method. That is, if the acquisition interval of the partial image by the fingerprint sensor 10 is sufficiently short, it can be assumed that the sweep direction and the sweep speed cannot change drastically as shown by arrows A1 and A2 in FIG. Therefore, as shown in FIG. 16, the next search range (sweep speed range d) can be predicted and limited based on the relative positions detected immediately before. As a result, the processing time for detecting the relative position can be shortened, and sufficient relative position detection performance can be obtained even if the search range is limited.

The sweep speed V is apparently calculated from the imaging interval I (K) of the partial image and the relative position Pr (K) as shown in the following equation (5).
V (K) = Pr (K) / I (K) (5)
Here, the imaging interval I (K) indicates the time from the start of capturing a certain partial image to the start of capturing the next partial image.

Then, the relative position Pr ′ (K) that is estimated to be detected next is calculated as the following equation (6) using the above equation (5).
Pr ′ (K) = V (K−1) * I (K), K: integer (6)
When the image acquisition interval is sufficiently short, the number of calculations can be further reduced and the risk of erroneous relative position detection can be reduced by regarding the search range as one ridge or less.

However, in the case where the imaging interval I (K) can be regarded as constant regardless of the imaging time, it is also possible to directly use the immediately preceding relative position Pr (K−1) as shown in the following equation (7). ) And the same effects can be obtained.
Pr ′ (K) = Pr (K−1) (7)
Such an estimation method is effective when the sweep speed changes very slowly and can be considered to be constant speed.

When the imaging interval I (K) can be regarded as constant and the sweep speed changes significantly, the relative position Pr ′ (K) may be estimated as in the following equation (8).
Pr '(K) = Pr (K-1) + [Pr (K-1) -Pr (K-2)] (8)
The above equation (8) is effective when the speed change [Pr (K-1) -Pr (K-2)] is constant, that is, it can be regarded as constant acceleration. For example, it is possible to set the search range appropriately by applying to the case where the sweep speed changes so as to gradually increase at the start of the sweep.

  Even when the imaging interval I (K) cannot be regarded as constant, the search range can be appropriately set by taking into account the speed change in the same way as described above. For example, by applying a change in speed to a quadratic expression by the method of least squares, it is possible to perform estimation in consideration of the change in speed. The expression for applying the speed change is not limited to the quadratic expression, and may be applied to a polynomial expression. In addition, as long as it can be applied to a polynomial, the estimation may be performed using other known methods without being limited to the least square method.

  When the relative position detection unit 30 detects the relative position between the partial images, as shown in FIG. 18, each partial image is divided into two or more partial areas (overlapping areas OW) having overlapping areas (overlapping areas OW). In FIG. 18, two search templates TPl and TPr are handled separately, and the relative positions of these two or more partial images are detected for each partial region, for example, as shown in FIG. Even when the finger is greatly deformed when sampling, the relative position of each part can be accurately detected.

17 and 18, f (T) is a partial image collected at time T, and f (T + 1) is a partial image collected at time T + 1. Further, as shown in the upper part of FIG. 18, the two search templates TPl and TPr are set so as to divide the partial image f (T) into left and right two parts and include an overlapping region OW that overlaps each other. Is set.
In the image in the search template TPL on the left side, there are relatively few features that determine the match as a matching result, and in the image in the search template TPr on the right side, there are branch points and end points that determine the match as the matching result. Such features are included.

  As shown in FIG. 17, the ridge image is greatly distorted due to friction between the finger 200 and the sensor surface 11 on the left side of the partial image f (T + 1) obtained next to the partial image f (T). In this case, as described above, by performing a search using two search templates TPl and TPr having mutually overlapping regions OW and detecting the relative positions, the relative positions of the respective parts as shown in the lower part of FIG. The position can be detected with high accuracy.

Here, when detecting the relative position, the reason for selecting two or more partial areas so as to overlap each other is that when one of the partial areas does not have sufficient features as in the example shown in FIG. This is to prevent the relative position from being properly detected and the detection accuracy from deteriorating.
In the example shown in FIG. 18, the partial image is divided into two partial areas each having an overlap area. However, the partial image is divided into three or more partial areas (templates) each having an overlap area. Similarly, the relative position may be detected.

[2-5] Image distortion due to delay characteristic of fingerprint sensor and correction method of the image distortion Next, image distortion due to the delay characteristic of fingerprint sensor 10 will be described with reference to FIGS. At the same time, a method for correcting the image distortion by the correction unit 40 of the present embodiment will be described.

When the fingerprint sensor 10 is configured to have a mechanism that simultaneously detects information of all pixels and saves it to a buffer or the like, image distortion due to delay characteristics does not occur, and correction by the correction unit 40 is unnecessary. Become.
However, in the fingerprint sensor 10 (see FIG. 19) that collects an image for each scanning line, as shown in FIG. 24, expansion / contraction distortion that changes the aspect ratio of the collected image occurs, and an image is captured for each pixel (cell). In the fingerprint sensor 10 (see FIG. 20) to be collected, skew distortion as shown in FIG. 26 occurs.

  Therefore, in the correction unit 40 of the present embodiment, the relative position detected by the relative position detection unit 30, the sampling time interval of each partial image by the fingerprint sensor 10, and the sampling delay time of each partial image by the fingerprint sensor 10 are used. Based on this, the position of the feature in each partial image is corrected so as to eliminate the distortion of each partial image accompanying the sampling delay of the fingerprint sensor 10. As a result, the image distortion (stretching distortion or skew distortion) as described above is removed from the feature information.

  Here, FIG. 19 is a block diagram showing a configuration example of the fingerprint sensor 10 capable of reading the imaging data for each line. As shown in FIG. 19, the fingerprint sensor 10 capable of reading the imaging data for each line includes: A plurality of line sensors 10a along the main scanning direction (left-right direction in the figure) are arranged in the sub-scanning direction (up-down direction in the figure). Each line sensor 10a is configured by arranging a large number of cells 10b in a line in the main scanning direction. The image information collected by the plurality of line sensors 10a is sequentially selected by the selector 10c, read out to the buffer 10d for each line sensor 10a (each scanning line), and output to the outside.

  FIG. 20 is a block diagram showing a configuration example of the fingerprint sensor 10 (an example of a sensor in which the delay time in units of pixels cannot be ignored) that can read out imaging data for each cell. As shown in FIG. The fingerprint sensor 10 capable of reading out image data is configured by arranging a large number of cells 10b in a matrix. Furthermore, the image information collected in a large number of cells 10b is sequentially selected by the two-stage selectors 10e and 10f, and read out to the outside for each cell 10b (each pixel).

In the fingerprint sensor 10 shown in FIG. 19, image information is read for each scanning line (for each line sensor 10 a). At that time, between reading one scanning line and reading the next scanning line. Delay time (biological information detection delay time: means a delay time caused by characteristics of the biometric information input means, that is, the fingerprint sensor 10).
In such a fingerprint sensor 10, when a partial image is taken with the finger 200 stationary as shown in FIG. 21, the aspect ratio of the partial image becomes 1 as shown in FIG.

  Here, for example, in the fingerprint sensor 10 shown in FIG. 19, the time required for reading information from each pixel (cell 10b) is sufficiently short on one scanning line and can be ignored, but when switching the scanning line for information reading. It is assumed that the delay time Δt is relatively large. In such a fingerprint sensor 10, when an image is acquired by sweeping the finger 200 in the sub-scanning direction, the aspect ratio changes according to the speed of the finger, and an expansion / contraction distortion occurs in the image. This is because the scanning line interval d is apparently d ′ (≠ d). As shown in FIG. 23, when the finger moves along the sub-scanning direction, the sub-scanning speed Vs apparently becomes Vs ′ depending on the speed Vf of the finger 200. Here, the apparent sub-scanning speed Vs ′ is given by the following equation (9).

Vs ′ = Vs−Vf (9)
The sub-scanning speed Vs is expressed by the following equation (10) using the scanning line interval d and the delay time Δt.
Vs = d / Δt (10)
The apparent sub-scanning speed Vs ′ is expressed by the following equation (11).
Vs ′ = d ′ / Δt (11)
If the expansion ratio in the sub-scanning direction is E,
d '= Ed (12)
E = d ′ / d = Vs′Δt / (VsΔt) = (Vs−Vf) / Vs (13)
As a result, the aspect ratio changes. The above formulas (11) and (12) indicate that the scanning line interval is apparently shortened as the finger 200 is moved in the sub-scanning direction and the speed is increased. Will appear stretched. Conversely, the above equations (11) and (12) represent that the finger 200 is moved in the direction opposite to the sub-scanning direction and the scanning line interval d is apparently longer as the speed increases, and the image is handled in the same manner. As shown in FIG. 24, the image appears to be shortened.

However, since the speed Vf of the finger 200 cannot be obtained directly from the image, the speed Vf is calculated using the apparent speed Vf ′ of the finger 200. Here, Vf ′ can be obtained from the relative position ΔY ′ detected by the relative position detector 30 and the input interval I of the fingerprint sensor 10 by the following equation (14).
Vf '= ΔY' / I (14)
Here, ΔY ′ is calculated by regarding d ′ as d. When the relationship between ΔY ′ and ΔY is corrected using the expansion / contraction rate E, the following equation (15) is obtained.

ΔY '= ΔY / E (15)
From the relationship of Vf = ΔY / I and the above equation (13), the following equation (16) is obtained.
Vf ′ = ΔY ′ / I = ΔY / (IE)
= Vf / E = VsVf / (Vs-Vf) (16)
Therefore, by solving the equation (16) for Vf, Vf is calculated by the following equation (17), and the velocity Vf of the finger 200 can be expressed by the sub-scanning velocity Vs and the apparent finger velocity Vf ′. .

Vf = VsVf ′ / (Vs + Vf ′) (17)
Further, since the relationship of the following equation (18) is established, the expansion / contraction ratio E is expressed by the following equation (19) using Vf ′.
ΔY / I = (ΔY ′ / I) (Vs / (Vs + Vf ′))
ΔY = (Vs / (Vs + Vf ′)) ΔY ′ (18)
E = Vs / (Vs + Vf ′) (19)
That is, when the relative position ΔY ′ is detected, it is possible to eliminate the influence of the change in the aspect ratio from the detected relative position ΔY ′ by multiplying ΔY ′ by Vs / (Vs + Vf ′). 17) shows. Therefore, the expansion / contraction distortion of the image can be removed and corrected by expanding the image by Vs / (Vs + Vf ′) times in the sub-scanning direction.

  On the other hand, as shown in FIG. 20, a case is considered in which the time Δtx required for reading information from the pixel (cell 10b) and the delay time Δt for switching the scanning line cannot be ignored. As shown in FIG. 25, it is assumed that there are X pixels (X cells) in one scanning line. At this time, if the finger 200 moves at the speed Vf, the finger 200 moves by VfXΔtx pixels while reading out pixel information from one end of the scanning line to the other. Assuming that the finger 200 is moving in parallel in the sub-scanning direction, the image is distorted at a skew angle φ given by the following equation (20).

φ = atan (VfΔtx) (20)
Here, Vf is given by the above equation (17), and Vs = d / (XΔtx + Δty). The correction of the skew distortion is realized by giving an offset of −Xatan (VfΔtx) in the sub scanning direction to the position of each pixel.
Consider the case where the finger 200 is moving in an oblique direction. If the main scanning speed is Vp and the apparent speed of the finger 200 in the main scanning direction is Vfp ′, the image needs to be expanded by Vfp′Vp / (Vp + Vfp ′) times in the main scanning direction in order to correct distortion. . That is, first, correction in the sub-scanning direction is performed, then skew distortion is corrected, and then correction in the main scanning direction is performed. All of the corrections can be realized by image deformation processing by affine transformation.

  When creating a reconstructed image for demonstration, the partial images are connected after correcting the distortion in each partial image by affine transformation as described above. However, when the imaging interval is sufficiently short, it can be considered that the distortion of successive partial images is hardly changed. Therefore, the processing time can be increased by locally performing the above correction after connecting the partial images. Can be reduced.

  As described above, the correction unit 40 corrects the position of the feature in each partial image so as to eliminate the distortion of each partial image due to the sampling delay of the fingerprint sensor 10 and obtains the relative position of the feature, thereby obtaining the fingerprint sensor. Even if there is a detection delay in the image 10, image distortion (expansion distortion or skew distortion) caused by the detection delay is removed from the feature information. As a result, characteristic information without distortion can be obtained, and not only deterioration of matching performance can be avoided, but also high-quality feature information can be checked during personal authentication. Reliability can be improved.

  Note that, as described with reference to FIGS. 17 and 18, the relative position detection unit 30 converts each partial image continuously collected by the fingerprint sensor 10 into two or more partial regions having overlapping regions. When separately handling and detecting the relative position between partial images for each partial region, the correction unit 40 eliminates the distortion of each partial image due to the collection delay of the fingerprint sensor 10 for each partial region in the same manner as described above. To correct the position of the feature. That is, the correction coefficient is obtained from the relative position detected in each partial area, and the distortion of the image is corrected. This is because the speed of the finger 200 varies depending on the contact position of the finger 200. For example, the area around the belly of the finger moves at a slower speed than the surroundings until the skin of the part is fully stretched, and moves at the same speed as the surroundings after the skin is fully stretched. In addition, when the skin contracts rapidly due to the reaction of the stretched skin, the part moves at a faster speed than the surroundings. Therefore, since the image distortion varies depending on the difference in speed, correction is performed for each region. Where the areas overlap, the correction coefficient is obtained from the average value of the relative positions obtained from the respective areas. Thus, even when the finger 200 is greatly deformed, the relative position of each part can be detected with high accuracy, and even if there is a detection delay in the fingerprint sensor 10, there is no image distortion (stretching distortion or skew distortion). Fingerprint data for registration or verification). Therefore, it contributes to improvement of collation performance, that is, improvement of reliability of personal authentication.

[2-6] Distortion due to finger deformation and correction method for the distortion Next, the distortion due to finger deformation will be described with reference to FIGS. 27 to 31 and the correction unit 40 according to the present embodiment. The distortion correction method will be described.
Here, FIGS. 27 to 29 are diagrams for explaining the change of the ridge image for each partial image, and FIGS. 30A to 30E are diagrams showing the transition of the deformation of the fingerprint during the sweep operation. 31 is a diagram illustrating an example of a strain distribution obtained in the present embodiment.

27 to 29, r (T) is a ridge image taken at time T, r (T + 1) is a ridge image taken at time T + 1, and r (T + 2) is taken at time T + 2. These ridge images r (T), r (T + 1), and r (T + 2) are the same ridge lines included in the three partial images successively collected by the fingerprint sensor 10. It is an image.
In the correction unit 40 of the present embodiment, a distortion amount is estimated from the relative position detected by the relative position detection unit 30 and the feature position in each partial image, and the feature position in each partial image is based on the distortion amount. Is corrected, the distortion caused by the deformation of the finger itself when the partial image of the fingerprint is collected is removed from the feature information.

More specifically, when the degree of deformation of the finger 200 is different at the time of capturing successive partial images, the deformation (distortion amount) of the finger 200 based on the relative distance between the matching features in the overlapping region of the consecutive partial images. ) And the position of the feature is corrected based on the estimation result.
Since the portion where the frictional force acts between the finger 200 and the sensor surface 11 moves as the finger 200 is swept, it matches when comparing the partial images as shown in FIG. The distances between the considered features are not the same. In the example shown in FIG. 27, the distance between the left branch points is larger than the distance between the right end points. That is, the amount of distortion due to the frictional force is not uniform within the partial image, but varies depending on the position within the partial image. However, correcting the distortion that differs depending on the position in the partial image significantly increases the time required for the correction process as compared to correcting the entire partial image uniformly. Therefore, the correction unit 40 according to the present embodiment corrects only the features extracted by the feature extraction unit 20 so as to not only improve the matching performance but also perform correction processing in a short time. Yes.

  As shown in FIGS. 28 and 29, it is considered that frictional force is generated along the sweep direction and deformation of the ridges occurs. Therefore, in the overlap region of the continuous partial images, matching features are arranged in the sweep direction. Explore along. Here, since the correlation of the part without deformation is lower than that of the part with deformation, the reference in the distortion amount measurement is the relative position detected by the relative position detection unit 30.

  At this time, as shown in FIGS. 30A to 30E, the distortion due to the deformation of the finger 200 propagates from the start to the end of the sweep operation. The position of the feature is corrected using the value obtained by integrating. By performing the above integration, for example, a distortion distribution as shown in FIG. 31 is obtained until the sweep operation is completed.

  In the strain distribution shown in FIG. 31, a region R1 is a region where a fingerprint image is taken at the start of sweeping, and is a central portion (abdominal side) region of the finger 200, and this central portion is a relatively soft portion. In this region R1, the amount of distortion gradually increases. The region R2 is a region following the region R1, and in this region R2, the change in sweep speed is reduced and the amount of distortion is also reduced. The region R3 is a region on the fingertip side following the region R2, and since the fingertip is relatively difficult to deform, the distortion in the region R3 is relatively small.

  Based on the distortion distribution estimated as described above, the correction unit 40 corrects the position of the feature in each partial image and obtains the relative position of the feature, whereby the finger itself is deformed when the fingerprint partial image is collected. For this reason, the distortion generated is removed from the feature information. As a result, distortion-free feature information (registration or collation fingerprint data) is obtained, and collation with high-quality feature information is possible at the time of personal authentication, improving collation performance, that is, reliability of personal authentication Improvement can be realized.

[2-7] Function of Moving Object Detection Unit Next, the function of the moving object detection unit 50 of the present embodiment will be described.
When a plurality of partial images are continuously collected using the sweep type fingerprint sensor 10 as in the present embodiment, as described in the item [2-2], the plurality of partial images obtained by the fingerprint sensor 10 are used. Based on the above, it is possible to separate a fingerprint image (ridge line image) and a pattern image (background image) that is not a moving object. The moving object detection unit 50 of the present embodiment uses the function described as the background removal function in item [2-2], and moves the moving object (for example, the finger 200; Line image) is detected. Thereby, it is not necessary to prepare a separate sensor for detecting the moving object, and the utility can be improved by detecting the moving object without incurring costs.

  Specifically, as described in the item [2-2], the moving object detection unit 50 obtains the weighted average image G (i, j, K) of the partial images collected by the fingerprint sensor 10 until immediately before. Calculated by the above equations (1) and (2). Then, the moving object detection unit 50 calculates a difference value between the calculated weighted average image G (i, j, K) and the latest partial image F (i, j, K), and the difference value is predetermined. It is determined that the presence of a moving object (ridge image, that is, the finger 200) is detected when the threshold value is exceeded.

At this time, when removing the non-moving pattern (background), if the weighted average image G (i, j, K) calculated by the removal function is used as it is, the amount of calculation in the authentication device (CPU 100) is increased. It is possible to detect a moving object without inviting.
In addition, by setting the above-mentioned predetermined threshold value, which is a reference for detecting moving objects, to be larger than the fluctuation value due to noise, it is possible to reliably prevent noise from being detected as moving objects, and to detect moving objects. Performance can be improved.

[2-8] Collation Processing by Collation Unit Next, collation processing by the collation unit 70 of the present embodiment will be described.
In the authentication apparatus according to the present embodiment, the features are sequentially extracted and generated from the previously input partial image group in accordance with the sweep operation, so that the matching unit 70 performs the matching process before obtaining the features of the entire fingerprint. It is possible to start.

Therefore, the collation unit 70 according to the present embodiment executes collation processing by using the feature obtained from the partial image with early imaging time and its relative position preferentially and confirms the verification result of the person to be authenticated. The collation process is terminated at
As a result, since the features are sequentially generated from the partial image group previously input by the sweep operation, the matching process can be started before obtaining the entire feature, while the matching result is confirmed. For example, when it is found that the person to be authenticated is not the person himself / herself, the verification process can be terminated early.

  In addition, since the collation unit 70 according to the present embodiment does not need to collate at least information other than the moving object, the collation unit 70 does not detect the presence of the moving object by the moving object detection unit 50. The collation process is not executed, that is, the collation process is not started while the moving object is not detected, and the collation process is performed only while the presence of the moving object is detected.

Thereby, the time and load required for calculation can be reduced. Further, since it is possible to avoid the feature information extracted from the noise being used for the matching process, the matching performance is not deteriorated.
Note that while the moving object is not detected by the moving object detection unit 50, the correction process by the correction unit 40 may not be executed, or the detection process by the relative position detection unit 30 may not be executed. This also reduces the time and load required for computation.

[2-9] Friction Force Reducing Unit Next, referring to FIGS. 32 to 35, the sensor surface of the fingerprint sensor 10 in order to reduce the friction force generated with the contact movement of the finger (subject) 200. The frictional force reducing means 15 to 18 provided around 11 will be described. Here, FIGS. 32 to 35 are perspective views schematically showing friction force reducing means 15 to 18 in the present embodiment, respectively.

The frictional force reducing means 15 shown in FIG. 32 is, for example, Teflon (registered) installed in the finger contact range 13 on the fingertip side (left side of the sensor surface 11 in FIG. 32) on the housing surface 12 of the fingerprint sensor 10. (Trademark) and other materials having a low friction coefficient (low friction coefficient material).
Thereby, at the time of the sweep operation, the frictional force reducing means 15 is arranged between the front end side (fingertip side) of the finger 200 and the housing surface 12, and even if pressure is applied to the fingertip during the sweep operation, It becomes possible to suppress the deformation of the finger 200 caused by the finger 200 being pulled by the frictional force.

Note that the above-described frictional force reducing means using the low coefficient of friction material is applied to the finger contact range 13 on the base side of the finger 200 (the right side of the sensor surface 11 in FIG. 32) on the casing surface 12 of the fingerprint sensor 10. In addition, the deformation of the finger 200 can be suppressed more reliably, and the deformation of the finger 200 can be reliably suppressed when the sweep direction of the finger 200 is not limited.
The frictional force reducing means 16 shown in FIG. 33 is provided on the housing surface 12 of the fingerprint sensor 10 provided with the frictional force reducing means 15 similar to that shown in FIG. 32 (on the right side of the sensor surface 11 in FIG. 33). ) Is configured as a protrusion.

  This protrusion 16 is in the shape of a long object arranged parallel to the sensor end (left end in FIG. 33), that is, parallel to the longitudinal direction of the sensor surface 11, and is separated from the fingerprint imaging target range 14 by an appropriate distance. Projecting from the housing surface 12 at the above position. In addition, the installation position of the protrusion 16 is a position where the first joint of the finger 200 is estimated to exist at the start of fingerprint collection (at the start of sweep). For example, the end of the fingerprint imaging target range 14 (FIG. 33). It is a position within a range of 4 to 10 mm from the left end in the middle.

  The height of the protrusion 16 is such that the sensor surface 11 and the finger 200 can sufficiently contact each other when the first joint of the finger 200 is placed on the protrusion 16 (for example, 0.2 to 1 mm). The width of the protrusion 16 is approximately the same horizontal width as the finger 200 (for example, around 10 mm). The shape of the cross section perpendicular to the longitudinal direction of the projection 16 is an inverted U shape so that the finger 200 can smoothly contact the projection 16.

  By performing the sweep operation of the finger 200 using the protrusion 16 as a fulcrum, not only pressure is not applied to the contact surface between the sensor surface 11 and the finger 200 but also the base side of the finger 200 and the housing surface 12. It can be avoided that the finger 200 is deformed so as to be contracted due to the friction with the. Further, when the sweep direction of the finger 200 is not limited, deformation due to friction between the portion immediately before being imaged and the housing surface 12 in accordance with the sweep operation can be reduced.

As the frictional force reducing means, instead of the protrusion 16 as shown in FIG. 33, a protrusion 17 as shown in FIG. 34 or a protrusion 18 as shown in FIG. 35 may be provided.
The protrusion 17 shown in FIG. 34 is formed so that the height near the center is lower than the height of both ends. By using the projection 17 having such a shape, the lateral movement of the finger 200 can be prevented during the sweep operation, fingerprint data (biological information) can be reliably acquired, and more stable collation is possible. Become. Further, according to the protrusion 17, in addition to the above-described actions and effects, an action of extending the bending of the belly of the finger 200 is obtained, and the finger 200 and the sensor surface 11 are more easily brought into contact with each other.

The protrusion 18 shown in FIG. 35 is formed such that the center portion is raised from both end portions. By using the protrusion 18 having such a shape, the effect of extending the bending of the belly of the finger 200 during the sweep operation is stronger than that of the protrusion 17.
The low friction coefficient material 15 and the protrusions 16 to 18 described above also function as a marker for instructing the sweep operation start position of the finger 200, and can improve convenience.

As described above, when the sweep type fingerprint sensor 10 is used, the sweep operation of the finger 200 can be easily performed by providing the friction force reducing means 15 to 18 that reduce the friction force generated with the contact movement of the finger 200. In addition to improving convenience, the deformation of the finger 200 can be reduced, and a highly reproducible biometric information image can be obtained and collation performance can be improved.
[2-10] Operation of Authentication Device of this Embodiment Next, referring to the flowchart (steps S11 to S22) shown in FIG. Will be described.

  First, initial adjustment of the fingerprint sensor 10 is performed (step S11). As the initial adjustment, gain adjustment for appropriately detecting finger information is performed. In addition, when the moving object detection unit 50 detects a moving object, the weighted time average of partial images continuously taken in the initial state by the fingerprint sensor 10 is prevented so that noise is not erroneously detected as a moving object. An image is created, the average image is stored in the volatile memory unit 90, and a threshold is set based on the average image so that noise is not detected as a moving object. At this time, the finger 200 may be placed on the sensor surface 11 of the fingerprint sensor 10 as long as it is stationary without starting the sweep operation.

After the initial adjustment, the partial images visualized by the fingerprint sensor 10 are sequentially stored in the volatile memory unit 90 (step S12). At this time, each time a partial image is collected, the imaging start time is read from the real-time clock 80, and the imaging start time and the partial image are associated with each other and stored in the volatile memory unit 90.
Then, the moving object detection unit 50 creates a weighted time average image of the partial images that are continuously input, performs background removal processing of the partial image, and the feature extraction unit 20 determines from the partial image after background removal, Foreground and edges are extracted as image features, and end points and branch points are extracted as fingerprint features and stored in the volatile memory unit 90 (step S13).

The relative position detection unit 30 detects a relative position between the partial images based on the feature extracted in step S13, and stores the relative position in the volatile memory unit 90 in association with the partial image (step S14). .
The correction unit 40 calculates the sweep speed using the relative position detected in step S14 and the time stored at the time of imaging, corrects the position of the feature according to the delay characteristic of the fingerprint sensor 10, and The degree of deformation (distortion distribution) is estimated and detected, and the position of the feature is further corrected according to the degree of deformation (step S15).

  During this time, the moving object detection unit 50 detects the moving object using the weighted time average image of the partial images that are continuously input (step S16). At this time, as the weighted time average image, the one calculated during the background removal process in step S13 is used. When the moving object is detected by the moving object detection unit 50 (YES route in step S17), the registration / collation data generation unit 60 registers the registration data corresponding to the person to be authenticated in the nonvolatile memory unit 91. It is confirmed whether it has been done (step S18). When the registration data is registered (YES route of step S18), the registration / collation data generation unit 60 associates the feature with the relative position stored in the volatile memory unit 90 in association with the partial image. Based on the above, collation data is generated and sent to the collation unit 70 in the input order of the partial images (step S19).

The collation unit 70 that has received the collation data reads the features from the nonvolatile memory unit 91 in the order generated when the registration data was created (step S20), and collates the collation data with the registration data (step S21). The collation process by the collation unit 70 is sequentially executed until all the registered data is read out.
When all of the registered data has been read from the non-volatile memory unit 91 and there is no feature matching the registered data and the verification data, it is assumed that verification has failed (personal authentication failure), and the process ends ( YES route of step S22). Further, at the time when the input of the verification data is completed along with the end of the sweep operation of the finger 200, even if the amount of verification data is insufficient, the verification is regarded as a verification failure (personal authentication failure) and the process ends (step S22). YES route).

  If the collation result is not confirmed / completed (NO route in step S22), the process returns to step S12 and the same process is repeated. If it is determined in step S17 that no moving object is detected (NO route), or if it is determined in step S18 that there is no registered data (NO route), the process proceeds to step S22 to complete the verification. Determine whether or not. Furthermore, before the completion of reading all of the registered data from the nonvolatile memory unit 91, if the collation result is determined by the collation process in step S21, the collation process is terminated at that time (YES route in step S22).

[3] Others The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.
For example, in the above-described embodiment, the case where the subject is a person's finger and the fingerprint image is collected as biological information has been described, but the present invention is not limited to this, and from the palm as the subject, The same applies to the case where a palm print image, a blood vessel pattern image, or the like is collected as biological information, and the same effect as described above can be obtained. The present invention is also applied in the same manner as described above even when a nose pattern image is collected as biological information from the nose of a cow or the like as a subject, and the same effects as described above can be obtained.

  In the above-described embodiment, the case where the fingerprint sensor is a capacitance type has been described, but an optical type may be used.

  As described above, according to the present invention, the relative position between two or more partial images is detected, and the feature in each partial image is based on the amount of distortion estimated from the relative position and the position of the feature in each partial image. The position of is corrected. As a result, characteristic information without distortion can be obtained, and collation with high-quality characteristic information can be performed at the time of personal authentication, so that the collation performance can be improved, that is, the reliability of personal authentication can be improved. At this time, since the feature information extracted from each partial image is corrected, it is not necessary to store each partial image as it is, and feature information without distortion can be obtained at a high speed with a small memory capacity.

  Therefore, the present invention is suitable for use in a system that performs personal authentication in a device that does not have sufficient space for incorporating a sensor, such as a small information device such as a mobile phone or a PDA, and its usefulness is considered to be extremely high. .

It is a block diagram which shows the function structure of the authentication apparatus by the biometric information as one Embodiment of this invention. It is a block diagram which shows the specific structure of the authentication apparatus by the biometric information as one Embodiment of this invention. It is a figure for demonstrating the fingerprint image collection operation | movement (sweep operation | movement) in a sweep type fingerprint sensor. It is a figure for demonstrating the partial image continuously extract | collected by the sweep type fingerprint sensor. It is a figure which shows an example of the fingerprint image obtained by a general fingerprint sensor. It is a figure which shows an example (what was collected at the time of fingerprint data registration) of the whole fingerprint image reconstructed from the partial image collected by the sweep type fingerprint sensor. It is a figure which shows the other example (what was extract | collected from the to-be-authenticated person at the time of collation) of the whole fingerprint image reconstructed from the partial image extract | collected by the sweep type fingerprint sensor. It is a figure which shows an example of the partial image extract | collected in the state (state which stopped the finger | toe) which does not perform a sweep operation | movement. It is a figure which shows an example of the partial image extract | collected during a sweep operation | movement. It is a figure which shows an example of the partial image extract | collected when a sensor surface has a defect and dirt. It is a figure which shows the average image of the partial image extract | collected when there exists a defect and dirt as shown in FIG. It is a figure which shows an example of the partial image from which the pattern (defect and dirt image) which does not move was removed. It is a figure which shows an example of the partial image which becomes difficult to detect a relative position only by foreground information. It is a figure which shows an example of the partial image which becomes difficult to detect a relative position only by edge information. It is a figure which shows the example of the feature point extracted in a partial image. It is a figure for demonstrating the reason why it is possible to limit the next search range based on the relative position detected until immediately before. It is a figure for demonstrating the relative position detection method in the case of handling each partial image divided into two or more partial area | regions which have the area | region which mutually overlaps. It is a figure for demonstrating the relative position detection method in the case of handling each partial image divided into two or more partial area | regions which have the area | region which mutually overlaps. It is a block diagram which shows the structural example of the sensor which can read imaging data for every line. It is a block diagram which shows the structural example (an example of a sensor which cannot ignore the delay time of a pixel unit) of the sensor which can read imaging data for every cell. It is a figure which shows the state which made the finger | toe stand still with the sensor shown in FIG. It is a figure which shows an example of the partial image extract | collected in the state shown in FIG. It is a figure which shows the state which swept the finger | toe with the sensor shown in FIG. It is a figure which shows an example (stretching distortion) of the partial image extract | collected in the state shown in FIG. It is a figure which shows the state which swept the finger | toe with the sensor shown in FIG. It is a figure for demonstrating distortion (skew distortion) of the partial image extract | collected in the state shown in FIG. It is a figure for demonstrating the change of the ridgeline image for every partial image. It is a figure for demonstrating the change of the ridgeline image for every partial image. It is a figure for demonstrating the change of the ridgeline image for every partial image. (A)-(E) are figures which show the transition of a deformation | transformation of the fingerprint at the time of a sweep operation | movement. It is a figure which shows an example of the distortion distribution obtained in this embodiment. It is a perspective view which shows typically an example of the frictional force reduction means in this embodiment. It is a perspective view which shows typically the other example of the frictional force reduction means in this embodiment. It is a perspective view which shows the 1st modification of the protrusion as a frictional force reduction means in this embodiment. It is a perspective view which shows the 2nd modification of the protrusion as a frictional force reduction means in this embodiment. It is a flowchart for demonstrating operation | movement of the authentication apparatus of this embodiment.

Claims (20)

Image collection means for imaging biometric information and continuously collecting a plurality of partial images of the biometric information;
Feature extraction means for extracting a feature in each partial image and a position of the feature from each of the plurality of partial images collected by the image collection means;
Relative detection of the relative position of the two or more partial images based on the feature existing in a region where two or more partial images of the plurality of partial images collected by the image collecting means overlap. Position detecting means;
A distortion amount of the feature is calculated based on the relative position detected by the relative position detection means and the position of the feature in each partial image, and the feature in each partial image is calculated based on the calculated distortion amount. A biometric information acquisition apparatus comprising correction means for correcting the position of the characteristic and obtaining a relative position of the characteristic. The image collection means continuously collects the plurality of partial images from a subject moving relative to the image collection means;
Before the feature extraction by the feature extraction means, further comprising a removal means for removing the images of the non-moving pattern included in the plurality of partial images collected by the image collection means; The biological information acquisition apparatus according to claim 1.   The biometric information acquisition apparatus according to claim 1, wherein the feature extraction unit extracts a foreground and an edge of the foreground in each partial image as the features.   The biometric information acquisition apparatus according to claim 1, wherein the feature extraction unit extracts an end point and a branch point in a pattern obtained by thinning a foreground in each partial image as the feature.   The relative position detecting means detects a relative position between the two or more partial images successively collected by the image collecting means with reference to one or more relative positions detected immediately before. The biological information acquisition apparatus according to claim 1.   The relative position detection means estimates the relative position between the two or more partial images successively acquired by the image acquisition means based on the one or more relative positions detected immediately before, The biological information acquisition apparatus according to claim 1, wherein the detection is performed based on a relative position to be detected.   The relative position detecting means handles the two or more partial images successively collected by the image collecting means by dividing the two or more partial images into two or more partial areas each having an overlapping area. The biological information acquisition apparatus according to claim 1, wherein a mutual relative position is detected for each of the two or more partial regions. The image collection means continuously collects the plurality of partial images from a subject moving relative to the image collection means;
A moving object detection means for detecting the presence or absence of a moving object moving with respect to the image acquisition means based on the plurality of partial images acquired by the image acquisition means is further provided. The biological information acquisition apparatus according to any one of claims 1 to 7. The image collecting means continuously collects the plurality of partial images from a subject that is moving relatively in contact with the image collecting means,
The biological information acquiring apparatus according to claim 1, further comprising a friction force reducing unit that reduces a friction force generated with the contact movement of the subject. An image sampling means for imaging the biometric information of the person to be authenticated and continuously collecting a plurality of partial images of the biometric information;
Feature extraction means for extracting a feature in each partial image and a position of the feature from each of the plurality of partial images collected by the image collection means;
Relative detection of the relative position of the two or more partial images based on the feature existing in a region where two or more partial images of the plurality of partial images collected by the image collecting means overlap. Position detecting means;
A distortion amount of the feature is calculated based on the relative position detected by the relative position detection means and the position of the feature in each partial image, and the feature in each partial image is calculated based on the calculated distortion amount. Correction means for correcting the position of the feature to obtain a relative position of the feature;
Using the feature extracted by the feature extraction means and the relative position of the feature obtained by the correction means, at least one of registration data and verification data for authenticating the person to be authenticated Generating means for generating
And a verification unit configured to perform verification processing to authenticate the person to be authenticated using at least one of registration data and verification data generated by the generation unit. Authentication device using biometric information. The image collection means continuously collects the plurality of partial images from the subject of the authentication subject who is moving relative to the image collection means;
Before the feature extraction by the feature extraction means, further comprising a removal means for removing the images of the non-moving pattern included in the plurality of partial images collected by the image collection means; The biometric information authentication apparatus according to claim 10.   11. The biometric information authentication apparatus according to claim 10, wherein the feature extraction unit extracts a foreground and an edge of the foreground in each partial image as the features.   The biometric information authentication apparatus according to claim 10, wherein the feature extraction unit extracts, as the features, end points and branch points in a pattern obtained by thinning the foreground in each partial image.   The relative position detecting means detects a relative position between the two or more partial images successively collected by the image collecting means with reference to one or more relative positions detected immediately before. The authentication apparatus using biometric information according to claim 10.   The relative position detection means estimates the relative position between the two or more partial images successively acquired by the image acquisition means based on the one or more relative positions detected immediately before, The biometric information authentication apparatus according to claim 10, wherein detection is performed based on a relative position to be detected.   The relative position detecting means handles the two or more partial images successively collected by the image collecting means by dividing the two or more partial images into two or more partial areas each having an overlapping area. The biometric information authentication apparatus according to claim 10, wherein a relative position of each other is detected for each of the two or more partial regions. The image collection means continuously collects the plurality of partial images from the subject of the authentication subject who is moving relative to the image collection means;
A moving object detection means for detecting the presence or absence of a moving object moving with respect to the image acquisition means based on the plurality of partial images acquired by the image acquisition means is further provided. The authentication apparatus by biometric information of any one of Claims 10-16.   The collation means executes the collation processing by preferentially using the feature obtained from the partial image whose imaging time is early by the image collection means and its relative position. The authentication apparatus by the biometric information of any one of Claim 17.   19. The biometric information authentication apparatus according to claim 18, wherein the verification means ends the verification process when the verification result of the person to be authenticated is confirmed. The image collection means continuously collects the plurality of partial images from the subject of the authentication subject who is relatively moving while in contact with the image collection means;
The biometric information authentication apparatus according to any one of claims 10 to 19, further comprising friction force reducing means for reducing a frictional force generated with the contact movement of the subject. .

Images