JP2004192633A - Pattern collation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance collation accuracy in a pattern collation device for collating a registration pattern with a collation pattern. <P>SOLUTION: A registered fingerprint is collated with a collation fingerprint by a correlation system (amplitude suppression correlation method) in a first collation part 20A, the registered fingerprint is collated with the collation fingerprint by a feature point system in a second collation part 20B, and collation results are transmitted to a collation decision part 20D. The collation decision part 20D decides that the registered fingerprint and the collation fingerprint are "coincident (matched)" when a collation result by any one system is "coincident (OK)". In addition, collation by the first collation part 20A is performed first and when its collation result is "noncoincident (NG)", collation by the second collation part 20B can be performed. In addition, which one of the collation systems should be first conducted can be specified. This device is similarly applied not only to two-dimensional pattern such as the fingerprint but to one-dimensional, three-dimensional patterns or the like. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to a pattern matching device that matches a registered pattern with a matching pattern.

  As a conventional pattern matching device, there is a pattern matching device that employs a matching algorithm called a correlation method based on a cross-correlation between a registered pattern and a matching pattern.

  In this pattern matching apparatus, a two-dimensional discrete Fourier transform is applied to a two-dimensional matching pattern to create matching Fourier pattern data, and a registration Fourier of a registered pattern created by performing the same processing as the matching Fourier pattern data. The two-dimensional discrete Fourier transform (or two-dimensional discrete inverse Fourier transform) is performed on the combined Fourier pattern data. Then, based on the correlation value obtained from the composite pattern data (correlation pattern data) subjected to the two-dimensional discrete Fourier transform (or the two-dimensional discrete inverse Fourier transform), the match / mismatch between the matching pattern and the registered pattern Is determined.

  The present applicant has previously described a pattern matching apparatus that performs matching of an N-dimensional pattern [for example, voice (one-dimensional), fingerprint (two-dimensional), three-dimensional (three-dimensional)] based on frequency characteristics or spatial frequency characteristics. It has been proposed (for example, see Patent Document 1).

  Japanese Patent Application Laid-Open No. H11-163873 discloses a type of amplitude suppression processing in which, in addition to performing amplitude suppression processing (for example, log processing) on a synthesized Fourier pattern in a spatial frequency space, a Fourier transform is performed on a registration and verification pattern. There is also mentioned a "phase-only correlation method" in which a correlation value between a registered pattern and a matching pattern is obtained based on only the phase component of the pattern data to obtain a matching result.

  On the other hand, there is a method called a feature point method with respect to the above-described correlation method. This feature point method extracts feature points of both patterns to be collated (for example, an end point which is an end of a fingerprint pattern, a branch point which is a branch of a pattern, a corner in a figure, and the like), and the micro-characteristic points thereof are obtained. This is a method of determining whether a matching pattern matches a registered pattern based on feature parameters representing information (position, direction, type, etc. of feature points) (for example, see Patent Documents 2 and 3).

JP-A-9-22406 (paragraphs [0019] to [0052], FIG. 4) JP-A-7-57084 (paragraphs [0007] to [0034], FIG. 15) JP-A 1-211184 Introduction to Computer Image Processing (Japan Industrial Technology Center, published by Soken Publishing Co., Ltd., pp. 44-45)

  The correlation method, particularly the amplitude suppression correlation method including the above-described phase only correlation method, is more effective than the general matching algorithm, which is a feature point method, in that the influence of environmental changes such as illuminance when the matching pattern is imported into the matching device, the registration pattern It has the advantage that it is strong against the influence of displacement between the reference pattern and the matching pattern, and that the matching accuracy is much higher.

  For example, when the amplitude suppression correlation method is used for fingerprint collation, it is possible to perform collation with high accuracy even if the image quality of the registered fingerprint or the collated fingerprint is poor due to drying, wetness, or rough hands. FIGS. 30 (a) and 30 (b) show images of a registered fingerprint and a collation fingerprint of a person whose hands are rough and whose fingerprint is disturbed. Even in such a case, since the matching is performed based on the spatial frequency characteristics in the amplitude suppression correlation method, it is possible to determine the match / mismatch between the two fingerprints. On the other hand, in the feature point method, since the end points and the branch points cannot be extracted, it is difficult to determine the match / mismatch between the two fingerprints.

  However, as a result of recent experiments, it has been found that there are patterns that cannot be correctly matched by the amplitude suppression correlation method, but can be correctly matched by the feature point method. For example, when the registered fingerprint is properly obtained as shown in FIG. 31A, but the verification fingerprint is only the fingertip as shown in FIG. Since the pattern is distorted, the amplitude suppression correlation method may not be able to determine the match / mismatch between the two fingerprints. On the other hand, in the feature point method, the end point and the branch point can be extracted even with the fingerprint of only the fingertip, so that the coincidence / mismatch between the two fingerprints can be determined.

  The present invention has been made in order to solve such a problem, and an object of the present invention is to combine the different types of matching methods, namely, the correlation method and the feature point method, to cover each other's weaknesses, and to independently cover each other's weak points. It is an object of the present invention to provide a pattern matching apparatus which can achieve a much higher matching accuracy than the case where the above method is used.

  In order to achieve such an object, a first invention (an invention according to claim 1) includes a first matching unit that performs matching between a registered pattern and a matching pattern based on a correlation value between the patterns, A second matching unit that executes matching between the registered pattern and the matching pattern based on a previously defined characteristic parameter; and at least one of a matching result by the first matching unit and a matching result by the second matching unit. If one of the comparison results indicates a match between the registered pattern and the matching pattern, a matching determining means for determining that the registered pattern matches the matching pattern is provided.

  According to the present invention, the first matching unit performs matching between the registered pattern and the matching pattern by the correlation method, and the second matching unit performs matching between the registered pattern and the matching pattern by the feature point method. When a matching result indicating a match is obtained from the matching means, it is determined that the registered pattern matches the matching pattern. In other words, even if it is not determined that they match in the correlation method, if it is determined that they match in the feature point method, it is determined that both patterns match. Also, even if it is not determined that they match in the feature point method, if it is determined that they match in the correlation method, it is determined that both patterns match. If it is not determined that there is a match by either the correlation method or the feature point method, it is determined that both patterns do not match.

According to a second invention (an invention according to claim 2), a first matching unit that executes matching between a registered pattern and a matching pattern based on a correlation value between these patterns, and performs matching between the registered pattern and the matching pattern in advance. If the second collation unit that executes based on the defined feature parameters and the collation result by the first collation unit is a collation result indicating the match between the registered pattern and the collation pattern, the collation by the second collation unit And a collation judging means for judging that the registered pattern matches the collation pattern without executing the above.
According to the present invention, first, the first collation unit performs collation between the registered pattern and the collation pattern by the correlation method. As a result, if a matching result indicating a match is obtained from the first matching unit, it is immediately determined that both patterns match. If a matching result indicating a match is not obtained from the first matching unit, the second matching unit compares the registered pattern with the matching pattern by the feature point method. Thus, if a matching result indicating a match is obtained from the second matching unit, it is determined that the two patterns match. If no matching result indicating a match is obtained from the second matching means, it is determined that the two patterns do not match.

According to a third invention (an invention according to claim 3), a first matching means for executing a comparison between a registered pattern and a matching pattern based on a correlation value between the patterns, and a method for comparing the registered pattern with the matching pattern in advance. A second matching unit that executes based on the defined feature parameter; and a matching unit that matches the registered pattern and the matching pattern when the matching result obtained by the second matching unit matches the registered pattern. And a collation judging means for judging that the registered pattern matches the collation pattern without executing the above.
According to the present invention, first, the second matching unit performs matching between the registered pattern and the matching pattern by the feature point method. As a result, if a matching result indicating a match is obtained from the second matching unit, it is immediately determined that both patterns match. If a matching result indicating a match is not obtained from the second matching unit, the first matching unit compares the registered pattern with the matching pattern by a correlation method. Thus, if a matching result indicating a match is obtained from the first matching unit, it is determined that the two patterns match. If no matching result indicating a match is obtained from the first matching means, it is determined that the two patterns do not match.

  A fourth invention (an invention according to claim 4) is a first matching means for executing matching between a registered pattern and a matching pattern based on a correlation value between these patterns, and performing matching between the registered pattern and the matching pattern in advance. It is possible to specify a second collation unit that executes based on the defined characteristic parameter, and to specify which collation to perform first, collation by the first collation unit or collation by the second collation unit, in the execution order. If the collation result by the execution order designating means to be executed and the collation means designated earlier by the execution order designation means is a collation result indicating the match between the registered pattern and the collation pattern, the collation designated later by the execution A collation determining means for determining that the registered pattern and the collation pattern match without executing collation by the means.

According to the present invention, the first matching unit matches the registered pattern and the matching pattern by the correlation method, and the second matching unit matches the registered pattern and the matching pattern by the feature point method. At this time, it is possible to specify in advance which of the two types of collation should be executed.
When the correlation method is specified first, first, collation by the first collation means (collation by collation method) is executed, and if a collation result indicating a match is obtained from the first collation means, the two collations are immediately performed. The pattern is determined to match. If a matching result indicating a match is not obtained from the first matching means, matching by the second matching means (matching by the feature point method) is performed. Thus, if a matching result indicating a match is obtained from the second matching unit, it is determined that the two patterns match. If no matching result indicating a match is obtained from the second matching means, it is determined that the two patterns do not match.
When the feature point method is specified first, first, the matching by the second matching means (matching by the feature point method) is executed, and if a matching result indicating a match is obtained from the second matching means, Immediately, it is determined that both patterns match. If a collation result indicating a match is not obtained from the second collation means, collation by the first collation means (collation by a correlation method) is performed. Thus, if a matching result indicating a match is obtained from the first matching unit, it is determined that the two patterns match. If no matching result indicating a match is obtained from the first matching means, it is determined that the two patterns do not match.

A fifth invention (an invention according to claim 5) is a first comparison means for executing a comparison between a registered pattern and a matching pattern based on a correlation value between these patterns, and a method for comparing a registered pattern with a matching pattern in advance. A second matching unit that executes based on the defined characteristic parameter; an image inspection unit that inspects an image of the matching pattern; and an image inspection unit that inspects the image of the matching pattern by the first inspection unit based on an inspection result of the image of the matching pattern. An execution order designating unit for designating which of the collation and the collation by the second collation unit is to be executed first, and a collation result by the collation unit designated to be executed first by the execution order designation unit. If the matching result indicates a match between the registered pattern and the matching pattern, the registered pattern and the matching pattern are not executed by the matching means designated to be executed later. Those in which the over down is provided and determining the match determination means and matched.
According to the present invention, the image of the collation pattern is inspected, and the collation by the first collation means (collation by the correlation method) is first executed based on the inspection result, or the collation by the second collation means (characteristic point) Is determined first. For example, as the inspection of the image of the collation pattern, it is inspected whether the collation pattern has a sufficient area and whether the image quality is good. In the case where the area of the matching pattern is small, the correlation method is used for the overall similarity, so that the possibility of being erroneously recognized as a mismatch increases. In this case, since the feature point method is more suitable, it is better to execute the matching by the feature point method first. Even when the image quality is good and the feature points are clear, if the distortion is large, the possibility of being erroneously recognized as mismatch by the correlation method increases. Even in such a case, it is better to first execute the matching by the feature point method instead of the correlation method.

  A sixth invention (an invention according to claim 6) is a registration Fourier pattern data creating means for creating an N-dimensional pattern data by performing an N-dimensional discrete Fourier transform on the N-dimensional pattern data of the registered pattern, Matching Fourier pattern data creating means for applying N-dimensional discrete Fourier transform to N-dimensional pattern data to create matching Fourier N-dimensional pattern data, and first amplitude suppression for performing amplitude suppression processing on registered Fourier N-dimensional pattern data Means, a second amplitude suppression means for performing amplitude suppression processing on the matching Fourier N-dimensional pattern data, and a coordinate system of the registered Fourier N-dimensional pattern data subjected to the amplitude suppression processing by the first amplitude suppression means to a polar coordinate system. The first polar coordinate system conversion means for converting and the illumination suppressed by the second amplitude suppression means. Second polar coordinate system conversion means for converting the coordinate system of the Fourier N-dimensional pattern data into a polar coordinate system, registered Fourier N-dimensional pattern data converted into a polar coordinate system by the first polar coordinate system conversion means, and a second polar coordinate system conversion First matching means for matching the matched Fourier N-dimensional pattern data converted into the polar coordinate system by the means by the amplitude suppression correlation method, and a position of the correlation peak obtained in the matching process by the first matching means. Rotational displacement measuring means for determining the rotational displacement, rotational displacement correcting means for correcting the rotational displacement of one of the registered pattern and the verification pattern based on the rotational displacement obtained by the rotational displacement measuring means, Second collating means for compensating the registered pattern and the collation pattern by the amplitude suppression correlation method after correcting the rotational deviation by the deviation compensating means. A vertical / horizontal displacement amount measuring means for obtaining the vertical and horizontal displacement amounts of the correlation peaks obtained from the positions of the correlation peaks obtained in the collating process by the second collating means; Rotation / vertical / horizontal deviation correction means for correcting rotational deviation / vertical / horizontal deviation in one of the registered pattern and the verification pattern based on the vertical / horizontal deviation amounts obtained by the vertical / horizontal deviation amount measuring means; A third matching unit that corrects the registration pattern against the matching pattern based on a predefined characteristic parameter after correcting the rotation shift and the vertical and horizontal shifts by the vertical and horizontal shift correcting unit; Of at least one of the matching results of the matching means, the second matching means, and the third matching means indicates that the registered pattern matches the matching pattern. If the result is a match, a matching determining means for determining that the registered pattern matches the matching pattern is provided.

According to the present invention, the N-dimensional pattern data (R) of the registered pattern is subjected to the N-dimensional discrete Fourier transform to generate the registered Fourier N-dimensional pattern data (R _F ), and the N-dimensional pattern data (I ) Is subjected to an N-dimensional discrete Fourier transform to generate collated Fourier N-dimensional pattern data ( _IF ). The registered Fourier N-dimensional pattern data (R _F ) and the collated Fourier N-dimensional pattern data ( _IF ) are Amplitude suppression processing such as log processing and √ processing is performed, and the coordinate system of the registered Fourier N-dimensional pattern data (R _FL ) and the collated Fourier N-dimensional pattern data ( _IFL ) subjected to the amplitude suppression processing is converted to a polar coordinate system. The registered Fourier N-dimensional pattern data (R _PL ) converted to the polar coordinate system and the collation Fourier N-dimensional pattern data (I _PL ) Matching (first matching) is performed by the suppression correlation method.
If a matching result indicating a match is not obtained by the first matching, the rotational deviation (Δθ) between the two is calculated from the position of the correlation peak obtained in the matching process of the first matching. Based on (Δθ), any one of the registered pattern and the collation pattern is subjected to rotational displacement correction, and the registered pattern and the collation pattern are collated again (second collation) by the amplitude suppression correlation method.
If a matching result indicating a match is not obtained by the second matching, the vertical and horizontal deviation amounts (ΔX, ΔY) between the two are obtained from the position of the correlation peak obtained in the matching process of the second matching. Any of the registered pattern and the matching pattern is determined based on the obtained vertical and horizontal deviation amounts (ΔX, ΔY) and the rotational deviation amount (Δθ) obtained from the position of the correlation peak obtained in the verification process of the first verification. After performing the rotational deviation and the vertical / horizontal deviation correction on one of them, the collation (third collation) between the registered pattern and the collation pattern is performed by the feature point method.

A seventh invention (an invention according to claim 7) is the sixth invention, wherein the sign of the phase is added to the amplitude for the registered Fourier N-dimensional pattern data and the collated Fourier N-dimensional pattern data subjected to the amplitude suppression processing, After extracting only the signed amplitude component, the coordinate system is converted to a polar coordinate system.
According to the present invention, the N-dimensional pattern data (R) of the registered pattern is subjected to the N-dimensional discrete Fourier transform to generate the registered Fourier N-dimensional pattern data (R _F ), and the N-dimensional pattern data (I ) Is subjected to an N-dimensional discrete Fourier transform to generate collated Fourier N-dimensional pattern data ( _IF ). The registered Fourier N-dimensional pattern data (R _F ) and the collated Fourier N-dimensional pattern data ( _IF ) are An amplitude suppression process such as a log process or a √ process is performed, and the sign of the phase of the registered Fourier N-dimensional pattern data (R _FL ) and the collated Fourier N-dimensional pattern data (I _FL ) subjected to the amplitude suppression process is changed. is added to the amplitude, after being extracted only signed amplitude components _{(R FL ', I FL'} ), the coordinate system is converted into the polar coordinate system, transform this polar coordinate system A registration Fourier N-dimensional pattern data (R _PL ') and the collation Fourier N-dimensional pattern data (I _PL') are matched by the amplitude suppression correlation method (first collation).

According to an eighth invention (an invention according to claim 8), in the sixth invention, after removing the phase components of the registered Fourier N-dimensional pattern data and matching Fourier N-dimensional pattern data, the registered Fourier N-dimensional pattern data and matching Fourier N The amplitude suppression processing is performed on the dimensional pattern data, and the coordinate system of the registered Fourier N-dimensional pattern data and the verification Fourier N-dimensional pattern data subjected to the amplitude suppression processing is converted to a polar coordinate system.
According to the present invention, the N-dimensional pattern data (R) of the registered pattern is subjected to the N-dimensional discrete Fourier transform to generate the registered Fourier N-dimensional pattern data (R _F ), and the N-dimensional pattern data (I ) Is subjected to an N-dimensional discrete Fourier transform to generate collated Fourier N-dimensional pattern data ( _IF ). The registered Fourier N-dimensional pattern data (R _F ) and the collated Fourier N-dimensional pattern data ( _IF ) are After removing the phase component, amplitude suppression processing such as log processing and √ processing is performed, and the registered Fourier N-dimensional pattern data (R _FL ') and the collated Fourier N-dimensional pattern data ( _IFL ') subjected to the amplitude suppression processing are performed. coordinate system is converted into the polar coordinate system, and the collation Fourier N-dimensional pattern converted registration Fourier N-dimensional pattern data into the polar coordinate system (R _PL ') Ndeta and (I _PL ') are matched by the amplitude suppression correlation method (first collation).

  According to the first invention, at least one of the collation result by the first collation unit that performs collation by the correlation method and the collation result by the second collation unit that performs collation by the feature point method is a registered pattern. If the matching result indicates that the registered pattern matches the matching pattern, it is determined that the registered pattern matches the matching pattern. , And the matching accuracy can be made much higher than in the case where each of them is executed independently.

  According to the second aspect, when the result of the first collation performed by the first collation unit that performs the collation by the correlation method is a result of the coincidence between the registered pattern and the collation pattern, the second collation that performs the collation by the feature point method Since it is determined that the registered pattern and the matching pattern match without executing the matching by the means, the matching determination is performed by using the amplitude suppression correlation method having a higher matching accuracy than the feature point method as the correlation method. You can get results faster. Further, even if the matching pattern cannot be correctly matched by the correlation method, the matching can be accurately performed by the feature point method, and the matching accuracy is improved.

  According to the third aspect, when the result of the comparison performed by the second matching unit that performs the matching by the feature point method is a result of matching indicating the match between the registered pattern and the matching pattern, the first matching that performs the matching by the correlation method Since it is determined that the registered pattern and the matching pattern match without executing the matching by the means, when the attribute of the matching pattern is suitable for the feature point method (for example, when the feature point is In the case of performing one-to-N matching (a method of matching one matching pattern with N registered patterns), the amount of calculation for matching is small. As a result, not only the matching accuracy is improved, but also the matching result can be obtained in a short time.

  According to the fourth aspect, by providing the execution order designating means for enabling the designation of the execution order of the matching by the correlation method and the matching by the feature point method, the compatibility with the two matching methods based on the attribute of the pattern to be matched is determined in advance. , It is possible to quickly obtain a collation determination result by designating that collation is performed first in a compatible manner. Further, even if the collation pattern cannot be correctly collated by the method executed first, the collation can be accurately performed by the method executed next, and the collation accuracy is increased.

  According to the fifth aspect, the image of the collation pattern is inspected, and it is determined whether the collation based collation or the feature point collation is performed first based on the inspection result. Therefore, when the area of the matching pattern is small or the image quality of the matching pattern is good, the matching by the matching method suitable for the current matching pattern is preferentially performed, such as performing the matching by the feature point method first. Thus, both the matching accuracy and the matching speed can be improved.

According to the sixth aspect, when a collation result indicating a match is not obtained by the first collation, the rotational deviation amount (Δθ) between the two is obtained from the position of the correlation peak obtained in the collation process of the first collation. After correcting the rotational deviation of either the registered pattern or the verification pattern based on the obtained rotational deviation (Δθ), the registered pattern and the verification pattern are verified again by the amplitude suppression correlation method (second verification). ) Is performed, and if a matching result indicating a match is not obtained by the second matching, the displacement amounts (ΔX, ΔY) in the vertical and horizontal directions from the position of the correlation peak obtained in the matching process of the second matching ) Is determined, and the registration pattern is determined based on the determined vertical and horizontal deviation amounts (ΔX, ΔY) and the rotational deviation amount (Δθ) determined from the position of the correlation peak obtained in the collation process of the first collation. And matching pattern After correcting the rotational deviation and the vertical / horizontal deviation for one of them, the registered pattern and the collation pattern are collated (third collation) by the feature point method, and the rotational deviation, the vertical direction, and the Even when there is a lateral shift, the matching can be performed with high accuracy.
When performing the third matching (feature point method), the rotational displacement and the longitudinal and lateral displacements have already been obtained in the matching process by the first matching and the second matching. Rotational deviation, vertical and horizontal deviations can be corrected, and the third collation can be performed quickly.

According to the seventh invention, the sign of the phase is added to the amplitude of the registered Fourier N-dimensional pattern data and the verification Fourier N-dimensional pattern data subjected to the amplitude suppression processing, and only the signed amplitude component is extracted. The coordinate system is converted to a polar coordinate system, making it less susceptible to errors due to phase discontinuities, and enabling high-accuracy matching even when there is an error such as positional deviation between registration and matching. Become.
According to the eighth invention, after removing the phase components of the registered Fourier N-dimensional pattern data and the collated Fourier N-dimensional pattern data, the registered Fourier N-dimensional pattern data and the collated Fourier N-dimensional pattern data are subjected to amplitude suppression processing. Since the coordinate system of the amplitude-suppressed registered Fourier N-dimensional pattern data and the collation Fourier N-dimensional pattern data is converted into a polar coordinate system, it is hardly affected by the change in illuminance. Even if there is a difference between them, it is possible to perform the matching with high accuracy. In addition, the performance at the time of obtaining the correlation peak by the amplitude suppression correlation method after the polar coordinate conversion is improved.

Hereinafter, the present invention will be described in detail with reference to the drawings.
[Embodiment 1: First to fifth inventions]
FIG. 1 is a block diagram of a fingerprint matching device according to an embodiment of the present invention. In FIG. 1, reference numeral 10 denotes an operation unit, and reference numeral 20 denotes a control unit. The operation unit 10 includes a numeric keypad 10-1, a display (LCD) 10-2, and a fingerprint sensor 10-3. The fingerprint sensor 10-3 includes a light source 10-31, a prism 10-32, and a CCD camera 10-33. The control unit 20 includes a control unit 20-1 having a CPU, a ROM 20-2, a RAM 20-3, a hard disk (HD) 20-4, a frame memory (FM) 20-5, and an external connection unit. An (I / F) 20-6 and a Fourier transform unit (FFT) 20-7 are provided, and a registration program and a collation program are stored in the ROM 20-2.

[Registration of fingerprint]
In this fingerprint matching device, a user's fingerprint (hereinafter, referred to as a registered fingerprint) to be a registered pattern is registered as follows. Before the operation, the user inputs the ID number assigned to the user using the numeric keypad 10-1 (step 101 shown in FIG. 2), and puts a finger on the prism 10-32 of the fingerprint sensor 10-3. Put. The prism 10-32 is irradiated with light from the light source 10-31. In the fingerprint concave portion (valley line portion) that does not contact the surface of the prism 10-32, the light from the light source 10-31 is totally reflected, Camera 10-33. Conversely, at the convex portion (ridge portion) of the fingerprint in contact with the surface of the prism 10-32, the condition of total reflection is broken, and light from the light source 10-31 is scattered. As a result, fingerprint valleys are bright, ridges are dark, and a fingerprint pattern with contrast is collected. The pattern of the collected fingerprint (registered fingerprint) is provided to the control unit 20 by A / D conversion, for example, as a grayscale image (image data: two-dimensional pattern data) having 512 × 512 pixels and 256 tones.

  The control unit 20-1 captures the image data of the registered fingerprint provided from the operation unit 10 into the frame memory 20-5 (Step 102), and calculates the area S and the image quality value Q of the captured registered fingerprint (Step 102). 103). The calculation processing of the area S and the image quality value Q is performed according to the flowchart shown in FIG.

  The control unit 20-1 extracts a boundary line between a region where the fingerprint pattern exists and a region where the fingerprint pattern does not exist, and calculates the number of pixels of the registered fingerprint including the boundary line as an area S with respect to the captured registered fingerprint. (Step 201). Further, the image of the registered fingerprint of 512 × 512 pixels is divided into blocks of 8 × 8 pixels and binarized (step 202), and the ridge direction (eight directions) of each block is calculated (step 203). Then, the continuity in the ridge direction between the blocks is evaluated to obtain an evaluation value (step 204). Then, the evaluation value is normalized by the area S to obtain an image quality value Q (step 205). The image quality value Q takes a value from 0 to 1, and the larger the image quality value Q, the worse the image quality.

  FIG. 4 shows a fingerprint image and a ridge direction block image obtained from the fingerprint image. FIG. 4A shows a fingerprint image and a ridge direction block image when the image quality value is obtained as Q = 0.13, and FIG. 4B shows a case where the image quality value is obtained as Q = 0.52. 3 shows a fingerprint image and a ridge direction block image of the image. The fingerprint image shown in FIG. 4B is for a person whose hands are rough and the pattern is disturbed, and the continuity in the ridge direction is impaired, which leads to deterioration of the image quality value.

  After calculating the area S and the image quality value Q of the registered fingerprint captured in this way, the control unit 20-1 compares the calculated area S with a predetermined threshold value Sth (step 104). If S ≦ Sth, the control unit 20-1 determines that the area of the fingerprint is small, returns to step 102, captures the image of the registered fingerprint again, and repeats step 103 and subsequent steps. If S> Sth, the control unit 20-1 determines that the area of the fingerprint is sufficient, and proceeds to step 105.

  In step 105, the number of captured images is checked, and the operations from step 102 are repeated until the number of captured images reaches N. In this way, the control unit 20-1 collects N registered fingerprint images whose area S exceeds Sth (YES in step 105), and the image quality value Q having the highest image quality among the N registered fingerprint images. The higher one is selected (step 106). Then, the image data of the selected registered fingerprint is filed in the hard disk 20-4 in correspondence with the ID number as original image data serving as a registered pattern (step 107).

[Fingerprint collation: collation method (1) (correlation method (amplitude suppression correlation method) + feature point method)]
In this fingerprint collation device, collation of the user's fingerprint is performed as follows. During operation, the user inputs his / her ID number using the numeric keypad 10-1 (step 301 shown in FIG. 5) and places his / her finger on the prism 10-32 of the fingerprint sensor 10-3. As a result, a pattern of a fingerprint (collation fingerprint) serving as a collation pattern is collected in the same manner as in the case of fingerprint registration, and is formed as a grayscale image (image data: two-dimensional pattern data) of 512 × 512 pixels and 256 gradations. It is provided to the control unit 20.

[Correlation method (amplitude suppression correlation method)]
When the ID number is given via the numeric keypad 10-1, the control unit 20-1 selects the registered fingerprint corresponding to the ID number from the image data of the registered fingerprint filed in the hard disk 20-4. The original image data is read (step 302). Then, a reduction process is performed on the read original image data of the registered fingerprint (step 303). This reduction processing is performed by thinning out pixel lines at a predetermined pixel pitch in the x direction (horizontal direction) and the y (vertical direction) direction of the original image data of 512 × 512 pixels and 256 gradations. For example, in the x direction and the y direction, pixel lines are thinned out at a pitch of 4 pixels each to obtain reduced data of 128 × 128 pixels.

  Then, the control unit 20-1 sends the reduced registered fingerprint image data (see FIG. 6A) to the Fourier transform unit 20-7, and converts the registered fingerprint image data into a two-dimensional discrete Fourier transform (DFT). ) (Step 304). As a result, the image data of the registered fingerprint shown in FIG. 6A becomes Fourier image data (registered Fourier image data) as shown in FIG. 6B.

  The control unit 20-1 captures the image data of the collation fingerprint given from the operation unit 10 via the frame memory 20-5 (step 305), and performs the acquired collation fingerprint image data in step 303. The same reduction processing as described above is performed (step 306).

  Then, the control unit 20-1 sends the reduced image data of the collation fingerprint (see FIG. 6E) to the Fourier transform unit 20-7, and converts the image data of the collation fingerprint into a two-dimensional discrete Fourier transform (DFT). ) (Step 307). Thereby, the image data of the collation fingerprint shown in FIG. 6E becomes Fourier image data (collation Fourier image data) as shown in FIG. 6F.

  The two-dimensional discrete Fourier transform is described in, for example, Non-Patent Document 1.

  Next, the control unit 20-1 combines the Fourier image data of the collation fingerprint obtained in Step 307 with the Fourier image data of the registered fingerprint obtained in Step 304 (Step 308) to obtain combined Fourier image data.

  Here, when the Fourier image data of the collation fingerprint is A · exp (jθ) and the Fourier image data of the registered fingerprint is B · exp (jφ), the registered Fourier image data of the collation fingerprint is used as the synthetic Fourier image data. A · B · exp (j (θ−φ)) obtained by multiplying the Fourier image data by the complex conjugate. However, A, B, θ, and φ are all functions of the spatial frequency (Fourier) space (u, v).

And AB exp (j (θ-φ)) is
A · B · exp (j (θ−φ)) = A · B · cos (θ−φ) + j · A · B · sin (θ−φ) (1)
Expressed as, if the A · exp (jθ) = α 1 + jβ 1, B · exp (jφ) = α 2 + jβ 2, A = (α 1 2 + β 1 2) 1/2, B = (α 2 2 _{^{+ β 2 2) 1/2, θ}} = tan -1 (β 1 / α 1), φ = tan becomes _{^{-1 (β 2 / α 2)}} . By calculating this equation (1), synthetic Fourier image data is obtained.

Note that ABexp (j (θ-φ)) = ABexp (jθ) exp (−jφ) = Aexp (jθ) Bexp (-jφ) = (α ₁ + jβ ₁ ) · (α ₂ −jβ ₂ ) = (α ₁ · α ₂ + β ₁ · β ₂ ) + j (α ₂ · β ₁ −α ₁ · β ₂ ), and the synthesized Fourier image data may be obtained. .

  Then, after obtaining the combined Fourier image data in this way, the control unit 20-1 performs an amplitude suppression process (step 309). In this embodiment, log processing is performed as amplitude suppression processing. That is, the logarithm of the amplitude A · B of A · B · exp (j (θ−φ)) which is the arithmetic expression of the above-described synthesized Fourier image data is obtained, and log (A · B) · exp (j (θ−φ )), The amplitude AB is suppressed to log (AB) (AB> log (AB)).

  FIG. 6D shows the synthesized Fourier image data after the amplitude suppression processing. In the synthesized Fourier image data subjected to the amplitude suppression processing, the influence of the illuminance difference between when the registered fingerprint is collected and when the collation fingerprint is collected is reduced. That is, by performing the amplitude suppression processing, the spectrum intensity of each pixel is suppressed, there is no outstanding value, and more information is effective. In addition, by performing amplitude suppression processing, feature points (end points and branch points) and ridge features (vortices and branches), which are personal information, of the fingerprint information are further emphasized, and ridges that are general fingerprint information are obtained. The overall flow and direction can be suppressed.

  In this embodiment, log processing is performed as amplitude suppression processing. However, Δ processing may be performed. Further, the processing is not limited to the log processing and the √ processing, and may be any processing as long as the amplitude can be suppressed. If all the amplitudes are set to, for example, 1 in the amplitude suppression, that is, if only the phase is used, there is an advantage that the amount of calculation can be reduced and that the amount of data decreases as compared with the log processing or the √ processing.

  After performing the amplitude suppression process in step 309, the control unit 20-1 sends the synthesized Fourier image data subjected to the amplitude suppression process to the Fourier transform unit 20-7, and again performs a two-dimensional discrete Fourier transform (DFT). Is performed (step 310). Thereby, the combined Fourier image data shown in FIG. 6D becomes the combined image data as shown in FIG. 6H. Although this image has suppressed amplitude in the frequency space, it can be basically considered as an image obtained by convolving the verification fingerprint and the registered fingerprint, and indicates the correlation between these two images.

  Then, the control unit 20-1 captures the composite image data obtained in step 310, scans the intensity (amplitude) of each pixel in a predetermined correlation component area from the composite image data, A histogram of the intensity of the correlation component of each pixel is obtained, and the upper n pixels (eight pixels in this embodiment) having a higher intensity of the correlation component are extracted from the histogram, and the intensity (correlation of the correlation component) of the extracted n pixels is extracted. The average of the peaks) is obtained as a correlation value (score) (step 311). Here, the correlation component area is defined as a region S0 surrounded by a white dotted line with respect to the combined Fourier image data shown in FIG.

  Then, the control unit 20-1 compares the correlation value obtained in step 311 with a predetermined threshold value (step 312), and if the correlation value is larger than the threshold value, checks by the amplitude suppression correlation method. The result is determined to be “match (OK)”. If the correlation value is equal to or smaller than the threshold value, the matching result by the amplitude suppression correlation method is determined to be “mismatch (NG)”.

[Feature point method]
On the other hand, the control unit 20-1 performs a binarization process on the original image data of the registered fingerprint read in step 302 (step 313), and applies the binarized registered fingerprint image data to the original image data. A thinning process is performed (step 314). Then, feature points (end points and branch points) are extracted from the image data of the registered fingerprint subjected to the thinning processing, and the position, direction, type, and the like are acquired as feature parameters (step 315).

  Further, the control unit 20-1 captures the image data of the collation fingerprint given from the operation unit 10 via the frame memory 20-5 (step 316), and corrects the positional deviation from the image data of the registered fingerprint (step 316). 317) The same binarization processing and thinning processing as performed in steps 313 and 314 are performed on the captured collation fingerprint image data (steps 318 and 319), and the binarization processing and thinning are performed. Feature points (end points and branch points) are extracted from the processed image data of the verification fingerprint, and the position, direction, type, and the like are acquired as feature parameters (step 320).

  Then, the control unit 20-1 determines each error value of the feature point parameters such as the position, direction, and type of the feature points of the registered fingerprint and the verification fingerprint extracted in steps 315 and 320 (for example, the feature point that should be an end point is branched). If it is a point, the error value is assumed to be 10), and this error value is added to obtain a collation score (step 321). Then, the obtained matching score is compared with a predetermined threshold value (step 322), and if the matching score is smaller than the threshold value, the matching result by the feature point method is determined to be “match (OK)”. I do. If the collation score is equal to or greater than the threshold value, the collation result by the feature point method is determined to be “unmatched (NG)”.

[OR of matching result by correlation method (amplitude suppression correlation method) and matching result by feature point method]
The control unit 20-1 performs final collation determination based on the collation result by the amplitude suppression correlation method obtained in Step 312 and the collation result by the feature point method obtained in Step 322 (Step 323). In this case, the control unit 20-1 determines that the registered fingerprint and the collated fingerprint match (match) if the collation result by any one of the methods is determined to be “match (OK)” (step S1). 324). On the other hand, if the matching result by any of the methods is determined to be “mismatch (NG)”, it is determined that the registered fingerprint and the matching fingerprint are “mismatch (mismatch)” (step 325).

  That is, in the matching method (1), even if it is not determined that the matching is performed by the amplitude suppression correlation method, if it is determined that the matching is performed by the feature point method, it is determined that the registered fingerprint matches the matching fingerprint, and the matching fingerprint is determined. Even if it is not determined that they match in the point method, if it is determined that they match by the amplitude suppression correlation method, it is determined that the registered fingerprint and the verification fingerprint match. If it is not determined that they match by either the amplitude suppression correlation method or the feature point method, it is determined that the registered fingerprint and the verification fingerprint do not match.

  Thus, even when the fingerprint is disturbed due to a rough hand, even when it is determined that the fingerprint does not match by the feature point method, it is determined that the fingerprint matches by the amplitude suppression correlation method, and the fingerprint is a part of the fingerprint, for example, the verification fingerprint of only the fingertip. In such a case, even when it is determined that they do not match by the amplitude suppression correlation method, they are determined to match by the feature point method. As described above, in the combination method according to the present embodiment, there is no risk of making a mistake in the collation determination even for a fingerprint that can be correctly collated in either one of the methods. This significantly improves the matching accuracy.

  It is also possible to combine two matching methods of the same type, such as combining the amplitude suppression correlation method and the cross-correlation method (normal correlation using the amplitude as it is) or combining two feature point methods defining different characteristic parameters. Conceivable. However, in this case, the matching accuracy is as low as the matching accuracy of the higher one, and a dramatic improvement in the matching accuracy cannot be expected. On the other hand, in this method, the correlation method and the feature point method are combined, and when one of them is determined to be “match (OK)”, the final collation determination result is determined to be “match (matching)”. Therefore, the matching accuracy is dramatically improved. The dramatic improvement in the matching accuracy can be seen from the following test results.

〔test〕
(1) Subjects In this test, in order to clarify the difference in performance with a small number of subjects, a large number of persons who intentionally had poor fingerprints and had difficulty in matching were collected. There were 12 subjects. There are eight men and four women, ages from the early 20s to late 30s. Fingerprints: 7 good people, 3 people who may be a little difficult to match in a dry system, 2 people who have difficulty in matching by the feature point method (1 person with severe hand roughening, 1 person with atopic dermatitis) Name). In this test, the percentage of persons who have difficulty in matching is 16%, which is more than 5 times higher than the ratio in the case of randomness. Therefore, it is assumed that the personal recognition rate will be worse by more than 5 times.
(2) Registration The right index finger was defined as one image for each person.

(3) Verification As confirmation data for personal confirmation, 10 images of the right index finger placed at different timings for each person are used (12 persons × 10 images = 120 images in total).
A total of 23 fingers including the right index finger and the right middle finger (11 persons × 2 = 22 fingers) of the other person and the right middle finger (1 finger) of the next finger of the person are used as the confirmation data for accepting another person. In general, different fingers are more similar to others. By using the next finger of the person, the smallness of the sample can be compensated and the reliability of the data received by another person can be further improved.
The collation count is as follows.
For identification verification: 12 persons × Person right index finger 10 images = 120 times collation For identification recognition of other persons: 12 persons × (11 others × 2 kinds of fingers (22 fingers) + Person right middle finger (1 finger)) = 276 Times collation

(4) Test Results Recognition performance is represented by two factors: a false rejection rate (FRR) and a false acceptance rate (FAR). The smaller the FRR and the FAR, the better the recognition performance. There is an expression method called ROC (Receiver Operating Characteristic) curve that can simultaneously express FRR and FAR.
FIG. 7 shows the amplitude suppression correlation method (characteristic I), the feature point method (characteristic II), and the combination method of the amplitude suppression correlation method and the characteristic point method (characteristic III according to the present invention) obtained from the test results described above. FIG.

  In the ROC curve, a point at which FRR and FAR become the same is called EER (Equal Error Rate), and is used as an index of recognition performance. The smaller the EER value, the better the performance. In FIG. 7, the EER (EER1) in the ROC curve I of the amplitude suppression correlation method is about 2.5%, the EER (EER2) in the ROC curve II of the feature point method is about 7%, and the EER (EER) in the ROC curve III of the combination method. EER3) is about 0.42%. As shown in FIG. 8, EER1, EER2, and EER3 are bar graphs, and the collation accuracy is dramatically improved by using a combination of the amplitude suppression correlation method and the feature point method. Was confirmed.

  FIG. 9 is a functional block diagram corresponding to the collation processing (collation method (1)) executed according to the flowchart shown in FIG. The control unit 20 includes, as functional blocks, a first matching unit 20A that performs matching by the amplitude suppression correlation method, a second matching unit 20B that performs matching by the feature point method, a registered fingerprint storage unit 20C, and a matching determination unit 20D. Having.

  The registered fingerprint input from the operation unit 10A is stored in the registered fingerprint storage unit 20C. When the collation fingerprint is input from the operation unit 10A, the collation fingerprint is given to the first collation unit 20A and the second collation unit 20B. The first collation unit 20A reads out the registered fingerprint from the registered fingerprint storage unit 20C, and executes collation between the registered fingerprint and the collation fingerprint from the operation unit 10A by the amplitude suppression correlation method. The second matching unit 20B reads out the same registered fingerprint from the registered fingerprint storage unit 20C, and executes matching between the registered fingerprint and the matching fingerprint from the operation unit 10A by the feature point method. The collation result in the first collation unit 20A and the collation result in the second collation unit 20B are provided to the collation determination unit 20D. The collation determination unit 20D determines that the registered fingerprint and the collated fingerprint match "" (matching) if the collation result by either method is "match (OK)".

[Fingerprint collation: collation method (2) (correlation method priority execution)]
In the matching method (1) according to the flowchart shown in FIG. 5, the matching by the amplitude suppression correlation method and the matching by the feature point method are executed. If the matching result by any one of the methods is “match (OK)”, It was determined that the registered fingerprint and the matching fingerprint "matched". On the other hand, in the matching method (2), the matching by the amplitude suppression correlation method is executed first, and if the matching result by the amplitude suppression correlation method is “match (OK)”, the matching by the feature point method is executed. Instead, it is determined that the registered fingerprint and the verification fingerprint have “matched”.

  As can be seen from the test results described above (FIGS. 7 and 8), since the amplitude suppression correlation method generally has higher matching accuracy than the feature point method, when the registered fingerprint and the matching fingerprint are the same, the amplitude suppression method is used. There is a higher possibility that the correlation method can be completed only once than the feature point method. The total time required for matching in the matching method (1) is the sum of the processing time required for matching by the amplitude suppression correlation method and the processing time required for matching by the feature point method (flow chart in FIG. 5). Also, in the functional block diagram of FIG. 9, the matching by the amplitude suppression correlation method and the matching by the feature point method are shown to be performed in parallel. However, actually, one CPU performs the processing operation. Is the sum of On the other hand, in the matching method (2), if the matching result by the amplitude suppression correlation method matches, the matching by the feature point method is not performed, so that the matching determination result is obtained earlier (the amplitude suppression correlation method uses the feature point method). This is often the case because the matching accuracy is generally higher than in the method). Further, even if the fingerprint cannot be correctly collated by the amplitude suppression correlation method, it can be collated accurately by the collation by the feature point method, so that the collation accuracy is improved.

  FIG. 10 shows a flowchart in the case where the matching is performed by the matching method (2). As shown in this flowchart, in the matching method (2), the matching by the amplitude suppression correlation method is executed in steps 401 to 412 corresponding to steps 301 to 312 shown in FIG. Is determined to be “match (OK)” (YES in step 413), it is immediately determined that the registered fingerprint and the collation fingerprint match (match) (step 414).

  On the other hand, when it is confirmed that the comparison result by the amplitude suppression correlation method is “mismatch (NG)” (NO in step 413), the feature points are obtained by steps 415 to 424 corresponding to steps 313 to 322 shown in FIG. Perform collation by method. When it is confirmed that the matching result by the feature point method is “match (OK)” (YES in step 425), it is determined that the registered fingerprint and the matching fingerprint match (match) (step 414). On the other hand, if the matching result by the feature point method is also “mismatch (NG)” (NO in step 425), it is determined that the registered fingerprint and the matching fingerprint are “mismatch (mismatch)” (step 426). .

  FIG. 11 is a functional block diagram corresponding to the collation processing (collation method (2)) executed according to the flowchart shown in FIG. The control unit 20 includes, as functional blocks, a first matching unit 20A that performs matching by the amplitude suppression correlation method, a second matching unit 20B that performs matching by the feature point method, a registered fingerprint storage unit 20C, and a matching determination unit 20D ′. And

  Further, a changeover switch SW1 is provided on the input line of the matching fingerprint to the first matching unit 20A and the second matching unit 20B, and a changeover switch SW2 is provided on the input line of the registered fingerprint to the first matching unit 20A and the second matching unit 20B. Is provided. The switch SW1 normally has a conduction path between its terminals c1 and a1 turned on, and the changeover switch SW2 normally has a conduction path between its terminals c2 and a2 turned on. In response to a command from the collation determination unit 20D ', the conduction paths are switched to the b1 and b2 sides, respectively.

  The registered fingerprint from the operation unit 10A is stored in the registered fingerprint storage unit 20C. When the collation fingerprint is input from the operation unit 10A, the collation fingerprint is given to the first collation unit 20A via the changeover switch SW1. The first matching unit 20A reads the registered fingerprint from the registered fingerprint storage unit 20C via the changeover switch SW2, compares the read registered fingerprint with the matching fingerprint from the operation unit 10A by the amplitude suppression correlation method, The collation result is sent to collation judging section 20D '. If the collation result from the first collation unit 20A is “match (OK)”, the collation determination unit 20D ′ determines that the registered fingerprint and the collation fingerprint match “match (match)”.

  On the other hand, if the collation result from the first collation unit 20A is “mismatch (NG)”, the collation determination unit 20D ′ sends a switching command to the changeover switches SW1 and SW2, and the terminals c1 and b1 of the changeover switch SW1. Is turned on, and the conduction path between the terminals c2 and b2 of the changeover switch SW2 is turned on. Thereby, the collation fingerprint is provided from the operation unit 10 to the second collation unit 20B via the changeover switch SW1. The second matching unit 20B reads the registered fingerprint from the registered fingerprint storage unit 20C via the changeover switch SW2, executes the matching between the read registered fingerprint and the matching fingerprint from the operation unit 10 by the feature point method, and performs the matching. The result is sent to collation determination section 20D '. If the collation result from the second collation unit 20B is “match (OK)”, the collation determination unit 20D ′ determines that the registered fingerprint and the collation fingerprint match “match (match)”.

[Fingerprint collation: collation method (3) (feature point method priority execution)]
In the matching method (2), the matching by the amplitude suppression correlation method is performed first, and if the matching result by the amplitude suppression correlation method is “match (OK)”, the registration is performed without executing the matching by the feature point method. It is determined that the fingerprint and the verification fingerprint have “matched”. On the other hand, in the matching method (3), the matching by the feature point method is executed first, and if the matching result by the feature point method is “match (OK)”, the matching by the amplitude suppression correlation method is not executed. Then, it is determined that the registered fingerprint and the verification fingerprint have “matched”.

  According to the matching method (3), when the attribute of the pattern to be matched is suitable for the feature point method (for example, the feature point is clear, the disturbance is strong, the pattern is not deformed, or the like), In the case where N matching (a method of matching one matching pattern with N registered patterns) is performed, the amount of calculation for matching is small, so that not only the matching accuracy is improved, but also the matching result is shortened in a short time. Can be obtained at That is, in the amplitude suppression correlation method, since calculation is performed to obtain a correlation value using all pixel data in the registered / matching pattern, it takes time to obtain a matching result. On the other hand, in the feature point method, since the calculation is performed using only the pixel data of the feature point in the registration / collation pattern, the amount of data to be handled is small and the collation result can be obtained in a short time. In particular, in the one-to-N collation, the time difference between the two increases at a faster rate than in the one-to-one collation.

  FIG. 12 shows a flowchart in the case of performing the matching by the matching method (3). As shown in this flowchart, in the collation method (3), collation by the feature point method is executed by steps 801 to 813 corresponding to steps 301, 302, and 313 to 322 shown in FIG. When it is confirmed that the collation result is “match (OK)” (YES in step 814), it is immediately determined that the registered fingerprint and the collation fingerprint match (match) (step 815).

  On the other hand, when it is confirmed that the matching result by the feature point method is “mismatch (NG)” (NO in step 814), the amplitude suppression correlation is performed by steps 816 to 824 corresponding to steps 303 to 312 shown in FIG. Perform a modal match. When it is confirmed that the collation result by the amplitude suppression correlation method is “match (OK)” (YES in step 825), it is determined that the registered fingerprint and the collation fingerprint match (match) (step 814). On the other hand, if the matching result by the amplitude suppression correlation method is also “mismatch (NG)” (NO in step 825), it is determined that the registered fingerprint and the matching fingerprint are “mismatch (mismatch)” (step 826). ).

  FIG. 13 is a functional block diagram corresponding to the collation processing (collation method (3)) executed according to the flowchart shown in FIG. In this functional block diagram, the switch SW1 normally has a conduction path between its terminals c1 and b1 turned on, and the changeover switch SW2 normally has a conduction path between its terminals c2 and b2. The conduction path is turned on, and the conduction path is switched to the a1 and a2 sides in response to a command from the collation determination unit 20D '.

  The registered fingerprint from the operation unit 10A is stored in the registered fingerprint storage unit 20C. When the collation fingerprint is input from the operation unit 10A, the collation fingerprint is given to the second collation unit 20B via the changeover switch SW1. The second matching unit 20B reads the registered fingerprint from the registered fingerprint storage unit 20C via the changeover switch SW2, compares the read registered fingerprint with the matching fingerprint from the operation unit 10A by the feature point method, and performs the matching. The result is sent to collation determination section 20D '. If the collation result from the second collation unit 20A is “match (OK)”, the collation determination unit 20D ′ determines that the registered fingerprint and the collation fingerprint match (match).

  On the other hand, if the collation result from the second collation unit 20B is “mismatch (NG)”, the collation determination unit 20D ′ sends a switching command to the changeover switches SW1 and SW2, and the terminals c1 and a1 of the changeover switch SW1. Is turned on, and the conduction path between the terminals c2 and a2 of the changeover switch SW2 is turned on. Thereby, the collation fingerprint is provided from the operation unit 10 to the first collation unit 20A via the changeover switch SW1. The first collation unit 20A reads the registered fingerprint from the registered fingerprint storage unit 20C via the changeover switch SW2, executes collation between the read registered fingerprint and the collation fingerprint from the operation unit 10 by the amplitude suppression correlation method, The collation result is sent to collation judging section 20D '. If the collation result from the first collation unit 20A is “match (OK)”, the collation determination unit 20D ′ determines that the registered fingerprint and the collation fingerprint match “match (match)”.

[Collation method (4) (designation of collation execution order)]
In the matching method (2) whose functional block diagram is shown in FIG. 11, matching by the amplitude suppression correlation method is always executed first. This is based on the assumption that fingerprint collation is performed. In the case of collating handwritten characters, the collation accuracy using the feature point method is higher than that using the amplitude suppression correlation method. In such a case, it is better to perform the matching by the feature point method first, and the matching determination result can be obtained earlier.

  In other words, if the compatibility between the two matching methods based on the attribute of the pattern to be matched (which is higher in matching accuracy when using the correlation method or the feature point method alone) is known in advance. By designating that the matching is performed first by the method, when the registered pattern and the matching pattern are the same, there is a high possibility that the matching is completed only once, and the matching determination result is obtained quickly. Therefore, in the matching method (4), when a plurality of types of patterns having different pattern attributes are matched by the same pattern matching device, the optimal execution order can be designated in a timely manner.

  FIG. 14 shows a functional block diagram when the collation method (4) is adopted. The pattern determining unit 40 includes, as functional blocks, a first matching unit 40A that performs matching by the amplitude suppression correlation method, a second matching unit 40B that performs matching by the feature point method, a registered pattern storage unit 40C, and a matching determining unit 40D. Of each function block.

  A changeover switch SW1 is provided on the input line of the collation pattern to the first collation unit 20A and the second collation unit 20B, and a changeover switch SW2 is provided on the input line of the registration pattern to the first collation unit 40A and the second collation unit 40B. Has been. The changeover switch SW1 is instructed by the collation determination unit 40D as to whether the common terminal c1 is connected to the terminal a1 (A mode) or to the terminal b1 (B mode). Similarly, the changeover switch SW2 is instructed by the collation determination unit 40D as to whether the common terminal c2 is connected to the terminal a2 (A mode) or to the terminal b2 (B mode).

[When matching by correlation method (amplitude suppression correlation method) is performed first]
When the execution order designating section 50 designates that the collation by the amplitude suppression correlation method is executed first (initial setting), the collation determining section 40D sends a command to the changeover switches SW1 and SW2, and sets both the changeover switches SW1 and SW2 together. The A mode is set as an initial state. That is, the conduction path between the common terminal c1 and the terminal a1 of the switch SW1 is turned on, and the conduction path between the common terminal c2 and the terminal a2 of the switch SW2 is turned on.

  In this case, when a matching pattern is input from the pattern input unit 30, the matching pattern is provided to the first matching unit 40A via the changeover switch SW1. The first matching unit 40A reads the registered pattern from the registered pattern storage unit 40C via the changeover switch SW2, and executes the matching between the read registered pattern and the matching pattern from the pattern input unit 40A by the amplitude suppression correlation method. The collation result is sent to the collation determination unit 40D. If the matching result from the first matching unit 40A is “match (OK)”, the matching determination unit 40D determines that the registered fingerprint and the matching fingerprint match (match).

  On the other hand, if the collation result from the first collation unit 40A is “mismatch (NG)”, the collation determination unit 40D sends a command to the changeover switches SW1 and SW2, and sets both the changeover switches SW1 and SW2 to the B mode. I do. That is, the conduction path between the terminals c1 and b1 of the changeover switch SW1 is turned on, and the conduction path between the terminals c2 and b2 of the changeover switch SW2 is turned on.

  As a result, the matching pattern from the pattern input unit 30 is provided to the second matching unit 40B via the changeover switch SW1. The second matching unit 40B reads the same registered pattern from the registered pattern storage unit 40C via the changeover switch SW2, and executes the matching between the read registered pattern and the matching pattern from the pattern input unit 30 by the feature point method, The collation result is sent to the collation determination unit 40D. If the collation result from the second collation unit 40B is “match (OK)”, the collation determination unit 40D determines that the registered fingerprint and the collation fingerprint match “match (match)”.

[When matching by the feature point method is performed first]
When the execution order designating section 50 designates that the collation by the feature point method is executed first (initial setting), the collation determining section 40D sends a command to the changeover switches SW1 and SW2, and initializes both the changeover switches SW1 and SW2. The state is set to the B mode. That is, the conduction path between the common terminal c1 and the terminal b1 of the switch SW1 is turned on, and the conduction path between the common terminal c2 and the terminal b2 of the switch SW2 is turned on.

  In this case, when a collation pattern is input from the pattern input unit 30, the collation pattern is provided to the second collation unit 40B via the changeover switch SW1. The second matching unit 40B reads the registered pattern from the registered pattern storage unit 40C via the changeover switch SW2, and executes the matching between the read registered pattern and the matching pattern from the pattern input unit 40A by the feature point method. The collation result is sent to the collation determination unit 40D. If the collation result from the second collation unit 40B is “match (OK)”, the collation determination unit 40D determines that the registered fingerprint and the collation fingerprint match “match (match)”.

  On the other hand, if the collation result from the second collation unit 40B is “mismatch (NG)”, the collation determination unit 40D sends a command to the changeover switches SW1 and SW2, and sets both the changeover switches SW1 and SW2 to the A mode. I do. That is, the conduction path between the terminals c1 and a1 of the changeover switch SW1 is turned on, and the conduction path between the terminals c2 and a2 of the changeover switch SW2 is turned on.

  As a result, the matching pattern from the pattern input unit 30 is provided to the first matching unit 40A via the changeover switch SW1. The first matching unit 40A reads the same registered pattern from the registered pattern storage unit 40C via the changeover switch SW2, and executes the matching between the read registered pattern and the matching pattern from the pattern input unit 30 by the amplitude suppression correlation method. , And sends the collation result to the collation determination unit 40D. If the matching result from the first matching unit 40A is “match (OK)”, the matching determination unit 40D determines that the registered fingerprint and the matching fingerprint match (match).

[Collation method (5) (automatic specification of collation execution order)]
In the matching method (5), when a plurality of types of patterns having different pattern attributes are matched by the same pattern matching device, the optimal execution order is automatically designated in a timely manner. By automatically specifying the execution order, if the area of the matching pattern is small or the image quality of the matching pattern is good, matching using the feature point method is performed first. Is performed preferentially, so that both the matching accuracy and the matching speed can be improved.

  FIG. 15 shows a flowchart in the case of performing collation by the collation method (5). As shown in this flowchart, in the collation method (5), first, the area S of the collation fingerprint is calculated (step 901). Then, the calculated collation area S is compared with a predetermined threshold value Sth, and if S ≦ Sth (NO in step 902: small area), the process proceeds to step 802 in FIG. 3) Perform fingerprint matching in (feature point method priority execution).

  If S> Sth (YES in step 902: large area), the image quality value Q of the verification fingerprint is calculated (step 903). The calculated image quality value Q is compared with a predetermined threshold value Qth. If Q ≦ Qth (NO in step 904: good image quality), the process proceeds to step 802 in FIG. (3) Perform fingerprint matching in (feature point method priority execution). If Q> Qth (YES in step 904: poor image quality), the flow advances to step 402 in FIG. 10 to perform fingerprint collation in collation method (2) (correlation method priority execution). Note that the calculation of the matching area S in step 901 and the calculation of the image quality value Q in step 903 are performed in the same manner as the method described with reference to the flowchart of FIG. 3, and a description thereof will be omitted.

  In the collation method (5), first, an image of the collation fingerprint is inspected to confirm whether the collation fingerprint has a sufficient area and whether the image quality is good. In the case where the area of the collation fingerprint is small, the amplitude suppression correlation method considers the overall similarity, so that there is a high possibility that the fingerprint is erroneously recognized as a mismatch. In this case, since the feature point method is more suitable, the matching by the feature point method is executed first. Even if the image quality is good and the feature points are clear, if the distortion is large, there is a high possibility that the amplitude suppression correlation method will mistakenly identify a mismatch. In such a case, the matching based on the feature point method is executed first instead of the amplitude suppression correlation method.

  FIG. 16 shows a functional block diagram when the collation method (5) is adopted. In this functional block diagram, the collation judging unit 40D is provided with an image inspection unit 40D1, an execution order designating unit 40D2, and a collation judging unit 40D3. The image inspection unit 40D1 forms a part of the collation determination unit 40D, inspects the area and image quality of the collation pattern, and sends the inspection result to the execution order designation unit 40D2. Based on the inspection result from the image inspection unit 40D1, the execution order designation unit 40D2 sends a command to the changeover switches SW1 and SW2 when the area of the collation pattern is small or the image quality is good, and sets both the changeover switches SW1 and SW2 to the B mode. And That is, the conduction path between the terminals c1 and b1 of the changeover switch SW1 is turned on, and the conduction path between the terminals c2 and b2 of the changeover switch SW2 is turned on. When the area of the collation pattern is large or the image quality is poor, a command is sent to the changeover switches SW1 and SW2, and both the changeover switches SW1 and SW2 are set to the A mode. That is, the conduction path between the terminals c1 and a1 of the changeover switch SW1 is turned on, and the conduction path between the terminals c2 and a2 of the changeover switch SW2 is turned on.

  In the flowchart shown in FIG. 5, the original image data of the registered fingerprint is read out at the time of collation (step 302), and the read-out original image data of the registered fingerprint is subjected to reduction processing and two-dimensional discrete Fourier transform. Although these processes are performed (steps 303 and 304), these processes may be performed on the original image data of the registered fingerprint at the time of registration, and this may be filed as registered data. By doing so, the collation time is reduced. The same applies to the flowcharts shown in FIGS.

  FIG. 17 shows, in order to shorten the collation time, the original image data of the registered fingerprint is subjected to reduction processing required for collation collation and binarization processing required for two-dimensional discrete Fourier transform and feature point collation. 2 is a flowchart in a case where the thinning processing and the feature point extraction are performed in advance, and are processed into the registration data for the correlation method and the registration data for the feature point method, and the registration is performed. After performing the processing of steps 501 to 506 corresponding to steps 101 to 106 in step (1), the image data of the registered fingerprint selected in step 506 is subjected to reduction processing and two-dimensional discrete Fourier transform (steps 507 and 508). Then, a file is created as registration data for the amplitude suppression correlation method (step 509). Also, the image data of the registered fingerprint selected in step 506 is subjected to binarization processing, thinning processing, and feature point extraction (steps 510, 511, 512), and is converted into a file as registered data for the feature point method. (Step 513).

  Further, in the flowchart shown in FIG. 5 (FIGS. 10 and 12), the two-dimensional discrete Fourier transform is performed in step 310 (410, 819). The inverse Fourier transform may be performed. That is, instead of performing the two-dimensional discrete Fourier transform on the synthesized Fourier image data subjected to the amplitude suppression processing, the two-dimensional discrete inverse Fourier transform may be performed. The two-dimensional discrete Fourier transform and the two-dimensional discrete inverse Fourier transform do not change the matching accuracy quantitatively. The two-dimensional discrete inverse Fourier transform is described in Non-Patent Document 1 above.

  In the flowchart shown in FIG. 5 (FIGS. 10 and 12), the two-dimensional discrete Fourier transform is performed by performing the amplitude suppression process on the synthesized Fourier image data (steps 309 and 310). (409, 410, 821, 822)), the synthesis may be performed after performing amplitude suppression processing on the Fourier image data of the registered fingerprint and the verification fingerprint before synthesis, respectively.

  The suppression rate of the amplitude of the combined Fourier image data at this time is smaller than the case where the amplitude suppression processing is performed after forming the combined Fourier image data. Therefore, when the amplitude suppression processing is performed on the synthesized Fourier image data before performing the amplitude suppression processing, the matching accuracy is improved compared to the method of performing the amplitude suppression processing on the synthesized Fourier image data. Note that, even when the combined Fourier image data is converted to the combined Fourier image data after performing the amplitude suppression processing, the combined Fourier image data may be subjected to a two-dimensional discrete inverse Fourier transform instead of the two-dimensional discrete Fourier transform.

  In the flowchart shown in FIG. 10, the correlation value obtained in step 411 is compared with a predetermined unique threshold value (step 412), and the correlation value is equal to or smaller than the unique threshold value. In this case, the matching result by the amplitude suppression correlation method is determined to be “mismatch (NG)”, and the matching by the feature point method is executed. However, the first threshold value and the second threshold value are determined ( First threshold> second threshold), the matching by the feature point method is executed only when the correlation value is between the first threshold and the second threshold. Yes. In this case, if the correlation value is equal to or less than the second threshold value, it is determined that there is almost no possibility that they match even in the feature point method, and the registered fingerprint and the verification fingerprint are “mismatch” (mismatch). Is determined.

  In the above-described embodiment, the amplitude suppression correlation is used as an example of the correlation method. However, the cross-correlation method (a normal correlation method using the amplitude as it is) or the correlation method based on the U-grid distance (the distance to the amplitude after the Fourier change) is used. Or a "rotation-invariant amplitude suppression correlation method" (amplitude suppression correlation method that corrects a rotational displacement between a registered pattern and a matching pattern) disclosed in Japanese Patent Application Laid-Open No. 10-124667. You may do so.

[Embodiment 2 (Sixth invention): Rotation-invariant amplitude suppression correlation method (with amplitude suppression + phase) + amplitude suppression correlation method + feature point method]
In the second embodiment, to create the registration Fourier image data R _F by performing two-dimensional discrete Fourier transform on the image data R of the registration fingerprint, the collation by performing two-dimensional discrete Fourier transform to the image data I collation fingerprint create a Fourier image data I _F, registration Fourier image data R _F and the collation Fourier image data I to the coordinate system of the _F converted into the polar coordinate system, registration Fourier image data R _P and the collation Fourier image data I is converted into the polar coordinate system _P is collated (coarse collation: first collation) by the amplitude suppression correlation method.
If a matching result indicating a match is not obtained by the first matching, the rotational deviation Δθ between the two is obtained from the position of the correlation peak obtained in the matching process of the first matching, and based on the obtained rotational deviation Δθ. After performing the rotational displacement correction on either the registered fingerprint or the collated fingerprint, the registered fingerprint and the collated fingerprint are collated again by the amplitude suppression correlation method (fine collation: second collation).
If a matching result indicating a match is not obtained by the second matching, the vertical and horizontal deviation amounts ΔX and ΔY of the two are obtained from the position of the correlation peak obtained in the matching process of the second matching. Based on the vertical and horizontal deviation amounts ΔX and ΔY and the rotation deviation amount Δθ obtained from the position of the correlation peak obtained in the first verification collation process, either one of the registered pattern and the verification pattern has a rotational deviation and a vertical / horizontal deviation. After the correction, the registered pattern is compared with the matching pattern by the feature point method (third matching).

Hereinafter, the fingerprint matching operation according to the second embodiment will be specifically described with reference to a flowchart.
[Registration of fingerprint]
In the fifth embodiment, as shown in the flowchart of FIG. 18, the processes of steps 601 and 602 are performed corresponding to steps 101 and 102 of FIG. 2, and the image data R of the registered fingerprint reduced in step 603 is registered. The original image data of the fingerprint is filed in association with the ID number (step 604). Incidentally, the registration Fourier image data R _F by performing two-dimensional discrete Fourier transform on the image data R of the registration fingerprint is filed in the ID number and to correspond as original registration fingerprint image data of the registration Fourier image data R _F You may do so.

[Fingerprint verification]
Fingerprint collation is performed as follows. When the ID number is input (step 701 shown in FIG. 20), the image data R of the registered fingerprint filed corresponding to the ID number is read (step 702: see FIG. 21A). Further, a collation fingerprint is input (step 703), and a reduction process is performed on the collation fingerprint (step 704) to obtain image data I of the collation fingerprint (see FIG. 21B). Then, read the registration Fourier image data R _F by performing two-dimensional discrete Fourier transform for the image data R of the registration fingerprint in step 702 (step 705: see FIG. 21 (c)), the collation fingerprint obtained in step 704 image data I by performing two-dimensional discrete Fourier transform on the collation Fourier image data I _F in (step 706: see FIG. 21 (d)).

The registered Fourier image data _RF and the collated Fourier image data _IF include an amplitude component and a phase component. The registered Fourier image data _RF and the collated Fourier image data _IF are in a Cartesian coordinate system, that is, an (x, y) coordinate system. Performs amplitude suppression processing for the registration Fourier image data R _F and the collation Fourier image data I _F (step 707 and 708), the coordinate system of the amplitude suppression processed registration Fourier image data R _FL and collation Fourier image data I _FL Is converted to polar coordinates (steps 709 and 710), and the registered Fourier image data R _PL and collation Fourier image data I _PL converted to polar coordinates are obtained (see FIGS. 21 (e) and 21 (f)).

Here, the polar coordinate conversion refers to a process of converting a Cartesian coordinate system (x, y) into a polar coordinate system (r, θ). That is, the Cartesian coordinate system (x = rcosθ, y = rsinθ) shown in FIG. 19A is changed to a polar coordinate system (r = (x ² + y ² ) ^1/2 , θ = tan ^-1 (y / x)).

[Rough collation (first collation)]
Then, the registered Fourier image data R _PL obtained as the polar coordinate system obtained in Step 709 and the collation Fourier image data I _PL obtained as the polar coordinate system obtained in Step 710 are collated by the amplitude suppression correlation method (Step 711). FIG. 22 shows the matching process.

In this case, a two-dimensional discrete Fourier transform is performed on the registered Fourier image data R _PL (see FIG. 24 (a)) and the collated Fourier image data I _PL (see FIG. 24 (b)) which are set as a polar coordinate system (step). 711-1, 711-2), registered Fourier image data R _PLF (see FIG. 24 (c)) and collation Fourier image data _IPLF (see FIG. 24 (e)).

Then, by synthesizing the registration Fourier image data R _PLF and collation Fourier image data I _PLF (step 711-3) to obtain synthesized Fourier image data. Then, an amplitude suppression process is performed on the synthesized Fourier image data (step 711-4: see FIG. 24G), and a two-dimensional discrete Fourier transform is performed on the synthesized Fourier image data on which the amplitude suppression process has been performed ( Step 711-5: FIG. 24 (h) and FIG. 21 (g), FIG. 24 (h) = FIG. 21 (g)).

In this example, it was subjected to amplitude suppressing processing to the synthesized Fourier image data with the R _PLF and I _PLF, R _PLF and I _PLF registration Fourier image data by performing amplitude suppression processing for the R _PLF 'and The collation Fourier image data I _PLF ′ may be used (see FIGS. _24D and _24F ), and this R _PLF ′ and _IPLF ′ may be combined. In FIGS. 24 (d), (f), and (g), all the amplitudes are set to 1, that is, only the phase is set by the amplitude suppression.

  Then, the intensity (amplitude) of the correlation component of each pixel in the predetermined correlation component area is scanned from the synthesized Fourier image data subjected to the two-dimensional discrete Fourier transform, and a histogram of the intensity of the correlation component of each pixel is obtained. From the histogram, upper n pixels having a higher correlation component intensity are extracted, and the average of the extracted correlation component intensities of the n pixels is obtained as a correlation value (score) (step 711-6). Here, if the obtained correlation value is higher than a predetermined threshold value (YES in step 711-7), it is determined that the registered fingerprint and the collation fingerprint are "coincidence (OK)" although roughly. . If the obtained correlation value is lower than the predetermined threshold value (NO in step 711-7), it is determined that the registered fingerprint and the collation fingerprint are “mismatch (NG)”.

  If it is determined in the first collation that the registered fingerprint and the collated fingerprint are “mismatch (NG)”, the pixel having the highest intensity of the correlation component is obtained from the synthesized Fourier image data subjected to the two-dimensional discrete Fourier transform in step 711-5. It is determined as a correlation peak, and the rotational deviation amount Δθ between the registered fingerprint and the verification fingerprint, that is, the rotational deviation amount Δθ between the image data R of the registered fingerprint and the image data I of the verification fingerprint is calculated from the position of the correlation peak (step 711-8). ).

  In FIG. 21 (g), P1 appears as a correlation peak. From the positional relationship between the correlation peak P1 and the center of the correlation area, the rotational deviation amount Δθ is determined. That is, the rotation deviation amount Δθ is obtained from the vertical position of the correlation peak P1 in the region in the drawing. In this case, the vertical upper limit position in the region indicates Δθ = + 180 °, and the lower limit position indicates Δθ = −180 °.

[Fine verification (second verification)]
When the first collation is determined to be “mismatch (NG)” and the rotational deviation Δθ between the image data R of the registered fingerprint and the image data I of the collation fingerprint is obtained, the collation fingerprint is obtained based on the obtained rotational deviation Δθ. The rotational deviation of the image data I is corrected, and the registered fingerprint and the collated fingerprint are collated again by the amplitude suppression correlation method (step 712). FIG. 23 shows the matching process.

In this case, it corrects the rotation offset amount Δθ to the image data I collation fingerprint (step 712-1) to obtain image data I _N which is matched image data R and the rotation angle of the registered fingerprint (FIG. 25 (See (a) and (b)). Then, two-dimensional discrete Fourier transform is performed on the image data I _N of the collation fingerprint (step 712-2) to obtain collation Fourier image data I _NF (see FIG. 25 (e)).

Then, the collation Fourier image data I _NF and the registered Fourier image data R _F (see FIG. 25C) obtained in the previous step 705 are combined (step 712-3) to obtain combined Fourier image data. . Then, the synthesized Fourier image data is subjected to amplitude suppression processing (step 712-4), and the synthesized Fourier image data (see FIG. 25 (g)) subjected to the amplitude suppression processing is subjected to a two-dimensional discrete Fourier transform. (Step 712-5).

  Then, the intensity (amplitude) of the correlation component of each pixel in the predetermined correlation component area is scanned from the synthesized Fourier image data (see FIG. 25 (h)) subjected to the two-dimensional discrete Fourier transform, and the correlation of each pixel is scanned. A histogram of the component intensities is obtained, the top n pixels having a high intensity of the correlation component are extracted from the histogram, and the average of the intensities of the extracted correlation components of the n pixels is obtained as a correlation value (score) (step 712-6). .

  Then, the correlation value obtained in step 712-6 is compared with a predetermined threshold value, and if the correlation value is equal to or larger than the threshold value (YES in step 712-7), the registered fingerprint and the verification fingerprint are compared. It is determined that "match (OK)" has occurred. If the correlation value is equal to or smaller than the threshold value (NO in step 712-7), it is determined that the registered fingerprint and the collation fingerprint are “mismatch (NG)”.

  If it is determined in the second verification that the registered fingerprint and the verification fingerprint do not match (NG), the pixel having the highest intensity of the correlation component from the synthesized Fourier image data subjected to the two-dimensional discrete Fourier transform in step 712-5 is determined. It is determined as a correlation peak, and from the position of the correlation peak, the vertical and horizontal deviation amounts ΔX and ΔY between the registered fingerprint and the verification fingerprint, that is, the vertical and horizontal deviations between the image data R of the registered fingerprint and the image data I of the verification fingerprint. The direction deviation amounts ΔX and ΔY are obtained (step 712-8).

In this example, it was subjected to amplitude suppressing processing to the synthesized Fourier image data with the R _F and I _NF, R _F and I _NF amplitude suppression processing performed by the registration Fourier image data R _F against 'and The reference Fourier image data I _NF ′ may be used (see FIGS. _25D and _25F ), and R _F ′ and I _NF ′ may be combined. In FIGS. 25D, 25F, and 25G, all amplitudes are set to 1, that is, only the phase by the amplitude suppression.

  Further, in FIG. 23, the rotational deviation is corrected for the image data I of the collation fingerprint, and the registered fingerprint and the collated fingerprint are collated again. However, the rotational deviation is corrected for the image data R of the registered fingerprint. May be performed to collate the registered fingerprint with the collation fingerprint again.

[Collation by feature point method (third collation)
When it is determined as “mismatch (NG)” in the second collation, the deviation amounts ΔX and ΔY in the vertical and horizontal directions between the image data R of the registered fingerprint and the image data I of the collation fingerprint are obtained. After correcting the rotational deviation and the vertical / horizontal deviation of the image data I of the verification fingerprint based on the direction deviations ΔX and ΔY and the rotational deviation Δθ obtained in the first verification, the registered fingerprint and the verification fingerprint are distinguished by the feature point method. Are collated again (step 713). FIG. 26 shows the matching process.

  In this case, the rough matching and the fine matching by the amplitude suppression correlation method are executed in steps 711 and 712 shown in FIG. 20, and the matching results of the rough matching and the fine matching by the amplitude suppression correlation method are both “mismatch (NG)”. When it is confirmed that the matching is performed, the matching according to the feature point method is executed in steps 713-1 to 713-10 corresponding to steps 313 to 322 shown in FIG.

  In the matching by the feature point method, the rotational deviation of the image data I of the verification fingerprint and the rotational deviation ΔX and ΔY obtained in the second verification and the rotational deviation Δθ obtained in the first verification and The vertical and horizontal displacement correction is performed in step 713-5. The rotation shift and the vertical and horizontal shift correction may be performed not on the image data I of the collation fingerprint but on the image data R of the registered fingerprint. In addition, the rotation shift and the vertical and horizontal shift correction do not always need to be performed after step 713-4. For example, when the process is performed on the image data R of the registered fingerprint, it can be inserted after any of the steps 713-1 to 713-8.

  When it is confirmed that the matching result by the feature point method is “match (OK)” (YES in step 713-10), it is determined that the registered fingerprint and the matching fingerprint match (match) (step 414). . On the other hand, if the matching result by the feature point method is also "mismatch (NG)" (NO in step 713-10), it is determined that the registered fingerprint and the matching fingerprint are "mismatch (mismatch)" (step S713). 425).

[Embodiment 3 (Seventh Invention): Rotation-invariant amplitude suppression correlation method (amplitude suppression + phase sign (±) is added to amplitude) + amplitude suppression correlation method + feature point method]
In the second embodiment, in the coarse matching, the coordinate system of the registered Fourier image data R _FL and the matching Fourier image data I _FL including the amplitude component and the phase component whose amplitude is suppressed is converted to the polar coordinate system ( Steps 709 and 710 in FIG. 20).

On the other hand, in the third embodiment, the sign of each phase is added to the amplitude of the registered Fourier image data R _FL and the matched Fourier image data I _FL that have been subjected to the amplitude suppression processing, and the signed amplitude component (R _FL ', I _FL '). The coordinate system of these R _FL 'and I _FL ' is converted to a polar coordinate system. FIG. 27 shows a flowchart in this case.

As can be seen from comparison with the flowchart shown in FIG. 20, in this embodiment, steps 713 and 714 are added, and the registered Fourier image data R _FL and the collated Fourier image data I _FL subjected to the amplitude suppression processing are respectively Are added to the respective amplitudes, and only the signed amplitude components (R _FL ′, I _FL ′) are extracted. Then, the coordinate system is converted into a polar coordinate system to obtain R _PL ′ and I _PL ′. (Steps 709 and 710).

According to the third embodiment, by respective codes of the phase with respect to the registration Fourier image data R _F and the collation Fourier image data I _F, respectively added to the amplitude, to extract only the signed amplitude components, phase Is less susceptible to errors due to the discontinuity of, and even if there is an error such as a positional deviation between registration and collation, it is possible to perform collation with high accuracy.

[Embodiment 4 (Eighth Invention): Rotation-invariant amplitude suppression correlation method (amplitude suppression + no phase) + amplitude suppression correlation method + feature point method]
In the second embodiment, performs amplitude suppression processing for the registration Fourier image data R _F and the collation Fourier image data I _F, this amplitude suppression processing has been registration Fourier image data R _FL and collation Fourier image data I _FL, the coordinates The system was converted to a polar coordinate system.

On the other hand, in the fourth embodiment, the registered Fourier image data R _F and the collated Fourier image data I _F are phase-removed, and the registered Fourier image data R _F ′ and the collation Fourier from which the phase components have been removed are removed. 'performs amplitude suppression processing for this amplitude suppression processing has been registration Fourier image data R _FL' image data I _F for and collation Fourier image data I _FL ', and converts the coordinate system into a polar coordinate system. However, in this amplitude suppression processing, log processing or √ processing is performed instead of amplitude suppression processing in which all amplitudes are set to 1. FIG. 28 shows a flowchart in this case.

As apparent from comparison with the flow chart of FIG. 20, in this embodiment, by adding a step 715 and 716, registration Fourier image data R _F and extract only the amplitude components with respect to the collation Fourier image data I _F (phase component (Steps 715 and 716), and an amplitude suppression process is performed on the registered Fourier image data R _F ′ and the collation Fourier image data I _F ′ from which the phase components have been removed (steps 707 and 708). The coordinate system of the suppressed Fourier image data R _FL ′ and the collated Fourier image data I _FL ′ is converted into a polar coordinate system to obtain R _PL ′ and I _PL ′ (steps 709 and 710).

According to the fourth embodiment, by performing amplitude suppression processing after removing phase components against registration Fourier image data R _F and the collation Fourier image data I _F, less affected by illumination changes, registration In addition to enabling high-precision matching even when there is a difference in illuminance between the time and the matching, the effect of improving the performance of obtaining a correlation peak by the polar coordinate conversion and the amplitude suppression correlation method is obtained. . That is, the continuity of each pixel is poor in the phase, and the continuity of each pixel is good in the amplitude. Therefore, by removing the phase component, the performance when polar conversion is performed and the correlation peak is obtained by the amplitude suppression correlation method is improved.

  However, at this time, as shown in FIG. 29, P1 and P2 appear as correlation peaks in the correlation component area. This is because the amplitude spectrum is point symmetric. One of the correlation peaks P1 and P2 is determined as a normal correlation peak indicating the rotational deviation Δθ including the rotation direction by performing the mask processing, and the rotational deviation Δθ is determined from the determined correlation peak. . For example, if the correlation peak P1 is determined to be a normal correlation peak, the rotational deviation amount Δθ is obtained from the vertical position of the correlation peak P1 in the region in the figure. In this case, the vertical upper limit position in the region indicates Δθ = + 180 °, and the lower limit position indicates Δθ = −180 °.

  In the first to fourth embodiments described above, the two-dimensional pattern matching such as fingerprint matching has been described. However, the same applies to N-dimensional pattern matching such as a one-dimensional pattern such as a voice and a three-dimensional pattern such as a three-dimensional image. It is possible to apply.

1 is a block diagram of a fingerprint matching device according to an embodiment of the present invention. 5 is a flowchart for explaining a fingerprint registration operation in the fingerprint matching device. 3 is a flowchart illustrating a process of calculating the area and image quality value of a registered fingerprint in the flowchart illustrated in FIG. 2. It is a photograph on a display showing a fingerprint image and a ridge direction block image obtained from the fingerprint image. It is a flowchart for demonstrating the fingerprint collation operation | movement (collation method (1)) in this fingerprint collation apparatus. 5 is a photograph on a display showing an image for explaining a fingerprint matching process in the fingerprint matching device. The ROC curves of the amplitude suppression correlation method (characteristic I), the feature point method (characteristic II), and the combination method of the amplitude suppression correlation method and the characteristic point method (characteristic III: the method according to the present invention) obtained from the test results are shown. FIG. It is the figure which showed the EER of each system obtained from this ROC curve in the bar graph. FIG. 6 is a functional block diagram corresponding to a matching process (a matching method (1)) executed according to the flowchart shown in FIG. 5. It is a flowchart for demonstrating the fingerprint collation operation | movement (collation method (2)) in this fingerprint collation apparatus. FIG. 11 is a functional block diagram corresponding to a matching process (matching method (2)) executed according to the flowchart shown in FIG. 10. It is a flowchart for demonstrating the fingerprint collation operation | movement (collation method (3)) in this fingerprint collation apparatus. FIG. 13 is a functional block diagram corresponding to a matching process (a matching method (3)) executed according to the flowchart shown in FIG. 12. FIG. 9 is a functional block diagram when a collation method (4) is adopted. It is a flowchart for demonstrating the fingerprint collation operation | movement (collation method (5)) in this fingerprint collation apparatus. FIG. 16 is a functional block diagram corresponding to a matching process (matching method (5)) executed according to the flowchart shown in FIG. 15. At the time of registration, the original image data of the registered fingerprint is subjected to the processing required for collation method matching and the required processing for feature point method matching, and processed into correlation method registration data and feature point method registration data. It is a flowchart at the time of making a score. 13 is a flowchart for describing a fingerprint registration operation in the second embodiment. FIG. 3 is a diagram for explaining conversion from a Cartesian coordinate system to a polar coordinate system. 9 is a flowchart for describing a fingerprint matching operation according to the second embodiment. FIG. 13 is a diagram for describing a coarse matching process according to the second embodiment. 21 is a flowchart showing the processing content (first collation) in step 711 shown in FIG. 20. 21 is a flowchart showing the processing content (second collation) in step 712 shown in FIG. 20. 5 is a photograph on a display showing an image for explaining a process after the polar coordinate conversion in a coarse matching (first matching) process. 9 is a photograph on a display showing an image for explaining a fine matching (second matching) process in the second embodiment. 13 is a flowchart for describing a matching (third matching) operation by the feature point method according to the second embodiment. 13 is a flowchart for describing a fingerprint collation operation in the third embodiment. 15 is a flowchart illustrating a fingerprint matching operation according to the fourth embodiment. 15 is a photograph on a display showing an image for explaining a coarse matching (first matching) process in Embodiment 4. 4 is a photograph on a display illustrating an image of a registered fingerprint and a collation fingerprint of a person whose hand is rough and whose crest is disturbed. It is a photograph on a display illustrating an image of a registered fingerprint and a collated fingerprint that cannot be correctly collated by the amplitude suppression correlation method but can be correctly collated by the feature point method.

Explanation of reference numerals

DESCRIPTION OF SYMBOLS 10 ... operation part, 20 ... control part, 10-1 ... numeric keypad, 10-2 ... display (LCD), 10-3 ... fingerprint sensor, 10-31 ... light source, 10-32, prism, 10-33 ... CCD camera , 20-1 ... Control unit, 20-2 ... ROM, 20-3 ... RAM, 20-4 ... Hard disk (HD), 20-5 ... Frame memory (FM), 20-6 ... External connection unit (I / F) ), 20-7: Fourier transform unit (FFT), 10A: operation unit, 20A: first collation unit, 20B: second collation unit, 20C: registered fingerprint storage unit, 20D, 20D ': collation determination unit, SW1, SW2: changeover switch, 30: pattern input section, 40: pattern determination section, 40A: first collation section, 40B: second collation section, 40C: registered fingerprint storage section, 40D: collation determination section, 40D1: image inspection means, 40D2 ... Execution order designation means, 40D3 ... Collation determination means, 50 ... Execution order designation unit.

Claims (8)

In a pattern matching device that matches a registered pattern with a matching pattern,
A first matching unit that performs matching between the registered pattern and the matching pattern based on a correlation value between the patterns;
A second matching unit that performs matching between the registered pattern and the matching pattern based on a predefined feature parameter;
When at least one of the comparison result by the first collation unit and the collation result by the second collation unit is a collation result indicating a match between the registered pattern and the collation pattern, A pattern matching device, comprising: matching determination means for determining that the registered pattern matches the matching pattern. In a pattern matching device that matches a registered pattern with a matching pattern,
A first matching unit that performs matching between the registered pattern and the matching pattern based on a correlation value between the patterns;
A second matching unit that performs matching between the registered pattern and the matching pattern based on a predefined feature parameter;
If the result of the comparison by the first matching means is a result of matching indicating the match between the registered pattern and the matching pattern, the matching by the registered pattern and the matching pattern is performed without executing the matching by the second matching means. And a matching determining means for determining that the pattern matches. In a pattern matching device that matches a registered pattern with a matching pattern,
A first matching unit that performs matching between the registered pattern and the matching pattern based on a correlation value between the patterns;
A second matching unit that performs matching between the registered pattern and the matching pattern based on a predefined feature parameter;
If the result of the comparison by the second matching means is a result of matching indicating the match between the registered pattern and the matching pattern, the matching by the registered pattern and the matching pattern is performed without executing the matching by the first matching means. And a matching determining means for determining that the pattern matches. In a pattern matching device that matches a registered pattern with a matching pattern,
A first matching unit that performs matching between the registered pattern and the matching pattern based on a correlation value between the patterns;
A second matching unit that performs matching between the registered pattern and the matching pattern based on a predefined feature parameter;
Execution order designating means for enabling designation of which one of the collation by the first collation means and the collation by the second collation means to be executed first;
If the collation result by the collation means designated first to be executed by the execution order designation means is a collation result indicating the match between the registered pattern and the collation pattern, the collation by the collation means designated to be executed later is executed. Pattern matching device, comprising: a matching determining unit that determines that the registered pattern matches the matching pattern without performing the matching. In a pattern matching device that matches a registered pattern with a matching pattern,
A first matching unit that performs matching between the registered pattern and the matching pattern based on a correlation value between the patterns;
A second matching unit that performs matching between the registered pattern and the matching pattern based on a predefined feature parameter;
Image inspection means for inspecting the image of the matching pattern,
An execution for designating an execution order of which of the collation by the first collation unit and the collation by the second collation unit is executed first based on the inspection result of the image of the collation pattern by the image inspection unit. Order designation means;
If the collation result by the collation means designated first to be executed by the execution order designation means is a collation result indicating the match between the registered pattern and the collation pattern, the collation by the collation means designated to be executed later is executed. Pattern matching device, comprising: a matching determining unit that determines that the registered pattern matches the matching pattern without performing the matching. Registered Fourier pattern data creating means for creating N-dimensional discrete Fourier transform by applying N-dimensional discrete Fourier transform to N (N ≧ 1) -dimensional pattern data of the registered pattern;
Matching Fourier pattern data creating means for creating N-dimensional pattern data by performing N-dimensional discrete Fourier transform on N (N ≧ 1) -dimensional pattern data of the matching pattern;
First amplitude suppression means for performing amplitude suppression processing on the registered Fourier N-dimensional pattern data;
Second amplitude suppression means for performing amplitude suppression processing on the matching Fourier N-dimensional pattern data;
First polar coordinate system conversion means for converting the coordinate system of the registered Fourier N-dimensional pattern data subjected to the amplitude suppression processing by the first amplitude suppression means into a polar coordinate system;
Second polar coordinate system converting means for converting the coordinate system of the collated Fourier N-dimensional pattern data subjected to the amplitude suppressing processing by the second amplitude suppressing means into a polar coordinate system;
The registered Fourier N-dimensional pattern data converted to the polar coordinate system by the first polar coordinate system converting means and the collated Fourier N-dimensional pattern data converted to the polar coordinate system by the second polar coordinate converting means are subjected to an amplitude suppression correlation method. First matching means for matching;
A rotational displacement amount measuring means for obtaining a rotational displacement amount between the two from a position of a correlation peak obtained in a collation process by the first collation means;
A rotational deviation correcting unit that performs a rotational deviation correction on one of the registered pattern and the verification pattern based on the rotational deviation amount obtained by the rotational deviation amount measuring unit;
A second matching unit that performs the rotation shift correction by the rotation shift correction unit, and then matches the registered pattern and the matching pattern by an amplitude suppression correlation method;
Vertical and horizontal shift amount measuring means for obtaining the vertical and horizontal shift amounts of the correlation peak from the position of the correlation peak obtained in the matching process by the second matching means;
The rotational deviation and the rotational deviation in any one of the registered pattern and the verification pattern based on the rotational deviation amount obtained by the rotational deviation amount measurement unit and the vertical and horizontal deviation amounts obtained by the vertical and horizontal deviation amount measurement unit. Rotation / vertical / horizontal deviation correction means for performing vertical / horizontal deviation correction;
Third correction in which the rotation pattern and the vertical and horizontal deviations are corrected by the rotation / vertical / horizontal deviation correction means, and then the registered pattern and the verification pattern are verified based on a predefined characteristic parameter. Means,
When at least one of the comparison results of the first, second, and third matching units is a matching result indicating that the registered pattern matches the matching pattern. A pattern matching device comprising: matching means for determining that the registered pattern matches the matching pattern. Registered Fourier pattern data creating means for creating N-dimensional discrete Fourier transform by applying N-dimensional discrete Fourier transform to N (N ≧ 1) -dimensional pattern data of the registered pattern;
Matching Fourier pattern data creating means for creating N-dimensional pattern data by performing N-dimensional discrete Fourier transform on N (N ≧ 1) -dimensional pattern data of the matching pattern;
First amplitude suppression means for performing amplitude suppression processing on the registered Fourier N-dimensional pattern data;
Second amplitude suppression means for performing amplitude suppression processing on the matching Fourier N-dimensional pattern data;
The sign of the phase is added to the amplitude of the registered Fourier N-dimensional pattern data subjected to the amplitude suppression processing by the first amplitude suppression means, only the signed amplitude component is extracted, and the coordinate system is changed to the polar coordinate system. First polar coordinate system converting means for converting;
The sign of the phase is added to the amplitude of the collated Fourier N-dimensional pattern data subjected to the amplitude suppression processing by the second amplitude suppression means, only the signed amplitude component is extracted, and the coordinate system is changed to the polar coordinate system. Second polar coordinate system converting means for converting;
The registered Fourier N-dimensional pattern data converted to the polar coordinate system by the first polar coordinate system conversion means and the collation Fourier N-dimensional pattern data converted to the polar coordinate system by the second polar coordinate system conversion means are subjected to an amplitude suppression correlation method. First matching means for matching;
A rotational displacement amount measuring means for obtaining a rotational displacement amount between the two from a position of a correlation peak obtained in a collation process by the first collation means;
A rotational deviation correcting unit that performs a rotational deviation correction on one of the registered pattern and the verification pattern based on the rotational deviation amount obtained by the rotational deviation amount measuring unit;
A second matching unit that performs the rotation shift correction by the rotation shift correction unit, and then matches the registered pattern and the matching pattern by an amplitude suppression correlation method;
Vertical and horizontal shift amount measuring means for obtaining the vertical and horizontal shift amounts of the correlation peak from the position of the correlation peak obtained in the matching process by the second matching means;
The rotational deviation and the rotational deviation in any one of the registered pattern and the verification pattern based on the rotational deviation amount obtained by the rotational deviation amount measurement unit and the vertical and horizontal deviation amounts obtained by the vertical and horizontal deviation amount measurement unit. Rotation / vertical / horizontal deviation correction means for performing vertical / horizontal deviation correction;
Third correction in which the rotation pattern and the vertical and horizontal deviations are corrected by the rotation / vertical / horizontal deviation correction means, and then the registered pattern and the verification pattern are verified based on a predefined characteristic parameter. Means,
A case where at least one of the matching results among the matching results by the first matching unit, the second matching unit, and the third matching unit is a matching result indicating a match between the registered pattern and the matching pattern A pattern matching device, comprising: matching means for determining that the registered pattern matches the matching pattern. Registered Fourier pattern data creating means for creating N-dimensional discrete Fourier transform by applying N-dimensional discrete Fourier transform to N (N ≧ 1) -dimensional pattern data of the registered pattern;
Matching Fourier pattern data creating means for creating N-dimensional pattern data by performing N-dimensional discrete Fourier transform on N (N ≧ 1) -dimensional pattern data of the matching pattern;
First amplitude suppression means for performing an amplitude suppression process on the registered Fourier N-dimensional pattern data after removing a phase component of the registered Fourier N-dimensional pattern data;
A second amplitude suppression unit that removes a phase component of the matching Fourier N-dimensional pattern data, and performs amplitude suppression processing on the matching Fourier N-dimensional pattern data;
First polar coordinate system conversion means for converting the coordinate system of the registered Fourier N-dimensional pattern data subjected to the amplitude suppression processing by the first amplitude suppression means into a polar coordinate system;
Second polar coordinate system converting means for converting the coordinate system of the collated Fourier N-dimensional pattern data subjected to the amplitude suppressing processing by the second amplitude suppressing means into a polar coordinate system;
The registered Fourier N-dimensional pattern data converted to the polar coordinate system by the first polar coordinate system conversion means and the collation Fourier N-dimensional pattern data converted to the polar coordinate system by the second polar coordinate system conversion means are subjected to an amplitude suppression correlation method. First matching means for matching;
A rotational displacement amount measuring means for obtaining a rotational displacement amount between the two from a position of a correlation peak obtained in a collation process by the first collation means;
A rotational deviation correcting unit that performs a rotational deviation correction on one of the registered pattern and the verification pattern based on the rotational deviation amount obtained by the rotational deviation amount measuring unit;
A second matching unit that performs the rotation shift correction by the rotation shift correction unit, and then matches the registered pattern and the matching pattern by an amplitude suppression correlation method;
Vertical and horizontal shift amount measuring means for obtaining the vertical and horizontal shift amounts of the correlation peak from the position of the correlation peak obtained in the matching process by the second matching means;
The rotational deviation and the rotational deviation in any one of the registered pattern and the verification pattern based on the rotational deviation amount obtained by the rotational deviation amount measurement unit and the vertical and horizontal deviation amounts obtained by the vertical and horizontal deviation amount measurement unit. Rotation / vertical / horizontal deviation correction means for performing vertical / horizontal deviation correction;
Third correction in which the rotation pattern and the vertical and horizontal deviations are corrected by the rotation / vertical / horizontal deviation correction means, and then the registered pattern and the verification pattern are verified based on a predefined characteristic parameter. Means,
When at least one of the comparison results of the first, second, and third matching units is a matching result indicating that the registered pattern matches the matching pattern. A pattern matching device comprising: matching means for determining that the registered pattern matches the matching pattern.

Images