WO2002073515A1

WO2002073515A1 - Biometric verification suitable for portable devices

Info

Publication number: WO2002073515A1
Application number: PCT/CA2002/000344
Authority: WO
Inventors: Alexei Stoianov; Colin Soutar
Original assignee: Bioscrypt Inc.
Priority date: 2001-03-14
Filing date: 2002-03-14
Publication date: 2002-09-19
Also published as: CA2340813A1

Abstract

This invention describes a method for fingerprint verification suitable for portable devices with low data memory requirements and low power consumption. The enrollment portion of the method includes sorting information contained in a fingerprint image into alignment information and verification information, to create an enrollment template. The verification portion of the method includes extracting alignment information from a candidate image, utilizing the alignment information to align the candidate image with the enrolled one to obtain the adjusted 'tile' locations for the candidate image. The next step includes extracting the 'tiles' of the same size and orientation as the template 'tiles' from the candidate image and matching each extracted 'tile' with the corresponding template 'tile'. A set of matching parameters is obtained for each matched 'tile'. The final match/non-match decision may be based on a multivariable decision boundaries approach.

Description

Biometric Verification Suitable For Portable Devices

BACKGROUND OF THE INVENTION

This invention relates to biometric verification, i.e. determining the match between a candidate sampled image of a user with a previously enrolled version of a user's sample image, as stored within an enrollment template. The invention also relates to biometric enrollment.

In the last decade, the art of fingerprint authentication has become so advanced that it can be considered for using in conjunction with various portable devices, such as cell phones, pagers, key fobs, or even smart cards. Such a device would have an embedded fingerprint sensor and use its internal processor both to store a fingerprint template and to effect the fingerprint authentication. Since this device would be in exclusive possession of its owner, it will provide a much higher level of security and maintain privacy at the same time.

However, unlike a common PC, a portable device has limited capabilities in terms of memory, processing speed, and power consumption. This makes the requirements for the fingerprint sensor and for the fingerprint verification algorithm much tougher than usual. For example, a fingerprint device used for an access control and/or connected to a common PC can use a high-resolution ( ~ 500 dpi) optical fingerprint sensor with a large power consumption and an algorithm requiring 2 MB of RAM, while a portable device may have only 16 KB of internal data' memory. During the past few years, new types of fingerprint sensors with low power consumption have appeared on the market, such as silicon sensors, thermal sensors, pressure sensors, etc. They can be used with a battery-operated device. However, the fingerprint image quality is usually lower compared to a "traditional" optical sensor. Combined with the limited processor capabilities, this will lead to a significant deterioration of the ability of the portable device to properly verify fingerprints. Moreover, most fingerprint verification algorithms known in the art simply will not work in the portable processor environment.

For example, fingerprint verification algorithms based on minutiae extraction usually require a high resolution (~ 500 dpi) image. Thus, a 300x300 gray scale level image (i.e. 300 pixel size in both x- and y-directions) will take 90 KB of data memory, which exceeds by far the capabilities of many portable devices. Other well performing algorithms based on various pattern recognition techniques also require a lot of memory to simultaneously store a number of arrays of the same size as the fingerprint image. Even if the latter is downsampled to, say, 200 dpi, and takes only - 10 - 15 KB of data memory, the simultaneous storage of three to five auxiliary arrays is typically required for the template and candidate fingerprint images, leading to the same ~ 100 KB of total memory.

Some methods known in the art try to binarize on-the-fly the incoming image to reduce the memory size required to store the image. However, this usually leads to a significant deterioration in performance.

Accordingly, there remains a need for an approach to biometric verification which is more suitable for portable devices. SUMMARY OF INVENTION

The present invention overcomes these difficulties by providing an algorithm that does a comparison of the candidate fingerprint (or other biometric) image with the enrolled template in a portable environment. The comparison is done in smaller parts of a fingerprint image (called "tiles"), preferably in a sequential order, and each step of the algorithm is within the processor capabilities. This method provides a high level of accuracy with low memory requirements and low power consumption. Thus, the method is suitable for portable devices.

The enrollment portion of the method includes sorting information contained in a fingerprint image into alignment information and verification information, to create an enrollment template. The verification information comprises a set of template "tiles" extracted from different parts of the image in pre-determined locations, the size of said "tiles" and of the alignment information may be considerably smaller than the size of the entire original image. An optimization procedure of generating a composite filter comprising several sampled versions of each tile, may be applied to create an enrollment template. The verification portion of the method includes extracting alignment information from a candidate image, utilizing the alignment information to align the candidate image with the enrolled one to obtain the adjusted "tile" locations for the candidate image. The next step includes extracting the "tiles" of the same size and orientation as the template "tiles" from the candidate image and matching each extracted "tile" with the corresponding template "tile". A set of matching parameters is obtained for each matched "tile". The final match/non-match decision may be based on a multivariable decision boundaries approach. With the alignment information and each "tile" processed in a sequential manner, the memory and, accordingly, the power consumption requirements for the hardware are significantly reduced. This, and/or the fact that the tiles and alignment information are considerably smaller than the size of the original image, allows the invention to be suitable for use with a portable device.

Accordingly, the present invention provides a method for biometric verification suitable for a portable device comprising: obtaining at least one sampled biometric image; extracting a plurality of sub-portions of said at least one sampled biometric image, each sub-portion being extracted from a pre-determined location, each said sub-portion comprising a tile; matching extracted tiles with corresponding template tiles.

The present invention further provides a portable biometric verification device, comprising; a biometric reader; memory storing a plurality of template tiles each associated with a location; a processor for: receiving at least one sampled biometric image from said biometric reader; extracting a plurality of sub-portions of said at least one sampled biometric image, each sub-portion being extracted from one said location, each said sub- portion comprising a tile; retrieving said template tiles from said memory and matching extracted tiles with corresponding template tiles.

The present invention further provides a method for biometric enrollment suitable for use with a portable device comprising: obtaining at least one sampled biometric image; extracting a plurality of sub-portions of said at least one sampled biometric image, each sub- portion being extracted from a pre-determined location, each said sub-portion comprising a tile; for each pre-determined location, storing a template tile in association with said each pre-determined location, each template tile derived from a composite of an extracted tile from said pre-determined location for each one of said at least one sampled biometric image.

BRIEF DESCRIPTION OF THE DRAWINGS

In the figures which illustrate an example embodiment of the invention, figure 1 A is a flow diagram illustrating an enrollment procedure; figure IB is a flow diagram illustrating a verification procedure; figure 2 illustrates the effect of fingerprint distortion on prior techniques and on techniques according to this invention, figure 3 illustrates the extraction of tiles from a fingerprint image, and figure 4 is a block diagram of a device adapted for use with this invention.

DETAILED DESCRIPTION

In this method for fingerprint verification suitable for portable devices, a matching algorithm, such as correlation, is performed in several small blocks (i.e. of 32x64 size) called "tiles", of the input images (see figure 1); They are extracted from the sampled images (i.e. of 128x128 size) in pre-defined positions. A set of parameters obtained as a result of correlation operations of each "tile" with a corresponding template "tile", such as the correlation ratio or the area at 20% of a maximum of the correlation peak, is used for the verification. Here the correlation ratio is defined as a height of a correlation peak divided by the total energy of the correlation function, and the area at 20% is the number of pixels of the correlation function that lie below 20% of the correlation peak. This set of correlations of smaller blocks is more tolerant to fingerprint deformations and rotations, including non- uniform ones, which yields a lower false rejection rate (FRR). Moreover, as a high cross- correlation score is less likely to occur in all the blocks simultaneously, this method also reduces the corresponding false acceptance rate (FAR).

Fig. 2 illustrates the advantages of this approach. A 128x128 fingerprint image, A, is stretched by 2% in X direction and by 3% in Y direction, which is typical for a sampled fingerprint images. The correlation ratio drops from 0.5415 to 0.0268 (i.e. 20 times) and the area at 20% of the maximum of the correlation peak increases from 1 to 12. On the other hand, the correlation ratio for the 64x32 tiles drops only from 0.4463 to 0.24 and the area at 20% increases from 1 to 2. The same is true for rotations and for irregular contrast changes. What is even more important is that the correlation of tiles is quite tolerant to the non- uniform, irregular distortions that cannot be compensated for the entire 128x128 image by any adaptive method. Moreover, an adaptive verification method (i.e. rotating or re-scaling a candidate image to get the best possible correlation) is more suitable to the tiles than to the entire image since the distortions in each tile are more uniform. Thus, the correlation in tiles reduces one of the major drawbacks of previous algorithms - a high sensitivity to the fingerprint distortions.

In a preferred version of the algorithm, two tiles of 32x64 and two tiles of 64x32 are extracted on enrollment in the pre-determined positions, as shown in Fig. 3. Note that the shape of the tiles can provide a degree of tolerance to relative translation between the candidate image and the image represented by the template. For example, the tiles may be chosen, as in this case, to be rectangular and aligned with the x and y axes, respectively, of the images. This provides a higher degree of tolerance to relative translations than a simple square tile shape would allow. Moreover, a rectangular tile is also more tolerant to lateral deformations than a square tile of the same total area. The tile centers are: (40,38), (75,28), (90,58), and (56,77). Filters are created separately for each tile. In general, the filter is defined as a Fourier transform of a tile. In some cases, only a phase-only part of the Fourier transform may be retained as a filter, or a so-called optimal filter may be created, as taught in U.S. patent 6,219,794. The filter may be also quantized to reduce the memory consumption.

In order to be able to extract the tiles on verification at the same or close positions, another block of information, for example, the central 64x128 part of the image, is extracted to generate its own filter (Fig. 3). This filter (representing the central part) is used to align an image to be verified with the image enrolled and to extract the tiles at the same positions.

Thus, the overall user template comprises 5 filters corresponding to the larger 64x128 block and 4 smaller (32x64 or 64x32) tiles. The total size of the user template is the same as it would be for a filter representing the entire 128x128 image.

To further reduce the memory requirements, the central 64x128 block may be downsampled to 32x64 both on enrollment and verification. This does not significantly affect the overall accuracy of the algorithm.

On verification, the first step includes extracting a central 64x128 block from the sampled candidate image and correlating this block with the related 64x128 filter. The location of the correlation peak indicates a relative shift between the two images (i.e. between the candidate and template images). Then the tiles are extracted from the candidate image in accordance with this shift, and each tile is correlated with the corresponding filter. Note that even before this, the image may be verified by the ratio in the central block, if this ratio exceeds a high threshold. About 90% of all auto-correlations are expected to be verified at the first step. Only if the ratio is lower than this high threshold, the tiles will be extracted. Unlike the central block, the correlation peaks in the tiles should be located exactly at the geometric centers (i.e. at (32,16) or (16,32) ). The shift by a few pixels indicates the presence of distortions, and a big shift usually occurs when there is no correlation. The values obtained from the tiles, such as ratios, areas at 20% of the maximum, and peak positions, are combined are combined to create an overall match result. As examples, this combination may be achieved by: the sequential update of a cumulative match result, which may trigger an early successful exit from the algorithm if a partial combination of values achieves required targets; the combination of all of the values from each of the tile matches as input to an arithmetic calculation such as the average; or by the combination of all of the values from each of the tile matches as input to a multivariate decision boundary. The method by which multivariate decision boundaries can be chosen is taught in "High-speed fingerprint verification using an optical correlator" by A. Stoianov, C. Soutar, and A. Graham, Optical Engineering, v. 38, No. 1, pp. 99 -107, 1999. This second step of verification together with the first step should allow more than 99% of all auto-correlations to be verified. If the image is still not verified, the third step of verification starts up. This step includes the adaptive verification for each tile. More specifically, the tile is rotated by an angle (for example, from -12° to +12° with a 3° increment) to obtain the best correlation, and so on for all the tiles. Scale variations may also be applied. After the adaptive verification is completed, the new values go through a set of multivariate decision boundaries (they may differ from the set used at the second step). In some cases, it may be applicable to form a composite filter from, for example, two or more images on enrollment:

- the images are averaged in the image domain;

- the images are co-aligned by the correlation of the central 64x128 blocks, then the central composite 64x128 block is created;

- the four tiles are extracted from each image according to their relative positions found during the alignment of the 64x128 blocks; after the tiles are extracted, their relative shift is checked again to check for local distortions, and if a shift exists for any tile, then a new tile is extracted from the original image at that shift, to compensate for the local distortion;

- after all four composite tiles and the central composite block are created, the Fast Fourier Transforms (FFTs) are performed. Then the magnitudes of the FFTs are

modified using D(u,v) and α terms according to U.S. Patent 6,219,794. The value is

different for the central block and the tiles; - all full-complex filters may be compressed, or quantized, using an operating curve with eight points as is taught in "Optimal trade-off filter for the correlation of fingerprints" by D. Roberge, C. Soutar, and B. N. K. Nijaya Kumar, Optical Engineering, v. 38, No. 1, pp. 108 - 113, 1999 (on verification, a dc block will be also applied);

- only half of each filter is taken, and these 5 halves are stored into one template of the same size as if the correlation with 128x128 images were performed.

Another feature of the algorithm includes dealing with images which are shifted too much (on verification) from the image enrolled. In this case some tiles lie (partially or even completely) outside the image area. If the area of overlapping of two tiles (i.e. being verified and enrolled) is less than about 20% of the total number of the pixels in the tile, this tile is excluded from the verification. Alternatively, if the total energy of the extracted tile is less than about 20% of the maximal total energy among all 4 tiles, this tile will be also excluded. It is required that at least 3 tiles should remain for the correlation analysis, otherwise, the image is rejected (it occurs usually in some cross-correlation cases).

There are many ways of combining the results from each of the matches of the tiles, such as neural networks or fuzzy logic. One particular and simple solution uses the mean arithmetic ratio and a mean geometric area (at 20%) calculated over all the values obtained from 4 tiles.

Also, when using the adaptive verification described above, it is possible to use some kind of a tree scheme to look for the best angle, which would reduce the processing time.

After the best angles for all 4 tiles have been found, the mean arithmetic ratio and the mean geometric area are calculated again.

It is also possible to set thresholds for each tile separately and to require a positive verification in at least 3 tiles from 4. This method, in general, works worse than the method using the mean values, but may provide an additional redundancy.

A second embodiment of the invention includes using another alignment technique, which is similar to that described in U.S. Patent 6,219,794. In particular, instead of storing a filter for the central part of the image into a template, as shown in Fig. 3, a, a set of pixel values at selected points of the image may be stored. In the preferred embodiment, 500 points are selected using a pseudo-random lookup table.

On verification, the 500 points are extracted from the same location of the candidate image, and the inner product of these points with the stored ones is computed. Then the lookup table is sequentially shifted both in x- and in y-direction to extract another set of 500 points from the candidate image, and the inner product is computed again. When all predetermined shifts (for example, 24 in x-direction and 48 in y-direction) are performed, the shift location with the maximum inner product is returned as a relative offset between the candidate and the template images. This offset is used as a reference point to extract 4 tiles from the candidate image, as described for the first embodiment. To speed up the processing and to reduce the memory requirements, the pixel values of the 500 points may be binarized, and a digital XOR operation may be used to compute the inner product. Also, the distortions, such as rotations and deformations, may be applied to the pseudo-random lookup table in addition to the shifts.

Thus, the aligning of extracted tiles with corresponding template tiles may be accomplished by extracting an image intensity at a plurality of pre-determined locations in the image and comparing these with corresponding stored intensities.

It is obvious that the same technique may be used in the next steps of the algorithm, i.e. to obtain matching scores for the tiles.

Capture of the sampled fingerprint images can be accomplished using many well known devices such as capacitive or optical fingerprint scanners, and using contact or non-contact means. It will be obvious to those skilled in the art that the invention does not utilize features or minutiae that are specific to fingerprint images, and is therefore also applicable to other biometric types, such as face, iris, retina, etc.

Other embodiments will be known to those skilled in the art and as such the invention is defined by the claims.

Claims

We claim:

1. A method for biometric verification suitable for a portable device comprising: obtaining at least one sampled biometric image; extracting a plurality of sub-portions of said at least one sampled biometric image, each sub-portion being extracted from a pre-determined location, each said sub-portion comprising a tile; matching extracted tiles with corresponding template tiles.

2. The method of claim 1 wherein said matching comprises sequentially matching extracted tiles with corresponding template tiles.

3. The method of claim 2 wherein said matching comprises aligning said extracted tiles with corresponding template tiles.

4. The method of claim 3 wherein said aligning comprises extracting a verification alignment tile from said at least one biometric image and matching said verification alignment tile with a template alignment tile.

5. The method of claim 4 wherein said verification alignment tile is centered with respect to said at least one biometric image.

6. The method of claim 2 wherein each sequential match updates a cumulative match result.

7. The method of claim 6 wherein if said cumulative match result exceeds a confidence threshold, ceasing said sequentially matching even where not all extracted tiles have been matched.

8. The method of claim 6 wherein said cumulative match result comprises an average of a result of each sequential match.

9. The method of claim 6 wherein the set of results from each said sequential match is input to a multivariate decision policy.

10. The method of claim 1 wherein said matching extracted tiles with corresponding template tiles comprises adjusting at least one extracted tile to account for distortion.

11. The method of claim 10 wherein said adjusting comprises rotating said at least one extracted tile.

12. The method of claim 10 wherein said adjusting comprises scaling said at least one extracted tile.

13. The method of claim 2 wherein each of said extracted tiles is rectangular.

14. The method of claim 13 wherein a length dimension of some of said extracted rectangular tiles is perpendicular to a length dimension of others of said extracted tiles.

15. The method of any of claims 1 to 14 wherein each said matching comprises a correlation operation.

16. The method of any of claims 1 to 15 wherein each template tile is associated with one said pre-determined location.

17. The method of any of claims 1 to 16 wherein said biometric image is a finge rint.

18. The method of any of claims 1 to 17 wherein each said template tile comprises a Fourier Transform of a tile extracted from at least one enrollment biometric image.

19. A portable biometric verification device, comprising; a biometric reader; memory storing a plurality of template tiles each associated with a location; a processor for: receiving at least one sampled biometric image from said biometric reader; extracting a plurality of sub-portions of said at least one sampled biometric image, each sub-portion being extracted from one said location, each said sub- portion comprising a tile; retrieving said template tiles from said memory and matching extracted tiles with corresponding template tiles.

20. A method for biometric enrollment suitable for use with a portable device comprising: obtaining at least one sampled biometric image; extracting a plurality of sub-portions of said at least one sampled biometric image, each sub-portion being extracted from-a pre-determined location, each said sub-portion comprising a tile; for each pre-determined location, storing a template tile in association with said each pre-determined location, each template tile derived from a composite of an extracted tile from said pre-determined location for each one of said at least one sampled biometric image.

21. The method of claim 20 wherein each said template tile is stored in a compressed form.

22. The method of claim 20 wherein each template tile is derived from a Fourier Transform of said composite.