EP4751244A1 - System and method for indexing a human fingerprint image - Google Patents

Abstract

The invention relates to a method of indexing a human fingerprint image. The method of the invention comprises the steps of defining a plurality of classes, each class corresponding to a local pattern of a fingerprint; building a ground truth by allowing a machine learning model to determine the inference probability to each class for acquired fingerprint images; inputting an original fingerprint image; assigning by the machine learning model to the original fingerprint image a multi-dimensional vector according to the inference probability determined for each class based on the ground-truth; generating an indexing key from the multi-dimensional vector of the original fingerprint; and indexing the fingerprint indexing key, the indexing comprising multi-stage checks including both exclusive and continuous classification checks.

Description

SYSTEM AND METHOD FOR INDEXING A HUMAN FINGERPRINT IMAGE

FIELD OF USE

The disclosure relates to a method of indexing a human fingerprint image , and to an automatic fingerprint identi fication system wherein the fingerprint images are indexed according to this method .

BACKGROUND

Fingerprints are widely used biometric traits for personal identi fication due to their uniqueness , persistence , and user acceptability . Many Automatic Fingerprint Identi fication Systems (AFIS ) are deployed worldwide since they emerged in 1980s . The si ze of fingerprint databases involved in AFIS is continuously growing .

Representative examples of civilian applications include the OBIM ( Of fice of Biometric Identity Management ) program, which holds more than 200 million unique identities and processes more than 300 , 000 biometric transactions per day, and the India' s Aadhar proj ect , which is now the largest biometrics deployment in the world with an enrolment that already exceeds 1 billion tenprints of supposedly unique individuals .

The identi fication of a person requires a comparison of one or more fingerprints with all the fingerprints in a such a massive database . Both high identi fication accuracy and high search ef ficiency are critical . The NIST (National Institute of Standards and Technology) FpVTE2012 ( Fingerprint Vendor Technology Evaluation 2012 ) is an evaluation of large 1-to-many fingerprint identification algorithms using enrolment sample sizes in the multimillions. It has different testing scenarios by using one, two, four, eight, or ten fingerprints for searching. From their report, it can be seen that, for one finger search, the most accurate submission D (from Cogent) achieved a FNIR (False Negative Identification Rate) of 1.97% for the left index finger and 1.9% for the right index finger searched against an enrolment set of 100,000 subjects. But for two fingers search, the best FNIR is 0.27% against an enrolment set of 1.6 million subjects. For ten fingers search, the best FNIR is 0.09% against an enrolment set of 3 million subjects. That's why most AFIS systems use two or more fingerprints for identification even though more storage and computing resources are required. From the report of the NIST MINEX III, an evaluation of large 1-to- 1 fingerprint verification, it can be seen that the most accurate submissions have a matching time of about 1 or 2 ms for single comparison. The time consumption is challenging when attempting to retrieve a huge database by such kind of 1-to-l matching. However, another observation in the FPVTE2012 report is that, in most accurate submissions, ten-finger searches are faster than two- finger searches since fingerprint indexing are deployed.

Fingerprint indexing is the fast filtering stage for large scale fingerprint identification system, cutting off a large portion of candidates quickly and maintaining a small portion for further accurate-but-slow checks to produce the final decision. However, an efficient fingerprint indexing algorithm should ensure that the true mate of the query is indeed present in the candidate list. The penetration rate (PR) , missing rate (MR) and indexing speed are generally used to measure the indexing performance . The penetration rate is the ratio of the passing list over the whole candidate list , while the missing rate is the ratio of the missing pairs in the indexing stage . An optimal fingerprint indexing aims to minimi ze both PR ( for speed or resources ) and MR ( for accuracy) with an extremely fast speed indexing .

The traditional way to shorten the candidate list is by implementing a fingerprint classi fication and, more speci fically, an exclusive classi fication . This technique splits the database into a fixed number of classes , such as Arch, Tented Arch, Left Loop, Right Loop, and Whorl , during the identi fication phase , and the searched fingerprint is compared only to fingerprints belonging to the same class . This approach is generally fast but cannot greatly reduce the number of candidates due to the small number of classes and uneven distribution among the classes . Some existing AFIS still use exclusive classi fication for tenprint-to-tenprint matching since the sequential ten- finger patterns bring far more discrimination and can shorten the candidate list to 12 % (penetration rate ) of whole reference database with low missing rate (below 0 . 1 % ) . Unlike the general fingerprint classi fication, fingerprint indexing ( also known as continuous classi fication) represents fingerprints as some stable and generally fixed length features for indexing .

Finally, in the past two decades , many indexing techniques have been proposed . They can be roughly grouped into two categories according to the features that have been used : either "global features" -based or " local features" - based . The global features define the global pattern of ridges, and typical approaches use direction field such as in the approaches disclosed in the documents entitled "Indexing Fingerprint Databases Based on Multiple Features", Johan de Boer et al., 29 Nov. 2011 and "Fingerprint classification by directional image partitioning" , R. Cappelli et al., Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , Volume: 21, Issue 5, May 1999, FingerCode such as in the abovereferenced document entitled "Indexing Fingerprint Databases Based on Multiple Features" , texture such as in the approach disclosed in the document entitled "A fingerprint orientation model based on 2D Fourier expansion (FOMFE) and its application to singular-point detection and fingerprint indexing" , Yi Wang et al., TPAMI, Volume 29, Issue 4, April 2007. Local features are usually minutiae and their neighbourhood, such as in the document entitled "Fingerprint indexing and matching : An integrated approach", Kai Cao et al., 2017, IEEE International Joint Conference on Biometrics (IJCB) , the minutiae and ridge flow curvature, such as in the patent document US6, 181, 807B, the minutiae and neighbours, such as in the patent document US6,941,003B [7] , or the minutiae local region group by gradient histogram, such as in the patent document US8 , 625, 861B .

SUMMARY

This disclosure intends to combine the advantages of exclusive classification, such as high level description, fast speed, short length vectors, together with the high discrimination of continuous classification. In this disclosure, using the class information as ground-truth, a deep learning based approach is proposed to determine a vector for each ten-print and reduce the penetration rate to below approximately 5% , with a low missing rate (below approximately 0 . 1 % ) , and produce a matching speed of over approximately 10 million subj ects comparison per second using single thread .

In accordance with a first aspect , the invention concerns a method of indexing a human fingerprint image comprising the steps of : defining a plurality of classes , each class corresponding to a local pattern of a fingerprint ; building a ground truth by allowing a machine learning model to determine the inference probability to each class for acquired fingerprint images ; inputting an original fingerprint image ; assigning by the machine learning model to the original fingerprint image a multi-dimensional vector according to the inference probability determined for each class based on the ground-truth; generating an indexing key from the multi-dimensional vector of the original fingerprint ; and indexing the fingerprint indexing key, the indexing comprising multi-stage checks including both exclusive and continuous classi fication checks .

Preferentially, - the machine learning model is a convolutional neural network; - the number of classes that are defined is comprised between 4 and 16 ; - the classes comprise at least four classes corresponding to the local patterns as follows : arch/tented arch, left loop, right loop and whorl ; - the classes comprise at least one unknown class ; - the length of the indexing keys is below 250 bytes , for a ten- fingerprint of a subj ect ; - for the implementation of an exclusive classi fication check, an unmated number of fingerprints that exceeds a threshold is determined for each class to cut of f the unmated number of fingerprints for each class that exceeds this threshold; a continuous classi fication check includes a determination of a Manhattan distance for each of the vectors ; - a continuous classi fication stage includes the determination of a cosine distance for each of the vectors ; and - a continuous classi fication stage comprises the determination of a fusion distance being a weighted sum of a Manhattan distance and the cosine distance for each of the vectors .

In accordance with a second aspect , the disclosure concerns a fingerprint identi fication system comprising a database wherein, in the database , the fingerprints are indexed according to the above method .

BRIEF DESCRIPTION OF THE DRAWINGS

Other features and aspects of the present disclosure will be apparent from the following description and the accompanying drawings , in which :

Fig . 1 demonstrates an exemplary fingerprint identi fication system according to the present disclosure ;

Figs . 2A, 2B, 20, 2D and 2E are exemplary original images that correspond to the predefined fingerprint classes arch/tented arch, left loop, right loop, whorl , and unknown implemented according to the method of the disclosure ;

Fig . 2 is an exemplary schematic representation that depicts the septs of the method according to the disclosure , which allows generation of a multi-dimensional vector, using a convolutional neural network, and starting from an original image of a fingerprint , the generation of the multi-dimensional vectors allowing to build a groundtruth;

Fig. 4 is an exemplary scheme that shows the steps that leads to the obtaining of a fusion vector according to the method of the disclosure;

Fig. 5 is an example of a table that shows how is generated an indexing key according to the method of the disclosure; and

Fig. 6 represents the multi-stage fingerprint indexing according to the method of the disclosure.

DETAILED DESCRIPTION

FIG. 1 demonstrates a fingerprint identification system 101 for indexing a human fingerprint in accordance with the disclosure. In an example, the fingerprint identification system may be or form part of an automatic fingerprint identification system (AFIS) . Such systems include a retrieval system(s) 104 that stores and retrieves features of fingerprint images. For instance, the retrieval system 104 may include storage 108 that may be one or more memory/storage devices, such as a hard drive, cloud drive, server, etc., and may include a database containing fingerprints that are indexed according to a method described in the disclosure herein. The retrieval system 104 may also include one or more application processors 109. For example, the retrieval system 104 may include circuitry such as, but not limited to, one or more single-core or multi-core processors. The processor (s) may include any combination of general-purpose processors and dedicated processors (e.g., graphics processors, application processors, etc.) . The processors may be coupled with or may include the memory/storage and may be configured to execute instructions stored in the memory/storage to enable various applications or operating systems to run on the retrieval system. The fingerprint identification system may include additional elements such as, for example, memory/storage, display, camera, sensor, or input/output (I/O) interface. In other embodiments, the components described may be included in more than one device (e.g., said circuitries may be separately included in more than one device) .

In an aspect of the disclosure, the fingerprint identification system 101 may optionally include a number of workstations, such as an input station 112. The input station 112 may further be connected with scanners, for example flat-bad scanners, or ten-print live scanners and digital cameras. The scanners or cameras are used to scan the fingerprints or capture latent prints that may be loaded into a microprocessor. The loaded prints are segmented and quality is assigned to each print. The prints may be further enhanced and processed. The matching features are also extracted, loaded into the microprocessor, and being sent to a distributed matcher controller. Based on the matching features, a filtered file database is loaded from the data storage and retrieval system to perform an initial matching of search and file records. The matching results are transmitted back to the distributed matcher controller for analysis. Index keys may be produced by the following method.

This disclosure aims to produce short vectors for indexing of fingerprints to accelerate fingerprint identification by fingerprint identification systems. Different from most existing approaches, which use features such as orientation field, minutiae, or singularities, that are extracted from a finger image to derive indexing keys, the disclosure allows to determine the indexing keys directly from the original fingerprint image. According to the disclosure, a "ground-truth" is built through assigning each fingerprint a vector according to its class, wherein the ground-truth is a set of acquired fingerprint images that have been matched to a class assignment. The class assignment me be a "true" or "correct" class assignment for the fingerprint image. For instance, the ground-truth may be verified, such as by human intervention. Hence, as shown in Figs. 2A, 2B, 2C, 2D and 2E, several classes are predefined, such as the five classes hereafter: arch/tented arch (Fig. 2A) , left loop (Fig. 2B) , right loop (Fig. 2C) , whorl (Fig. 2D) , and unknown (Fig. 2E) . The unknown label is for those fingerprints that are hard to recognize due extremely low quality, and those ambiguous cases. Fewer classes may be used, such as that only several of the five previously named classes may be used, or a different set of classes may be produced. It is also considered that more than five classes may be used. For example, additional classes may include any of a variety of whorls, such as plain whorl, twin-loop whorl, central pocket loop whorl, and accidental whorl. The classes may be defined in different ways, such as to the specifications of the needs or requirements of the developers or researchers, including from four classes to eight classes, and fewer or more classes are considered as long as the description is consistent.

The assigned multidimensional vector, in this case a fivedimensional vector, is shown in Fig. 3. Each dimensional vector represents the inference probability of belonging to a corresponding class. For instance, in Fig. 3, the fingerprint is definitely of an arch type, so the first element is 1 for the ground-truth . A machine learning model may be used to assign the multidimensional vector to the image. For instance, the machine learning model may have trained multi-scale models to represent a global information of fingerprints. The machine learning model, in an example, is implemented as a neural network (NN) model that is trained and/or configured to detect a predefined class within a fingerprint image. As discussed herein, a neural network (NN) is a computing model that uses a collection of connected nodes to process input data based on machine learning techniques. Neural networks are referred to as networks because they may be represented by connecting together different operations. A model of a NN (e.g., feedforward neural network) may be represented as a graph representing how the operations are connected together from an input layer, through one or more hidden layers, and finally to an output layer, with each layer including one or more nodes, and where different layers perform different types of operations on respective input. In one or more implementations, the machine learning model is implemented as a convolutional neural network (CNN) . As discussed herein, a CNN refers to a particular type of neural network, but uses different types of layers made up of nodes existing in three dimensions where the dimensions may change between layers. In a CNN, a node in a layer may only be connected to a subset of the nodes in a previous layer. The final output layer may be fully connected and be sized according to the number of classifiers. A CNN may include various combinations, and in some instances, multiples of each, and orders of the following types of layers: the input layer, convolutional layers, pooling layers, rectified linear unit layers (ReLU) , and fully connected layers. Part of the operations performed by a convolutional neural network includes taking a set of filters (or kernels) that are iterated over input data based on one or more parameters.

Fig 4 depicts an embodiment of the ground-truth building wherein two models having the same structure but different input sizes are trained. The model structure may be predefined with layers of filters, however the parameters may be continuously updated with training images. Larger size images may need more computations than smaller images. The multi-scale processing and fusion of vectors can produce complementation for the final performance. One is of an input size W1 x Hl, the other is of an input size W2 x H2. Both use CNNs, and output an n-d vector. The Resnet- 18 convolutional neural network is an example of network structure. The final indexing key is the fusion of the two vectors. The fusion vector, which is detailed in Fig. 5, aims to get the complementary information from the two vectors, and derive some features to accelerate indexing, such as class labels and normalization factors of the vectors .

The indexing is a pre-filtering stage before an accurate check stage. In this disclosure, the fingerprint indexing by itself is a multi-stage process. As shown in Fig 6, in the indexing phase, subjects are picked from the enrolment database and passed to multi-stage checks (Pass stage 1 to Pass stage n) . The produced vector for each fingerprint can be used for exclusive classification (choose the class with highest possibility) and continuous classification. For a ten-print application, the stage using exclusive classification will count the unmatched number of fingerprints for each candidate subject and make a decision. The stage using continuous classification will compute a distance between two subjects and their vectors. This distance will be compared with a predefined threshold to make a decision. The indexing stages may include

1) an exclusive classification stage to cut off, for example, unmated number of fingerprints (by class) exceeds a threshold numThres. The unmated means corresponding classes of corresponding fingerprints are of different classes, which is to say that for ten-print (i.e., 2 or more fingerprints for each person/sub j ect ) applications, two fingerprints being unmated means that their respective classes (such as the classes with highest possibilities in the vectors) are different. If the answer is yes (Y in Fig. 6) , the comparison of the probe subject and the gallery subject ends with a similarity score of zero, which means that there is not a match present. Otherwise (answer is no - N in Fig. 6) , it is passed to the next stage. 2) a continuous classification stage using, for example, Manhattan Distance d. For n-d vectors X and Y, the Manhattan Distance is defined as: d=X"₁|-V_f -hl

If the distance is below the threshold mdThres, it is passed to the next stage, otherwise returns a similarity score of zero.

3) the continuous classification stage using Cosine Distance. Cosine Distance = 1 - Cosine Similarity, and Cosine Similarity is defined as:

Cosine Similarity

If the Cosine Distance is below cdThres, it is passed to the next stage, otherwise returns a similarity score of zero .

4) the continuous classification stage to check the fusion distance, which is a weighted sum of Manhattan Distance and Cosine Distance. If the distance is below fdThres, the candidate subject will be sent to next stage, otherwise returns a similarity score of zero. If the candidate passes all of the above stages, it is passed to further accurate check stage (such as minutiae based accurate matching, which uses two sets of minutiae, which are several feature points for a fingerprint, to conduct point matching between two fingerprints) , otherwise the candidate is supposed to be a non-match.

In an aspect of this disclosure using five predefined classes for model training and inference and four stages indexing, the indexing key extraction time using CPU (2.3GHZ) may be approximately 300ms and GPU (T4) may be approximately 64ms for ten prints of a person. The indexing speed may be approximately 10 million comparisons per second using single thread. For the database NIST14, with a reference database of 1.06M subjects enrolled, this embodiment may achieve a penetration rate of approximately 4.86% with zero missing rate. Hence, the method according to the disclosure is an end to end approach that uses the original fingerprint images as input, and the index key as the output, avoiding the complex and error propagation image pre-processing, orientation\singularities\minutiae extraction, etc. The method is robust to noise, transformation, low quality, and distortion, with an easy to build ground truth that is scalable .

Claims

1 . A method of indexing a human fingerprint image comprising the steps of : defining a plurality of classes , each class corresponding to a local pattern of a fingerprint ; building a ground truth by allowing a machine learning model to determine the inference probability to each class for acquired fingerprint images ; inputting an original fingerprint image ; assigning by the machine learning model to the original fingerprint image a multi-dimensional vector according to the inference probability determined for each class based on the ground-truth; generating an indexing key from the multi-dimensional vector of the original fingerprint ; and indexing the fingerprint indexing key, the indexing comprising multi-stage checks including both exclusive and continuous classi fication checks .

2 . The method of claim 1 , wherein the machine learning model is a convolutional neural network .

3 . The method of claims 1 or 2 , wherein the number of classes that are defined is comprised between 4 and 16 .

4 . The method of one of the claims 1-3 , wherein the classes comprise at least four classes corresponding to the local patterns as follows : arch/tented arch, left loop, right loop and whorl .

5 . The method of one of the claims 1-4 , wherein the classes comprise at least one unknown class .

6 . The method of one of the previous claims , wherein the length of the indexing keys is below 250 bytes , for a ten- fingerprint of a subj ect .

7. The method of one of the previous claims, wherein, for the implementation of an exclusive classification check, an unmated number of fingerprints that exceeds a threshold is determined for each class to cut off the unmated number of fingerprints for each class that exceeds this threshold.

8. The method of one of the previous claims, wherein a continuous classification check includes a determination of a Manhattan distance for each of the vectors.

9. The method of one of the previous claims, wherein a continuous classification stage includes the determination of a cosine distance for each of the vectors.

10. The method of any one of the claims 8 or 9, wherein a continuous classification stage comprises the determination of a fusion distance being a weighted sum of a Manhattan distance and the cosine distance for each of the vectors .

11. A fingerprint identification system (101) comprising a database wherein, in the database, the fingerprints are indexed according to the method of one of the previous claims.