WO2024030105A1

WO2024030105A1 - Multi-stage fusion matcher for dirty fingerprint and dirty palm

Info

Publication number: WO2024030105A1
Application number: PCT/TR2023/050763
Authority: WO
Inventors: Ismail KILINC; Emre BASESKI; Yusuf Oguzhan Artan
Original assignee: Havelsan Hava Elektronik San. Ve Tic. A.S.
Priority date: 2022-08-02
Filing date: 2023-08-02
Publication date: 2024-02-08

Abstract

The invention relates to a fusion matcher (FM) method that uses the scale, rotation and quality characteristics of the minutiae to optimize the speed, memory and accuracy metrics in the matching process of previously minutiae (key points) crime scene fingerprints or palm prints, previously minutiae crime scene fingerprints or palm prints, or previously minutiae sensor (rolled or flat print) fingerprints or palm print.

Description

MULTI-STAGE FUSION MATCHER FOR DIRTY FINGERPRINT AND DIRTY PALM

Technical Field

The invention relates to a fusion matcher (FM) method that uses the scale, rotation and quality characteristics of the minutiae to optimize the speed, memory and accuracy metrics in the matching process of previously extracted minutiae (key points) of crime scene fingerprints or palm prints, previously extracted minutiae crime scene fingerprints or palm prints, or previously extracted minutiae sensor (rolled or flat print) fingerprints or palm prints.

Prior Art

A fingerprint is a unique physical characteristic of a person that does not change throughout the person's lifetime. Fingerprints are one of the most widely used biometric features for the identification of individuals. A fingerprint is a pattern of lines and valleys (grooves) on the top of a person's fingers. Fingerprints have key points called minutiae points located at the end of the lines and at the line separations. The fingerprint recognition process is generally similar for fingerprints obtained from the crime scene and fingerprints collected with a sensor. However, due to many factors such as poor image quality and the size of the prints, the success performance of fingerprints obtained from crime scenes is lower than that of sensor data. Due to the poor image quality of fingerprints obtained at the crime scene, fingerprints/ palm prints obtained from the crime scene are called latent fingerprints, dirty fingerprints, unqualified fingerprints or source fingerprints. Fingerprints registered in databases are called sensor fingerprints or target fingerprints.

Latent fingerprints are used as forensic evidence by law enforcement agencies to solve crimes. However, due to deformations in latent fingerprint images, the performance of automatic latent recognition systems is far from the desired levels.

In basic matchers, the fingerprint image of the person is obtained with the help of a scanner. Then, the fingerprint image is enhanced by various pre-processing activities (enhancing low quality images, histogram equalization, etc.). On these enhanced images, minutiae and other key points that will be used in the fingerprint identification stage are detected. For each detected minutiae (end of line and fork), a feature vector representing the minutiae is obtained by using the spatial and angular difference information of the minutiae with other minutiaes in its neighborhood. After minutiae detection (feature extraction) with the FingerNet algorithm, numerical results are obtained using the matching algorithm in the comparison stage. Many methods mentioned in the literature can be used to generate feature vectors [1], The matching algorithm compares this set of feature vectors with other fingerprint templates (set of vectors) stored in the database. The fingerprint represented by the vector set/template/cylinder with the highest similarity in the database is presented to the user as a match result.

The Chinese patent document CN108416262 A in the known state of the art relates to a fingerprint image characteristic matching algorithm based on multiple characteristic values. It consists of four steps. Step 1 is to compare the print shape of a fingerprint image with a registered fingerprint template; Step 2 is to compare the characteristic point number of the fingerprint image with the registered fingerprint template; Step 3 is to simultaneously perform distance matching, ridge line number matching and included angle matching of the fingerprint image and the registered fingerprint template; Step 4 is to determine that the fingerprint image is matched with the registered fingerprint template if the matching in the three steps is successful. Fingerprint image identification accuracy is high.

The Chinese patent document CN111695386A in the known state of the art relates to removing the background texture of a valid fingerprint image and obtaining a clean fingerprint image. It includes the steps of performing a first preprocessing on the clean fingerprint image where the background is removed to obtain a first preprocessed image, performing directional field estimation and directional field correction on the active area, and performing a second preprocessing on the active area subjected to directional field correction to obtain a fingerprint-enhanced image. The regularization method reduces fingerprint acquisition times, improves the quality of the fingerprint image, and reduces the complexity of fingerprint acquisition. The fingerprint identification is completed according to the comparison between the characteristic data and the characteristic data of the fingerprint template. The database contains information about the fingerprint such as minutiae, protrusion, etc.

In the known state of the art, a lot of memory is used in fingerprint queries and therefore the matching process is slow. In addition, there are methods that are not sensitive to scale differences. In the state of the art, there is no fusion matching method that focuses on angle, scale and minutiae quality parameters for dirty fingerprint detection and prioritizes high accuracy, high speed and memory usage. Therefore, there is a need to develop a multi-stage fusion matcher that prioritizes speed, accuracy and memory according to certain parameters.

Purposes of the Invention

The purpose of the present invention is to provide a multi-stage fusion matching method that prioritizes high accuracy, high speed and low memory usage in the detection of dirty fingerprints.

Another purpose of the invention is to provide a multi-stage fusion matching method that prioritizes high accuracy, high speed and low memory usage, focusing on angle, scale and minutiae quality parameters.

Another purpose of the invention is to provide a multi-stage fusion matching method that can achieve better results than basic matching.

Detailed Description of the Invention

The multi-stage fusion matcher realized to achieve the purposes of the invention is shown in the attached figures.

These figures are;

Figure 1 : A schematic representation of a multi-stage fusion matcher showing the template creation stage and the search stage.

Figure 2: A schematic representation showing the 4 steps involved in the search stage of a multi-stage fusion matcher.

Figure 3: Illustration of an example matching using the FM 6 configuration of the multistage fusion matcher.

The method in the figures is numbered individually and the equivalents of these numbers are given below. 1. Multi-stage fusion matcher

2. Template creation stage

3. Search stage

4. Target fingerprint image

5. Source fingerprint image

6. Target fingerprint template

7. Source fingerprint template

8. Stage 1

9. Stage 2

10. Stage 3

11. Stage 4

The invention is a multi-stage fusion matcher method and includes the following steps; fingerprint image acquisition using a fingerprint reading or scanning device or crime scene equipment, improving image distortions of the obtained fingerprint image in the preprocessing stage, detection of key points (minutiae) on the enhanced fingerprint image using a minutiae detection algorithm, proceeding to the template creation stage for the obtained 2D minutiae cloud, for source images in the template creation stage, creation of the resulting minutiae clouds by scaling the source scale (SS) for each scale value, for the source images in the template creation stage, creation of the resulting minutiae clouds by rotating the source rotation (SR) counterclockwise for each rotation value, for source images in the template creation stage, creation of a new minutiae cloud for source images by thresholding the SR and SS transformed templates for the number of quality values specified in the source quality (SQ), for the target images in the template creation stage, creation of the resulting target minutiae cloud by scaling the target scale (TS) for a number of scale values, for the target images in the template creation stage, thresholding the minutiae clouds obtained from the TS transformed templates for the number of quality values specified in TQ and creating a new minutiae cloud for minutiae values above the threshold, obtaining target and source fingerprint templates for minutiae clouds as a result of the template creation stage,

- transition of the new minutiae templates to stage 1, in the 1 st stage, determining the source scaling, source rotation and source quality values from the created templates and eliminating according to the matching quality and maximum source matching parameters, in stage 1, determining the target scaling, target scale and target quality values from the templates created and performing the elimination process by using the cylinder information of the minutiae number specified in the maximum target matching parameter, at the end of stage 1, performing matching on the source fingerprint image and the target fingerprint images (there may be a total of 20 fingerprints for each person in the target, 10 rolled prints and 10 flat prints), in the matching process, using the target and source cylinder information, calculating similarity scores between minutiae as in Equation 1, td tA' 0 ^1 <X 1 ® ^vill s(vi, vj) =] ^{0 p} — ⁰ f_or ||vi|| + || vj|| (Equation 1)

I otherwise Q creating a similarity matrix using similarity scores, identifying nP minutiae pairs from the similarity matrix using the local similarity assignment algorithm (LSA), matching using LSS-R multiscale parameters and calculating the match score value, if the resulting match score value is below the value specified for the step thresholds parameter, the template for the target does not proceed to the next step,

- transmitting the target fingerprints with the 2 highest match scores from the obtained score values to the 2nd stage, in stage 2 for the source image, selecting templates according to source scaling (SS), source rotation (SR) and source quality (SQ) values from the templates created in stage 1, in stage 2 for the source image, eliminating templates based on matching quality and maximum source matching parameters, in stage 2 for the target image, selecting templates according to target scaling (TS) and target quality (TQ) values from the templates created in the 1st stage, in stage 2 for the target image, eliminating templates according to the maximum target matching parameter, as a result of stage 2, matching using the matching axis and LSS multiscale parameters on a total of 2 fingerprints, target image and source image, and obtaining the matching score value using Equation 1, if the match score value is below the value specified for the step thresholds parameter, the template of the target does not proceed to the next step, comparison is performed using the source and 2 fingers from the target and forwarding the target finger with the highest match score to stage 3 and applying the comparison as in stage 2 using the stage 3 parameters, at the end of stage 3, determining the most suitable template from the minutiae of the source and target images,

- transmitting target and source finger images to stage 4, selecting the best source scaling value and source rotation value between -40 and +40 degrees of the best source rotation value obtained in stage 3 among the source templates in the 1st additional stage of stage 4, selecting the templates with the best target scale found in stage 3, matching between the selected source templates by increasing the source minutiae match quality thresholds upwards (15 for template quality threshold less than 70, 10 for 70-80, 5 for 80-96) and decreasing the target minutiae number downwards (40 for more than 120 minutiae, 30 for 120-80, 20 for 80-50, 10 for 50-20) and finding the source match quality value and target minutiae number that gives the best score, if the score obtained is not less than a4, proceed to the next stage, selecting a set of constrained templates with the best source minutiae threshold and the best target minutiae counts, identifying the source templates whose source scale range is within 0.8 and 1.2 times of the best source scale value found in stage 3, determining the source template that produces the best score using the source templates found with the highest scoring target template, the obtained source matching quality threshold values and the number of target matching minutiae, if the score obtained is greater than a5, proceed to the next stage, the 2nd additional stage of stage 4, selecting among the source templates those with the best source scaling value and source rotation value between -60 and +60 degrees of the best source rotation value obtained in additional stage 1 of stage 4, selection of templates with the best target scale found in additional stage 1 of stage 4, making matches within the selected source templates by increasing the source minutiae count match quality thresholds upwards (5 by 5 up to 96 from the source template quality threshold) and decreasing the target minutiae count downwards (5 by 5) and finding the source match quality value and target minus count that gives the best score, selecting constrained templates with the best source minutiae threshold and the best target minutia counts, determining the source templates with a source scale range within 0.75 and 1.25 multiples of the best source scale value found in additional stage 1 of stage 4, and template quality thresholds within +10 and -10 of the source template quality threshold found in additional stage 2 of stage 4, determining the target templates with a target scale value between 0.9 and 1.1 times the best target scale value found in additional stage 1 of stage 4 and a target template quality value between +10 and -10 times the target template quality threshold found in additional stage 2 of stage 4, determining the source and target templates that produce the best score by using the source and target templates selected in the 2nd additional stage of stage 4, the source matching quality threshold values and the target matching minutiae number that give the highest score found in the 2nd additional stage of stage 4

The purpose of the proposed multi-stage fusion matcher algorithm is to produce a robust match for rotation and scale changes while combining various combinations of scale, rotation and quality parameters into an optimal configuration that will yield the highest match score in a cost-effective manner.

The fusion matcher and the base matcher use minutiae templates generated by the FingerNet minutiae extractor, which extracts the minutiae spatial coordinates, angles and quality values for each minutiae. Although the base matcher in the known state of the art works well for fingerprints obtained through sensors, its performance degrades for latent fingerprints with strong deformations/ distortions and poor image quality. The fusion matcher addresses the problem of inherent distortion in latent images by creating multiple feature vector (cylinder) templates for each source (latent) and target (rolled/flat) fingerprint image. The fusion matcher method produces these cylinder templates by performing an affine transform on the 2D point cloud of minutiae (for both latent (source) and sensor (target) images) and quality thresholding on the extracted minutiae (for both source and target images). Moreover, the proposed FM method has several configurations (FM23/3/6//8). This allows users (operators) to vary the speed, accuracy and memory requirements. For example, FM23 and FM3 are designed for situations where we operate at high search speed with a small loss in search accuracy, and FM6/8 prioritizes accuracy over speed and memory requirements.

The FM algorithm consists of 2 main stages. The first one is template creation and the second one is the search stage.

In the template creation process, using the rolled/flat-print target image and source image sets, rolled/flat-print target image templates and source image templates are created by using quality thresholding and affine transformations on the rolled/flat-print target image and source image sets. In the template creation stage, the fusion matcher method creates new 2D point clouds from the original minutiae template to be used as minutiae templates for the source and rolled/flat-print target images. For this purpose, the fusion matcher method applies affine transform operations using Equation 2 and Equation 3 on the 2D minutiae point cloud (m), in particular rotation (R) and scale (S), to create a new transformed 2D point cloud (mnew). lcos(0)sin(0)l „ sO] _ZT7 .• R⁼ U(s)_ms(e)l ^{ve S =} loJ ^{<Equatlon 2>}

9new ⁼ 9org + 9 ve mnew = S x R x rh^T (Equation 3)

In the search stage, the similarity score value between the rolled/flat-print target image templates and the source image templates is calculated using Equation 1 and the LSS-R method. rh^T = [(xo, yo); . . . ; (XN, yx)], represents the spatial coordinates partition of m for N number of minutiae in the original set. (When 9 = 9 and 5 = 1 the original minutiae point cloud (m) is obtained). Then, using these newly transformed 2D minutiae point clouds (mnew and 0new) and their associated quality values, new cylinder descriptors are created to reduce the effects of distortion and orientation mismatch. In terms of quality values, source and target cylinder quality thresholds can be used separately in the cylinder template creation process. Therefore, during the template creation stage, the following parameters (listed in Table 1) are used to incorporate scale, rotation and quality values into the template creation process.

Table 1. Parameters and search values used in the template creation stage for the base matcher, FM23, FM3, FM6 and FM8

The expression d_e (0t, O ) in Equation 1 shows the difference between (angle in the source image) and 0j (angle in the target image). 6e is a parameter that controls the maximum rotation between two minutiae. The © operation represents the bit-level XOR operation. XOR is a bit-level operator. It performs a logical operation. If the input bits are the same, the output will be 0 when false or will be 1 when true, "vi" is the feature vector obtained for each minutiae from the scene (source) image, "vj" represents the feature vector obtained for each minutiae from the rolled/flat print (target) image. The expression s(vi, Vj) indicates the score value for the similarity between the specified minutiae feature vectors.

The proposed multi-stage fusion matcher method consists of two stages. These are the template creation stage and the search stage. The parameters used in the template creation stage are listed below.

The source scale (SS) parameter corresponds to the range of scale values (s in Equation 2) selected from the source scale search (SS Search) listed in Table 1. These values are used to scale the source scale (latent image) at minutiae locations (x, y) according to Equation 3. The scale values range from 0.8 to 1.20 as listed in Table 1. For example, when the Source Scale (SS) value is 1, a scale value of 1 is used in the template creation process. Similarly, if the Source Scale index value is 3, a scale value of [1, 1.2, 0.8] is used in the template creation process.

The source rotation (SR) parameter is used to rotate the source minutiae locations (x, y) according to Equations 2 and 3 using the values listed in the source rotation search (SR Search) values in Table 1. For example, when the SR value is 3, the first three rotation values in the SR Search [0, -90, 90] are used in the template creation process. If the SR value is 8, the first 8 SR Search values are used in the template creation.

The source quality (SO) parameter is the minutiae quality threshold applied to the source (latent) minutiae when generating cylinder templates. The source image minutiae can be ignored or used according to this quality threshold, and the specified number of cylinders is produced and used for each step. Table 1 lists the source minutiae quality levels used in the template creation stage. The minutiae quality values created with the minutiae extractor are between 50 and 100, and the search values are selected according to this range.

The target scale (TS) parameter is used to transform the 2D target (x, y) minutiae locations according to Equations 2 and 3. Target scale values range from 0.8 to 1.2 as listed in Table 1.

The target quality (TO) parameter is the target (rolled/flat) image minutiae quality values used in the cylinder template creation process. Target image minutiae can be ignored or used according to this quality threshold level. Table 1 lists the target minutiae quality levels (TQ Search) used in the stencil creation process. The Multiple Radius (MR) parameter is used to create cylinder templates for each radius value listed in the Multiple Radius Search in Table 1. The radius (R) affects the selection of the minutia neighbor during cylinder creation.

The source generation model (SGM) limits the number of cylinders produced during the template creation stage for the source. If the mode is 0, all rotation values for scale 1 (in SR Search) and all scale values for rotation 0 (in SS Search in Table 1) are used to create cylinders. If mode is 1, all scale values for the first 3 rotations (0, 90, -90) are used.

The maximum source fusion element

specifies the maximum number of source cylinders to be generated per latent image by the fusion matcher method. The number of cylinders may be less than the minimum number of minutiae for high source quality (SQ) values and therefore the actual number of elements may be less than the SFIM value. In the developed method, the minimum number of minutiae is set to 7 as shown in Table 2. (Denklem 4)

ASME = SFIM x SC x (SMA+1) x 80 (Equation 5)

The Application Resource Memory Estimate (ASME) in bits can be calculated with Equation 5. This value is used to estimate the memory requirement of an application with specified parameters. SC indicates the number of source (latent) images. SMA is the source minutia average and 80 is the single-cylinder memory allocation value in bytes. A value is added to SMA when there is one header file per resource. fusion item maximum (TFIM) specifies the maximum number of target cylinders per number of fingers used for the fusion method. The memory allocation for a template is multiplied by this value. The TFIM value is adjusted to 1 for FM23 to create a minimum memory solution.

TFIM = TS x TQ x (MR+1) (Equation 6)

ATME = TFIM x TC x (TMA+1) x TFA x 80 (Equation 7) The Application Target Memory Estimate (ATME) in bytes is calculated by Equation 7. This value is used to estimate the memory requirement for an application with the specified parameters. TC is the number of targets. TMA is the target minutia average and 80 is the single cylinder memory value in bits. A value is added to TMA when there is a header per number of fingers. TFA is the target finger average per person. This value is 10 for the fully rotated fingerprint and 20 for the full flat and rotated dataset.

TFIM has a significant impact on the general recognition process as it specifies the amount of memory used for a large target set. If the target image size is in the tens of millions, memory requirements will be an issue. In the template creation process, only 80 bits per minutiae are allocated for uncompressed templates. For example, a set of 100 million (TC) dial and flat prints (each person is assumed to have 10 dial and 10 flat fingerprints for a total of 20 TFA fingerprints) requires 12.8 x TFIM Terabytes of memory or disk space, with an average of 79 minutiae (TMA). The source cylinder size may be important for large latent image database search with flat/rolled, but the latent image database sizes are expected to be relatively small compared to flat and rolled database. Once the template creation process for the rolled/flat and latent fingerprint images is completed, the fusion matcher method starts to be used in the search stage.

The fusion matcher creates a similarity score between source and target images.

The search stage of the fusion matcher is a sequential multistage fusion system consisting of 4 steps. Each step of this architecture aims to keep the retrieval rate high while eliminating mismatches by using a set of parameters and threshold levels to control the input of the matching process to the next steps. Each step of the fusion matcher method optimizes the use of the following parameters.

Matching quality is used to eliminate cylinders belonging to minutiae cylinders with quality values lower than this value during the matching process. Low quality minutiae that are not used in the matching process may not have cylinders, but may contribute to the cylinder perimeter of high quality minutiae. From Step 1 to Step 3 the default match quality values are used as shown in Table 1 and Table 2. But the extra step is looking for the best match quality to get the highest score value. The base matcher produces the best identification rates according to the default value.

Maximum source matching limits the maximum number of source minutiae used in the matching process. The number of quality minutiae for the latent image is typically between 20 and 30. Too many minutiae can lead to false detections. Since the number of minutiae used affects the baseline matcher speed, a small number of minutiae is used in the first step (Step 1) and increased in subsequent steps.

Maximum target matching limits the maximum number of minutiae in the target image used in the matching process. The average number of quality minutiae for the rotated print is typically about 80, and too many minutiae can cause false detection. This value is higher than the expected average number of minutiae minutiae in all steps. The extra step looks for the maximum best minutiae that gives the highest score.

The match axis is used to control the number of matches by using all values for scales and rotations or only some scales and rotations. The use of the match axis means that all source scale values are used if the target scale is 1 and all target scale values are used only if the source scale is 1. Call numbers are controlled and can be used mainly in the partial step of FM8 and FM23 (step 2) and FM3 as well as in all steps in FM6. If the source scale and target scale for cylinders are simultaneously less than 1.0 or greater than 1.0, these cylinders are not compared to avoid unexpected score generation.

Reducing the number of fingers indicates the number of fingers used after the first step (Step 1) is completed.

LSS multi scale is a scaling method applied after the pairwise similarity calculation and before the LSS-R stage in base matching. In the LSS-R calculation stage of the base matcher, if a threshold (a0 = 0.135) is exceeded for the match candidates and score level fusion is applied, the minutiae positions are scaled for 1.1 and 0.9 for the target and 1.05 and 0.95 for the source.

Step thresholds contain threshold values to balance the number of match calls between steps. There are 5 thresholds (al, a2, a3, a4, a5) of which the first three (al, a2, a3) are used sequentially from step 1 to step 3 and the last two a values are used in step 4.

The multi-radius threshold (a6) value controls the use of multi-radius cylinder templates.

Minimum minutiae indicates the minimum number of minutiae required for matching score calculation. Maximum Number of Basic Matcher Calls indicates the maximum expected number of basic matcher calls for matching at each step and summed for a single latent image with 10 flat and 10 rolled template data sets. The number of calls from step 1 to step 3 does not vary with the number of minutiae calls, but the number of extra step calls depends on the quality of the hidden and sensor image and the number of minutiae calls.

Step Thresholds check and block unwanted base matcher calls for potential unmatched candidates from step 2 to step 4. While step 4 base matcher call counts are very high compared to other steps, step 4 threshold values are close to the expected match score average and only high match candidates are shown for step 4 calls.

In the latent - rolled/flat print match process, a latent source is expected to match only two fingers. Therefore, when searching for a match of a latent image, in step 1 (the first step), matches with scores lower than the step threshold al are eliminated. Then the remaining rotated/flat finger matches are reduced to 2 fingers with the best matching score values for each person. In step 2 (partial step), the number of two fingers is reduced to one for step 3 by selecting the fingers with the highest matching scores. The fusion parameters (rotation angle, scale values, etc.) and the number of matcher calls start with small numbers and increase from step 1 to step 4 as shown in Table 2. If the multi-radius fusion matcher configuration option is selected, all steps are executed again if the multi-radius threshold level is exceeded.

The main objective in the first step (stage 1) is to find the two most suitable fingerprint targets across all target fingers for a single person. Therefore, only a limited set of scale, rotation and quality values are used in this step. For example, Table 2 shows that FM 23/3/6 uses only 1 target scale (1.0) and target quality (60), 3 source scales (1.2, 1.0, 0.8), 3 source rotations (-90, 0, 90) and a single source quality (60), while FM 6/8 uses 3 additional target scales (1.0, 0.88, 1.08). The first step also produces a score multiplier value of 0.975 if the top two fingers have different rotated and straight indices. This score multiplier is the value that will multiply the match score from step 1 to step 4.

Similarly, the partial step (step 2) aims to find a single target without a deep search. FM3 uses 3 target scales (1.0, 0.88, 1.08), FM6, 3 source qualities (60, 85, 50), FM23 2 source qualities (60, 85), 5 source scales (1.0, 1.2, 0.8, 1.12, 0.88) and 6 source rotations (0, -90, 90, 180, -30, 30), FM8, 3 source cylinder qualities (60, 85, 50), two target cylinder qualities (60, 85), 5 source scales (1.0, 1.2, 0.8, 1.12, 0.88), 6 source rotations (0, -90, 90, 90, 180, - 30, 30) and 4 target scales (1.0, 0.88, 1.08, 0.94).

The normal step (step 3) uses most of the fusion parameters with the default match quality threshold. In this step, FM3 has 2 source qualities (60, 85), FM23/6/8 has 4 source qualities (60, 85, 50, 70) in this step. Also in this step, FM23 has one target quality (60), FM 3/6 has two target cylinder qualities (60, 85) and FM8 has 4 target qualities (60, 85, 50, 75). There are 3 source scales for FM3 (1.2, 1.0, 0.8), 5 source scales for FM6/FM8 (1.0, 1.2, 0.8, 1.12, 0.88) and 7 source scales for FM23 (1.0, 1.2, 0.8, 1.12, 0.88, 1.04, 0.96). There is 1 target scale (1.0) for FM23, 3 target scales (1.0, 0.88, 1.08) for FM3, 4 target scales (1.0, 0.88, 1.08, 0.94) for FM6 and 5 target scales (1.0, 0.88, 1.08, 0.94, 1.04) for FM8. There are 3 source rotations for FM3 (0, -90, 90), 6 source rotations for FM6/8 (0, -90, 90, 180, -30, 30) and 8 source rotations for FM23 (0, -90, 90, 180, -30, 30, -60, 60).

The extra step (step 4) uses all cylinders and adds additional logic to find the optimal parameters. As shown in Table 2, FM3 has 4 source quality values (60, 85, 50, 70), FM6 has 8 source quality values (60, 85, 50, 70, 55, 65, 75, 80) and FM8/23 has 10 source quality values (60, 85, 50, 70, 55, 65, 65, 75, 80, 90, 95). FM8 has 4 target cylinder qualities (60, 85, 50, 75) and FM23 has 1 target cylinder quality (60), while FM3/6 has 2 target cylinder qualities (60, 85). FM3/6 has 5 source scales (1.0, 1.2, 0.8, 1.12, 0.88), FM 23/8 has 7 source scales (1.0, 1.2, 0.8, 1.12, 0.88, 1.04, 0.96), FM23 has one target scale (1.0), FM3/6/8 has 5 target scales (1.0, 0.88, 1.08, 0.94, 1.04).

The extra step (step 4) search process is relatively more extensive compared to steps 1-3 and includes some control values and thresholds to find the optimal matching parameters. Stage 4 consists of 2 additional steps.

The 1st additional stage is the additional stage of finding the best matching source minutiae quality threshold and the best target minutiae count. Among the source templates, those with the best source scaling value obtained in stage 3 and source rotation value between -40 and +40 degrees of the best source rotation value found in stage 3 are selected. Target templates with the best target scale found in stage 3 are selected. Within the selected source templates, the number of source minutiae matching quality thresholds are increased upwards (15 for a template quality threshold less than 70, 10 for 70-80, 5 for 80-96) and the number of target minutiae matching quality thresholds are decreased downwards (40 for more than 120 minutiae matching quality thresholds, 30 for 120-80, 20 for 80-50, 10 for 50-20) to find the source matching quality value and the number of target minutiae matching quality values that give the best score. If the obtained score is not less than a4, the next stage is started. The template is selected again constrained by the best source minutiae threshold and the best target minutiae counts. Within the source templates, the source scale range is determined as the ones within 0.8 and 1.2 times of the best source scale value found in stage 3. In the 1st additional stage, the target template with the highest score and the source template with the best score are found by using the source matching quality threshold values and the number of target matching minutiae found in the 1st additional stage. If the obtained score is greater than a5, the next stage is started.

The 2nd additional stage is the additional stage of finding the best matching source minutiae quality threshold and the best target minutiae count. Among the source templates, the ones with the best source scaling value obtained in the 1st additional stage and the source rotation value between -60 and +60 degrees of the best source rotation value found in the 1st additional stage are selected. Target templates are selected with the best target scale found in the 1st additional stage. Within the selected source templates, the number of source minutiae matching quality thresholds are increased upwards (5 by 5 up to 96 from the source template quality threshold) and the number of target minutiae matching quality thresholds are decreased downwards (5 by 5 by 5) to find the source matching quality value and the number of target minutiae matching quality values that give the best score. A constrained retemplate is selected with the best source minutiae threshold and the best target minutiae counts. Among the source templates, the source scale range is between 0.75 and 1.25 times of the best source scale value found in the 1st additional stage and the template quality thresholds are between +10 and -10 times of the source template quality threshold found in the 2nd additional stage. Among the target templates, those with target scale values between 0.9 and 1.1 times the best target scale value found in the 1st additional stage and target template quality values between +10 and -10 times the target template quality threshold found in the 2nd additional stage are selected. In the 2nd additional stage, the source and target templates are selected, and the source and target template that produces the best score is found by using the source matching quality threshold values and the target matching minus number. Table 2. shows the parameters of the basic matcher, FM3, FM6, FM8, FM23 search stage. For each parameter, the step column lists the number of steps of the fusion matcher method. The selected values for steps 1-4 are indicated as - / - / - / - / - / -

The quality -related parameters depend on the minutiae extractor properties and other fusion matcher parameters are valid for all minutiae subtractors, marked manual minutiae and special basis matches with scale and rotation intolerance. In this work, FingerNet algorithm is used for minutiae extraction.

In summary, the method for developing a new fusion matching includes the following steps. 1. Preparing a dataset of latent and sensor data that will be used to compute quality searches and step thresholds 2. Extraction of minutiae from hidden and sensor images using one of the minutiae extraction methods

3. Finding the best quality thresholds for the source and target used in template creation by analyzing minutiae quality distributions and also by trial and error using the dataset and a custom basis matcher

4. Finding the best matching threshold by trial and error using custom base matches in the selected dataset

5. Creating templates and implementing the multi-stage fusion matcher method

6. Using additional radius values for FM6 and FM8 if a custom base mapper is available and allows radius selection in the template creation used for regional minutiae.

The minutiae data set from all fingerprints and the latent, dirty minutiae set are fed into the fusion matcher. The fusion matcher creates templates with the used datasets. This part is called the template creation stage. The dirty fingerprint and all the fingerprints are inserted back into the fusion matcher and the search stage is performed. The final matching score value is obtained.

Furthermore, the use of palm data has an important role in solving criminal cases. Approximately 30% of the prints collected from crime scenes are palm prints [2], Similar to the fingerprint recognition task, minutiae-based matching methods are mostly used for the identification of latent palm prints [3], Palm prints have more details than fingerprints.

The original size of crime scene and sensor palmprint images is more than 2000 pixels in width and height. Since minutiae extractors are not designed for such large size images, in the developed multi-stage fusion matcher, the images are divided into overlapping sizes. Therefore, the palmprint images are converted into 512x512 pixels, which are suitable for minutiae extraction. After minutiae detection is performed from 512x512 images, the developed multi-stage fusion matcher method is performed.

A search request in the multi-stage fusion matcher returns the best matching source and destination index of the cylinder buffer in the template, the best match quality for source and destination, the best maximum minutiae used for source and destination, and matches the search counts at each step with the total score value and optional minutiae mappings. Each cylinder in the template has a decodable link value that stores the scale, rotation, quality threshold, finger index and a uid value (used to link a buffer to a memory location) in addition to the minutiae local structures. Figure 3 shows the similarity view for a source image from a dataset and a target image corresponding to the source image. Figure 3(a) shows the baseline matcher similarity and Figure 3(b) shows FM 6 similarity calculations of matched minutiae pairs. Red minutiae are not used in the matching calculations because they are below the cylinder quality thresholds. Yellow minutiae are used in cylinder production but not for matching. Green details have cylinders and contribute to the matching score. The matching lines are colored from green to red and numbered from 1 to 12 according to their contribution value. 5 minutiae for the source and 11 minutiae for the target (colored in red in Figure 3) are discarded for cylinder production. In FM6, some minutiae (colored yellow in Figure 3) were discarded for matching. The best source cylinder producing the maximum score has a scale of 1.0, a rotation of -60, a cylinder quality of 70 and a match quality of 95 for the FM6 configuration (Figure 3(b)). The best target cylinder for this case has a scale of 0.88, a cylinder quality of 85 and a maximum minutiae number of 81. Since there is a match, the matching process follows all the steps in the search stage, including the extra step. Since the extra step gradually reduces the maximum minutiae number and is set to 81 to achieve the highest score without using any match quality threshold, the target match quality is zero. Similarly, the default value for the source match quality value has been increased from 84 to the optimum 95 for FM6.

The advantage obtained with the developed method is mentioned below.

The proposed method provides better results than basic matching in detecting dirty fingerprints due to its high accuracy, high speed and memory utilization prioritization.

REFERENCES

[1] R. Cappelli, M. Ferrara and D. Maltoni, "Minutia Cylinder-Code: A New Representation and Matching Technique for Fingerpring Recognition” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010. [2] S.K. Dewan, Elementary, Watson: Scan a palm, find a clue p. 1. The New York Times,

2003.

[3] J. Rodriguez-Ruiz, M. A. Medina-Perez, R. Monroy, "A survey on minutiae-based palmprint feature representations, and a full analysis of palmprint feature representation role in latent identification performance," Expert Systems with Applications, vol. 131, pp. 30-44,

Claims

1. The invention is a multi-stage fusion matcher method comprising processing steps;

- taking fingerprint images using a fingerprint reading device or crime scene equipment, improving image distortions in the preprocessing process of the acquired fingerprint image, detecting key points (minutiae) on the enhanced fingerprint image with minutiae detection algorithm, characterized by comprising the process steps; proceeding to the template creation stage for the obtained 2D minutiae cloud, for the source images in the template creation stage, creation of the resulting minutiae clouds scaled for a number of SS scale values, for the source images during the template creation stage, creation of the obtained minutiae clouds by rotating the point clouds counterclockwise for a number of SR rotation values, for source images during the template creation stage, thresholding the SR and SS transformed templates for the number of quality values specified in SQ and creating a new minutiae cloud for minutiae templates above the threshold, for the target images in the template creation stage, creation of the resulting target minutiae cloud by scaling it for TS scale values, for the target images in the template creation stage, thresholding the minutiae clouds obtained from the TS transformed templates for the number of quality values specified in TQ and creating a new minutiae cloud for minutiae values above the threshold, obtaining the template for minutiae clouds as a result of the template creation stage, comparison stage (stage 1) of the new minutia templates obtained,

In stage 1, determining source scaling, source rotation and source quality values from the templates created for source images and eliminating according to matching quality and maximum source matching parameters, In stage 1, determining the target scaling, target scale and target quality values from the templates created for the target images and performing the elimination process using the cylinder information of the minutiae number specified in the maximum target matching parameter,

At the end of stage 1, performing a matching process on the source fingerprint image and the target fingerprint images (up to 20 fingerprints of the target person, up to 10 rolled and up to 10 flat prints),

In the matching process, calculating the similarity scores between minutiae using the target and source cylinder information as in Equation 1, td tA' 0 ^1 <X 1 ® vj II s(vi, vj) =] ^{0 p} M — ⁰ f_or ||vi|| + || vj|| (Equation 1)

( otherwise Q

Creating a similarity matrix using similarity scores,

Identifying nP minutiae pairs from the similarity matrix using the local similarity assignment algorithm (LSA), matching using LSS-R multiscale parameters and calculating the match score value, if the resulting match score value is below the value specified for the step thresholds parameter, the template for the target does not proceed to the next step,

- transmitting the target fingerprints with the 2 highest match scores from the obtained score values to the 2nd stage, in stage 2, selecting templates for the source image according to source scaling (SS), source rotation (SR) and source quality (SQ) values from the templates created in stage 1, in stage 2, eliminating template for the source image based on matching quality and maximum source matching parameters, in stage 2, selecting templates for the target image according to target scaling (TS) and target quality (TQ) values from the templates created in stage 1, in stage 2, eliminating templates for the target image according to the maximum target matching parameter, as a result of the 2nd stage, Matching using the matching axis and LSS multiscale parameters on a total of 2 fingerprints, target image and source image, and obtaining the matching score value, if the match score value is below the value specified for the step thresholds parameter, the template for the target does not proceed to the next step, comparing the source and 2 fingers from the target and forwarding the target finger with the highest matching score to stage 3 and performing a comparison as in stage 2 using the parameters of stage 3, at the end of stage 3, identifying the template from the minutiae of the source and target images,

- transmission of target and source finger images to stage 4, in the 1st additional stage of stage 4, selecting among the source templates the best source scaling value and source rotation value obtained in stage 3, which are between -40 and +40 degrees of the best source rotation value found in stage 3, selecting the templates with the best target scale found in stage 3, matching between the selected source templates by increasing the source minutiae match quality thresholds upwards (15 for template quality threshold less than 70, 10 for 70-80, 5 for 80-96) and decreasing the target minutiae number downwards (40 for more than 120 minutiae, 30 for 120-80, 20 for 80-50, 10 for 50-20) and finding the source match quality value and target minutiae number that gives the best score, if the score obtained is not less than a4, proceed to the next stage, selecting a set of constrained templates with the best source minutiae threshold and the best target minutiae counts, identifying the source templates whose source scale range is within 0.8 and 1.2 times of the best source scale value found in stage 3, determining the source template that produces the best score by using the source templates found with the highest scoring target template, the obtained source matching quality thresholds and the number of target matching minutiae, if the score obtained is greater than a5, proceed to the next stage, the 2nd additional stage of stage 4, selecting among the source templates the best source scaling value and source rotation value between -60 and +60 degrees of the best source rotation value found in the additional stage 1 of stage 4, selection of templates with the best target scale found in additional stage 1 of stage 4, matching the number of source minutiae within the selected source templates by increasing the match quality thresholds upwards (5 by 5 up to 96 from the source template quality threshold) and decreasing the number of target minutiae downwards (5 by 5 by 5) and finding the source match quality value and target number of minutiae that gives the best score, selecting a set of constrained templates with the best source minutia threshold and the best target minutiae counts, determining the source templates with a source scale range between 0.75 and 1.25 times the best source scale value found in additional stage 1 of stage 4 and a template quality threshold between +10 and -10 times the source template quality threshold found in additional stage 2 of stage 4, - determining the target templates with a target scale value between 0.9 and 1.1 times the best target scale value found in additional stage 1 of stage 4 and a target template quality value between +10 and -10 times the target template quality threshold found in additional stage 2 of stage 4, determining the source and target templates that produce the best score by using the source and target templates selected in the 2nd additional stage of stage 4, the source matching quality threshold values and the number of target matching minutiae that give the highest score found in the 2nd additional stage of stage 4.