CN115830645A - Large-scale fingerprint retrieval method based on point cloud registration - Google Patents

Abstract

The invention discloses a large-scale fingerprint retrieval method based on point cloud registration, which can be used for efficiently and accurately retrieving input fingerprints to be retrieved to obtain a fingerprint library sample set which most possibly has the same matching relation. The scheme can be divided into three layers on the whole: a preliminary screening layer, a fine registration layer, and a robust optimization layer. Three layers are interdependent, serially processed, screened layer by layer and lack of one, and a set of complete fingerprint retrieval algorithm is formed together. After the fingerprint to be retrieved is input, the preliminary screening layer completes rapid large-scale preliminary screening by using global features, the fine registration layer realizes further screening by using local registration and provides a matching relation between minutiae for the robust optimization layer, the robust optimization layer judges whether the matching relation of the minutiae is correct or incorrect according to geometric consistency check and a maximum clustering algorithm and outputs a retrieval result by using comprehensive matching scores.

Description

Large-scale fingerprint retrieval method based on point cloud registration

Technical Field

The invention relates to the technical field of fingerprint retrieval, in particular to a large-scale fingerprint retrieval method based on point cloud registration.

Background

Due to the rapid development of artificial intelligence technology, biometric identification is becoming popular in identity identification applications. Compared with the traditional identity identification modes such as certificates, passwords, access control cards and the like, the biological characteristic identification has the advantages of convenience, anti-counterfeiting performance, difficulty in losing and the like. Fingerprints are one of the most widely used biometrics in the field of biometric identification at present due to their uniqueness and persistence. However, the application of fingerprint identification techniques also suffers from several problems:

one is the time efficiency problem of fingerprint retrieval identification. According to the technical scheme, in the application of criminal investigation fingerprint automatic identification systems, large-scale fingerprint attendance systems, access control systems and the like, the input inquiry fingerprints need to be compared with the registered fingerprints in the fingerprint database one by one until the registered fingerprints with the best similarity are found or the conclusion that the corresponding registered fingerprints do not exist is given after the whole fingerprint database is searched. However, as the application field of the automatic fingerprint identification technology is continuously expanded, the scale of the fingerprint database is also continuously expanded, and the identity fingerprint database of residents in China reaches the hundred million people level. This results in a long time for comparing the fingerprint to be queried with the fingerprints in the fingerprint database, and thus the method cannot be applied to practice.

Secondly, the fingerprint images recorded by the same finger twice may have differences, resulting in failure of fingerprint retrieval. The whole fingerprint identification process can be divided into two links of feature extraction and comparison. Feature extraction nodes extract fingerprint features for fingerprint identification, the most common of which are "minutiae" features. However, during fingerprint entry, the results of multiple entries of the same fingerprint may differ. Possible reasons include that the skin of the finger is soft, and the fingerprint image acquired by the pressing mode can be deformed in an irregular elastic manner to a certain extent, so that the fingerprint image is deformed. In addition, the finger may have factors such as temporary molting and wrinkles, which cause changes such as fingerprint texture loss. These all affect the extraction of features such as "minutiae" in the fingerprint identification process, resulting in identification errors and failure of fingerprint retrieval.

The existing minutiae-based fingerprint retrieval technology is mainly divided into two types according to a basic idea. One is a learning-based approach that uses neural networks or deep learning to find similarities between two fingerprints. The method mainly depends on training of large-batch truth value data to obtain appropriate network parameters, so that the prediction performance of the network is improved. The other type is a characteristic-based method, and an index of a fingerprint database is constructed by using the characteristics of point cloud. And comparing the fingerprints to be retrieved by using the characteristic indexes to determine the fingerprint sequences which are relatively similar to each other so as to obtain a retrieval result.

Both of the above two types of methods have their own drawbacks:

the learning-based method requires a large number of training samples to be obtained in advance, which is laborious. And the network has inexplicability, and the generalization performance is difficult to control. When the fingerprint minutiae have large-range noise, the matching precision slides down.

The method based on the characteristics can extract the fingerprint characteristics of the minutiae in an off-line manner, and the retrieval speed is accelerated. However, the original minutiae data of the fingerprint is not retained any more by the feature-based method, and a large amount of information is lost, so that the retrieval accuracy is greatly reduced.

At present, no scheme can ensure matching precision and improve fingerprint retrieval speed.

Disclosure of Invention

In view of the above, the invention provides a large-scale fingerprint retrieval method based on point cloud registration, which can take fingerprint minutiae as two-dimensional point cloud with directions, and perform fingerprint retrieval by using a point cloud registration method, so as to improve the retrieval speed of fingerprints while ensuring the matching accuracy.

In order to achieve the purpose, the technical scheme of the invention comprises the following steps:

the method comprises the following steps: before online retrieval, fingerprint samples in the fingerprint database are input into a primary screening layer, the primary screening layer inputs the fingerprint samples in the fingerprint database into a DAA global feature descriptor extraction module, DAA features of the fingerprint samples are extracted, and the DAA features are stored in an offline fingerprint feature database as sample features; after the retrieval is started, the fingerprint to be retrieved is input to a primary screening layer, wherein a DAA global feature descriptor extraction module extracts DAA features of the current fingerprint to be retrieved, compares sample features in an offline fingerprint feature library to obtain similarity scores of the DAA features of the current fingerprint to be retrieved and DAA features of all samples, sorts the sample features according to the similarity scores from high to low, selects fingerprint samples corresponding to the first N sample features as output of the primary screening layer, and sends the output to a fine registration layer.

Step two: the fine registration layer carries out random relative translation on the fingerprint to be retrieved and the screened fingerprint samples, finds out the optimal initial state by calculating the registration scores after translation, then carries out iterative optimization by utilizing an ICP (inductively coupled plasma) algorithm, determines the transformation relation between the fingerprint to be retrieved and all the fingerprint samples, calculates to obtain the corresponding objective function value by utilizing the registration result of the ICP, and determines the matching relation between the minutiae points by a nearest neighbor searching and matching method; the objective function values and the matching relationships between minutiae are fed into the robust optimization layer.

Step three: the robustness optimization layer uses geometric consistency check for the matching relationship between the detail nodes, then judges the quality of the matching relationship between the detail nodes according to a maximum clique algorithm, and filters wrong detail point matching pairs; calculating to obtain a final comprehensive matching score of the fingerprint sample according to the proportion of the correct minutiae matching pair and an objective function value given by the fine matching layer; and sorting by utilizing the comprehensive matching scores, and realizing screening at any filtering level to obtain a final screening result.

Further, the DAA global feature descriptor extraction module is used for extracting distance-angle-area DAA features; the distance is used for describing distance information between any two minutiae; dividing the range into n segments by taking the minimum distance and the maximum distance as boundaries to count the information of all the distances D; and adding one to the count value of the corresponding section of each distance obtained, and finally carrying out normalization processing on the distance obtained to obtain a histogram of the distance information.

An angle, which is used for describing an included angle between a minutiae and a connecting line of other two minutiae, is set to be [0, pi ], and is replaced by a monotonous cosine function in the range, and a final value range is clamped to be (-1,1 ]; and splitting the final value domain into n sections, counting all included angles, adding the included angles in the corresponding sections, and finally performing normalization processing to obtain a histogram of the angle information.

The area is used for describing the area of a triangle constructed by any three minutiae, the area A of the triangle is obtained through preliminary estimation, the area range is divided into n sections, all the areas are counted and added in corresponding sections, normalization processing is carried out, and a histogram of area information is obtained.

And combining the histograms of the distance information, the angle information and the area information to obtain the DAA characteristic of each fingerprint sample.

Further, comparing sample features in the offline fingerprint feature library to obtain similarity scores of the DAA features of the current fingerprint to be retrieved and all sample DAA features, specifically:

the Manhattan distance is used as an index for measuring the DAA feature descriptor difference, and the calculation formula of the similarity score is as follows:

wherein DAA ₁ Is the DAA feature of the fingerprint sample,

data representing the pth column of DAA features of a fingerprint sample, S being the benchmark maximum score; DAA ₂ Is the DAA feature of the fingerprint currently to be retrieved.

And comparing the fingerprint to be retrieved with the DAA characteristics of all the fingerprint samples in the library one by one, and calculating the similarity.

Further, the fine registration layer carries out random relative translation on the fingerprint to be retrieved and the screened fingerprint sample, and finds out the optimal initial state by calculating the registration score after translation, and the following specific steps are adopted:

and selecting fingerprint samples with the number of minutiae exceeding a set threshold value A as reference fingerprints from the fingerprint samples screened from the primary screening layer, and selecting fingerprint samples with the number of minutiae below a set threshold value B as matching fingerprints, wherein the threshold value B is less than the threshold value A.

And (3) adopting a position random translation strategy, randomly translating the positions of the matched fingerprint minutiae set in four directions of an upper direction, a lower direction, a left direction and a right direction in a certain area range by a given step length, calculating the nearest neighbor minutiae matching pair cost between the matched fingerprint and the reference fingerprint, and selecting the position with the lowest cost as the initial relative registration position of the fingerprint.

Further, in the second step, an ICP algorithm is used for iterative optimization, the transformation relation between the fingerprint to be retrieved and all fingerprint samples is determined, the corresponding objective function value is obtained by calculation according to the registration result of the ICP, and the matching relation between the minutiae points is determined by a nearest neighbor search and matching method, and the specific steps are as follows:

set minutiae sets in the matched fingerprints after filtering outliers as

The minutiae set after filtering the outliers in the reference fingerprint is

Defining the centroids of the two sets of points P and Q as P and Q, respectively:

then the simplified objective function is constructed as:

where R is a rotation matrix and T is a translation matrix.

Based on the simplified objective function, the ICP solution step is:

step a. Calculating the centroid positions of the two sets of points, then calculating the centroid-removed coordinates p 'of each point' _i ，q′ _i ：

p′ _i ＝p _i -p，q′ _i ＝q _i -q

B, calculating an optimized rotation matrix R according to the following optimization problem ^* ：

The matrix W is defined as:

carrying out SVD on W to obtain

W＝U∑V ^T

Wherein, Σ is a diagonal matrix composed of singular values, diagonal elements are arranged from large to small, and U and V are diagonal matrices; when W is full rank, R is R = UV ^T 。

C, according to the optimized rotation matrix R ^* Computing an optimized translation matrix T ^* ：T ^* ＝q-R ^* p。

If then R ^* If the determinant is negative, then take-R ^* As the optimum value.

Using the obtained R ^* And T ^* And (4) carrying out pose transformation on the P, if the error is greater than a set error threshold, carrying out iteration until the iteration times reach the iteration threshold or the error is less than the error threshold, obtaining a nearest neighbor matching result of minutiae of the matched fingerprint and the reference fingerprint, and finally outputting a matching relation between the minutiae within the threshold range and an objective function value to a robust optimization layer for further retrieval.

Has the beneficial effects that:

1. the invention provides a large-scale fingerprint retrieval method based on point cloud registration, which treats the minutiae of a fingerprint as sparse discrete two-dimensional point cloud, thereby realizing the application of a plurality of point cloud-based knowledge in the field of fingerprint retrieval, such as global feature descriptors of point cloud, ICP (inductively coupled plasma) algorithm and the like, and realizing the effect of improving the retrieval speed of the fingerprint while ensuring the matching precision.

2. The invention provides a brand-new feature descriptor: the ESF (an advanced global feature descriptor of a three-dimensional point cloud) is reduced to two dimensions, so that the feature description of the fingerprint minutiae is adapted. By comprehensively considering the distance, the angle and the area, the fingerprint minutiae processing method has higher robustness on translation, rotation and expansion of the fingerprint minutiae.

3. The invention improves the classic ICP algorithm: aiming at the condition that the classical ICP algorithm is difficult to register low-overlapping-degree and high-noise points, the fine registration work of the fingerprints with large differences is realized by searching an optimal iteration initial value and an outlier minutiae filtering mode, and the extracted matching points are more accurate.

4. The invention has a robust optimization layer: and under the condition of primarily finding the matching points, screening again, and filtering out wrong matching points by using geometric consistency and a maximum clustering algorithm, so that the transformation relation between the fingerprints is more accurate. And the scoring is carried out according to the result, so that the correct fingerprint sample is guaranteed to be screened out.

Drawings

FIG. 1 is a frame diagram of an overall algorithm of a large-scale fingerprint retrieval method based on point cloud registration provided by the invention;

FIG. 2 is a flow chart of a preliminary screening layer;

fig. 3 is a fine registration layer flow diagram;

FIG. 4 is a diagram of robust optimization layer input data;

FIG. 5 is a robust optimization layer flow diagram;

FIG. 6 is a schematic diagram of matching point pairs;

FIG. 7 is a graph of edge consistency.

Detailed Description

The invention is described in detail below by way of example with reference to the accompanying drawings.

The invention provides a layered robust large-scale fingerprint retrieval algorithm based on feature descriptors and geometric consistency, which can efficiently and accurately retrieve input fingerprints to be retrieved to obtain a fingerprint library sample set most possibly having the same matching relation. The algorithm can be divided into three layers as a whole: a preliminary screening layer, a fine registration layer, and a robust optimization layer. The three layers are interdependent, serially processed, screened layer by layer, and lack of one is not enough, so that a set of complete fingerprint retrieval algorithm is formed together. The overall framework of the algorithm is as shown in fig. 1, after a fingerprint to be retrieved is input, a primary screening layer completes rapid large-scale primary screening by using global features, a fine registration layer realizes further screening by using local registration and provides a matching relation between minutiae for a robust optimization layer, the robust optimization layer judges whether the matching relation of the minutiae is correct or incorrect according to geometric consistency test and a maximum clustering algorithm, and outputs a retrieval result by using comprehensive matching scores.

Fig. 1 shows an overall algorithm framework of a large-scale fingerprint retrieval method based on point cloud registration, which includes the following three layers according to a data flow sequence:

primary screening layer: before on-line retrieval, fingerprint samples in the fingerprint database are input into a DAA global feature descriptor extraction module, DAA features of the fingerprint samples are extracted, and the DAA features refer to distance-angle-area features, so that an off-line fingerprint feature database with a simpler storage format is obtained. And after the retrieval is started, inputting the fingerprint to be retrieved into a DAA global feature descriptor extraction module to obtain the features of the current fingerprint to be retrieved. This feature is used to compare sample features in the off-line fingerprint feature library, followed by a simple numerical operation to obtain a similarity score for all samples in the library. Finally, the fingerprint sample ID with higher score is transmitted to the fine matching layer after sorting. Where the fingerprint samples themselves are stored in minutiae form.

Fine matching layer: and inputting the screening result of the primary screening layer, and performing improved ICP matching. Firstly, random relative translation is carried out on a fingerprint to be retrieved and a fingerprint sample after primary screening in a fingerprint database, and an optimal initial state is found by calculating a registration score after translation, which mainly aims at the condition that the translation amount of the sample with the same matching relation is large. And then, carrying out iterative optimization by utilizing an ICP (inductively coupled plasma) algorithm to further determine the transformation relation between the fingerprint to be retrieved and all fingerprint samples, wherein the transformation relation is used for processing the condition of large rotation amount. And calculating to obtain a corresponding objective function value by utilizing the registration result of ICP, determining the matching relation between the detail nodes through nearest neighbor search and matching, and simultaneously using the objective function value to complete further screening.

Robust optimization layer: and (3) performing geometric consistency check on the input minutiae matching relationship, judging the quality of the matching relationship between the minutiae according to a maximum clustering algorithm, and filtering out wrong minutiae matching pairs (most of the minutiae matching pairs). And calculating to obtain the final comprehensive matching score of the sample according to the correct matching point pair occupation ratio and the objective function value given by the fine matching layer. And sorting by using the score, and realizing screening of any filtering level to obtain a final screening result. In fact, the optimal result obtained by screening is basically the fingerprint samples with the same matching relationship.

First layer-preliminary screening layer

Fig. 2 shows the algorithm flow of the preliminary screening layer in the present invention. The function of the layer is to utilize the advantage of fast query of the off-line library to preliminarily filter fingerprint samples which are least likely to have the same relation, so as to improve the time efficiency as much as possible.

Two-dimensional point cloud global feature descriptor DAA

In order to realize fingerprint retrieval efficiently, the fingerprints in the database need to be initially screened by using global features. The minutiae of the fingerprint can be regarded as two-dimensional point cloud according to the coordinates (the directions of the minutiae are different due to rotation, so that the method is not considered in the link). The descriptors of point clouds can be roughly divided into two main categories: global feature descriptors and local feature descriptors. The local feature descriptors are generally used for online matching of point clouds, are suitable for online matching of fingerprints, and occupy large storage space; the global feature descriptors represent global information of the fingerprints, the storage is convenient, and the off-line matching of the fingerprints is suitable, so that the global feature descriptors are selected for the layer.

There are many global feature descriptors that can be used for point clouds, such as VFH, CVFH, ESF, GFPFH, etc. However, these algorithms are based on three-dimensional point clouds and do not focus on the feature descriptors of two-dimensional point clouds. The layer is subjected to dimension reduction treatment on the basis of ESF, a global feature descriptor DAA (Distance-Angle-Area) suitable for two-dimensional point cloud is designed, features of all minutiae are expressed, and statistics is carried out in a form of a combined histogram of 3 multiplied by 64 columns. Because the number of minutiae extracted per fingerprint image is on average about 20-40, and the number is small, the DAA traverses all minutiae to ensure the accuracy of the extracted features to the maximum extent. The meaning of DAA in the present invention is specifically:

a. distance between two adjacent plates

The distance describes information of the distance between any two minutiae points. This range is split into 64 segments with the minimum and maximum distances as boundaries to count the information for all distances D. Every time a distance is obtained, the counting value of the corresponding section is added by one, and finally the distance is normalized, so that a histogram of distance information can be obtained.

b. Angle of rotation

The angle is defined as the angle between a minutia and the line connecting two other minutiae, and the range is set to [0, π ]. For statistical convenience, a cosine function that is monotonic in this range may be used instead. Therefore, the final value range is clamped to (-1,1.) similarly, the final value range is divided into 64 sections, the resolution of each section is 0.032, all included angles are counted and added in the corresponding section, and finally normalization processing is carried out.

c. Area of

Due to the stable structure of triangles, the method is often used for the task of searching fingerprints. In view of this, DAA also uses the area of a triangle constructed by any three minutiae as feature information. The area of the triangle has a certain range by preliminary estimation. It was divided into 64 segments and counted using the same method as described above.

The three pieces of characteristic information all have strong translation invariance and rotation invariance, are very suitable for the characteristic extraction work of fingerprint minutia data, and the angle A ¹ Having robustness to length, A ² Has robustness to length and angle. Can still well perform in the case of elastic deformation of fingerprintsAnd (6) processing. In addition, a histogram form is used in statistical information, so that not only can robustness on characteristic information be maintained, but also information of fingerprint nodes can be more comprehensively reflected compared with a method using index values. DAAs therefore have great potential in extracting minutiae features of fingerprints.

Combining the histograms obtained from the above three features to obtain the DAA feature of each fingerprint sample.

Offline fingerprint feature database construction

To enable fast retrieval of fingerprints, those least likely fingerprint samples may be initially filtered using global features. An off-line fingerprint database may well fulfill this requirement. And constructing all fingerprint samples of the database by using the global feature sub DAA, and representing by using a combined histogram of the DAA instead of storing the sample point information of each fingerprint.

Each fingerprint is pre-processed by using DAA as a feature descriptor to derive its features. Combining these features results in an offline fingerprint feature database. The pre-treatment step is performed completely off-line and is disposable, possibly taking a relatively long time. However, the time cost spent in the process is high, the difficulty of primary screening of the online fingerprints is reduced, and the online fingerprint screening can be almost finished in real time.

On-line fingerprint feature preliminary screening

After the database construction is completed, the database can be used as a comparison baseline to find a similar set of fingerprints to be retrieved. This link is done on-line, and for the fingerprint to be retrieved which is conveyed in, the characteristics of the fingerprint are extracted by using DAA. And comparing the extracted features with an offline feature library one by one, wherein the problem of similarity measurement is involved.

The Manhattan distance (Manhattan distance) is used as an index for measuring the difference of DAA feature descriptors, so that the similarity calculation formula is as follows:

wherein

Data representing the pth column of fingerprint DAA with ID 1, and S is the benchmark maximum score. By using the method, the fingerprint to be searched is compared with all the fingerprint characteristics in the library one by one, and the similarity between the fingerprint and the fingerprint characteristics is calculated. The higher the value, the greater the probability of representing belonging to the same fingerprint. All scores are sorted and screened using the TopN (top N) algorithm to filter out those fingerprints that are least likely to match.

Second layer-Fine registration layer

Fig. 3 shows the algorithm flow of the fine registration layer. The layer has the functions of performing fingerprint detail point nearest neighbor matching based on improved ICP according to the screening result of the primary screening layer, determining the matching relation among detail points and the objective function value to complete further screening, and outputting the result to the next layer for retrieval.

Fingerprint initial relative registration position optimization

After the screened matched fingerprint sample and the fingerprint to be retrieved obtained by the primary screening layer processing are input, the fingerprint to be retrieved and the screened matched fingerprint sample are registered, the matching relation among fingerprint nodes is found, and whether the fingerprint is the same or not is determined. However, because the number of minutiae extracted from each fingerprint is different, the number of minutiae having the same relationship is also different. Therefore, determining the registration direction is advantageous for improving the registration accuracy. The fingerprint with more minutiae is used as a reference fingerprint, and the fingerprint with less minutiae is used as a matching fingerprint for registration, so that the registration time can be shortened, and the registration accuracy can be improved.

Meanwhile, due to the fact that rotation and translation relations exist among the fingerprint images, rotation and translation also exist among the corresponding extracted minutiae features, if the initial relative registration position error of the fingerprint minutiae features is large, registration time and algorithm complexity are increased additionally. Therefore, it is important to optimize the initial relative registration position of the fingerprints to have a small cost. The layer adopts a position random translation strategy, randomly translates the positions of the matched fingerprint minutiae sets in four directions of up, down, left and right within a certain area range by a given step length, calculates the nearest neighbor minutiae matching pair cost between the matched fingerprint and the reference fingerprint, and selects the position with the lowest cost as the initial relative registration position of the fingerprint. The method comprises the following specific steps:

firstly, randomly translating the positions of the matched fingerprint minutiae sets, and calculating the least square distance from each matched fingerprint minutia to the nearest neighbor reference fingerprint minutia after translation, namely

Where M is a set of matched fingerprint minutiae, Q is a set of reference fingerprint minutiae, M _i For the ith matched fingerprint minutiae, q _i For reference fingerprint minutiae middle distance m _i Plane Euclidean distance closest point, T _ξ Is a random translation matrix, n _M To match the number of fingerprint minutiae, n _Q For reference to the number of fingerprint minutiae, D (M, Q, T) _ξ ) Random translation T for matching fingerprint minutiae sets _ξ The least squares distance of each subsequent matched fingerprint minutia to its nearest reference fingerprint minutia.

Meanwhile, counting the number of the minutiae pairs smaller than a certain distance range according to the Euclidean distance between nearest neighbor minutiae. Taking the number of the minutiae pairs in a certain distance range and the least square distance of the nearest neighbor minutiae as the cost of the nearest neighbor minutiae matching pairs, wherein the number of the minutiae pairs in the certain distance range is less, and the cost is lower; if the number of the minutiae pairs in a certain distance range is the same, the smaller the least square distance of the nearest neighbor minutiae is, the lower the cost is.

Within a certain area range, conducting xi-order given step length and non-repeated random translation on the matched fingerprint minutiae set, selecting a translation matrix with the lowest cost as a translation matrix T of the initial registration position of the matched fingerprint _init The initial registration position of the reference fingerprint is unchanged:

the relation of each minutia may be disordered before the registration position is optimized, but after the registration position is optimized, the matching relation among the fingerprint minutiae can be enhanced, the cost of the nearest neighbor minutiae matching pair between the matching fingerprint and the reference fingerprint is effectively reduced,

fingerprint minutiae nearest neighbor search and matching

The above steps perform a rough optimization on the translation relationship between the fingerprint images, and the optimization on the rotation and fine translation relationship between the fingerprint images is completed in the present step. Firstly, the initial relative registration position optimization of the fingerprints only considers most minutiae in the matched fingerprints, outliers may exist in the matched fingerprints, and the outliers in the matched fingerprints are filtered out, so that the correct matching of the subsequent minutiae is facilitated. The outlier minutiae filtering strategy employed is: and deleting minutiae in the matched fingerprint of which the distance between nearest neighbor minutiae pairs in the optimized initial relative registration position of the fingerprint is greater than a certain threshold, and only keeping the matched fingerprint minutiae in the threshold range. Meanwhile, before filtering the outlier minutiae, if the number of the minutiae pairs in a certain distance range is too small or the least square distance of the nearest neighbor minutiae is too large, the fingerprint to be matched is filtered, and the subsequent outlier minutiae filtering and the nearest neighbor matching and searching of the fingerprint minutiae are not participated in, so that the time complexity of the layer is reduced.

Next, the nearest neighbor matching and searching of the reference fingerprint and the matched fingerprint minutiae is realized by using a two-dimensional ICP (iterative closest point) method, which includes the following steps:

set minutiae sets in the matched fingerprints after filtering outliers as

Construct an objective function of

Wherein, R is a rotation matrix, T is a translation matrix, and an Euclidean transformation is formed together.

The centroids of the two groups of points P and Q are defined as:

subsequently, the following process is performed on the formula (4):

due to (q) _i -q-R(p _i P)) is zero after summation, the objective function can be simplified to

Therefore, based on the simplified objective function of equation (7), the ICP solution step is:

a. the centroid positions of the two sets of points are calculated, and then the centroid-removed coordinates of each point are calculated:

p′ _i ＝p _i -p，q′ _i ＝q _i -q (8)

b. the rotation matrix is calculated according to the following optimization problem:

c. calculating T from R of the above formula:

T ^* ＝q-Rp (10)

for the solution of the rotation matrix R, the expansion (9) can be derived

Wherein the content of the first and second substances,

independently of R, R ^T R is 0, the actual optimized objective function is

The matrix W is defined as:

carrying out SVD on W to obtain

W＝U∑V ^T (14)

Where Σ is a diagonal matrix composed of singular values, diagonal elements are arranged from large to small, and U and V are diagonal matrices. When W is of full rank, R is

R＝UV ^T (15)

After R is solved, T can be solved according to the formula (10), and if the determinant of R is negative at the moment, R is taken as an optimal value.

And performing pose transformation on P by using the obtained R and T, if the error is greater than the threshold, performing iteration until the iteration times reach the threshold or the error is less than the threshold, obtaining the nearest neighbor matching result of the minutiae of the matched fingerprint and the reference fingerprint, and finally outputting the matching relation between the minutiae within the threshold range and the objective function value to a robust optimization layer for further retrieval.

And a third layer: robust optimization layer

FIG. 4 illustrates a robust optimization layer input data diagram. The input of the robust optimization layer is the output of the nearest neighbor matching of the previous layer (fine registration layer), namely the retrieval result of each ID fingerprint after being filtered by the previous layer, wherein each retrieval result of a certain ID fingerprint is composed of matching point pairs of a plurality of minutiae points. The schematic diagram of the input of the layer is shown in the following figure. The layer is used for further filtering the fingerprint sample retrieval result output by the fine registration layer and ensuring high retrieval precision, so that the high penetration rate of fingerprint retrieval can be ensured under different filtering levels. The flow chart of this layer is shown in fig. 5.

Geometric consistency

The minimum data unit input into the layer is a plurality of minutiae matching point pairs of two fingerprints, and the layer starts from the geometric consistency of the minutiae matching point pairs and judges the probability of correct matching of the two fingerprints. And then sequencing the probability values according to the sizes, and selecting the fingerprint corresponding to the probability value of TopN as a final output result according to the requirement of the filtering level.

Let u1, u2, u3 correspond to the matching pair of minutiae points being correct, and u4 corresponds to the matching pair of minutiae points being incorrect. Then, a relationship can be given in which | a '-c' | represents the length of a two-point line connecting a 'and c', and the remaining equations have similar meanings.

|a'-c'|≈|a-c| (16)

|b'-c'|≈|b-c| (17)

|a'-b'|≈|a-b| (18)

This is because for a line segment where any two points in a two-dimensional point set are connected, any rotation and translation of the point set in the plane will not change the length of the line segment. Therefore, if a correct matching point pair is generated between two point sets, these matching point pairs necessarily satisfy the geometric relationships like equations (16) to (18). For u4, the probability that the geometric relationship expressed by equation (19) holds for an incorrect matching point pair is not 0, but the probability value is small.

|d'-c'|≈|d-c| (19)

In general, taking u1 and u2 as an example, if the matching point pair corresponding to u1 and u2 satisfies the geometric relationship of equation (16), u1 and u2 are said to have "geometric consistency"; on the contrary, since the matching point pair corresponding to u1 and u4 does not satisfy the geometric relationship of the similar equation (16), u1 and u4 are said not to have "geometric consistency".

The above-mentioned "geometric consistency" applies equally to the minutiae directions, since the minutiae directions are related to the texture of the fingerprint of the finger, even if there is translation and rotation when the same finger enters the fingerprint twice,but the directions of the same minutiae extracted twice are not changed greatly basically. Suppose that the minutiae point direction of the minutiae point a is defined as θ _a The orientation definitions of the other minutiae points are similar, and the following geometrical relationships between the minutiae point orientations can be obtained:

|θ _a' -θ _c' |≈|θ _a -θ _c | (20)

|θ _b' -θ _c' |≈|θ _b -θ _c | (21)

|θ _a' -θ _b' |≈|θ _a -θ _b | (22)

considering that there may be dust on the finger, and when a fingerprint is recorded, the finger is too soft and is easy to deform, and the like, the direction or position of a small number of minutiae points is different from the actual direction or actual position of the minutiae points in the fingerprint, so it is not enough to consider whether the matching pairs of two minutiae points have "geometric consistency", but rather, the comprehensive consideration of "geometric consistency" among a plurality of pairs of matching points is needed.

The "geometric consistency" between pairs of matching points is defined as follows. Also taking u1, u2, and u3 as an example, when each pair of matching points of u1, u2, and u3 has a relationship as shown in formula (16), u1, u2, and u3 are said to satisfy "geometric consistency". The geometric consistency between multiple pairs of matching point pairs plays an important role in the layer, and for two fingerprints with the same relation, if the two fingerprints do not have too large fingerprint difference and minutiae difference, more correct matching point pairs can be constructed between the minutiae of the two fingerprints in the fine matching layer, and a larger number of matching point pairs have the geometric consistency relation with each other; on the contrary, if there are two fingerprints without the "same" relationship, there will be a small number of matching point pairs having a "geometric consistency" relationship with each other in the fine matching layer although there is a small probability that matching point pairs between minutiae satisfying the relationship of equation (16) are constructed. It is thus possible to better distinguish whether two fingerprints have an "identical" relationship.

Maximum clique algorithm

In the following, it is assumed that two are givenMatching point pairs of all minutiae points of the fingerprint are solved, and the maximum number of the multiple pairs of matching point pairs in the geometric consistency relation between the multiple pairs of matching point pairs is met. This is illustrated in schematic form by a small number of dot and edge representations, as shown in fig. 6. The circles in FIG. 6 represent minutiae points in two fingerprints, denoted by a, b, c and a ', b', c ', d', respectively, and θ _a The direction of the minutiae point a is shown, and the mathematical representation of the directions of other minutiae points is similar; the line segments are connecting lines after the minutiae matching, are respectively denoted by u1, u2, u3, u4 and u5, and are called edges. Whether or not there is a "geometric consistency" relationship between the edges is judged by the following equation (23). If a ', b' and a, b satisfy equation 23, the edges u1, u2 are said to have a "geometric consistency" relationship. In formula (23)

And σ is a threshold value other than 0, in the sense that: it is ensured that when there is a difference in fingerprints between two presses of the same finger, correctly matching pairs of fingerprint minutiae are still identified.

As a result of the determination of equation (23), matching pairs respectively represented by the sides u1, u2, and u4 in fig. 6 have a "geometric consistency" system, and matching pairs respectively represented by the sides u3 and u5 have a "geometric consistency" relationship, that is, a "geometric consistency" relationship between different multi-matching pairs exists in the above figure. Thus, the "geometric consistency" relationship between different multi-matched pairs cannot be established simultaneously, so that there must be a pair of mismatched minutiae.

Therefore, a consistency graph related to the edges u1, u2, u3, u4 and u5 can be constructed, and the subsequent maximum clique algorithm solution is facilitated, as shown in FIG. 7. Wherein the blue circles u1, u2, u3, u4, u5 correspond to the 5 edges in fig. 6, and the black line segment indicates that there is a "geometric consistency" relationship between the two edges. For example, line segments are connected between u1 and u2, u2 and u4, and u1 and u4 respectively, which indicate that the edges have a "geometric consistency" relationship between each other. If two of the blue circles u1, u2, u3, u4, u5 have no connecting line segment between them, it means that there is no "geometric consistency" relationship between the two edges. For example, u2 and u3, and u3 and u4 have no connecting line segment therebetween. Therefore, fig. 7 and fig. 6 have a correspondence relationship related to "geometric consistency". FIG. 7 is a graph of edge consistency. In fig. 7, there are multiple pairs of edges in the "geometric consistency" relationship, and two subsets of two edges, i.e., two subsets u1, u2, u4 and u3, u5, are formed, and the "geometric consistency" relationship of the two subsets may not be true, otherwise there should be only one subset. Therefore, it is necessary to find the subset satisfying the "geometric consistency" relationship with the largest number of edges through an algorithm and remove the erroneous subset with the smaller number of edges.

Firstly, a weight value is given to each line segment to enable the line segment to become a weighted undirected graph, then an adjoint matrix of the weighted undirected graph is constructed, then a maximum clustering algorithm is used for solving the matrix, and whether each edge is correct or not is judged.

The line segments are given a weight of 1 to form a 5 × 5 matrix M as shown in formula (24), and the rows and columns of the matrix sequentially represent u1, u2, u4, u3, and u5, respectively. When the edges ui and uj are connected by a wired segment, the corresponding matrix elements M (i, j) =1 and M (j, i) =1; on the contrary, when the edges ui and uj are not connected by a line segment, the corresponding matrix elements M (i, j) =0 and M (j, i) =0. The diagonal element M (i, i) =1 of the matrix, which is seen to be a symmetric matrix.

Up to this point, the minutiae point diagram in which there is a matching relationship between the two fingerprints is converted into the adjoint matrix in fig. 7 and equation (24). The subset of the most numerous edges of consistency can be found by solving the problem represented by equation (25) below.

The optimization variable u is a binary vector with the length of n (the value of each element can only be 0 or 1), when the edge corresponding to a certain element in u has consistency, the element is 1, otherwise, the element is 0. Since u is binary and subject to the constraint that when M (i, j) =0, uiuj =0, ui and uj should not occur simultaneously in the coherency edge of the final output.

Since the elements in the M matrix are 1 or 0 and are symmetric matrices, the problem represented by equation (25) can be transformed into the classical maximum clique problem as equation (26):

in the example, u = [1,1,0,1,0 is finally output] ^T The subset of edges for which u1, u2, u4 in fig. 7 are solved for consistency is shown, while u3, u5 are discarded. This result also indicates that a and a ', b and b ', c and c ' in fig. 6 are correctly matched pairs of minutiae, whereas b and c ', c and d ' are not correctly matched pairs of minutiae. It can therefore be calculated that the proportion of correctly matched pairs of minutiae in fig. 6 to all matched pairs of minutiae is:

scoring function and TopN ranking

And then, scoring and sequencing the fingerprint retrieval results of the same fingerprint ID in the fingerprint database, so as to output corresponding retrieval results according to different filtering levels required by the sequencing results and the contest questions, and count and complete the calculation of the fingerprint penetration rate under different filtering levels. And outputting the objective function value of nearest neighbor iteration in each fingerprint retrieval result at the fine matching layer.

Defining a scoring function score (i) whose score for a fingerprint search result with ID i is calculated as follows:

the above equation shows that the smaller the average cost of each matched minutiae pair, the larger the proportion of correctly matched minutiae pairs, the smaller the score, and the higher the probability that the fingerprint retrieval result is the result of 'same finger'. And when TopN sorting is carried out, the scores of all fingerprint retrieval results are sorted from large to small. And selecting a certain number of fingerprint retrieval results from the tail part of the sequence as output according to the requirement of the filtering level.

In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (5)

1. A large-scale fingerprint retrieval method based on point cloud registration is characterized by comprising the following steps:

the method comprises the following steps: before online retrieval, inputting a fingerprint sample in a fingerprint library into a primary screening layer, inputting the fingerprint sample in the fingerprint library into a DAA global feature descriptor extraction module by the primary screening layer, extracting DAA features of the fingerprint sample, and storing the DAA features as sample features into an offline fingerprint feature library; after the retrieval is started, the fingerprint to be retrieved is input into a primary screening layer, wherein a DAA global feature descriptor extraction module extracts DAA features of the current fingerprint to be retrieved, compares sample features in an offline fingerprint feature library to obtain similarity scores of the DAA features of the current fingerprint to be retrieved and the DAA features of all samples, sorts the sample features from high to low according to the similarity scores, selects fingerprint samples corresponding to the first N sample features as the output of the primary screening layer, and sends the fingerprint samples into a fine registration layer;

step two: the fine registration layer carries out random relative translation on the fingerprint to be retrieved and the screened fingerprint samples, finds out the optimal initial state by calculating the registration scores after translation, then carries out iterative optimization by utilizing an ICP (inductively coupled plasma) algorithm, determines the transformation relation between the fingerprint to be retrieved and all the fingerprint samples, calculates to obtain the corresponding objective function value by utilizing the registration result of the ICP, and determines the matching relation between the minutiae points by a nearest neighbor search and matching method; the objective function value and the matching relation between the detail nodes are sent to a robust optimization layer;

step three: the robustness optimization layer uses geometric consistency check for the matching relationship among the detail nodes, then judges the quality of the matching relationship among the detail nodes according to a maximum clustering algorithm, and filters wrong detail point matching pairs; calculating to obtain a final comprehensive matching score of the fingerprint sample according to the proportion of the correct minutiae matching pair and an objective function value given by the fine matching layer; and sequencing by utilizing the comprehensive matching score, and realizing screening of any filtering level to obtain a final screening result.

2. The method for large-scale fingerprint retrieval based on point cloud registration as claimed in claim 1, wherein the DAA global feature descriptor extraction module is used for extracting distance-angle-area DAA features;

the distance is used for describing distance information between any two minutiae; dividing the range into n segments by taking the minimum distance and the maximum distance as boundaries to count the information of all the distances D; every time a distance is obtained, adding one to the count value of the corresponding section, and finally carrying out normalization processing on the distance to obtain a histogram of distance information;

the angle is used for describing an included angle between one minutiae and a connecting line of other two minutiae, the range is set to be [0, pi ], a monotonous cosine function in the range is adopted for replacing, and the final value range is clamped to be (-1,1); splitting the final value domain into n sections, counting all included angles, adding the included angles in corresponding sections, and finally performing normalization processing to obtain a histogram of angle information;

the area is used for describing the area of a triangle constructed by any three minutiae, the area A of the triangle is obtained through preliminary estimation, the area range is divided into n sections, all the areas are counted and added in corresponding sections, normalization processing is carried out, and a histogram of area information is obtained;

and combining the histograms of the distance information, the angle information and the area information to obtain the DAA characteristic of each fingerprint sample.

3. The point cloud registration-based large-scale fingerprint retrieval method according to claim 1 or 2, wherein the comparison of the sample features in the offline fingerprint feature library to obtain the similarity scores of the DAA feature of the current fingerprint to be retrieved and all the DAA features of the sample is specifically:

the Manhattan distance is used as an index for measuring the DAA feature descriptor difference, and the calculation formula of the similarity score is as follows:

wherein DAA ₁ Is the DAA feature of the fingerprint sample of,

data representing the pth column of DAA features of a fingerprint sample, S being the benchmark maximum score; DAA ₂ The DAA characteristic of the current fingerprint to be retrieved;

and comparing the fingerprint to be retrieved with the DAA characteristics of all the fingerprint samples in the library one by one, and calculating the similarity.

4. The large-scale fingerprint retrieval method based on point cloud registration as claimed in claim 1 or 2, wherein the fine registration layer performs random relative translation on the fingerprint to be retrieved and the screened fingerprint sample, and finds the optimal initial state by calculating the registration score after translation, adopting the following specific steps:

selecting fingerprint samples with the number of minutiae exceeding a set threshold value A as reference fingerprints and selecting fingerprint samples with the number of minutiae below a set threshold value B as matching fingerprints from the fingerprint samples screened by the primary screening layer; the threshold B is less than the threshold A;

and (3) adopting a position random translation strategy, randomly translating the positions of the matched fingerprint minutiae set in four directions of an upper direction, a lower direction, a left direction and a right direction in a certain area range by a given step length, calculating the nearest neighbor minutiae matching pair cost between the matched fingerprint and the reference fingerprint, and selecting the position with the lowest cost as the initial relative registration position of the fingerprint.

5. The point cloud registration-based large-scale fingerprint retrieval method as claimed in claim 4, wherein in the second step, an ICP algorithm is used for iterative optimization, the transformation relation between the fingerprint to be retrieved and all fingerprint samples is determined, the corresponding objective function value is calculated by using the registration result of ICP, the matching relation between the minutiae points is determined by a nearest neighbor search and matching method, and the specific steps are as follows:

set minutiae sets in the matched fingerprints after filtering outliers as

The minutiae set after filtering the outliers in the reference fingerprint is

Defining the centroids of the two sets of points P and Q as P and Q, respectively:

then the simplified objective function is constructed as:

wherein R is a rotation matrix and T is a translation matrix;

based on the simplified objective function, the ICP solution step is:

step a. Calculating the centroid positions of the two sets of points, then calculating the centroid-removed coordinates p 'of each point' _i ，q′ _i ：

p′ _i ＝p _i -p，q′ _i ＝q _i -q

B, calculating an optimized rotation matrix R according to the following optimization problem ^* ：

The matrix W is defined as:

carrying out SVD on W to obtain

W＝U∑V ^T

Wherein, Σ is a diagonal matrix composed of singular values, diagonal elements are arranged from large to small, and U and V are diagonal matrices; when W is full rank, R is R = UV ^T

C, according to the optimized rotation matrix R ^* Computing an optimized translation matrix T ^* ：T ^* ＝q-R ^* p；

If then R ^* If the determinant is negative, then take-R ^* As an optimum value;

using the obtained R ^* And T ^* And (4) carrying out pose transformation on the P, if the error is greater than a set error threshold, carrying out iteration until the iteration times reach the iteration threshold or the error is less than the error threshold, obtaining a nearest neighbor matching result of minutiae of the matched fingerprint and the reference fingerprint, and finally outputting a matching relation between the minutiae within the threshold range and an objective function value to a robust optimization layer for further retrieval.

Images