WO2020047717A1

WO2020047717A1 - Multi-source fingerprint image enhancement and synthesis method and related fingerprint sensor

Info

Publication number: WO2020047717A1
Application number: PCT/CN2018/103829
Authority: WO
Inventors: 许志隆; 王浩任; 李宗德
Original assignee: 深圳市汇顶科技股份有限公司
Priority date: 2018-09-03
Filing date: 2018-09-03
Publication date: 2020-03-12
Also published as: CN109196523A

Abstract

A multi-source fingerprint image enhancement and synthesis method applied to a fingerprint sensor, comprising: obtaining a plurality of first local feature values of a first input fingerprint image and a plurality of second local feature values of a second input fingerprint image respectively; determining a spatial disparity relationship between the first input fingerprint image and the second input fingerprint image according to the plurality of first local feature values and the plurality of second local feature values; and synthesizing the first input fingerprint image and the second input fingerprint image according to the spatial disparity relationship to generate an output fingerprint image.

Description

Multi-light source fingerprint image enhancement and synthesis method and related fingerprint sensor

Technical field

The present application relates to a method for enhancing and synthesizing fingerprint images and related fingerprint sensors, and in particular, to a method for enhancing and synthesizing fingerprint images with multiple light sources and related fingerprint sensors.

Background technique

Invisible Fingerprint Sensor (IFS), Optical Fingerprint Sensor or UnderDisplay Fingerprint Sensor, etc., integrate the optical fingerprint sensor into the display screen, allowing users to identify fingerprints through the display screen. The applicant has noticed that if multiple light sources are used to capture optical fingerprint images separately, the quality of fingerprint images captured at different light source positions is inconsistent, thereby affecting the accuracy of fingerprint interpretation.

Therefore, how to provide a multi-light source fingerprint image enhancement and synthesis method and related fingerprint sensors has become one of the goals of the industry.

Summary of the Invention

Therefore, the main purpose of the present application is to provide a method for enhancing and synthesizing multi-light source fingerprint images and related fingerprint sensors.

In order to solve the above technical problems, an embodiment of the present application provides a method for enhancing and synthesizing a multi-light source fingerprint image for a fingerprint sensor, which is characterized in that it includes obtaining a plurality of first local features of a first input fingerprint image respectively. Value and a plurality of second local feature values of a second input fingerprint image; judging the first input fingerprint image and the second input fingerprint image based on the plurality of first local feature values and the plurality of second local feature values A spatial parallax relationship between the two; and synthesizing the first input fingerprint image and the second input fingerprint image to generate an output fingerprint image according to the spatial parallax relationship.

For example, a stereo matching is performed on the first input fingerprint image and the second input fingerprint image to generate the plurality of pixel stereo disparity estimates of the first input fingerprint image and the second input fingerprint image.

For example, calculating the global displacement reference vector of the first input fingerprint image and the second input fingerprint image according to the plurality of first local feature values and the plurality of second local feature values; and according to the global displacement reference vector, A plurality of pixels of the first input fingerprint image and the second input fingerprint image are compensated to generate a first compensated fingerprint image and a second compensated fingerprint image.

In order to solve the above technical problems, an embodiment of the present application further provides a multi-light source fingerprint sensor, which is characterized by comprising: a first light source; a second light source; and an image acquisition unit, wherein when the first light source is turned on and the When the second light source is off, the image acquisition unit acquires a first input fingerprint image; when the first light source is off and the second light source is on, the image acquisition unit acquires a second input fingerprint image; and an image processing unit, Coupled to the image acquisition unit and configured to perform a multi-light source fingerprint image enhancement and synthesis process on the first input fingerprint image and the second input fingerprint image to generate an output fingerprint image, the multi-light source fingerprint image enhancement and synthesis The process includes the steps of the method for enhancing and synthesizing the multi-light source fingerprint image.

When a multi-light source fingerprint sensor is used, this application can obtain the offset, rotation, scaling, etc. of multiple input fingerprint images by acquiring the local feature values of multiple input fingerprint images (such as the coordinate positions of the endpoints of multiple lines). Spatial parallax relationship. Synthesizing multiple input fingerprint images according to the spatial parallax relationship) can make the image quality of multiple input fingerprint images complementary to each other to improve the problem of uneven quality of fingerprint images caused by image shift and uneven light intensity.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a fingerprint sensor according to an embodiment of the present application.

FIG. 2 is a flowchart of a multi-light source fingerprint image enhancement and synthesis process according to the first embodiment of the present application.

FIG. 3 is a flowchart of a multi-light source fingerprint image enhancement and synthesis process according to a second embodiment of the present application.

detailed description

In order to make the purpose, technical solution, and advantages of the present application clearer, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

Please refer to FIG. 1, which is a schematic diagram of a fingerprint sensor 1 according to an embodiment of the present application. The fingerprint sensor 1 includes an image processing unit 10, an image acquisition unit 12, and light sources L1 and L2. The image processing unit 10 includes an image disparity estimation unit 100 and an image synthesis unit 102.

The light sources L1 and L2 can provide light sources with different incident angles to irradiate the fingers, and the image acquisition unit 12 can obtain a fingerprint image to generate input fingerprint images F1 and F2 corresponding to the light sources L1 and L2, respectively. For example, when the light source L1 is on and the light source L2 is off, the image acquisition unit 12 may obtain the input fingerprint image F1; when the light source L1 is off and the light source L2 is on, the image acquisition unit 12 may obtain the input fingerprint image F2. The image processing unit 10 is coupled to the image acquisition unit 12 and is configured to perform image processing on the input fingerprint images F1 and F2 to generate an output fingerprint image F_out.

The image parallax estimation unit 100 is coupled to the image acquisition unit 12 and is configured to perform image analysis on the input fingerprint images F1 and F2 to obtain local feature values corresponding to the input fingerprint images F1 and F2, respectively. The image synthesizing unit 102 is coupled to the image parallax estimating unit 100 and is configured to synthesize the input fingerprint images F1 and F2 according to the local feature values corresponding to the input fingerprint images F1 and F2 to generate an output fingerprint image F_out. In one embodiment, the local feature values are the recognition characteristics of the input fingerprint patterns F1 and F2, including the fingerprint pattern (for example: ring, bow, spiral, etc.) and the detailed features (for example, the starting point, end point, Join points, bifurcation points, short lines (isolated points, etc.).

When a plurality of light sources L1 and L2 are used to acquire a fingerprint image, because the incident angles and distances of the light sources are different, the images acquired by the image acquisition unit 12 at the same imaging position are different. That is, there is a spatial parallax relationship between the input fingerprint images F1 and F2, so that there is a spatial parallax relationship between the input fingerprint images F1 and F2 such as translation, rotation, and scaling.

In addition, when the light sources L1 and L2 are point light sources, the light intensity gradually decreases from the center of the light source, and the imaging position with a stronger light intensity can obtain a clearer image, while the imaging position with a weaker light intensity can obtain a more blurred image. image. Therefore, uneven light intensity will cause uneven image quality of the input fingerprint images F1 and F2.

Therefore, based on the spatial parallax relationship of the input fingerprint images F1 and F2, combining the two fingerprint images F1 and F2 can make the image quality of the two input fingerprint images F1 and F2 complementary to each other to improve image offset and uneven light intensity. The problem of uneven fingerprint image quality.

In an embodiment, the image processing unit 10 may perform a multi-light source fingerprint image enhancement and synthesis process 2 according to the input fingerprint images F1 and F2 to generate an output fingerprint image F_out. FIG. 2 is a flowchart of a multi-light source fingerprint image enhancement and synthesis process 2 according to an embodiment of the present application. The process 2 can be compiled into a program code and stored in a memory unit, and is used to instruct the image processing unit 10 to perform the following steps.

Step 21: Obtain multiple first local feature values of a first input fingerprint image and multiple second local feature values of a second input fingerprint image.

Step 22: Perform a stereo matching on the first input fingerprint image and the second input fingerprint image according to the plurality of first local feature values and the plurality of second local feature values to generate a first input fingerprint image and a second input fingerprint image. Of multiple pixel stereo disparity estimates.

Step 23: Perform a pixel-to-pixel superposition on the first input fingerprint image and the second input fingerprint image according to the estimated stereo disparity to generate an output fingerprint image.

In the process 2, the image parallax estimation unit 100 may respectively obtain a plurality of first local feature values of the first input fingerprint image F1 and a plurality of second local feature values of the second input fingerprint image F2 (step 21); The first local feature value and the plurality of second local feature values perform stereo matching on the first input fingerprint image F1 and the second input fingerprint image F2 to generate a first input fingerprint image and a second input fingerprint image. A plurality of pixel stereo disparity estimates (step 22). The image synthesizing unit 102 may perform pixel-to-pixel superposition on the first input fingerprint image F1 and the second input fingerprint image F2 according to the stereo disparity estimation value to generate an output fingerprint image F_out (step 23).

Specifically, according to the coordinate positions (i.e., local feature values) of multiple line end points in the two input fingerprint images F1 and F2, it is possible to know the offset, rotation, and scaling between the two input fingerprint images F1 and F2. Spatial parallax relationship (ie, stereo parallax estimates). Pixel-to-pixel superimposition (or point-to-point superimposition) of two input fingerprint images F1 and F2 according to the spatial parallax relationship can make the image quality of the two input fingerprint images F1 and F2 complementary to each other to improve image offset and light intensity Uneven quality of fingerprint images caused by unevenness.

In step 21, the method for obtaining the local feature values of the input fingerprint images F1 and F2 by the image parallax estimation unit 100 is as follows. Take a single input fingerprint image as an example: open a local feature window (size n * n); calculate the local feature The average value of a central window (size m * m, m <n) in the window is used as a reference value ref_value for calculating local eigenvalues. The average value of the reference value ref_value is to reduce the impact of noise, but Not limited to this; and generating a local eigenvector function F [p]. Each pixel in the local feature window can be represented by an index p, where f is the local feature value produced by the index p by the local feature vector function F, that is, F [p] = f. Therefore, through the above calculation, the image parallax estimation unit 100 can obtain the local feature vector functions F1 and F2 of the input fingerprint images F1 and F2 and the corresponding local feature values F1 [p] = f1, F2 [p] = f2, where The feature vector functions F1 and F2 can be used to describe the spatial disparity relationships of the two input fingerprint images F1 and F2, such as the offset, rotation, and scaling.

Assume that the size of the local feature window is 5 * 5 and includes pixels P (0) to P (24), and the size of the central window is 3 * 3 and includes 9 pixels. The image parallax estimation unit 100 can calculate the average value of the 9 pixels. (Ie, the reference value ref_value) and each pixel P (0) -P (24) in the local feature window, respectively, corresponding local feature values f (0) -f (24) are generated. For example, a comparison result is generated by comparing the size of each pixel P (0) -P (24) and the reference value ref_value to determine the distribution directionality of each pixel P (0) -P (24) in the local feature window. To generate a local feature vector F [p]. In different embodiments, the local feature vector function F is generated by pushing back and forth the pixel index p through the results of functions such as edge response, pixel value response, and the like. Those skilled in the art may determine the local feature vector according to circumstances. The function F is generated to describe different spatial parallax relationships.

In step 22, the method for obtaining the parallax estimation value between the input fingerprint images F1 and F2 by the image parallax estimation unit 100 is as follows: selecting one of the input fingerprint images F1 and F2 as a reference image; for each of the non-reference image Pixels, scan horizontally relative to the relative position of the reference image (for example, in Figure 1, if the input fingerprint image F1 on the left is a reference image, the input fingerprint image F2 on the right scans each pixel horizontally on the left; if The input fingerprint image F2 on the right is a reference image, then the input fingerprint image F1 on the right left scans each pixel horizontally to the right); calculate the local feature values F1 [p] = f1, F2 [of the input fingerprint images F1 and F2 [ p] = f2's difference Diff [p] = f1-f2; and select a minimum value from the difference Diff [p] = f1-f2 corresponding to all pixels as the parallax estimate between the input fingerprint images F1 and F2 value.

In step 23, the image synthesis unit 102 performs pixel-to-pixel superposition on the input fingerprint images F1 and F2 according to the parallax estimation value to generate an output fingerprint image F_out as follows: For each pixel p1 of the input fingerprint images F1 and F2 , P2, calculate the consistency parameters p1_const, p2_const corresponding to pixels p1, p2 to calculate the blend ratio W1, W2 corresponding to each pixel of the input fingerprint image F1, F2, where the sum of the blend ratios W1, W2 is equal to 1 (that is, W1 + W2 = 1); and perform pixel-to-pixel superposition on each pixel p1 and p2 of the input fingerprint images F1 and F2 according to the mixing ratios W1 and W2, where the result F_out (p) of the pixel-to-pixel superposition can be used as the following function Means, but is not limited to this.

F_out (p) = W1 * p1 + W2 * p2;

W1 = p1_const / (p1_const + p2_const);

W2 = (1-W1).

The consistency parameters p1_const and p2_const are used to describe the consistency of the fingerprints of the pixels p1 and p2 on the respective image input fingerprint images F1 and F2 with the fingerprint characteristics of the surrounding pixels. In most cases, the consistency parameter of the poor signal region will be relatively low, so the pixel with lower consistency parameter will have a lower mixing ratio when pixel-to-pixel superposition. For example, when p1_const = 20 and p2_const = 30, the image synthesizing unit 102 superimposes the pixels p1 and p2 with the mixing ratios W1 = 0.4 and W2 = 0.6, that is, F_out (p) = 0.4 * p1 + 0.6 * p2.

Therefore, through the process 2, the image processing unit 10 can perform multi-light source fingerprint image enhancement and synthesis according to the input fingerprint images F1 and F2, so that the image quality of the two input fingerprint images F1 and F2 is complementary to improve image offset and light intensity. Uneven quality of fingerprint images caused by unevenness.

FIG. 3 is a flowchart of a multi-light source fingerprint image enhancement and synthesis process 3 according to an embodiment of the present application. The process 3 can be compiled into a program code and stored in a memory unit, and is used to instruct the image processing unit 10 to perform the following steps.

Step 31: Obtain multiple first local feature values of a first input fingerprint image and multiple second local feature values of a second input fingerprint image.

Step 32: Calculate a global displacement reference vector of the first input fingerprint image relative to the second input fingerprint image according to the plurality of first local feature values and the plurality of second local feature values.

Step 33: Globally translate multiple pixels of the first input fingerprint image according to the global displacement reference vector to generate a first compensated fingerprint image.

Step 34: Determine a plurality of representative pixels from the plurality of pixels of the first compensated fingerprint image and the plurality of pixels of the second input fingerprint image to synthesize the first compensated fingerprint image and the second input fingerprint image to generate an output fingerprint. image.

In the process 3, the image parallax estimation unit 100 may obtain a plurality of first local feature values of the first input fingerprint image F1 and a plurality of second local feature values of the second input fingerprint image F2 (step 31); A first local feature value and a plurality of second local feature values are used to calculate a global displacement reference vector of the first input fingerprint image relative to the second input fingerprint image (step 32). Assuming that the first fingerprint image F1 is a non-fixed image and the second fingerprint image F2 is a fixed image, the image synthesis unit 102 may globally translate a plurality of pixels of the first input fingerprint image F1 according to a global displacement reference vector to generate a First compensated fingerprint image F1_comp (step 33); judging a plurality of representative pixels from a plurality of pixels of the first compensated fingerprint image F1_comp and a plurality of pixels of the second input fingerprint image F2 to synthesize the first compensated fingerprint image F1_comp and The second input fingerprint image F2 is used to generate an output fingerprint image F_out (step 34).

Specifically, the method in step 31 can refer to step 21 in FIG. 2. In step 32, the image parallax estimation unit 100 may calculate a global displacement reference vector F_ref according to a plurality of local feature values F1 [p] = f1, F2 [p] = f2 of the input fingerprint images F1 and F2.

In step 33, the image synthesis unit 102 may select one of the input fingerprint images F1 and F2 as a fixed image and the other as a non-fixed image; and according to the global displacement reference vector F_ref, the global translation is not fixed All pixels of the image are used to generate a compensated fingerprint image F_comp.

If the input fingerprint image F1 is a non-fixed image and the input fingerprint image F2 is a fixed image, the image synthesis unit 102 may globally translate all pixels of the input fingerprint image F1 according to the global displacement reference vector F_ref to generate a first compensated fingerprint image F1_comp . If the input fingerprint image F1 is a fixed image and the input fingerprint image F2 is a non-fixed image, the image synthesis unit 102 may globally translate all pixels of the input fingerprint image F2 according to the global displacement reference vector F_ref to generate a second compensated fingerprint image F2_comp . For example, assuming that the input fingerprint image F2 is a fixed image, and the global displacement reference vector of the input fingerprint image F1 relative to the input fingerprint image F2 is F_ref = (6,6), the image synthesis unit 102 according to the displacement reference vector F_ref = (6, 6) After global translation of all pixels of the input fingerprint image F1, the total average difference between the compensated fingerprint image F1_comp and all pixels of the input fingerprint image F2 (total averaged pixel difference) can be minimized.

In step 34, the image synthesis unit 102 determines a plurality of representative pixels from the plurality of pixels of the compensated fingerprint image F1_comp and the plurality of pixels of the input fingerprint image F2 to synthesize the compensated fingerprint image F1_comp and the input fingerprint image F2 to generate an output. The method of the fingerprint image F_out is as follows: Assume that the pixels p1_comp of the compensated fingerprint image F1_comp, the pixels p2 of the input fingerprint image F2, and the representative pixels p_out of the output fingerprint image F_out correspond to the same position (x, y). The local areas near p1_comp and p2 are searched in a small area, and one is selected from a plurality of pixels in the local area (for example, the pixel with the smallest difference is selected through a small area search) as the representative pixel p_out.

For example, suppose that the pixel p1 (0,0) of the input fingerprint image F1 is translated to the pixel p1_comp (6,6) of the compensated fingerprint image F1_comp through the displacement reference vector F_ref. Take a small range to search nearby to determine the representative pixel p_out. For example, the image synthesizing unit 102 may search within a range of positive and negative n unit pixels centered on the pixel p1_comp (6,6), for example, in a range of positive and negative 3 unit pixels (3,3) to (9,9) of n = 3 ) To search for a pixel with smaller error than the pixels p1_comp (6,6) and p2 (6,6) as the representative pixel p_out. That is, the error ERR_out of the pixel p_out and the pixel p2 of the input fingerprint image F2 (or the pixel p_comp of the compensation fingerprint image F1_comp) is not greater than the error ERR_comp of the pixel p2 of the input fingerprint image F2 and the pixel p_comp of the compensation fingerprint image F1_comp.

If the error between the pixel p1_comp (5,6) of the compensation fingerprint image F1_comp and the pixel p2 (6,6) of the input fingerprint image F2 is smaller, the image synthesis unit 102 may select the pixel p1_comp (5,6) or p2 (6, 6) As a representative pixel p_out. Alternatively, referring to step 23, the image synthesizing unit 102 may mix pixels p1_comp (5,6) and p2 (6,6) as the representative pixels p_out according to an appropriate ratio, but is not limited thereto.

Therefore, through the process 3, the image processing unit 10 can perform multi-light source fingerprint image enhancement and synthesis according to the input fingerprint images F1 and F2, so that the image quality of the two input fingerprint images F1 and F2 is complementary to improve image offset and light intensity. Uneven quality of fingerprint images caused by unevenness.

In other embodiments, the image synthesizing unit 102 may determine a mixing ratio of two or more input fingerprint images according to pixel brightness and contrast, and calculate a new synthesized pixel point to generate an output fingerprint image. For example, in step 34 of process 3, when the local representative pixel is selected from the second compensated fingerprint image F2_comp, the second compensated fingerprint image F2_comp may use a higher blend ratio and the first compensated fingerprint image F1_comp may use a lower blend ratio. To calculate a new composite pixel, but it is not limited to this.

In other embodiments, the number of input fingerprint images is not limited, and more than two input fingerprint images can be used for multi-source fingerprint image enhancement and synthesis. For example, when three light sources are used to acquire a fingerprint image, three input fingerprint images obtained by the three light sources may be used to enhance and synthesize multi-light source fingerprint images.

It is worth noting that the prior art uses a single image to enhance the original input image with a poor signal-to-noise ratio for image enhancement, which may enhance the noise together and cause subsequent fingerprint recognition errors. The multi-image synthesis step used in this application can retain the better resolution part and fingerprint details in the fingerprint image. Compared with the single-image SNR, the composite image has a better SNR, which can reduce subsequent fingerprint recognition. The burden of the module can also reduce the error rate of continued fingerprint recognition.

In summary, this application obtains the local feature values of multiple input fingerprint images (such as the coordinate positions of the endpoints of multiple lines), and can know the spatial disparity relationships such as offset, rotation, and scaling between multiple input fingerprint images. Synthesizing multiple input fingerprint images according to the spatial parallax relationship) can make the image quality of multiple input fingerprint images complementary to each other to improve the problem of uneven quality of fingerprint images caused by image shift and uneven light intensity.

The above description is only part of the embodiments of this application, and is not intended to limit this application. Any modification, equivalent replacement, and improvement made within the spirit and principles of this application shall be included in the protection scope of this application. within.

Claims

A multi-light source fingerprint image enhancement and synthesis method for a fingerprint sensor is characterized in that it includes:

Acquiring a plurality of first local feature values of a first input fingerprint image and a plurality of second local feature values of a second input fingerprint image, respectively;

Determining a spatial parallax relationship between the first input fingerprint image and the second input fingerprint image according to the plurality of first local feature values and the plurality of second local feature values; and

According to the spatial parallax relationship, the first input fingerprint image and the second input fingerprint image are synthesized to generate an output fingerprint image.
The method of claim 1, wherein the fingerprint sensor comprises a first light source, a second light source, and an image acquisition unit, wherein when the first light source is on and the second light source is on When the light source is off, the image acquisition unit acquires the first input fingerprint image; when the first light source is off and the second light source is on, the image acquisition unit acquires the second input fingerprint image.
The method for enhancing and synthesizing a multi-light source fingerprint image according to claim 1, characterized in that a plurality of first local feature values of the first input fingerprint image and a plurality of second local features of the second input fingerprint image are respectively obtained. The steps of eigenvalues include:

Respectively opening a first local feature window and a second local feature window to calculate a first reference value corresponding to the first local feature window and a second reference value corresponding to the second local feature window; and

Generate a first local feature vector function based on the first reference value and a plurality of pixels of the first input fingerprint image, and generate a second local feature vector function based on the second reference value and a plurality of pixels of the second input fingerprint image Local eigenvector function

The pixels of the first input fingerprint image can generate the first local feature values through the first local feature vector function, and the pixels of the second input fingerprint image can be generated through the second local feature vector function. The plurality of second local feature values are generated.
The method for enhancing and synthesizing a multi-light source fingerprint image according to claim 3, wherein the first reference value is an average value of a plurality of pixels of a first central window within the first local feature window, and the first The two reference values are an average of a plurality of pixels of a second central window in the second local feature window.
The method for enhancing and synthesizing a multi-light source fingerprint image according to claim 3, wherein the spatial disparity relationship is a plurality of pixel stereo disparity estimates of the first input fingerprint image and the second input fingerprint image, and is based on The step of determining the spatial parallax relationship between the first input fingerprint image and the second input fingerprint image by the plurality of first local feature values and the plurality of second local feature values includes:

Selecting one of the first input fingerprint image and the second input fingerprint image as a reference image and the other as a non-reference image; and

For each pixel of the non-reference image, in the horizontal direction of the relative position of the reference image, according to the first local feature vector function and the second local feature vector function, the plurality of first local feature values and the A plurality of difference values of the plurality of second local feature values to calculate the plurality of pixel stereo disparity estimation values;

The multiple pixel stereo disparity estimates are used to describe the first input fingerprint image

And the plurality of pixel stereo disparity estimates of the second input fingerprint image.
The method for enhancing and synthesizing a multi-light source fingerprint image according to claim 5, wherein the estimated value of the stereo disparity of the plurality of pixels is a minimum value of the plurality of differences.
The method of claim 5, wherein the step of synthesizing the first input fingerprint image and the second input fingerprint image to generate the output fingerprint image according to the spatial parallax relationship includes: :

Calculating a plurality of first consistency parameters corresponding to the plurality of pixels of the first input fingerprint image and a plurality of second corresponding to the plurality of pixels of the second input fingerprint image according to the plurality of pixel stereo disparity estimated values A consistency parameter to calculate a plurality of first blending ratios and a plurality of second blending ratios corresponding to the plurality of pixels of the first input fingerprint image and the second input fingerprint image; and

Performing a pixel-to-pixel superimposition of the first input fingerprint image and the second input fingerprint image according to the plurality of first mixing ratios and the plurality of second mixing ratios to generate the output fingerprint image;

The plurality of first blending ratios and the plurality of second blending ratios are used to describe the respective numbers of pixels of the first input fingerprint image and the plurality of pixels of the second input fingerprint image on respective input fingerprint images. The fingerprint is consistent with the fingerprint characteristics of surrounding pixels;

The sum of the plurality of first mixing ratios and the plurality of second mixing ratios is equal to 1.
The method of claim 3, wherein the spatial disparity relationship is a global displacement reference vector of the first input fingerprint image and the second input fingerprint image, and according to the multiple The first local feature values and the plurality of second local feature values, and the step of judging the spatial parallax relationship between the first input fingerprint image and the second input fingerprint image includes:

Calculating the global displacement reference vector according to the first local feature values of the first input fingerprint image and the second local feature values of the second input fingerprint image;

Selecting one of the first input fingerprint image and the second input fingerprint image as a fixed image and the other as a non-fixed image; and

Translate a plurality of pixels of the non-fixed image globally to generate a compensated fingerprint image according to the global displacement reference vector;

The total average difference between the pixels of the compensated fingerprint image and the pixels of the fixed image is the smallest.
The method of claim 8, wherein the method further comprises judging a plurality of representatives from the plurality of pixels of the compensated fingerprint image and the plurality of pixels of the fixed image. Pixels to synthesize the compensated fingerprint image and the fixed image to generate the output fingerprint image, including:

Perform a small-area search in a local area corresponding to one of the plurality of pixels of the compensated fingerprint image and one of the plurality of pixels of the fixed image, respectively, and select one from the plurality of pixels in the local area to Respectively as the plurality of representative pixels;

The local area is a positive and negative n unit pixel range at the center of one of the plurality of pixels of the compensated fingerprint image, and n is a positive integer greater than 0;

An error between the plurality of representative pixels and the fixed image or the plurality of pixels of the compensation fingerprint image is not greater than an error between the plurality of pixels of the fixed image and the plurality of pixels of the compensation fingerprint image.
The method of claim 9, wherein the step of synthesizing the compensated fingerprint image and the fixed image to generate the output fingerprint image comprises:

Calculating a plurality of first mixing ratios and a plurality of second mixing ratios corresponding to the plurality of pixels of the compensation fingerprint image and the fixed image according to the pixel brightness and contrast of the compensation fingerprint image and the fixed image; and

A pixel-to-pixel overlay is performed according to the plurality of first mixing ratios of the compensated fingerprint image and the plurality of second mixing ratios of the fixed image to generate the output fingerprint image.
A multi-light source fingerprint sensor is characterized in that it includes:

A first light source;

A second light source;

An image acquisition unit, wherein when the first light source is on and the second light source is off, the image acquisition unit obtains a first input fingerprint image; when the first light source is off and the second light source is on, the image acquisition unit Acquiring a second input fingerprint image; and

An image processing unit coupled to the image acquisition unit is used to perform a multi-light source fingerprint image enhancement and synthesis process on the first input fingerprint image and the second input fingerprint image to generate an output fingerprint image. The multi light source The fingerprint image enhancement and synthesis process includes:

Acquiring a plurality of first local feature values of a first input fingerprint image and a plurality of second local feature values of a second input fingerprint image, respectively;

Determining a spatial parallax relationship between the first input fingerprint image and the second input fingerprint image according to the plurality of first local feature values and the plurality of second local feature values; and

According to the spatial parallax relationship, the first input fingerprint image and the second input fingerprint image are synthesized to generate an output fingerprint image.
The multi-light source fingerprint sensor according to claim 11, wherein the steps of obtaining a plurality of first local feature values of the first input fingerprint image and a plurality of second local feature values of the second input fingerprint image include: :

Respectively opening a first local feature window and a second local feature window to calculate a first reference value corresponding to the first local feature window and a second reference value corresponding to the second local feature window; and

Generate a first local feature vector function based on the first reference value and a plurality of pixels of the first input fingerprint image, and generate a second local feature vector function based on the second reference value and a plurality of pixels of the second input fingerprint image Local eigenvector function

The pixels of the first input fingerprint image can generate the first local feature values through the first local feature vector function, and the pixels of the second input fingerprint image can be generated through the second local feature vector function. The plurality of second local feature values are generated.
The multi-light source fingerprint sensor according to claim 12, wherein the first reference value is an average value of a plurality of pixels of a first central window in the first local feature window, and the second reference value is the An average of a plurality of pixels of a second central window in the second local feature window.
The multi-light source fingerprint sensor according to claim 12, wherein the spatial disparity relationship is a plurality of pixel stereo disparity estimates of the first input fingerprint image and the second input fingerprint image, and according to the plurality of first The step of determining the spatial parallax relationship between the first input fingerprint image and the second input fingerprint image by the local feature values and the plurality of second local feature values includes:

Selecting one of the first input fingerprint image and the second input fingerprint image as a reference image and the other as a non-reference image; and

For each pixel of the non-reference image, in the horizontal direction of the relative position of the reference image, according to the first local feature vector function and the second local feature vector function, the plurality of first local feature values and the A plurality of difference values of the plurality of second local feature values to calculate the plurality of pixel stereo disparity estimation values;

The plurality of pixel stereo disparity estimation values are used to describe the plurality of pixel stereo disparity estimation values of the first input fingerprint image and the second input fingerprint image.
The multi-light source fingerprint sensor according to claim 14, wherein the estimated stereo disparity of the plurality of pixels is a minimum value among the plurality of differences.
The multi-light source fingerprint sensor according to claim 14, wherein the step of synthesizing the first input fingerprint image and the second input fingerprint image to generate the output fingerprint image according to the spatial parallax relationship comprises:

Calculating a plurality of first consistency parameters corresponding to the plurality of pixels of the first input fingerprint image and a plurality of second corresponding to the plurality of pixels of the second input fingerprint image according to the plurality of pixel stereo disparity estimated values A consistency parameter to calculate a plurality of first blending ratios and a plurality of second blending ratios corresponding to the plurality of pixels of the first input fingerprint image and the second input fingerprint image; and

Performing a pixel-to-pixel superimposition of the first input fingerprint image and the second input fingerprint image according to the plurality of first mixing ratios and the plurality of second mixing ratios to generate the output fingerprint image;

The plurality of first blending ratios and the plurality of second blending ratios are used to describe the respective numbers of pixels of the first input fingerprint image and the plurality of pixels of the second input fingerprint image on respective input fingerprint images. The fingerprint is consistent with the fingerprint characteristics of surrounding pixels;

The sum of the plurality of first mixing ratios and the plurality of second mixing ratios is equal to 1.
The multi-light source fingerprint sensor according to claim 12, wherein the spatial disparity relationship is a global displacement reference vector of the first input fingerprint image and the second input fingerprint image, and according to the plurality of first local features And the plurality of second local feature values, the step of determining the spatial parallax relationship between the first input fingerprint image and the second input fingerprint image includes:

Calculating the global displacement reference vector according to the first local feature values of the first input fingerprint image and the second local feature values of the second input fingerprint image;

Selecting one of the first input fingerprint image and the second input fingerprint image as a fixed image and the other as a non-fixed image; and

Translate a plurality of pixels of the non-fixed image globally to generate a compensated fingerprint image according to the global displacement reference vector;

The total average difference between the pixels of the compensated fingerprint image and the pixels of the fixed image is the smallest.
The multi-light source fingerprint sensor according to claim 17, wherein the method further comprises determining a plurality of representative pixels from the plurality of pixels of the compensated fingerprint image and the plurality of pixels of the fixed image to synthesize the plurality of pixels. Compensating the fingerprint image and the fixed image to generate the output fingerprint image, including:

Perform a small-area search in a local area corresponding to one of the plurality of pixels of the compensated fingerprint image and one of the plurality of pixels of the fixed image, respectively, and select one from the plurality of pixels in the local area to Respectively as the plurality of representative pixels;

The local area is a positive and negative n unit pixel range at the center of one of the plurality of pixels of the compensated fingerprint image, and n is a positive integer greater than 0;

An error between the plurality of representative pixels and the fixed image or the plurality of pixels of the compensation fingerprint image is not greater than an error between the plurality of pixels of the fixed image and the plurality of pixels of the compensation fingerprint image.
The multi-light source fingerprint sensor according to claim 18, wherein the step of synthesizing the compensation fingerprint image and the fixed image to generate the output fingerprint image comprises:

Calculating a plurality of first mixing ratios and a plurality of second mixing ratios corresponding to the plurality of pixels of the compensation fingerprint image and the fixed image according to the pixel brightness and contrast of the compensation fingerprint image and the fixed image; and

A pixel-to-pixel overlay is performed according to the plurality of first mixing ratios of the compensated fingerprint image and the plurality of second mixing ratios of the fixed image to generate the output fingerprint image.