CN100517371C - Stitching Method of Fingerprint Image Frame Sequence Based on Waveform Matching - Google Patents

Abstract

The invention relates to a finger print image jointing method for image processing control. For each picture, it only selects part of it as the reference area to get the grey binary wave and to extract the wave jumping information of the current picture with the jointed image overlapping area to realize the jointing. It has small amount of computation, simple in realization, quick to complete the jointing of finger print image, applicable for all kinds of scrape finger print image collection module.

Description

Stitching Method of Fingerprint Image Frame Sequence Based on Waveform Matching

technical field

The invention relates to an image processing process control method, in particular to a splicing method of fingerprint image frames.

Background technique

Now the automatic fingerprint identification system is more and more widely used, such as mobile phones and computers, especially when the automatic fingerprint identification system is applied to mobile phones, it is necessary to use a small fingerprint sensor, otherwise it will affect the appearance layout and volume of the mobile phone. In terms of product cost, a small fingerprint sensor requires a smaller integrated circuit chip area and lower cost. Therefore, the scratch sensor has been widely used.

The working principle of the scratch-type fingerprint sensor is: collect many frames of fingerprint images, and there is a certain overlapping area between adjacent frames. It is necessary to stitch each frame of images into a complete fingerprint image through image stitching. Therefore, the fingerprint image mosaic algorithm is of great significance to the scratch-type fingerprint sensor.

In practical applications, the response time of the automatic fingerprint identification system is required to be relatively short, so the fingerprint image mosaic algorithm is required to be fast, and the mosaic image can meet the requirements of the fingerprint identification algorithm.

After searching the prior art documents, it was found that Chinese Patent Publication No. CN1694118A, published on November 9, 2005, was named "Seamless Splicing Method of Sliding Fingerprint Sequences with Extended Phase Correlation", and the application disclosed a A method for seamless stitching of swipe fingerprint sequences with extended phase correlation. Its disadvantage is that it needs to perform Fourier transform on each image frame to achieve splicing.

Chinese Patent Publication No. CN 1804862A, the disclosure date was July 19, 2006. The title of the invention is "the splicing method of fingerprint image frames", and the application discloses a splicing method of fingerprint image frames. Its shortcoming is that it needs to calculate the variance of gray level for each image frame to achieve splicing.

Contents of the invention

The present invention overcomes the above-mentioned shortcomings and provides a splicing method of fingerprint image frames that only processes the binarized reference area of each image frame during the splicing process of fingerprint images, thereby reducing the amount of calculation and speeding up the splicing speed.

The technical scheme adopted by the present invention to solve the technical problem is: a splicing method of fingerprint image frames, and splicing each frame of image only according to the binary waveform jump information extracted from its reference area.

May include the following steps:

1) After receiving the fingerprint image frame collected by the fingerprint image acquisition module, read in the first frame image;

2) The image frame is reduced, stitched into the spliced image, each reference area extraction template is used in turn to extract the reference reference area, and the waveform jump information of the extracted reference reference area is calculated. If the waveform jump information of the reference area can be obtained, read Enter the next frame of image, enter step 3); if the waveform jump information of the reference area cannot be obtained, read in the next frame of image, return to step 2);

3) The image frame is reduced, and the reference area of the current image frame is extracted using the reference area extraction template adopted by the previous image frame, and the waveform jump information of the reference area is calculated;

4) Determining the overlapping area between the current image frame and the spliced image;

5) If there is an image area that can be used as a reference area in the image area above the current image frame reference area calibration repetition line, use it as a new reference area, update the waveform jump information of the reference area, stitch the current image frame into the spliced image, and read Enter the next image frame, go to step 3); otherwise, go to step 6);

6) Select other reference area templates to extract the new reference area, calculate the waveform transition information of the new reference area, obtain the waveform transition information of the new reference area, extract the new reference area, and calculate the waveform transition information of the new reference area until passing Step 4) and 5) enter step 3); On the contrary, if all reference region extraction templates are used, still cannot enter step 3), then enter step 7) to complete splicing;

7) If all image frames cannot extract the waveform jump information of the reference area, or all image frames have been processed, or a certain image frame cannot be spliced into the spliced image, or the size of the spliced image has reached the preset threshold, the splicing algorithm is terminated; if spliced If successful, the spliced image will be output, otherwise, the splicing will fail.

The reduction processing of the image frame is to delete the uppermost and the lowermost row respectively from the image frame collected by the fingerprint image collection module.

The reference area refers to the image area extracted from the reference area extraction template in the image frame or spliced image, and its size is 1/6-1/3 of the image frame.

Described step 4) realizes by following steps:

1) Overlay the current image frame on the spliced image so that the top line coincides with the top line of the reference area, and calculate the waveform similarity between the overlapping part of the current image frame and the reference area and the reference area;

2) Move the current image frame upwards, one line at a time, until the bottom line of the current image frame coincides with the bottom line of the reference area; every time it moves, calculate the waveform similarity between the overlapping part of the current image frame and the reference area and the reference area ;

3) The current image frame area corresponding to the maximum value of the waveform similarity of all areas is the calibration repetition area;

4) The image line with the largest similarity to the line waveform of the corresponding reference line in the calibration repetition area is the calibration repetition line of the current image frame, and the corresponding reference behavior is the calibration reference line;

5) Move the waveform corresponding to the calibrated repetitive row in the horizontal direction, and calculate the similarity of the row waveform with the calibrated reference row for each pixel point, and the horizontal movement of the calibrated repetitive row waveform corresponding to the maximum value of the obtained row waveform similarity The amount is the horizontal offset of the current image frame.

When the current image frame is stitched into the spliced image in said step 5), the weighted average of the gray values of two overlapping pixels is taken as the splicing result in the overlapping part, and the weights of the gray values of the two overlapping pixels are non-negative, and is 1.

The reference area refers to 3 consecutive rows of images in the reference region of the mosaic image, where each row of images contains at least one waveform jump information.

The waveform jump information refers to the position and sequence of the waveform jumps from 0 to 1 or from 1 to 0 in the binary waveform corresponding to each image row.

The waveform jump information refers to the position and sequence of the waveform jump from 1 to two consecutive 0s in the binary waveform.

The calculation of the row waveform similarity is determined by the position, number and order of the waveform jumps contained in the two waveforms, and the two waveforms participating in the calculation each time belong to the reference area and the reference area of the current image frame respectively.

The calculation rules for the row waveform similarity are as follows:

1) If the two transition positions differ by 3 or 4, add 1 to the similarity, and if the two transitions are in the same order in their respective waveforms, add 1 to the similarity;

2) If the difference between the two jump positions is 2, the similarity is added by 2, and if the order of the two jumps in their respective waveforms is the same, the similarity is added by 1;

3) If the difference between two transition positions is 1, add 3 to the similarity, and if the order of the two transitions in their respective waveforms is the same, add 2 to the similarity;

4) If the positions of the two transitions are the same, add 4 to the similarity, and if the order of the two transitions in their respective waveforms is the same, add 2 to the similarity;

5) If two waveforms that satisfy one or more rules in rule 1) to rule 4) contain the same number of transitions, add 2 to the similarity.

Description of drawings

Fig. 1 is a work flow diagram of the present invention

Figure 2 is the program flow chart for calculating the waveform jump information in the reference area when the program is initialized

Figure 3 is a program flow chart for splicing a single image frame

Fig. 4 is the image frame sequence gathered by the AES2510 scraping type sensor of Authentec company that the present invention adopts

果Fig. 5 is the splicing obtained according to Embodiment 1 of the splicing method of the present invention

fruit

Fig. 6 is the splicing result obtained according to Embodiment 2 of the splicing method of the present invention

Fig. 7 is the splicing result obtained according to embodiment 3 of the splicing method of the present invention

Detailed ways

The present invention will be further described below in conjunction with embodiment.

The number of image frames collected by the scratch-type fingerprint sensor is not fixed each time, and there will be overlapping areas in two adjacent image frames that match or can be approximately regarded as coincidence. The degree of this agreement is measured by waveform similarity. The measurement is based on the similarity between the waveform jump information of the current image frame and the waveform of the reference area. The waveform similarity refers to the similarity between the waveform corresponding to the current image frame and the waveform corresponding to the reference area. According to the similarity of the waveform, the calibration repetition line and the horizontal offset of the current image frame are determined to locate the overlapping area. The similarity used in this embodiment is determined by the waveform jump offset, number and sequence number.

In the process of image frame splicing, the uppermost and lowermost lines of the image frame, which are easily affected by noise, sweat stains and smudges, are deleted, which improves the accuracy of the splicing algorithm and reduces the amount of calculation.

Partial regions of the pruned image frames are extracted for splicing operations, which further reduces the amount of computation.

Wherein, the image frame reduction process described below is to delete the topmost and bottommost lines of the image frames collected by the fingerprint image collection module.

The calculation rules for the similarity of the row waveforms are as follows:

1) If the two transition positions differ by 3 or 4, add 1 to the similarity, and if the two transitions are in the same order in their respective waveforms, add 1 to the similarity;

2) If the difference between the two jump positions is 2, the similarity is added by 2, and if the order of the two jumps in their respective waveforms is the same, the similarity is added by 1;

3) If the difference between two transition positions is 1, add 3 to the similarity, and if the order of the two transitions in their respective waveforms is the same, add 2 to the similarity;

4) If the positions of the two transitions are the same, add 4 to the similarity, and if the order of the two transitions in their respective waveforms is the same, add 2 to the similarity;

5) If two waveforms that satisfy one or more rules in rule 1) to rule 4) contain the same number of transitions, add 2 to the similarity.

Example 1

The image splicing process in this embodiment will be described in detail below in conjunction with FIGS. 1 , 2 , and 3 .

First of all, it is assumed that the image size of each frame is 16 rows, 192 columns, and the gray level is 0-255. The maximum size of the image after normal splicing is 192 rows, 192 columns. A total of 3 reference area extraction templates are used, which extract the 65th to 128th columns, the 1st to 64th columns and the 129th to 192nd columns in the 1st to 14th rows of the reduced image frame respectively.

After receiving the first image frame, it is reduced, and the remaining part is stitched into the bottom of the stitched image. Select a reference area extraction template to extract the benchmark reference area, and calculate the binarized gray threshold as the average gray value of the reference area minus 16. Binarize the benchmark reference area according to this threshold, convert each image line into a binary sequence, and obtain the corresponding binary waveform. Scan each waveform, and record the position of the first 0 when it encounters a jump from 1 to two consecutive 0s. If the waveforms corresponding to three consecutive lines of images all contain waveform jump information, these three lines of images are used as the reference area, and the waveform jump information contained therein is the waveform jump information of the reference area.

After receiving the second frame of image, use the reference region extraction template used in the previous frame of image to extract the reference region, calculate the binarization gray threshold, and binarize the reference region to obtain the corresponding binary waveform. Scan each waveform, and record the position of the first 0 when it encounters a jump from 1 to two consecutive 0s. Obtain the waveform transition information of each row in the reference area.

Calculate the waveform similarity between the waveform jump information in the reference area and the waveform jump information in the reference area. Calculate the similarity between any three consecutive waveforms in the reference area and the waveform jump information in the reference area, and the three consecutive image lines in the reference area corresponding to the maximum value of all waveform similarities are the second The calibrated repeat region of the image frame. The three waveforms correspond to the waveform similarities of the three lines respectively, and the image line corresponding to the maximum value is the calibration repetition line of the second image frame, and the corresponding reference line is the calibration reference line. Move and calibrate the waveform jump information corresponding to the repeated rows in the horizontal direction, and calculate the waveform similarity of each row respectively. The movement amount of the waveform jump information corresponding to the maximum value of all similarities is the horizontal offset of the second image frame .

After obtaining the calibrated repetition line position and horizontal offset of the second image frame, the overlapping area of the second image frame is determined, and three consecutive positions above the calibrated repetition line are selected from the reference area of the second image frame. Image lines, and each image line contains waveform jump information, so as to obtain the new reference area and the waveform jump information of the new reference area. Stitch the second image frame into the spliced image. Firstly, coincide the calibrated repeat line with the calibrated reference line, and then move the second frame image to the left or right according to the horizontal offset. The non-overlapping area is directly stitched into the stitched image. In the overlapping area, when calculating the weighted average value of the gray value, the weight of the gray value of the pixel point of the spliced image is 0.6, and the weight of the gray value of the pixel point of the current image frame is 0.4. The blank area of the spliced image caused by the movement of the image frame is filled with pixels with a grayscale of 0.

After the image processing of the second frame is completed, the subsequent image frames are read in, and the processing method is the same as that of the second frame. Finally, the image shown in Figure 5 is output.

Example 2

The image splicing process in this embodiment will be described in detail below in conjunction with FIGS. 1 , 2 , and 3 .

First of all, it is assumed that the image size of each frame is 16 rows, 192 columns, and the gray level is 0-255. The maximum size of the image after normal splicing is 192 rows, 192 columns. A total of 3 reference area extraction templates are used, which extract the 73rd to 120th columns, the 25th to 72nd columns and the 121st to 168th columns in the 1st to 14th rows of the reduced image frame respectively.

After receiving the first image frame, it is reduced, and the remaining part is stitched into the bottom of the stitched image. Select a reference area extraction template to extract the benchmark reference area, and calculate the binarized gray threshold as the average gray value of the reference area minus 16. Binarize the benchmark reference area according to this threshold, convert each image line into a binary sequence, and obtain the corresponding binary waveform. Scan each waveform, and record the position of the first 0 when it encounters a jump from 1 to two consecutive 0s. If the waveforms corresponding to three consecutive lines of images all contain waveform jump information, these three lines of images are used as the reference area, and the waveform jump information contained therein is the waveform jump information of the reference area.

After receiving the second frame of image, use the reference region extraction template used in the previous frame of image to extract the reference region, calculate the binarization gray threshold, and binarize the reference region to obtain the corresponding binary waveform. Scan each waveform, and record the position of the first 0 when it encounters a jump from 1 to two consecutive 0s. Obtain the waveform transition information of each row in the reference area.

Calculate the waveform similarity between the waveform jump information in the reference area and the waveform jump information in the reference area. Calculate the similarity between any three consecutive waveforms in the reference area and the waveform jump information in the reference area, and the three consecutive image lines in the reference area corresponding to the maximum value of all waveform similarities are the second The calibrated repeat region of the image frame. The three waveforms correspond to the waveform similarities of the three lines respectively, and the image line corresponding to the maximum value is the calibration repetition line of the second image frame, and the corresponding reference line is the calibration reference line. Move and calibrate the waveform jump information corresponding to the repeated rows in the horizontal direction, and calculate the waveform similarity of each row respectively. The movement amount of the waveform jump information corresponding to the maximum value of all similarities is the horizontal offset of the second image frame .

After obtaining the calibrated repetition line position and horizontal offset of the second image frame, the overlapping area of the second image frame is determined, and three consecutive positions above the calibrated repetition line are selected from the reference area of the second image frame. Image lines, and each image line contains waveform jump information, so as to obtain the new reference area and the waveform jump information of the new reference area. Stitch the second image frame into the spliced image. Firstly, coincide the calibrated repeat line with the calibrated reference line, and then move the second frame image to the left or right according to the horizontal offset. The non-overlapping area is directly stitched into the stitched image. In the overlapping area, when calculating the weighted average value of the gray value, the weight of the gray value of the pixel point of the spliced image is 0.5, and the weight of the gray value of the pixel point of the current image frame is 0.5. The blank area of the spliced image caused by the movement of the image frame is filled with pixels with a grayscale of 0.

After the image processing of the second frame is completed, the subsequent image frames are read in, and the processing method is the same as that of the second frame. Finally, the image shown in Figure 6 is output.

Example 3

The image splicing process in this embodiment will be described in detail below in conjunction with FIGS. 1 , 2 , and 3 .

First of all, it is assumed that the image size of each frame is 16 rows, 192 columns, and the gray level is 0-255. The maximum size of the image after normal splicing is 192 rows, 192 columns. A total of 3 reference area extraction templates are used to extract the 81st to 112th columns, the 49th to 80th columns and the 113th to 144th columns in the 1st to 14th rows of the reduced image frame respectively.

After receiving the first image frame, it is reduced, and the remaining part is stitched into the bottom of the stitched image. Select a reference area extraction template to extract the benchmark reference area, and calculate the binarized gray threshold as the average gray value of the reference area minus 16. Binarize the benchmark reference area according to this threshold, convert each image line into a binary sequence, and obtain the corresponding binary waveform. Scan each waveform, and record the position of the first 0 when it encounters a jump from 1 to two consecutive 0s. If the waveforms corresponding to three consecutive lines of images all contain waveform jump information, these three lines of images are used as the reference area, and the waveform jump information contained therein is the waveform jump information of the reference area.

After receiving the second frame of image, use the reference region extraction template used in the previous frame of image to extract the reference region, calculate the binarization gray threshold, and binarize the reference region to obtain the corresponding binary waveform. Scan each waveform, and record the position of the first 0 when it encounters a jump from 1 to two consecutive 0s. Obtain the waveform transition information of each row in the reference area.

Calculate the waveform similarity between the waveform jump information in the reference area and the waveform jump information in the reference area. Calculate the similarity between any three consecutive waveforms in the reference area and the waveform jump information in the reference area, and the three consecutive image lines in the reference area corresponding to the maximum value of all waveform similarities are the second The calibrated repeat region of the image frame. The three waveforms correspond to the waveform similarities of the three lines respectively, and the image line corresponding to the maximum value is the calibration repetition line of the second image frame, and the corresponding reference line is the calibration reference line. Move and calibrate the waveform jump information corresponding to the repeated rows in the horizontal direction, and calculate the waveform similarity of each row respectively. The movement amount of the waveform jump information corresponding to the maximum value of all similarities is the horizontal offset of the second image frame .

After obtaining the calibrated repetition line position and horizontal offset of the second image frame, the overlapping area of the second image frame is determined, and three consecutive positions above the calibrated repetition line are selected from the reference area of the second image frame. Image lines, and each image line contains waveform jump information, so as to obtain the new reference area and the waveform jump information of the new reference area. Stitch the second image frame into the spliced image. Firstly, coincide the calibrated repeat line with the calibrated reference line, and then move the second frame image to the left or right according to the horizontal offset. The non-overlapping area is directly stitched into the stitched image. In the overlapping area, when calculating the weighted average value of the gray value, the weight of the gray value of the pixel point of the spliced image is 0.3, and the weight of the gray value of the pixel point of the current image frame is 0.7. The blank area of the spliced image caused by the movement of the image frame is filled with pixels with a grayscale of 0.

After the image processing of the second frame is completed, the subsequent image frames are read in, and the processing method is the same as that of the second frame. Finally, the image shown in Figure 7 is output.

The gray levels of each image frame can be in any range, such as 0-255 or 0-15. For image frames other than 0-255, it is only necessary to convert the gray-scale range to 0-255. For For a binary image with a gray level of 0 to 1, the waveform jump information can be directly extracted without binarization. The present invention is also applicable to other image acquisition and imaging processes that adopt a method similar to the scraping fingerprint image acquisition method.

The method for mosaicing fingerprint image frames based on waveform matching provided by the present invention has been introduced in detail above, and the principles and implementation methods of the present invention have been explained by applying specific embodiments in the text. The descriptions of the above embodiments are used to help understand the present invention methods and ideas. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims (10)

1. the joining method of a fingerprint image frame, it is characterized in that: at first fingerprint image frame to be spliced is reduced processing, from the picture frame after the reduction, select reference zone again, extract reference zone waveform saltus step information and carry out the splicing of fingerprint image frame, comprise the steps:

1) receive the fingerprint image frame that the fingerprint image acquisition module collects after, read in first two field picture;

2) picture frame is reduced processing, piece together to go into stitching image, adopt each reference zone to extract template extraction reference zone successively, the waveform saltus step information in the reference zone that calculating is extracted, if can obtain reference area waveform saltus step information, read in the next frame image, enter step 3); If can't obtain reference area waveform saltus step information, read in the next frame image, return step 2);

3) picture frame is reduced processing, the reference zone that adopts a last picture frame to be adopted extracts the reference zone of template extraction current image frame, calculates the waveform saltus step information of reference zone;

4) determine the overlapping region of current image frame and stitching image;

5) existence can be as the image-region of reference area in the above image-region of repeated rows if the current image frame reference zone is demarcated, it as new reference area, is upgraded reference area waveform saltus step information, current image frame is pieced together into stitching image, read in next picture frame, change step 3) over to; Otherwise, enter step 6);

6) select other the new reference of reference zone template extraction zone for use, calculate new reference zone waveform saltus step information, obtain new reference area waveform saltus step information, extract new reference zone, calculate the waveform saltus step information of new reference zone, up to by step 4) and 5) enter step 3); Otherwise,, then enter step 7) and finish splicing if after all reference zones extraction templates are all used, still can not enter step 3);

7) if all images frame all can't extract reference area waveform saltus step information, perhaps handle all images frame, perhaps certain picture frame can not be pieced together into stitching image, and perhaps the stitching image size has reached predetermined threshold value, stops stitching algorithm; If splice successfully, the output stitching image, otherwise, output splicing failure.

2. the joining method of fingerprint image frame according to claim 1 is characterized in that: the reduction of described picture frame handle be picture frame deletion that the fingerprint image acquisition module is collected topmost and bottom each delegation.

3. the joining method of fingerprint image frame according to claim 1 is characterized in that: described reference zone is meant in picture frame or the stitching image and extracts the image-region that template extraction is come out by reference zone that size is 1/6～1/3 of a picture frame.

4. the joining method of fingerprint image frame according to claim 1, it is characterized in that: described step 4) realizes as follows:

1) current image frame is covered on the stitching image, the lastrow of its lastrow and reference area is overlapped, calculate the wave-form similarity of the overlapping part of current image frame and reference area and reference area;

2) current image frame that moves up, each mobile delegation overlaps up to the next line of current image frame bottom line and reference area; Whenever move once, calculate the wave-form similarity of the overlapping part of current image frame and reference area and reference area;

3) the current image frame zone of the maximal value correspondence of All Ranges wave-form similarity is the demarcation repeat region;

4) image line of demarcating the capable wave-form similarity maximum of reference row pairing with it in the repeat region is the demarcation repeated rows of current image frame, the benchmark behavior demarcation reference row corresponding with it;

5) move the waveform of demarcating the repeated rows correspondence in the horizontal direction, whenever move a pixel, calculate once and the capable wave-form similarity of demarcating reference row, the amount of moving horizontally of the pairing demarcation repeated rows of the maximal value of the capable wave-form similarity of gained waveform is the horizontal offset of current image frame.

5. the joining method of fingerprint image frame according to claim 1, it is characterized in that: when piecing together current image frame into stitching image in the described step 5), lap is got the weighted mean value of two superposition image vegetarian refreshments gray-scale values as the splicing result, the weights of two superposition image vegetarian refreshments gray-scale values are non-negative and be 1.

6. the joining method of fingerprint image frame according to claim 1 is characterized in that: described reference area is meant the continuous 3 row images in the stitching image reference zone, and wherein every capable image all comprises a waveform saltus step information at least.

7. the joining method of fingerprint image frame according to claim 1 is characterized in that: described waveform saltus step information is meant in the pairing binary waveform of each image line by 0 and changes to 1 or by 1 position and the order that changes to 0 waveform saltus step.

8. the joining method of fingerprint image frame according to claim 7 is characterized in that: described waveform saltus step information is meant in the binary waveform position and the order that changes to continuous two 0 waveform saltus step by 1.

9. the joining method of fingerprint image frame according to claim 4, it is characterized in that: the calculating of described capable wave-form similarity is to be determined by the position of two waveform saltus steps that waveform comprised, number and order, and each two waveforms that participate in calculating belong to reference area and current image frame reference zone respectively.

10. the joining method of fingerprint image frame according to claim 9, it is characterized in that: the computation rule of described capable wave-form similarity is as follows:

1) if two saltus step positions differ 3 or 4, similarity adds 1, and if two saltus steps order in waveform separately identical, similarity adds 1;

2) if two saltus step positions differ 2, similarity adds 2, and if two saltus steps order in waveform separately identical, similarity adds 1;

3) if two saltus step positions differ 1, similarity adds 3, and if two saltus steps order in waveform separately identical, similarity adds 2;

4) if two saltus step positions are identical, similarity adds 4, and if two saltus steps order in waveform separately identical, similarity adds 2;

5) if satisfy rule 1)～rule 4) in the saltus step number that comprises of two waveforms of one or more rules identical, similarity adds 2.

Images