WO2022127225A1

WO2022127225A1 - Image stitching method and apparatus, and device and storage medium

Info

Publication number: WO2022127225A1
Application number: PCT/CN2021/118350
Authority: WO
Inventors: 吴桐; 梁嘉骏
Original assignee: 北京迈格威科技有限公司
Priority date: 2020-12-14
Filing date: 2021-09-14
Publication date: 2022-06-23
Also published as: CN112686802B; CN112686802A

Abstract

The present application provides an image stitching method and apparatus, and a device and a storage medium. The method comprises: acquiring an original image to be processed, the original image consisting of a plurality of sub-images collected by a plurality of collectors; in the original image, dividing the image of a non-overexposed region into an overlapping region and a non-overlapping region, the overlapping region being a region where the pixel coordinates of the plurality of sub-images overlap each other in the original image; separately performing pixel processing on the overlapping region and the non-overlapping region to generate first pixel information of the overlapping region and second pixel information of the non-overlapping region; and performing image stitching on the overlapping region and the non-overlapping region according to the first pixel information and the second pixel information to generate a stitched image of the original image. According to the present application, the grid noise of the stitched image is reduced, so that the stitched image is more continuous and smoother.

Description

Image stitching method, device, equipment and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese Patent Application No. 202011477009.3 and entitled "Image Mosaic Method, Apparatus, Equipment and Storage Medium" filed with the China Patent Office on December 14, 2020, the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the technical field of image processing, and in particular, to an image stitching method, apparatus, device and storage medium.

Background technique

With the improvement of hardware technology, the current research and development direction of mobile phone appearance is gradually moving towards thinner and lighter direction. Fingerprint recognition has become the mainstream configuration of mobile phone functions, which requires the use of off-screen fingerprint recognition technology while the mobile phone is thinner and lighter.

The under-screen fingerprint hardware module solution of ultra-thin mobile phones usually uses an array lens module or a collimator imaging module. The array lens module is composed of a plurality of small lenses spliced together, so the original raw image obtained by the lens array is also spliced from the original images obtained by the multiple small lenses. However, due to the short distance between each small lens, the imaging area overlaps, and some fingerprint information also overlaps.

In the existing image preprocessing technology, the sub-images collected by each small lens are usually selected and spliced into an output image for subsequent processing. Stitched images are prone to undesirable conditions such as blocking and grid noise. Therefore, how to improve the fingerprint stitching algorithm is an urgent problem to be solved in the image preprocessing stage.

SUMMARY OF THE INVENTION

The embodiments of the present application provide an image stitching method, device, device, and storage medium, which make full use of the image information of the overlapping area and the non-overlapping area between sub-images acquired by each small lens (collector), and comprehensively consider the two The image features of the region are processed separately, and then the stitched image of the original image is generated, which reduces the grid noise of the stitched image and makes the stitched image more continuous and smooth.

Some embodiments of the present application provide an image stitching method, which may include: acquiring an original image to be processed, where the original image is composed of multiple sub-images collected by multiple collectors; The image of the exposure area is divided into an overlapping area and a non-overlapping area; the overlapping area is an area where the pixel coordinates of the multiple sub-images overlap each other in the original image; the overlapping area and the non-overlapping area are respectively Perform pixel processing to generate first pixel information of the overlapping area and second pixel information of the non-overlapping area; according to the first pixel information and the second pixel information, combine the overlapping area and the non-overlapping area Image stitching is performed on the overlapping area to generate a stitched image of the original image.

In one embodiment, the method may further include: judging whether there is an overexposed area in the original image; if the overexposed area exists in the original image, selecting the overexposed area for the overexposed area in the original image. An area where the pixel coordinates of the sub-images do not overlap each other is an effective area; the third pixel information of the effective area is calculated; image stitching is performed according to the first pixel information, the second pixel information and the third pixel information, The stitched image of the original image is generated.

In one embodiment, the judging whether there is an overexposed area in the original image may include: calculating the gradient information of each pixel in the original image; finding whether the gradient information exists in the original image; If there is a first pixel that is smaller than the preset gradient threshold and whose pixel value is within the preset pixel range, the area where the first pixel is located is the overexposed area.

In an embodiment, the step of performing pixel processing on the overlapping area to generate the first pixel information of the overlapping area may include: separately calculating the overlapping times of each overlapping pixel point in the overlapping area; The pixel value of the overlapping pixel points and the overlapping times are used to calculate the average pixel value of the overlapping pixel points, respectively, and the first pixel information includes the average pixel value of each overlapping pixel point.

In an embodiment, the step of performing pixel processing on the non-overlapping area to generate the second pixel information of the non-overlapping area may include: a pixel value of each non-overlapping pixel in the non-overlapping area. A preset weight is assigned to generate a weighted pixel value of each of the non-overlapping pixel points, and the second pixel information includes the weighted pixel value of each of the non-overlapping pixel points.

In one embodiment, the preset weight may be greater than 1.

In an embodiment, dividing the image of the non-overexposed area into an overlapping area and a non-overlapping area in the original image may include: calculating the plurality of collectors respectively for the non-overexposed area. Obtain the pixel coordinates of each of the sub-images; select an image area where the pixel coordinates of the sub-images overlap each other as the overlapping area, and the remaining image areas are the non-overlapping areas.

Other embodiments of the present application provide an image stitching apparatus, which may include: an acquisition module, configured to acquire an original image to be processed, where the original image is composed of multiple sub-images collected by multiple collectors; a division module, configured with In the original image, the image of the non-overexposed area is divided into an overlapping area and a non-overlapping area; the overlapping area is an area where the pixel coordinates of a plurality of the sub-images overlap each other in the original image; processing a module for performing pixel processing on the overlapping area and the non-overlapping area respectively, to generate the first pixel information of the overlapping area and the second pixel information of the non-overlapping area; For the first pixel information and the second pixel information, image splicing is performed on the overlapping area and the non-overlapping area to generate a spliced image of the original image.

In one embodiment, it may further include: a determination module configured to determine whether there is an overexposed area in the original image; a selection module configured to be configured to determine if the overexposed area exists in the original image , for the overexposed area in the original image, select an area where the pixel coordinates of the sub-image do not overlap each other as an effective area; a calculation module, configured to calculate the third pixel information of the effective area ; the stitching module is further configured to perform image stitching according to the first pixel information, the second pixel information and the third pixel information to generate the stitched image of the original image.

In one embodiment, the judging module may be configured to: calculate the gradient information of each pixel in the original image; in the original image, find out whether the gradient information is smaller than a preset gradient. The threshold value and the first pixel point whose pixel value is within the preset pixel range, if it exists, the area where the first pixel point is located is the overexposed area.

In one embodiment, the processing module may be configured to: respectively calculate the overlapping times of each overlapping pixel point in the overlapping area; The average pixel value of the overlapping pixel points, and the first pixel information includes the average pixel value of each of the overlapping pixel points.

In one embodiment, the processing module may be further configured to: assign a preset weight to the pixel value of each non-overlapping pixel in the non-overlapping area, and generate a pixel value of each non-overlapping pixel. A weighted pixel value, the second pixel information includes the weighted pixel value of each of the non-overlapping pixel points.

In one embodiment, the preset weight may be greater than 1.

In one embodiment, the dividing module may be configured to: for the non-overexposed area, calculate the pixel coordinates of each of the sub-images obtained by the plurality of collectors respectively; select the sub-images; The image area where the pixel coordinates of the overlap each other exist is the overlapping area, and the remaining image area is the non-overlapping area. Still other embodiments of the present application provide an electronic device, including: a memory for storing a computer program; and a processor for executing the method of any one of the some embodiments of the present application, to perform a plurality of original sub-images for image stitching.

Still other embodiments of the present application provide a non-transitory electronic device-readable storage medium, comprising: a program that, when executed by an electronic device, causes the electronic device to perform any of the foregoing embodiments of the present application The method of an embodiment.

The image stitching method, device, device and storage medium provided by the present application can adopt different processing methods for the light leakage area and non-light leakage area of the original image. Divide the image into the overlapping area image and the non-overlapping area image, and then perform different pixel processing on the images of the two areas, and finally integrate the pixel information of the two areas to perform image stitching to generate a stitched image. In this way, for the image of the non-light leakage area , make full use of the image information of the overlapping area and non-overlapping area between the sub-images acquired by each collector, comprehensively consider the image characteristics of the two areas, and perform pixel processing respectively, which reduces the grid noise of the spliced image and makes the splicing The image is more continuous and smoother.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, therefore It should not be regarded as a limitation of the scope. For those of ordinary skill in the art, other related drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the application;

2A is a schematic diagram of a fingerprint identification scene according to an embodiment of the present application;

2B is a schematic diagram of an original image according to an embodiment of the present application;

2C is a schematic diagram of a stitched image according to an embodiment of the application;

3 is a schematic flowchart of an image stitching method according to an embodiment of the application;

4 is a schematic flowchart of an image stitching method according to an embodiment of the application;

5 is a schematic diagram of a stitched image according to an embodiment of the application;

6A to 6B are schematic diagrams of fingerprint images corresponding to the stitched images according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an image stitching apparatus according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. In the description of the present application, the terms "first", "second" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

As shown in FIG. 1 , this embodiment provides an electronic device 1 , which may include: at least one processor 11 and a memory 12 . In FIG. 1 , one processor is used as an example. The processor 11 and the memory 12 can be connected through the bus 10, and the memory 12 can store instructions that can be executed by the processor 11, and the instructions are executed by the processor 11, so that the electronic device 1 can execute all or all of the methods in the following embodiments. Part of the process to perform image stitching on multiple original sub-images.

In one embodiment, the electronic device 1 may be a mobile phone, a notebook computer, a desktop computer, or a computing system composed of multiple computers.

In an embodiment, the electronic device 1 may further include a plurality of image collectors 13 , and each collector 13 may be connected to the processor 11 and the memory 12 through the bus 10 , respectively. For example, the multiple collectors 13 may be camera arrays, which are used to collect multiple original sub-images.

Please refer to FIG. 2A , which is a fingerprint identification scene according to an embodiment of the application, which may include: an electronic device 1, and a fingerprint identification module 14 may be provided on the electronic device 1. The fingerprint identification module 14 may be composed of a small camera array, and a collector 13 Can be a small camera array. When the user's finger touches or presses the fingerprint recognition module 14, the camera array can be triggered to collect the original image of the user's finger, and multiple cameras collect multiple original sub-images to form the original image to be processed. After acquiring the original image, the electronic device 1 re-splices the original image to generate a spliced image, and the spliced image can be used as a fingerprint image for subsequent fingerprint identification.

In one embodiment, taking a 5*5lens array (lens array) as an example, assuming that the sub-image obtained by each small lens (lens) is 20*20pixel (pixel), the original raw image obtained by the device is a piece of 5 The original image of size 100*100 formed by splicing a 20*20 matrix (as shown in Figure 2B). In the actual scene, the overlapping area may be uneven. It is assumed that the overlapping area between the two lenses in the middle is 6 pixels, and the overlapping area between the lenses on both sides is 3 pixels, that is, the pixel positions of the five lenses are: [0 :20][17:37][31:51][45:65][62:82], then according to the above method, each small lens non-overlapping area can be selected and spliced into a [20*2+(20-3- 3)*3] ² = a stitched image with a size of 82*82 pixels (as shown in FIG. 2C ).

In one embodiment, the overlapping area of the five lenses may also be uniform. For example, the pixel positions of the five lenses are: [0:20][17:37][34:54][51:71][68 :88], according to the above method, each small lens non-overlapping area can be selected and stitched into a stitched image with a size of 88*88pixel.

However, the distance between each camera is relatively close, and the image acquisition range overlaps, resulting in overlapping parts between sub-images. In the process of image stitching, the image features of the overlapping area and the non-overlapping area are comprehensively considered, and pixel processing is performed separately. Reduce the grid noise of the stitched image, making the stitched image more continuous and smoother.

Please refer to FIG. 3 , which is an image stitching method according to an embodiment of the present application. The method can be executed by the electronic device 1 shown in FIG. 1 , and can be applied to the fingerprint recognition scene shown in FIG. 2A to FIG. Image stitching is performed on multiple original sub-images collected by the camera array. The method may include the following steps:

Step 301 : Acquire an original image to be processed, and the original image may be composed of multiple sub-images collected by multiple collectors 13 .

In this step, the multiple collectors 13 may be the camera array of the fingerprint identification module 14 of the mobile phone, or may be the image collectors 13 externally connected to the mobile phone, and the collection ranges of the multiple collectors 13 overlap. The collector 13 can collect multiple sub-images of the target object (finger) in real time, and the multiple sub-images constitute the original image of the target object.

Step 302: In the original image, the image of the non-overexposed area is divided into an overlapping area and a non-overlapping area. The overlapping area is an area where pixel coordinates of multiple sub-images overlap each other in the original image.

In this step, the size of the overlap range between the collection ranges of the multiple collectors 13 may be preset, for example, the size of the overlap range between the sub-images collected by each camera of the camera array may be preset. The image of the non-overexposed area may be the fingerprint image of the non-light-leakage area during the pattern acquisition process. Accordingly, in the original image, the image of the non-overexposed area can be divided into the image of the overlapping area and the image of the non-overlapping area.

Step 303: Perform pixel processing on the overlapping area and the non-overlapping area, respectively, to generate first pixel information of the overlapping area and second pixel information of the non-overlapping area.

In this step, there are different sub-image information in the overlapping area in the original non-overexposed area image, and the non-overlapping area only contains the information of a single sub-image, so it has different image characteristics, and different sub-images can be used respectively. By performing pixel processing on the two in a way, the obtained first pixel information and the second pixel information can clearly represent the actual image of the user's fingerprint to the greatest extent possible.

Step 304: According to the first pixel information and the second pixel information, image stitching is performed on the overlapping area and the non-overlapping area to generate a stitched image of the original image.

In this step, the first pixel information can be used to represent the pixel feature of each pixel in the overlapping area, and the second pixel information can be used to represent the pixel feature of each pixel in the non-overlapping area. The first pixel information and the second pixel information can be spliced according to the position of the pixel points on the original image, and then the overlapping area and the non-overlapping area can be image spliced to generate a spliced image of the original image. In this way, the spliced image contains The image features of the overlapping area and the non-overlapping area are saved, so as to avoid wasting the collected information of the collector 13 .

The above image stitching method, for the non-overexposed area, first divides multiple original sub-images collected by the camera array into overlapping area images and non-overlapping area images, and then performs different pixel processing on the images of the two areas, and finally integrates the images. The pixel information of the two areas is stitched to generate a stitched image. In this way, the image information of the overlapping area and the non-overlapping area between the sub-images obtained by each camera is fully utilized, and the image characteristics of the two areas are comprehensively considered. Pixel processing reduces the grid noise of the stitched image, making the stitched image more continuous and smoother.

Please refer to FIG. 4 , which is an image stitching method according to an embodiment of the present application. The method can be executed by the electronic device 1 shown in FIG. 1 , and can be applied to the fingerprint recognition scene shown in FIG. 2A to FIG. Image stitching is performed on multiple original sub-images collected by the camera array. The method includes the following steps:

Step 401 : Acquire an original image to be processed, and the original image may be composed of multiple sub-images collected by multiple collectors 13 . For details, refer to the description of step 301 in the above embodiment.

Step 402 : Determine whether there is an overexposed area in the original image, if yes, go to Step 405 , otherwise, go to Step 403 .

In this step, in the actual fingerprint image collection scene, the user's finger may fail to face the collection area or the azimuth shift when pressing, etc., at this time, some cameras will be overexposed or light leaked, and the collected image will be overexposed. The area will affect the non-overexposed area, forming a transition area with gradually changing brightness, which is not conducive to the result of fingerprint recognition. In order not to magnify the influence of the overexposed area on the non-overexposed area, first distinguish the light leakage area in the original image. , that is, the area where the overexposed first pixel is located is selected as the overexposed area, and then step 405 is entered for subsequent processing. The area where the remaining second pixel points are located is regarded as a non-overexposed area, and then proceeds to step 403 for subsequent processing.

In one embodiment, step 402 may include: calculating gradient information of each pixel in the original image. In the original image, find the first pixel whose gradient information is less than the preset gradient threshold and whose pixel value is within the preset pixel range, the area where the first pixel is located is the overexposed area, and the remaining second pixel is located area as a non-overexposed area.

In this step, taking 5*5lens array (lens array) as an example, assuming that the sub-image obtained by each small lens (lens) is 20*20pixel (pixel), the original raw image obtained by the device is a piece of 5 20 pixels The original image of size 100*100 formed by splicing of *20 matrices (as shown in Figure 2B). In the actual scene, since the overexposed area of the raw image is relatively smooth, the gradient information (the difference between a certain pixel value and its eight neighboring pixels) can be used to divide each small lens into two states of light leakage and non-light leakage, and then distinguish the original Overexposed and non-overexposed areas of the raw image. First, the gradient information of each pixel in the original image is calculated. When the minimum gradient information in the eight neighborhoods of the pixel is less than the gradient threshold, and the pixel value is within the preset pixel range, the pixel is in the overexposed area. the first pixel of . Areas where other pixels that do not meet the above criteria are considered non-overexposed areas.

In one embodiment, the gradient threshold may be 12, which may be obtained by statistics based on gradient information of historical overexposed area images. The principle is that the gradient threshold is small enough to represent image information collected in a light leak scene.

In one embodiment, assuming that the maximum value of raw image pixels is H, the preset pixel range may be 9/10H to H. The double judgment standard can make the overexposed area distinguished more accurately.

Step 403: For the non-overexposed area, calculate the pixel coordinates of each sub-image obtained by the multiple collectors 13 respectively.

In this step, for the image of the non-overexposed area, the image of the user's fingerprint can be comprehensively collected, and there is no adverse effect caused by light leakage. The pixels occupied by the sub-images collected by each camera can be calculated separately according to the setting parameters of the camera. coordinate. Taking a square camera as an example, the calculation process can be as follows:

For each lens of the non-overexposed area, the following formula is used to calculate pixel coordinates (d _x , _dy ) pixel by pixel:

Among them, d _xj represents the pixel coordinate starting amount of the small lens in the jth column in the x direction, and d _yi represents the pixel coordinate starting amount of the small lens in the ith row in the y direction. lens_size represents the size of each small lens. In this embodiment, each small lens may be 20*20 pixels, then lens_size=20. Overlap represents the size of the overlapped area between sub-images of the lens, and in this embodiment, the size of the overlapped area inside the middle is 6 pixels. lens_number represents the number of lenses in each row and column of the lens array, which is 5 in this embodiment. i represents the number of lines of the original image, i=0, 1, 2 . . . in this embodiment. j represents the number of columns of the original image, and in this embodiment, j=0, 1, 2 . . .

In one embodiment, since only a part of the small lenses in the outermost circle of the lens array overlaps with other lenses, the size of the overlapping area of the small lenses in the outermost circle in this embodiment is 3 pixels, so it should be considered separately when calculating the offset. The lens in the middle will have two adjacent lenses, left and right, and the lens at the outermost corner (when i=0 and j=0 or when i=lens number-1 and j=lens number-1) is only adjacent to one lens Adjacent, so there is only half of the offset. When i=0 and j=0, the above d _xj and _{dy yi} formulas can be used in combination with the following formulas to calculate:

When i=lens number-1 and j=lens number-1, the above formulas of d _xj and d _yi can be used in combination with the following formulas to calculate:

Among them, d _x0 represents the pixel coordinate starting amount of the small lens in the 0th column in the x direction, d _y0 represents the pixel coordinate starting amount of the small lens in the 0th row in the y direction, and dx _{lens number-1} represents the lens. The starting pixel coordinates of the small lens in the number-1 column in the x direction, and dy _{lens number-1} represents the starting pixel coordinate of the small lens in the y direction of the dy _{lens number-1} row.

Step 404 : Select the image area where the pixel coordinates of the sub-images overlap each other as the overlapping area, and then go to step 406 . The remaining image areas are non-overlapping areas, and then step 408 is entered.

In this step, after the pixel coordinates of the sub-images are calculated, it can be clearly calculated which sub-image pixel points overlap each other. For example, the value range of d _x in the pixel coordinates of sub-image A is 0 to 18 , the value range of d _x in the pixel coordinates of sub-image B is 10 to 28, then the overlapping range of sub-image A and sub-image B in the d _x direction is 10 to 18, so the overlapping range in the _dy direction can be calculated. , and then the image area where the pixel coordinates of the sub-images overlap each other can be selected as the overlapping area, and step 406 is performed for subsequent calculation. The image area that does not meet the overlapping standard is the non-overlapping area, and step 408 is executed for subsequent calculation.

Step 405: For the overexposed area in the original image, select the area where the pixel coordinates of the sub-images do not overlap each other as the effective area, and calculate the third pixel information of the effective area. Go to step 409.

In this step, if there is an overexposed area in the original image, for the image of the light leakage area, in order to avoid the influence of magnifying the overexposed area on the overexposed area, the useless overexposure information will not be spliced and fused, so only sub-images are selected. The pixel coordinates of , where there is no overlapping area with each other, are used as the effective area, and the third pixel information of the effective area is calculated for subsequent image stitching, and then goes to step 409 .

In one embodiment, for the overexposed area, the third pixel information of the effective area may be calculated as shown in FIG. 2B to FIG. 2C . See the description of the above-mentioned embodiment for details. Step 406: Calculate the overlapping times of each overlapping pixel point in the overlapping area respectively.

In this step, a mask matrix of the same size as the raw image can be generated, and the matrix value is all 1. The same pixel processing algorithm is adopted as the raw image, and the accumulation times of each pixel point are recorded. One overlap is the overlapping area of the two lenses. For one accumulation, the mask here is increased by 1, so that the overlapping times of each overlapping pixel in the overlapping area can be calculated.

Step 407 : Calculate the average pixel value of the overlapping pixel points respectively according to the pixel value and the overlapping times of the overlapping pixel points. The first pixel information includes the average pixel value of each overlapping pixel point, and then go to step 409 .

In this step, taking the pixel point Q as an example, it is assumed that the number of times of overlapping at the pixel point Q calculated in step 406 is n (n is an integer greater than or equal to zero), and the pixel values of all the overlapping pixel points Q are respectively P ₁ , P ₂ ......P _n , the following formula can be used to calculate the average pixel value P _{Q of the pixel point Q} :

Step 408: Assign a preset weight to the pixel value of each non-overlapping pixel in the non-overlapping area, generate a weighted pixel value of each non-overlapping pixel, the preset weight is greater than 1, and the second pixel information includes each non-overlapping pixel. The weighted pixel value of the overlapped pixel point goes to step 409 .

In this step, for the non-overlapping area of each small lens in the non-light leakage area, a higher weight will be given during pixel processing, and a preset weight will be given to the pixel value of each non-overlapping pixel, and the preset weight will be greater than 1 , the default weight can be 1.2, making it a higher proportion. In this embodiment, the weight of 14 pixels in the non-overlapping area is increased, and the corresponding weighted pixel value can be obtained by directly multiplying the pixel value of each pixel in the non-overlapping area by 1.2.

Step 409: Perform image stitching according to the first pixel information, the second pixel information and the third pixel information to generate a stitched image of the original image.

In one embodiment, taking the raw image shown in FIG. 2B as an example, the following formula can be used to calculate the size L (unit pixel) of the spliced image:

L=[lens_size*2+(lens_size-overlap)*(lens_number-2)] ²

Then, the size of the stitched image of the raw image shown in FIG. 2B is 20*2+(20-6)*3=82*82pixel.

As shown in FIG. 5 , in the pixel distribution map obtained after processing by the above pixel processing method in this embodiment, the pixel processing of the non-overlapping area S1 is enhanced by more weights, that is, a preset weight value is assigned in step 408 1.2, directly increase its signal strength. For the overlapping area S2, if there are two pixel points Q in the overlapping area of lens 1 and lens 2, the pixel value of the pixel point Q in the actual stitched image is (pixel1+pixel2)/2. Here 2 is the number of overlaps, which is averaged with the information of the two images.

In one embodiment, the original image takes the fingerprint image as an example. Since the noise in the raw image is stronger than that of the fingerprint signal, it is difficult to distinguish the difference between the noise and fingerprint ridges between the two stitching algorithms from the analysis of the legend. The denoising algorithm converts the raw image into a denoised image that is easier to compare.

The above image stitching method can maximize the use of the fingerprint information obtained by each small lens by fusing the overlapping area of each small lens. For the non-overlapping area of each small lens, a higher weight will be given during stitching , making it a higher proportion. Compared with the stitching scheme that only uses the non-overlapping area of each small lens, on the basis of using the same denoising algorithm subsequently, the fingerprint image obtained in this embodiment can significantly improve the image quality of weak fingerprint scenes such as dry and cold, and reduce image noise. At the same time, considering the particularity of the lens array module, in the light leakage scene, the overexposed area of the image will have an impact on the non-overexposed area, forming a transition area with gradually changing brightness. In the light leakage scene, only the non-overlapping part of the image is reserved for image splicing, and the useless overexposure information is not spliced and fused to improve the accuracy of the spliced image.

As shown in Figure 6A, it is a schematic diagram of comparison after denoising the raw image of the strong fingerprint signal, the image in the area S3 is the splicing image effect obtained by directly selecting the non-overlapping area splicing algorithm, and the image in the area S4 is as shown in Figure 3 and/ Or the effect of the stitched image obtained by the image stitching method in Figure 4.

As shown in FIG. 6B , it is a schematic diagram of the comparison after the raw image of the weak fingerprint signal is denoised. The image in the area S5 is the stitched image effect obtained by directly selecting the non-overlapping area stitching algorithm, and the image in the area S6 is the effect of the stitched image obtained by the image stitching method as shown in FIG. 3 and/or FIG. 4 .

Comparing the schematic diagrams in FIGS. 6A to 6B , in a strong fingerprint scenario, the fingerprint signal obtained by the image stitching algorithm of this embodiment is more stable, the noise is much reduced, and the fingerprint is smoother. In a weak fingerprint scenario, the image splicing algorithm splicing and fusion in this embodiment can greatly reduce noise in the image, and the fingerprint signal is more stable, improving the fingerprint identification capability of the weak fingerprint scenario.

Please refer to FIG. 7 , which is an image stitching apparatus 700 according to an embodiment of the application, which can be applied to the electronic device 1 shown in FIG. 1 , and can be applied to the fingerprint recognition scene shown in FIG. 2A to FIG. 2B , Image stitching is performed on multiple original sub-images collected by the camera array. The apparatus may include: an acquisition module 701, a division module 702, a processing module 703 and a splicing module 704, and the principle relationship of each module may be as follows:

The acquisition module 701 is configured to acquire an original image to be processed, and the original image is composed of multiple sub-images collected by multiple collectors 13 . For details, refer to the description of step 301 in the above embodiment.

The dividing module 702 is configured to divide the image of the non-overexposed area into an overlapping area and a non-overlapping area in the original image. The overlapping area is an area where pixel coordinates of multiple sub-images overlap each other in the original image. For details, refer to the description of step 302 in the above embodiment.

The processing module 703 is configured to perform pixel processing on the overlapping area and the non-overlapping area, respectively, to generate first pixel information of the overlapping area and second pixel information of the non-overlapping area. For details, refer to the description of step 303 in the above embodiment.

The stitching module 704 is configured to perform image stitching of the overlapping area and the non-overlapping area according to the first pixel information and the second pixel information, to generate a stitched image of the original image. For details, refer to the description of step 304 in the above embodiment.

In one embodiment, it may further include: a determination module 705, configured to determine whether there is an overexposed area in the original image; a selection module 706, configured to be configured to, for the original image, if there is an overexposed area in the original image The overexposure area in the sub-image is selected as the effective area where the pixel coordinates of the sub-image do not overlap each other; the calculation module 707 is configured to calculate the third pixel information of the effective area; the splicing module 704 is also configured to be used according to Image splicing is performed on the first pixel information, the second pixel information and the third pixel information to generate a spliced image of the original image.

In one embodiment, the judging module 705 may be configured to: calculate the gradient information of each pixel in the original image; in the original image, find out whether there is gradient information smaller than a preset gradient threshold and the pixel value is in the preset value. If the first pixel in the set pixel range exists, the area where the first pixel is located is an overexposed area.

In one embodiment, the processing module 703 may be configured to: calculate the overlapping times of each overlapping pixel point in the overlapping area respectively; calculate the average pixel value of the overlapping pixel points according to the pixel value and the overlapping times of the overlapping pixel points, respectively. , the first pixel information includes the average pixel value of each overlapping pixel point.

In one embodiment, the processing module 703 may be further configured to: assign a preset weight to the pixel value of each non-overlapping pixel point in the non-overlapping area, generate a weighted pixel value of each non-overlapping pixel point, and The two-pixel information includes the weighted pixel value of each non-overlapping pixel point.

In one embodiment, the preset weight may be greater than 1.

In one embodiment, the dividing module 702 may be configured to: for the non-overexposed area, calculate the pixel coordinates of each sub-image obtained by the multiple collectors respectively; select the image area where the pixel coordinates of the sub-images overlap each other as: Overlapping areas, the remaining image areas are non-overlapping areas. For a detailed description of the above image stitching apparatus 700, please refer to the description of the relevant method steps in the above embodiment.

Embodiments of the present application also provide a non-transitory electronic device-readable storage medium, which may include: a program, when running on the electronic device, enables the electronic device to execute all or part of the processes of the methods in the foregoing embodiments. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, Abbreviation: HDD) or Solid-State Drive (SSD), etc. The storage medium may also include a combination of the aforementioned kinds of memories.

Although the embodiments of the present application have been described in conjunction with the accompanying drawings, various modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and variations all fall within the scope of the appended claims within the limited range.

Industrial Applicability

The present application provides an image stitching method, device, device and storage medium, the method includes: acquiring an original image to be processed, the original image is composed of multiple sub-images collected by multiple collectors; , the image of the non-overexposed area is divided into an overlapping area and a non-overlapping area; the overlapping area is an area where the pixel coordinates of a plurality of the sub-images overlap each other in the original image; Perform pixel processing on the non-overlapping area to generate first pixel information of the overlapping area and second pixel information of the non-overlapping area; according to the first pixel information and the second pixel information, the overlapping area Perform image stitching with the non-overlapping area to generate a stitched image of the original image. The present application realizes the reduction of grid noise of the stitched images, so that the stitched images are more continuous and smoother.

Furthermore, it is understood that the image stitching method, apparatus, device and storage medium of the present application are reproducible and can be used in a variety of industrial applications. For example, the image stitching method, apparatus, device and storage medium of the present application can be applied to any device that performs image stitching processing.

Claims

An image stitching method, comprising:

acquiring an original image to be processed, the original image is composed of multiple sub-images collected by multiple collectors;

In the original image, the image of the non-overexposed area is divided into an overlapping area and a non-overlapping area; the overlapping area is an area where the pixel coordinates of the plurality of sub-images overlap each other in the original image;

Perform pixel processing on the overlapping area and the non-overlapping area, respectively, to generate first pixel information of the overlapping area and second pixel information of the non-overlapping area;

According to the first pixel information and the second pixel information, image splicing is performed on the overlapping area and the non-overlapping area to generate a spliced image of the original image.
The method of claim 1, further comprising:

Determine whether there is an overexposed area in the original image;

If the overexposed area exists in the original image, for the overexposed area in the original image, select an area where the pixel coordinates of the sub-image do not overlap each other as an effective area;

calculating the third pixel information of the effective area;

Image stitching is performed according to the first pixel information, the second pixel information and the third pixel information to generate the stitched image of the original image.
The method according to claim 2, wherein the judging whether there is an overexposed area in the original image comprises:

Calculate the gradient information of each pixel in the original image;

In the original image, find out whether there is a first pixel whose gradient information is less than a preset gradient threshold and whose pixel value is within a preset pixel range, and if so, the area where the first pixel is located is the overexposed area.
The method according to any one of claims 1 to 3, wherein the step of performing pixel processing on the overlapping area to generate the first pixel information of the overlapping area includes:

Calculate the overlapping times of each overlapping pixel point in the overlapping area respectively;

According to the pixel value of the overlapping pixel point and the overlapping times, the average pixel value of the overlapping pixel point is calculated respectively, and the first pixel information includes the average pixel value of each overlapping pixel point.
The method according to any one of claims 1 to 3, wherein the step of performing pixel processing on the non-overlapping area to generate the second pixel information of the non-overlapping area includes:

assigning a preset weight to the pixel value of each non-overlapping pixel point in the non-overlapping area, generating a weighted pixel value of each of the non-overlapping pixel points, and the second pixel information includes each of the non-overlapping pixel points the weighted pixel value of the pixel point.
The method according to claim 5, wherein the preset weight is greater than 1.
The method according to any one of claims 1 to 5, wherein, in the original image, dividing the image of the non-overexposed area into an overlapping area and a non-overlapping area, comprising:

For the non-overexposed area, separately calculating the pixel coordinates of each of the sub-images obtained by the multiple collectors;

An image area where pixel coordinates of the sub-images overlap each other is selected as the overlapping area, and the remaining image areas are the non-overlapping area.
An image stitching device, comprising:

an acquisition module, configured to acquire an original image to be processed, the original image is composed of a plurality of sub-images collected by a plurality of collectors;

a dividing module, configured to divide the image of the non-overexposed area into an overlapping area and a non-overlapping area in the original image; the overlapping area is a plurality of pixel coordinates of the sub-images in the original image overlapping areas in

a processing module configured to perform pixel processing on the overlapping area and the non-overlapping area, respectively, to generate first pixel information of the overlapping area and second pixel information of the non-overlapping area;

The stitching module is configured to perform image stitching on the overlapping area and the non-overlapping area according to the first pixel information and the second pixel information to generate a stitched image of the original image.
The image stitching device according to claim 8, further comprising:

a judgment module, configured to judge whether there is an overexposed area in the original image;

A selection module, configured to select an area where the pixel coordinates of the sub-images do not overlap each other for the over-exposed area in the original image to be effective if the over-exposed area exists in the original image area;

a calculation module configured to calculate the third pixel information of the effective area;

Wherein, the stitching module is further configured to perform image stitching according to the first pixel information, the second pixel information and the third pixel information to generate the stitched image of the original image.
The image stitching device according to claim 9, wherein the judging module is configured to: calculate the gradient information of each pixel in the original image; If the gradient information is less than the preset gradient threshold and the pixel value is within the preset pixel range of the first pixel, if there is, the area where the first pixel is located is the overexposed area.
The image stitching apparatus according to any one of claims 8 to 10, wherein the processing module is configured to: respectively calculate the overlap times of each overlapped pixel point in the overlapped area; The average pixel value of the overlapping pixel points is calculated respectively according to the pixel value of the overlapping pixel points and the overlapping times, and the first pixel information includes the average pixel value of each overlapping pixel point.
The image stitching apparatus according to any one of claims 8 to 10, wherein the processing module is further configured to: assign a pixel value to each non-overlapping pixel point in the non-overlapping area A preset weight is used to generate a weighted pixel value of each of the non-overlapping pixel points, and the second pixel information includes the weighted pixel value of each of the non-overlapping pixel points.
The image stitching apparatus according to claim 12, wherein the preset weight is greater than 1.
The image stitching apparatus according to any one of claims 8 to 13, wherein the dividing module is configured to: for the non-overexposed area, respectively calculate each image obtained by the plurality of collectors pixel coordinates of the sub-images; selecting an image area where the pixel coordinates of the sub-images overlap each other as the overlapping area, and the remaining image areas as the non-overlapping area.
An electronic device, comprising:

a memory configured to store a computer program;

A processor configured to perform the method of any one of claims 1 to 7 for image stitching of the plurality of original sub-images.
A non-transitory electronic device readable storage medium, characterized by comprising: a program which, when run by the electronic device, causes the electronic device to execute the method of any one of claims 1 to 7 .