CN117911263A - Method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering - Google Patents
Method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering Download PDFInfo
- Publication number
- CN117911263A CN117911263A CN202410098683.2A CN202410098683A CN117911263A CN 117911263 A CN117911263 A CN 117911263A CN 202410098683 A CN202410098683 A CN 202410098683A CN 117911263 A CN117911263 A CN 117911263A
- Authority
- CN
- China
- Prior art keywords
- pyramid
- image
- sml
- cuda
- fsml
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 44
- 230000004927 fusion Effects 0.000 title claims abstract description 36
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000001133 acceleration Effects 0.000 claims abstract description 13
- 238000005070 sampling Methods 0.000 claims abstract description 9
- 238000004364 calculation method Methods 0.000 claims description 49
- 238000010586 diagram Methods 0.000 claims description 42
- 238000007500 overflow downdraw method Methods 0.000 claims description 19
- 238000011156 evaluation Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000000903 blocking effect Effects 0.000 claims description 2
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 claims 17
- 230000000694 effects Effects 0.000 abstract description 12
- 230000000875 corresponding effect Effects 0.000 description 19
- 230000006870 function Effects 0.000 description 11
- 238000012545 processing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 241000282693 Cercopithecidae Species 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000011960 computer-aided design Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guided filtering. Secondly, CUDA parallel acceleration Laplacian pyramid sampling is adopted, so that time loss caused by multi-layer nested circulation is avoided. And then, rapid guided filtering is adopted, so that the speed of guided filtering in sampling is improved; and the effect of guided filtering in delivering fine gradient values is preserved. Finally, the computation of the image multi-layer channel is involved, such as SML computation, laplacian pyramid computation, gaussian pyramid computation, rapid guided filtering computation and the like, the external circulation computation is not needed, the built-in API of the CUDA acceleration OpenCV is adopted to complete, and the OpenCV internally optimizes the multi-channel computation process, so that the final computation speed is higher; the invention can be well applied in some real-time tasks, such as multi-layer scanning of a microscope.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering.
Background
The limited depth of field of the vision sensor makes it difficult to acquire a fully focused image and also presents challenges for analysis and understanding of the image. In order to describe images of all objects in the same scene, a method of shooting with different focal lengths can be adopted for a plurality of times, images with different focal lengths are obtained, the images are fused into a single image, and clear parts in each image are reserved.
The multi-focal image fusion method can be divided into a plurality of kinds mathematically, mainly pixel-based and region-based distinction. Among them, the pixel-based fusion method tends to have a problem in that pixels are not independent of each other but are related to each other. Therefore, more methods are currently used, which tend to be based on region fusion; however, the method based on the region fusion needs to calculate the definition of the region, and in order to obtain better effect, an image pyramid is constructed in the process, and the definition evaluation is carried out on the multi-layer image, so that the method has higher requirements on processing equipment, and the general processing equipment has slower calculation speed and cannot be used in tasks with higher time requirements; especially, for microscope pictures with image resolution reaching 500W pixels, synchronous real-time fusion is difficult to achieve.
Disclosure of Invention
The invention provides a method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering, aiming at improving image fusion efficiency. The invention is realized by the following technical scheme.
The method for accelerating the fusion of the multi-focus images based on the CUDA rapid guided filtering is characterized by comprising the following steps of:
Step 1: inputting an image I A and an image I B of the same scene focused on different targets;
Step 2: taking out the V components V (I A) and V (I B) of the images I A and I B, respectively, and constructing laplace energy and pyramids SML V(PyramidA) and SML V(PyramidB of the V components of the images I A and I B by V (I A) and V (I B), respectively; taking out the S components S (I A) and S (I B) of the respective images I A and I B, and constructing the respective images I A and I B by S (I A) and S (I B) based on the laplace energy of the S component and the pyramids SML S(PyramidA) and SML S(PyramidB, respectively;
Step 3: fetching the first layers SML V(PyramidA)1 and SML S(PyramidA)1 of SML V(PyramidA) and SML S(PyramidA), summing the two to obtain an initial focus decision map SSML A of image I A; fetching the first layers SML V(PyramidB)1 and SML S(PyramidB)1 of SML V(PyramidB) and SML S(PyramidB), summing the two to obtain an initial focus decision map SSML B of image I B;
Step 4: taking the focusing decision diagram SSML A generated in the step 3 as an input diagram, taking the V component V (I A) of the image I A as a guide diagram, and performing rapid guide filtering to obtain a fine initial focusing decision diagram FSML A; taking the focusing decision diagram SSML B generated in the step 3 as an input diagram, taking the V component V (I B) of the image I B as a guide diagram, and performing rapid guide filtering to obtain a fine initial focusing decision diagram FSML B;
step 5: gaussian pyramids FSML (Pyramid A) and FSML (Pyramid B) are constructed for focus decision maps FSML A and FSML B, respectively;
Step 6: multiplying the Gaussian Pyramid FSML (Pyramid A) by the Laplacian energy of the V component constructed in the step 2 and the Pyramid SML V(PyramidA to obtain a fusion decision graph Pyramid D (Pyramid A) of the image I A; multiplying the Gaussian Pyramid FSML (Pyramid B) by the Laplacian energy of the V component constructed in the step 2 and the Pyramid SML V(PyramidB to obtain a fusion decision graph Pyramid D (Pyramid B) of the image I B; taking MAX (D (Pyramid A),D(PyramidB)) as a Pyramid D (Pyramid) of the final fusion decision diagram;
Step 7: constructing Laplacian pyramids L (Pyramid A) and L (Pyramid B) according to channels on the original pictures of the image I A and the image I B;
Step 8: sampling the corresponding Laplacian Pyramid obtained in the step 7 by utilizing the final fusion decision diagram Pyramid D (Pyramid) obtained in the step 6 to obtain a final Laplacian Pyramid F (Pyramid);
Step 9: and (3) reconstructing the Laplacian pyramid F (LPyramid) obtained in the step (8) to obtain a multi-focus picture fusion result.
In step 2, the image I A and the image I B are converted into HSV format by using an OpenCV library, and the V components V (I A) and V (I B) of the image I A and the image I B are taken out, and the gaussian Pyramid A and Pyramid B are respectively made; improved laplace energy and SML V are calculated for each layer of the resulting pyramid, resulting in a focus evaluation, thereby constructing image I A and image I B, respectively, based on the laplace energy of the V component and the pyramid SML V(PyramidA) and SML V(PyramidB).
In step 2, the calculation formulas of the laplace energy and pyramid SML V corresponding to the V components of the image I A and the image I B are as follows:
Wherein W represents the calculated energy and the size of the window, (i, j) represents the coordinates of the picture in the window, (x, y) represents the coordinates of the pixel point of the image; the calculation formula of ML V is:
Where step represents the variable distance between pixels, step=l, and I v (x, y) represents the V component value of the input image at the pixel coordinates (x, y).
In step 2, the image I A and the image I B are converted into HSV format by using an OpenCV library, and the S components S (I A) and S (I B) of the image I A and the image I B are taken out, and the gaussian Pyramid A and Pyramid B are respectively made; improved laplace energy and SML S are calculated for each layer of the resulting pyramid, resulting in a focus evaluation, thereby constructing image I A and image I B, respectively, based on the laplace energy of the S component and the pyramid SML S(PyramidA) and SML S(PyramidB).
In step 2, as a preferred technical solution, the calculation formulas of the laplace energy and the pyramid SML S corresponding to the S component of the image I A and the image I B are:
wherein W represents the calculated energy and the size of the window, (i, j) represents the coordinates of the picture in the window, (x, y) represents the coordinates of the pixel point of the image; the calculation formula of ML S is:
in step 2, a linear filter function of CUDA accelerating OpenCV is adopted to accelerate the calculation of the corresponding laplace energy and SML in parallel.
In step 2, when parallel acceleration is performed on calculation of corresponding Laplace energy and SML by using a linear filter function of CUDA acceleration OpenCV, a linear filter is created by using cv: CUDA: CREATELINEARFILTER, corresponding ML is calculated by using Laplace operator as convolution kernel, and then corresponding SML is calculated by using identity matrix of 3*3 as convolution kernel;
Wherein the laplace operator is:
As a preferred technical solution, in step 3, the calculation formulas of SSML A and SSML B are as follows:
SSMLA＝SMLV(PyramidA)1+SMLS(PyramidA)1；
SSMLB＝SMLV(PyramidB)1+SMLS(PyramidB)1。
as a preferred technical solution, in step 4, the calculation formulas of FSML A and FSML B are as follows:
FSMLA＝FastGuideFilterr,ε(V(IA),SSMLA)；
FSMLB＝FastGuideFilterr,ε(V(IB),SSMLB)；
Wherein FastGuideFilter r,ε (·) represents the fast steering filter function, r is the filter radius, ε is the penalty coefficient.
As an optimal technical scheme, CUDA is adopted to accelerate the fast guiding filtering in parallel in the step 4; the method comprises the following steps:
starting a kernel function according to the columns, and calculating the accumulated sum of each row in each column in parallel; starting a kernel function according to the rows, and calculating the accumulated sum of each column in each row in parallel to obtain an integral graph of the guide graph and the input graph; block computing a guide map mean value by integrating the map and the filter radius Mean value in input graph region/>In-region mean/>, of the product of the guide map and the input mapWherein I represents a corresponding input diagram, G represents a guide diagram, and W represents a region window;
The coefficient a r and the bias b r are calculated by the following formula:
the result of the guided filtering is calculated as follows:
FSML＝ar·G+br。
In step 8, as a preferred technical solution, the calculation formula of each layer of the final laplacian pyramid F (LPyramid) is as follows:
F(LPyramid)l(x,y)＝
LPyramidl(argmaxl(D(PyramidA)(x,y),D(PyramidB)(x,y)))；
Wherein l represents the layer number of the pyramid and (x, y) represents the pixel position of the image; LPyramid l denotes argmax l(D(PyramidA)(x,y),D(PyramidB) (x, y)) corresponds to L (Pyramid) of the image.
As a preferable technical scheme, in the step 8, CUDA is adopted to accelerate sampling of the Laplacian pyramid; specific: starting a kernel function according to pixel points, and calculating an index value of an original image corresponding to each pixel point in parallel to be used as a Mask; using Mask, a final laplacian pyramid was obtained.
Compared with the prior art, the invention has the following beneficial technical effects:
Firstly, the invention adopts the Laplace energy and the SML based on the area improvement to carry out focusing evaluation, and has better fusion effect compared with a pixel-based mode; secondly, rapid guiding filtering is adopted, so that the speed of the traditional guiding filtering is improved; and the effect of guided filtering in delivering fine gradient values is preserved. Thirdly, the CUDA parallel acceleration Laplacian pyramid sampling is adopted, so that time loss caused by multi-layer nested circulation is avoided; finally, the calculation task of the method of the invention relates to the calculation of the image multi-layer channel, such as SML calculation, laplacian pyramid calculation, gaussian pyramid calculation, rapid guided filtering calculation and the like, and the calculation can be completed by adopting a built-in API of CUDA (compute unified device architecture) accelerating OpenCV (open computer aided design) without external cyclic calculation, and the OpenCV internally optimizes the multi-channel calculation process, so that the final calculation speed is higher. The invention can be well applied in some real-time tasks, such as multi-layer scanning of a microscope.
Drawings
Fig. 1 is a flowchart of an image fusion method according to an embodiment of the present invention.
Fig. 2 illustrates an image I A input in the image fusion method according to the embodiment of the present invention.
Fig. 3 illustrates an image I B input in the image fusion method according to the embodiment of the present invention.
Fig. 4 is an effect diagram corresponding to laplace energy of the V component of the image I A and the pyramid SML V(PyramidA) in the image fusion method provided by the embodiment of the present invention.
Fig. 5 is an effect diagram corresponding to laplace energy of the V component of the image I B and the pyramid SML V(PyramidB) in the image fusion method provided by the embodiment of the present invention.
Fig. 6 is an initial focus decision diagram FSML (Pyramid A) of an image I A in an image fusion method according to an embodiment of the present invention.
Fig. 7 is an initial focus decision diagram FSML (Pyramid B) of an image I B in an image fusion method according to an embodiment of the present invention.
Fig. 8 is a final focus decision diagram D (Pyramid A) of an image I A in an image fusion method according to an embodiment of the present invention.
Fig. 9 is a final focus decision diagram D (Pyramid B) of an image I B in an image fusion method according to an embodiment of the present invention.
Fig. 10 is a laplacian Pyramid L (Pyramid A) of an image I A in an image fusion method according to an embodiment of the present invention.
Fig. 11 is a laplacian Pyramid L (Pyramid B) of an image I B in an image fusion method according to an embodiment of the present invention.
Fig. 12 is a schematic diagram of an image merging method according to an embodiment of the present invention, where the image merging is performed with a final laplacian pyramid F (LPyramid).
Fig. 13 is an image corresponding to a final fusion result in the image fusion method according to the embodiment of the present invention.
Note that: most of the images in the embodiment of the invention are actually color images, but are limited by patent text format, and can only be presented through gray images, so that only schematic effects can be achieved, and real effects cannot be fully presented.
Detailed Description
In order to make the technical solution of the present invention clearer, technical advantages will be more apparent, the technical solution of the present invention will be clearly and completely described below in connection with specific embodiments, it is obvious that the described embodiments are only some embodiments of the present invention, but not all embodiments, and the technical features of the embodiments may be combined on the premise of not conflicting with each other.
As shown in fig. 1, the present embodiment provides a method for accelerating multi-focus image fusion based on CUDA fast guiding filtering, which mainly includes the following steps:
Step 1: the same scene is input focused on images I A and I B of different objects.
Taking the image shown in fig. 2 as an example, the image I A and the image I B as examples, and taking the image shown in fig. 3 as an example, the two images are respectively images of the same scene focused on different targets, and in fig. 2, a small monkey with a foreground is in a focusing area, the definition is high, and a building with a background is blurred; while in the building with the background in fig. 3, the resolution is high and the foreground is blurred.
Step2: taking out the V components V (I A) and V (I B) of the images I A and I B respectively, and constructing the Laplace energy and pyramid SML V of the V components of the images I A and I B through V (I A) and V (I B) respectively; and, the S components S (I A) and S (I B) of the image I A and the image I B, respectively, are taken, and the laplace energy and pyramid SML S of the image I A and the image I B, respectively, based on the S components are constructed by S (I A) and S (I B), respectively.
Step 2a: the respective V components V (I A) and V (I B) of the images I A and I B are taken out, and the Laplace energy and pyramid SML V of the V components of the images I A and I B are respectively constructed through V (I A) and V (I B), and the specific procedures are as follows:
Specifically, the image I A and the image I B are converted into HSV formats by using an OpenCV library, the V components V (I A) and V (I B) of the image I A and the image I B are taken out, and the Gaussian pyramids Pyramid A and Pyramid B are respectively made; improved laplace energy and SML V are calculated for each layer of the resulting pyramid, and a focus evaluation is obtained, thereby constructing image I A and image I B, respectively, based on the laplace energy of the V component and the pyramid SML V(PyramidA) and SML V(PyramidB), the corresponding effect maps are shown in fig. 4 and 5, respectively.
The calculation formulas of the laplace energy and pyramid SML V of the V component of the image I A and the image I B are as follows, respectively.
The calculation formula of the laplace energy and pyramid SML V is:
Where W represents the calculated energy and the size of the window, (i, j) represents the coordinates of the picture in the window, and (x, y) represents the pixel coordinates of the image.
The calculation formula of ML V is:
In this embodiment, step=l, and I v (x, y) represents a V component value of the input image at the pixel coordinates (x, y).
Based on the above formula, the laplace energy and pyramid SML V(PyramidA) and SML V(PyramidB of the image I A and the image I B, respectively, based on the V component can be acquired, respectively.
Step 2b: the S components S (I A) and S (I B) of the images I A and I B, respectively, are taken, and the laplace energy and pyramid SML S of the images I A and I B, respectively, based on the S components are constructed by S (I A) and S (I B), respectively, as follows:
Converting the image I A and the image I B into HSV formats by using an OpenCV library, and taking out S components S (I A) and S (I B) of the image I A and the image I B respectively to prepare a Gaussian Pyramid A and a Gaussian Pyramid B respectively; improved laplace energy and SML S are calculated for each layer of the resulting pyramid, resulting in a focus evaluation, thereby constructing image I A and image I B, respectively, based on the laplace energy of the S component and the pyramid SML S(PyramidA) and SML S(PyramidB).
The formulas of the laplace energy and pyramid SML S of the S component of the image I A and the image I B are as follows.
The calculation formula of the laplace energy and pyramid SML S is:
Where W represents the calculated energy and the size of the window, (i, j) represents the coordinates of the picture in the window, and (x, y) represents the pixel coordinates of the image.
The calculation formula of ML S is:
Wherein step represents a variable distance between pixels, in this embodiment step=l, I s (x, y) represents an S component value of the input image at the pixel coordinates (x, y);
Based on the above formula, the laplace energy and pyramid SML S(PyramidA) and SML S(PyramidB of each of the image I A and the image I B based on the S component can be acquired, respectively.
The execution sequence of the step 2a and the step 2b may be different, the step 2a may be executed first and then the step 2b may be executed, the step 2b may be executed first and then the step 2a may be executed, or the step 2a and the step 2b may be executed simultaneously.
In addition, in step 2a and step 2b, parallel acceleration is performed on the corresponding calculation of the laplace energy and the SML by adopting a linear filter function of CUDA acceleration OpenCV; specifically, a linear filter is created by using cv: cuda: CREATELINEARFILTER, a Laplacian operator is adopted as a convolution kernel to calculate corresponding ML, and then an identity matrix of 3*3 is used as the convolution kernel to calculate corresponding SML;
Wherein the laplace operator is:
Step 3: taking out the first layers SML V(PyramidA)1 and SML S(PyramidA)1 of SML V(PyramidA) and SML S(PyramidA) generated in step 2, summing the two to obtain an initial focus decision map SSML A of image I A; taking out the first layers SML V(PyramidB)1 and SML S(PyramidB)1 of SML V(PyramidB) and SML S(PyramidB) generated in step 2, summing the two to obtain an initial focus decision map SSML B of image I B;
Wherein, the calculation formula of SSML A is:
SSMLA＝SMLV(PyramidA)1+SMLS(PyramidA)1；
wherein, the calculation formula of SSML B is:
SSMLB＝SMLV(PyramidB)1+SMLS(PyramidB)1。
Step 4: taking the focusing decision diagram SSML A generated in the step 3 as an input diagram, taking the V component V (I A) of the image I A as a guide diagram, and performing rapid guide filtering to obtain a fine initial focusing decision diagram FSML A; taking the focusing decision diagram SSML B generated in the step 3 as an input diagram, taking the V component V (I B) of the image I B as a guide diagram, and performing rapid guide filtering to obtain a fine initial focusing decision diagram FSML B;
The formulas for FSML A and FSML B are as follows:
FSMLA＝FastGuideFilterr,ε(V(IA),SSMLA)；
FSMLB＝FastGuideFilterr,ε(V(IB),SSMLB)；
Wherein FastGuideFilter r,ε (·) represents the fast steering filter function, r is the filter radius, ε is the penalty coefficient.
In addition, CUDA is adopted to accelerate the rapid guided filtering in parallel in the step 4; the method comprises the following steps:
firstly, starting a kernel function according to columns, and calculating the accumulation sum of each row in each column in parallel; starting a kernel function according to the rows, and calculating the accumulated sum of each column in each row in parallel to obtain an integral graph of the guide graph and the input graph; by integrating the graph and the filter radius r, the average value of the guide graph is calculated in a blocking mode Mean value in input graph region/>In-region mean/>, of the product of the guide map and the input mapWherein I represents a corresponding input diagram, G represents a guide diagram, and W represents a region window;
The coefficient a r and the bias b r are calculated by the following formula:
the result of the guided filtering is calculated as follows:
FSML＝ar·G+br。
Step 5: gaussian pyramids FSML (Pyramid A) and FSML (Pyramid B) are constructed for focus decision maps FSML A and FSML B generated in step 4, respectively, as shown in fig. 6 and 7, respectively.
Step 6: multiplying the Gaussian Pyramid FSML (Pyramid A) constructed in the step 5 by the Laplacian energy of the V component constructed in the step 2 and the Pyramid SML V(PyramidA to obtain a fusion decision graph Pyramid D (Pyramid A) of an image I A, as shown in fig. 8; multiplying the Gaussian Pyramid FSML (Pyramid B) constructed in the step 5 by the Laplacian energy of the V component constructed in the step 2 and the Pyramid SML V(PyramidB to obtain a fusion decision graph Pyramid D (Pyramid B) of an image I B, as shown in fig. 9; and taking MAX (D (Pyramid A),D(PyramidB)) as the final fusion decision map Pyramid D (Pyramid).
Step 7: the original pictures of the images I A and I B are constructed as laplacian pyramids L (Pyramid A) and L (Pyramid B) according to channels, as shown in fig. 10 and 11;
Step 8: and (3) sampling the corresponding Laplacian Pyramid obtained in the step (7) by utilizing the final fusion decision diagram Pyramid D (Pyramid) obtained in the step (6) to obtain a final Laplacian Pyramid F (LPyramid), as shown in fig. 12.
Wherein, the calculation formula of each layer of the final laplacian pyramid F (LPyramid) is as follows:
F(LPyramid)l(x,y)＝
LPyramidl(argmaxl(D(PyramidA)(x,y),D(PyramidB)(x,y)))；
wherein l represents the layer number of the pyramid and (x, y) represents the pixel position of the image; LPyramid l denotes argmax l(D(PyramidA)(x,y),D(PyramidB) (x, y)) corresponds to L (Pyramid) of the image; for example, when D (Pyramid A) (x, y) is greater than D (Pyramid B) (x, y), LPyramid l selects L (Pyramid A),D(PyramidA) (x, y) to be less than D (Pyramid B) (x, y), LPyramid l selects L (Pyramid B).
In addition, in the step 8, CUDA is adopted to accelerate sampling of the Laplacian pyramid; specifically, starting a kernel function according to pixel points, and calculating an index value of an original image corresponding to each pixel point in parallel to be used as a Mask; using Mask, a final laplacian pyramid was obtained.
Step 9: reconstructing the laplacian pyramid F (LPyramid) obtained in the step 8 to obtain a multi-focus picture fusion result, as shown in fig. 13.
Compared with the prior art, the invention has the following beneficial technical effects:
The invention adopts the improved Laplace energy and SML based on the region to carry out focusing evaluation, and has better fusion effect compared with a pixel-based mode; the rapid guiding filtering is adopted, so that the speed of the guiding filtering during the adoption is improved; and the effect of guided filtering in delivering fine gradient values is preserved. In addition, the invention only calculates the guiding filtering of the first layer of the focusing image, and other layers are obtained by adopting Gaussian filtering downsampling, so that the time loss caused by calculation and evaluation of each layer is avoided; in addition, the calculation task of the method of the invention relates to the calculation of the image multi-layer channel, such as SML calculation, laplacian pyramid calculation, gaussian pyramid calculation, rapid guided filtering calculation and the like, and the calculation can be completed by adopting an API (application program interface) built in OpenCV (open computer program language) without external circulation calculation, and the OpenCV internally optimizes the multi-channel calculation process, so that the final calculation speed is higher. So that the invention can be well applied in some real-time tasks, such as multi-layer scanning of a microscope.
The above embodiments are only preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, but any insubstantial changes and substitutions made by those skilled in the art on the basis of the present invention are intended to be within the scope of the present invention as claimed.
Claims (12)
1. The method for accelerating the fusion of the multi-focus images based on the CUDA rapid guided filtering is characterized by comprising the following steps of:
Step 1: inputting an image I A and an image I B of the same scene focused on different targets;
Step 2: taking out the V components V (I A) and V (I B) of the images I A and I B, respectively, and constructing laplace energy and pyramids SML V(PyramidA) and SML V(PyramidB of the V components of the images I A and I B by V (I A) and V (I B), respectively; taking out the S components S (I A) and S (I B) of the respective images I A and I B, and constructing the respective images I A and I B by S (I A) and S (I B) based on the laplace energy of the S component and the pyramids SML S(PyramidA) and SML S(PyramidB, respectively;
Step 3: fetching the first layers SML V(PyramidA)1 and SML S(PyramidA)1 of SML V(PyramidA) and SML S(PyramidA), summing the two to obtain an initial focus decision map SSML A of image I A; fetching the first layers SML V(PyramidB)1 and SML S(PyramidB)1 of SML V(PyramidB) and SML S(PyramidB), summing the two to obtain an initial focus decision map SSML B of image I B;
Step 4: taking the focusing decision diagram SSML A generated in the step 3 as an input diagram, taking the V component V (I A) of the image I A as a guide diagram, and performing rapid guide filtering to obtain a fine initial focusing decision diagram FSML A; taking the focusing decision diagram SSML B generated in the step 3 as an input diagram, taking the V component V (I B) of the image I B as a guide diagram, and performing rapid guide filtering to obtain a fine initial focusing decision diagram FSML B;
step 5: gaussian pyramids FSML (Pyramid A) and FSML (Pyramid B) are constructed for focus decision maps FSML A and FSML B, respectively;
Step 6: multiplying the Gaussian Pyramid FSML (Pyramid A) by the Laplacian energy of the V component constructed in the step 2 and the Pyramid SML V(PyramidA to obtain a fusion decision graph Pyramid D (Pyramid A) of the image I A; multiplying the Gaussian Pyramid FSML (Pyramid B) by the Laplacian energy of the V component constructed in the step 2 and the Pyramid SML V(PyramidB to obtain a fusion decision graph Pyramid D (Pyramid B) of the image I B; taking MAX (D (Pyramid A),D(PyramidB)) as a Pyramid D (Pyramid) of the final fusion decision diagram;
Step 7: constructing Laplacian pyramids L (Pyramid A) and L (Pyramid B) according to channels on the original pictures of the image I A and the image I B;
Step 8: sampling the corresponding Laplacian Pyramid obtained in the step 7 by utilizing the final fusion decision diagram Pyramid D (Pyramid) obtained in the step 6 to obtain a final Laplacian Pyramid F (Pyramid);
Step 9: and (3) reconstructing the Laplacian pyramid F (LPyramid) obtained in the step (8) to obtain a multi-focus picture fusion result.
2. The method for accelerating multi-focus image fusion based on CUDA fast guiding filtering according to claim 1, wherein in step 2, the image I A and the image I B are converted into HSV format by using OpenCV library, and the V components V (I A) and V (I B) of the image I A and the image I B are extracted, and the gaussian Pyramid A and Pyramid B are respectively made; improved laplace energy and SML V are calculated for each layer of the resulting pyramid, resulting in a focus evaluation, thereby constructing image I A and image I B, respectively, based on the laplace energy of the V component and the pyramid SML V(PyramidA) and SML V(PyramidB).
3. The CUDA-based fast guided filtering acceleration multi-focal image fusion method according to claim 2, wherein in step 2, the calculation formulas of laplace energy and pyramid SML V corresponding to the V components of the image I A and the image I B are as follows:
Wherein W represents the calculated energy and the size of the window, (i, j) represents the coordinates of the picture in the window, (x, y) represents the coordinates of the pixel point of the image; the calculation formula of ML V is:
Where step represents the variable distance between pixels, step=l, and I v (x, y) represents the V component value of the input image at the pixel coordinates (x, y).
4. The method for accelerating multi-focus image fusion based on CUDA fast guiding filtering according to claim 2, wherein in step 2, the image I A and the image I B are converted into HSV format by using OpenCV library, and the S components S (I A) and S (I B) of the image I A and the image I B are extracted, and the gaussian Pyramid A and Pyramid B are respectively made; improved laplace energy and SML S are calculated for each layer of the resulting pyramid, resulting in a focus evaluation, thereby constructing image I A and image I B, respectively, based on the laplace energy of the S component and the pyramid SML S(PyramidA) and SML S(PyramidB).
5. The CUDA-based fast guided filtering acceleration multi-focal image fusion method according to claim 4, wherein in step 2, the calculation formulas of laplace energy and pyramid SML S corresponding to S components of the image I A and the image I B are:
wherein W represents the calculated energy and the size of the window, (i, j) represents the coordinates of the picture in the window, (x, y) represents the coordinates of the pixel point of the image; the calculation formula of ML S is:
6. The method for accelerating multi-focus image fusion based on CUDA fast steering filtering according to claim 3 or 5, wherein in step 2, the calculation of the corresponding laplace energy and SML is accelerated in parallel by using a linear filter function of CUDA accelerating OpenCV.
7. The method for accelerating multi-focus image fusion based on CUDA rapid guided filtering according to claim 6, wherein in step 2, when parallel acceleration is performed on the calculation of the corresponding Laplacian energy and SML by adopting a linear filter function of CUDA accelerating OpenCV, a linear filter is created by using cv: CUDA: CREATELINEARFILTER, the corresponding ML is calculated by adopting the Laplacian operator as a convolution kernel, and then the corresponding SML is calculated by adopting an identity matrix of 3*3 as a convolution kernel;
Wherein the laplace operator is:
8. The CUDA-based fast guided filtering acceleration multi-focal image fusion method according to claim 1, wherein in step 3, the calculation formulas of SSML A and SSML B are:
SSMLA＝SMLV(PyramidA)1+SMLS(PyramidA)1；
SSMLB＝SMLV(PyramidB)1+SMLS(PyramidB)1。
9. The CUDA-based fast guided filter acceleration multi-focal image fusion method of claim 8, wherein in step 4, the calculation formulas of FSML A and FSML B are as follows:
FSMLA＝FastGuideFilterr,ε(V(IA),SSMLA)；
FSMLB＝FastGuideFilterr,ε(V(IB),SSMLB)；
Wherein FastGuideFilter r,ε (·) represents the fast steering filter function, r is the filter radius, ε is the penalty coefficient.
10. The method for accelerating multi-focus image fusion based on CUDA fast steering filtering according to claim 9, wherein in step 4, CUDA is adopted to accelerate fast steering filtering in parallel; the method comprises the following steps:
Starting a kernel function according to the columns, and calculating the accumulated sum of each row in each column in parallel; starting a kernel function according to the rows, and calculating the accumulated sum of each column in each row in parallel to obtain an integral graph of the guide graph and the input graph; by integrating the graph and the filter radius r, the average value of the guide graph is calculated in a blocking mode Mean value in input graph region/>In-region mean/>, of the product of the guide map and the input mapWherein I represents a corresponding input diagram, G represents a guide diagram, and W represents a region window;
The coefficient a r and the bias b r are calculated by the following formula:
the result of the guided filtering is calculated as follows:
FSML＝ar·G+br。
11. the CUDA-based fast guided filtering acceleration multi-focal image fusion method according to claim 1, wherein in step 8, a calculation formula of each layer of the final laplacian pyramid F (LPyramid) is as follows:
F(LPyramid)l(x,y)＝LPyramidl(argmaxl(D(PyramidA)(x,y),D(PyramidB)(x,y)));
Wherein l represents the layer number of the pyramid and (x, y) represents the pixel position of the image; LPyramid l denotes argmax l(D(PyramidA)(x,y),D(PyramidB) (x, y)) corresponds to L (Pyramid) of the image.
12. The method for accelerating multi-focus image fusion based on CUDA fast steering filtering according to claim 11, wherein CUDA is adopted to accelerate sampling of laplacian pyramid in step 8; specific: starting a kernel function according to pixel points, and calculating an index value of an original image corresponding to each pixel point in parallel to be used as a Mask; using Mask, a final laplacian pyramid was obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410098683.2A CN117911263A (en) | 2024-01-24 | 2024-01-24 | Method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410098683.2A CN117911263A (en) | 2024-01-24 | 2024-01-24 | Method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117911263A true CN117911263A (en) | 2024-04-19 |
Family
ID=90686748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410098683.2A Pending CN117911263A (en) | 2024-01-24 | 2024-01-24 | Method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117911263A (en) |
-
2024
- 2024-01-24 CN CN202410098683.2A patent/CN117911263A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111007661B (en) | Microscopic image automatic focusing method and device based on deep learning | |
| US10353271B2 (en) | Depth estimation method for monocular image based on multi-scale CNN and continuous CRF | |
| CN111340844B (en) | Multi-scale characteristic optical flow learning calculation method based on self-attention mechanism | |
| CN109782414B (en) | Automatic focusing method based on definition of non-reference structure | |
| JP5777367B2 (en) | Pattern identification device, pattern identification method and program | |
| CN112581379A (en) | Image enhancement method and device | |
| CN111126385A (en) | Deep learning intelligent identification method for deformable living body small target | |
| CN111784620B (en) | Light field camera full-focusing image fusion algorithm for guiding angle information by space information | |
| CN103440662A (en) | Kinect depth image acquisition method and device | |
| CN113052755A (en) | High-resolution image intelligent matting method based on deep learning | |
| CN113344958B (en) | Microscopic imaging scanning method and scanning system | |
| WO2018129692A1 (en) | Image refocusing | |
| CN111105452A (en) | High-low resolution fusion stereo matching method based on binocular vision | |
| Wang et al. | A simple and robust super resolution method for light field images | |
| CN112419191A (en) | Image motion blur removing method based on convolution neural network | |
| CN111882581A (en) | Multi-target tracking method for depth feature association | |
| CN112633248B (en) | Deep learning full-in-focus microscopic image acquisition method | |
| CN112132771B (en) | Multi-focus image fusion method based on light field imaging | |
| CN113763300A (en) | Multi-focus image fusion method combining depth context and convolution condition random field | |
| CN110766609B (en) | Depth-of-field map super-resolution reconstruction method for ToF camera | |
| CN117911263A (en) | Method for accelerating multi-focus image fusion based on CUDA (compute unified device architecture) rapid guide filtering | |
| Wang et al. | Shape-from-focus reconstruction using block processing followed by local heat-diffusion-based refinement | |
| CN113469286A (en) | Spacecraft multi-local component detection method based on regional convolutional neural network | |
| Tran et al. | Variational disparity estimation framework for plenoptic images | |
| CN112419180A (en) | High-dynamic star point extraction method for unknown direction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |