CN117218016A - Image blind deblurring method, system, equipment and medium based on edge enhancement - Google Patents
Image blind deblurring method, system, equipment and medium based on edge enhancement Download PDFInfo
- Publication number
- CN117218016A CN117218016A CN202311063076.4A CN202311063076A CN117218016A CN 117218016 A CN117218016 A CN 117218016A CN 202311063076 A CN202311063076 A CN 202311063076A CN 117218016 A CN117218016 A CN 117218016A
- Authority
- CN
- China
- Prior art keywords
- image
- original
- clear image
- formula
- blind deblurring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 239000011159 matrix material Substances 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000005070 sampling Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000005498 polishing Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000013459 approach Methods 0.000 claims description 3
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000015556 catabolic process Effects 0.000 claims description 3
- 238000006731 degradation reaction Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 claims description 2
- 238000011084 recovery Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Processing (AREA)
Abstract
An image blind deblurring method, system, equipment and medium based on edge enhancement, wherein the method comprises the following steps: acquiring a blurred image and layering the pyramid to construct an image pyramid, obtaining a regular term of an original clear image by using a weight matrix and a gradient of the acquired original clear image, establishing an image blind deblurring model, solving the original clear image by using the model, solving a blur kernel under the same layer of scale in the image pyramid, up-sampling the blur kernel to serve as an initial blur kernel of the next layer, performing circulation until a final blur kernel is obtained, and processing the original image by using the final blur kernel to obtain a final clear image; the system, the device and the medium are used for realizing an image blind deblurring method based on edge enhancement; the invention has the advantages of high accuracy, strong restoration performance and clear restored image.
Description
Technical Field
The invention relates to the technical field of image processing and pattern recognition, in particular to an image blind deblurring method, an image blind deblurring system, image blind deblurring equipment and an image blind deblurring medium based on edge enhancement.
Background
With the popularization of imaging devices such as digital cameras, smart phones and cameras, image acquisition becomes easier, however, in the exposure time of the cameras, factors such as environment, defocus of the cameras and relative movement between the cameras and a target object can blur the acquired image, and in practical application, clear images are required, so that image deblurring is an important research direction in image processing, and the image blurring process can be generally described as follows:wherein->The two-dimensional linear convolution operator is represented, B, k, I and n respectively represent a blurred image, a blur kernel, a clear image and independent and equidistributed Gaussian white noise, the purpose of image deblurring is to recover the original clear image I from the observed blurred image B, if the blur kernel is known, the image deblurring is called image non-blind deblurring, otherwise, the image blind deblurring is called image blind deblurring, and in practical application, the information of the blur kernel is generally difficult to obtain.
Image blind deblurring is a highly ill-posed problem, and the current mainstream method for solving the image blind deblurring is maximum posterior probability estimation, and in 2009, levin et al point out: simultaneously estimating a blur kernel and a clean image which cannot obtain an ideal result, and providing an idea of estimating the blur kernel first and then carrying out image recovery through a non-blind deblurring method, wherein the estimation of the blur kernel is a key for solving the problem of image deblurring; cho, lee and the like explicitly extract sharp edges by using bilateral filtering and impact filtering, estimate fuzzy kernels by using the edges, and then obtain clean images by using fast Fourier transform; xu and the like consider that the remarkable edge is not completely favorable for the accurate estimation of the fuzzy core, an additional edge selection step is still needed, and an image edge selection method capable of improving the estimation performance of the fuzzy core is provided; aiming at deblurring a text image, pan and the like, an L0 sparse regularization model of image gray scale and gradient thereof is provided; then, through statistical feature analysis of natural images, pan and the like propose a new image priori-dark channel priori (marked as DCP), and design an image blind deblurring model based on the DCP; in 2019, chen et al proposed another image prior, the mode of the largest gradient in the local neighborhood of the image (denoted as LMG), and established an LMG model.
The method shows that the significant edge of the original clear image is the key of fuzzy kernel estimation, and the accuracy of the fuzzy kernel influences the definition of the final restored image; while effective in recovering significant edges, the LMG model can widen the image edges to such an extent that artifacts exist near the edges in the original sharp image, resulting in estimated blur kernels that are far from the true blur kernels, and thus failing to recover the sharp image, where the width of the image edge expansion depends on the size of the neighborhood, a problem that arises from the definition of LMG.
The patent application of publication number CN114998146a discloses an image semi-blind deblurring method of deconvolution total least squares with bias correction, comprising selecting a regularization term based on an image blur imaging model; constructing an image semi-blind deblurring model based on deconvolution total least square; solving a potentially clear image of parameters in the image semi-blind deblurring model; performing deviation correction; solving a potential clear image of parameters in the image semi-blind deblurring model if the parameters are not converged or the maximum iteration times are not reached, otherwise, outputting a final recovered clear image; however, because the regular term of the original clear image in the model can bring about artifacts, the model cannot estimate the original clear image with clear and significant edges, so that the fuzzy core cannot be effectively estimated, and the clear image cannot be restored.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention aims to provide an image blind deblurring method, a system, equipment and a medium based on edge enhancement, which measure the edge strength by calculating the local second order statistic HOS of each pixel point of an original clear image, obtain an optimized model for estimating the clear image and a fuzzy kernel by using the HOS as a weight, solve the model to obtain a final fuzzy kernel, and obtain the final clear image by adopting a non-blind deblurring method for the fuzzy image.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
an image blind deblurring method based on edge enhancement comprises the following steps:
step 1, acquiring an original blurred image, converting the blurred image into a gray image if the blurred image is a color image, and layering the pyramid by downsampling to construct an image pyramid; if the blurred image is a black-and-white image, pyramid layering is directly carried out on the blurred image through downsampling, and an image pyramid is constructed;
step 2, acquiring an original clear image, and calculating a local second order statistic HOS value on each pixel point of the original clear image so as to obtain a weight matrix of the original clear image; obtaining a regular term of the original clear image by using the weight matrix and the gradient of the original clear image, and establishing an image blind deblurring model;
step 3, taking the blurred image under each layer of scale in the image pyramid in the step 1 as an initial original clear image of the layer, calculating an initial weight matrix, and solving the original clear image under the layer according to the image blind deblurring model constructed in the step 2;
step 4, solving the original clear image under the layer obtained by solving in the step 3 to obtain a fuzzy kernel of the original clear image under the same layer scale;
step 5, up-sampling the fuzzy core of the original clear image obtained by solving in the step 4, transmitting the up-sampled fuzzy core to the next layer of scale in the image pyramid in the step 1, and taking the up-sampled fuzzy core as the initial fuzzy core of the next layer, and circularly and alternately executing the step 3 and the step 4 according to the set times until the final fuzzy core is obtained on the finest scale;
and 6, processing the original blurred image by using the final blurred image estimated in the step 5 to obtain a final clear image.
The step 2 of establishing an image blind area fuzzy model specifically comprises the following steps:
2.1 the regular term of the original sharp image is shown as formula (1):
in the formula (1), I represents an original clear image, H i,j Representing the local second order statistic HOS value at each pixel point of the original clear image, which is defined asR i,j A rectangular region having a size of m×n centered on the pixel (i, j) is shown, and the average pixel value in the region is: />In which H is more than or equal to 0 i,j Is less than or equal to 1; when R is i,j When the pixel is positioned in the smooth area, the pixel in the area is not changed greatly, and the corresponding H i,j Smaller; when R is i,j When the edge region is included, the pixels in R (p) change drastically, and the corresponding H i,j Larger; thus, when (i, j) is in the smooth region, H, while minimizing the objective function described in step two i,j →0，1-H i,j 1, such that->Polishing noise in the region; when (i, j) is at the edge position, H i,j →1，1-H i,j 0, such that->Enhancing the image edges;
2.2, establishing an image blind deblurring model according to a regular term of an original clear image, wherein the image blind deblurring model is shown as a formula (2):
in equation (2), regularization parameters γ, τ, λ、β＞0；Representing a two-dimensional linear convolution operator; B. k and I respectively represent a blurred image, a blur kernel and an original clear image; h i,j Representing the value of the local second order statistic HOS at the (I, j) position in the original sharp image I; />The gradient of the pixel point (I, j) on the blur kernel I is represented by: />Wherein,and->Representing the partial derivatives of the image I in the horizontal and vertical directions, respectively; gradient->Is->Is defined as the sum of the absolute values of its two components: />Then there are: /> L of (2) 0 Norms mean +.>Number of non-zero elements: />
The original clear image is solved in the step 3, specifically:
initializing regularization parameters and blur kernels in equation (2), as shown in equation (3):
in the formula (3), I t And k t The results of the t-th iteration of the original sharp image I and the blur kernel k are shown respectively,is I t The local second order statistic HOS value of the pixel point (i, j) is solved by adopting a half-quadratic splitting method and an alternating minimization mode, and the method comprises the following steps:
3.1 introduction of auxiliary parameter eta instead ofg＝(g h ,g v ) Approximation->As shown in formula (4):
in the formula (4), the parameter alpha 1 ,α 2 ＞0，η i,j Is the (i, j) element of eta,Λ represents a modulo operation:thus (S)>
Equation (4) can be solved by alternatively minimizing the following three equations:
3.2 hypothesis I t+1,s Updating eta is known t+1,s+1 As shown in formula (8):
in the formula (8), 1 represents a matrix of all 1 elements, and the size and I t+1,s In accordance with the method, the device and the system,from I t HOS value of (i, j);
3.3 introducing a parameter J to approach I, deforming equation (7) to equation (9):
in the formula (9), the parameter alpha 3 Equation (9) can be solved by alternate minimization to yield two sub-problems as shown in equation (10) and equation (11):
in the formulas (10) and (11), the parameter α 3 ＞0；
3.4 based on η obtained in step 3.2 t+1,s+1 Suppose I t+1,s+1,d Update J is known t+1,s+1,d+1 As shown in formula (12):
3.5 hypothesis I t+1,s+1,d+1,r Update g is known t+1,s+1,d+1,r+1 As shown in formula (13):
3.6 based on J obtained in step 3.4 and step 3.5 t+1,s+1,d+1 And g t+1,s+1,d+1,r+1 Updating I by fast Fourier transform t+1,s+1,d+1,r+1 As shown in formula (14):
in formula (14), F (&) and F -1 (·) represent the fast fourier transform and its inverse,representing complex conjugate operator and having +.>
In the step 4, according to the original clear image under the layer obtained by solving in the step 3, a fuzzy core of the original clear image under the same layer scale is obtained by solving, and the specific steps are as shown in the formula (15):
in the formula (15), u t+1 Representing the original sharp image I t+1 The expression of which is shown in formula (16):
in the formula (16), the parameter epsilon > 0, and p represents a pixel point in the image u; d (D) h (p) and D v (p) represents the total variation of the window in the horizontal direction and the vertical direction, respectively: r (p) represents a variation region centered on p pixels, q represents a pixel in the region, g p,q Gaussian weighting function representing standard deviation sigma, i.e.>p i And p j Respectively representing the position coordinates of the p point in the horizontal direction and the vertical direction, q i And q j Respectively representing the position coordinates of the q point in the horizontal direction and the vertical direction; l (L) h (p) and L v (p) represents inherent window degradation: />
Solving the formula (15) by fast fourier transform as shown in the formula (17):
estimating a fuzzy kernel k t+1 Then, the negative element of the fuzzy core is set to 0, and the next step is carried outAnd (5) normalization treatment.
And in the step 6, the final blur kernel estimated in the step 5 is used for processing the original image to obtain a final clear image, and the specific steps are as follows:
6.1 estimating the original clear image by using the Laplacian prior method according to the blurred image and the final blur kernel estimated in the step 5, and recording as I L ；
6.2 estimating the distinct image using equation (18) as I 0 ：
6.3 calculation of the original clear image I L And clear image I 0 A difference image between the two images, and removing artifacts by using bilateral filtering;
6.4 in the original clear image I L And subtracting the filtered difference image to obtain a final clear image.
An edge-enhanced based image blind deblurring system, comprising:
the image acquisition preprocessing module is used for acquiring an original blurred image, converting a color image into a gray image, and carrying out pyramid layering on the blurred image through downsampling to construct an image pyramid; collecting an original clear image, and calculating a local second order statistic HOS value on each pixel point of the original clear image so as to obtain a weight matrix of the original clear image;
the image blind deblurring model construction module is used for obtaining a regular term of the original clear image by utilizing the weight matrix of the original clear image and the gradient of the original clear image obtained by the image acquisition preprocessing module, and establishing an image blind deblurring model;
the image blind deblurring model solving module is used for carrying out model solving on the image blind deblurring model established by the image blind deblurring model establishing module to obtain an original clear image;
the fuzzy core solving module is used for solving according to the original clear image obtained by the image blind deblurring model solving module to obtain an estimated fuzzy core;
and the clear image generation module is used for obtaining a final clear image by adopting a non-blind deblurring method according to the blurred image and the blurred kernel estimated by the blurred kernel solving module.
An edge-enhancement-based image blind deblurring apparatus, comprising:
a memory for storing a computer program;
a processor for performing an edge-enhanced based image blind deblurring method in the computer program.
A computer readable medium storing a computer program which, when executed by a processor, is capable of blindly deblurring an image based on edge enhancement.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the method, the local second order statistic HOS value on each pixel point of the original clear image is calculated in the step 2 to obtain the weight matrix, and the regular term of the original clear image is obtained by utilizing the weight matrix, so that the image edge is enhanced while the interference of polishing noise is realized, and the restoration performance of the image is improved.
2. According to the invention, the fuzzy core of the original clear image is solved through the step 4 and the step 5, and the final fuzzy core is estimated on the finest scale, so that the accuracy of fuzzy core estimation is improved.
3. According to the invention, the final fuzzy check estimated in the step 5 is utilized in the step 6, and the original fuzzy image is subjected to a non-blind deblurring method to obtain a clear image, so that the recovery of the fuzzy image is clearer.
In summary, the method and the device have the advantages of strong image restoration performance, high estimation accuracy of the blur kernel and clear restored image, by calculating the HOS value of the local second order statistic on each pixel point of the clear image to obtain the weight matrix, utilizing the regular term of the original clear image obtained by the weight matrix to establish the image blind deblurring model, obtaining the final blur kernel after the model is solved, and utilizing the final blur kernel to obtain the clear image by adopting the non-blind deblurring method.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a graph showing the comparison between the average peak signal-to-noise ratio PSNR values of deblurred images obtained by other methods according to an embodiment of the present invention.
FIG. 3 is a diagram showing the visual effect of the image restored by other methods according to the embodiment of the present invention, wherein FIG. 3 (a) is a blurred image and FIG. 3 (b) is a method using L 0 The regularized model restored image, fig. 3 (c) is an image restored using the DCP model, fig. 3 (d) is an image restored using the LMG model, fig. 3 (e) is an image restored using the method of the present invention, and fig. 3 (f) is a true clear image.
Detailed Description
The present invention will be described in detail with reference to the accompanying drawings.
The present invention provides an embodiment.
Referring to fig. 1, an image blind deblurring method based on edge enhancement includes the steps of:
step 1, collecting original blurred images, wherein in the embodiment, the collected blurred images are 351×502 in size, pyramid layering is performed through downsampling, an image pyramid is constructed, and the sizes of images on thick to thin layers are respectively: 123×176, 174×249, 247×353, 351×502;
step 2, acquiring an original clear image, and calculating a local second order statistic HOS value on each pixel point of the original clear image so as to obtain a weight matrix of the original clear image; the method comprises the steps of obtaining a regularization term related to an original clear image by using a weight matrix and gradients of the original clear image, and establishing an image blind deblurring model, and specifically comprises the following steps:
2.1 regular terms of the original sharp image, as shown in formula (1):
in the formula (1), HOS value H i,j Is defined asR i,j A rectangular region having a size of m×n centered on the pixel (i, j) is shown, and the average pixel value in the region is: />In which H is more than or equal to 0 i,j Is less than or equal to 1; when R is i,j When the pixel is positioned in the smooth area, the pixel in the area is not changed greatly, and the corresponding H i,j Smaller; when R is i,j When the edge region is included, the pixels in R (p) change drastically, and the corresponding H i,j Larger; thus, when (i, j) is in the smooth region, H, while minimizing the objective function described in step two i,j →0，1-H i,j 1, such that->Polishing noise in the region; when (i, j) is at the edge position, H i,j →1，1-H i,j 0, such that->Enhancing the image edge, wherein the regular term on the original clear image realizes the effect of polishing noise and enhancing the image edge at the same time;
2.2, establishing an image blind deblurring model according to a regular term of an original clear image, wherein the image blind deblurring model is shown as a formula (2):
in the formula (2), regularization parameters gamma, tau, lambda and beta are more than 0;representing a two-dimensional linear convolution operator; B. k and I respectively represent a blurred image, a blur kernel and an original clear image; h i,j Representing the value of the local second order statistic HOS at the (I, j) position in the original sharp image I; />The gradient of the pixel point (I, j) on the blur kernel I is represented by: />Wherein,and->Representing the partial derivatives of the image I in the horizontal and vertical directions, respectively; gradient->Is->Is defined as the sum of the absolute values of its two components: />Then there are: /> L of (2) 0 Norms mean +.>Number of non-zero elements: />The HOS value on each pixel point of the original clear image is calculated in the step 2 to obtain a weight matrix, and the weight matrix is utilized to obtain a regular term of the original clear image, so that the edge of the image is enhanced while the noise is polished, and the restoration performance of the image is improved;
step 3, initializing regularization parameters and a blur kernel of an objective function, taking a blurred image under each layer of scale in an image pyramid as an initial original clear image of the layer, calculating an initial weight matrix, and solving the original clear image under the layer as shown in a formula (3) according to a constructed image blind deblurring model:
wherein I is t And k t The results of the t-th iteration of the original sharp image I and the blur kernel k are shown respectively,is I t The local second order statistic HOS value for the upper (i, j) pixel point, in this embodiment, the parameter is set to: λ=γ=0.003, β=0.5; solving the formula (3) by adopting a semi-quadratic splitting method and an alternating minimization mode, and specifically comprising the following steps of:
3.1 introduction of auxiliary parameter eta instead ofg＝(g h ,g v ) Approximation->And changing the formula (3) to:
wherein the parameter alpha 1 ,α 2 ＞0，η i,j Is the (i, j) element of eta,Λ represents a modulo operation:thus (S)>
The above equation (4) can be solved by alternately minimizing the following three sub-problems:
3.2 hypothesis I t+1,s Updating eta is known t+1,s+1 Then there is formula (8):
in the formula (8), 1 represents a matrix of all 1 elements, and the size and I t+1,s In accordance with the method, the device and the system,from I t HOS value of (i, j);
3.3 introducing the parameter J to approach I, converting equation (7) to equation (9):
wherein the parameter alpha 3 Equation (9) can be solved by alternate minimization, two sub-problems are shown below:
in the formulas (10) and (11), the parameter α 3 ＞0；
3.4 based on η obtained in step 3.2 t+1,s+1 Suppose I t+1,s+1,d Update J is known t+1,s+1,d+1 ：
3.5 hypothesis I t+1,s+1,d+1,r Update g is known t+1,s+1,d+1,r+1 ：
3.6J based on the above steps 3.4 and 3.5 t+1,s+1,d+1 And g t+1,s+1,d+1,r+1 Updating I by fast Fourier transform t+1,s+1,d+1,r+1 ：
In formula (14), F (&) and F -1 (·) represent the fast fourier transform and its inverse,representing complex conjugate operator>
Step 4, solving to obtain a fuzzy kernel of the original clear image under the same layer scale according to the original clear image under the layer obtained by solving in the step 3:
in the formula (15), u t+1 Representing the original sharp image I t+1 The expression is as follows:
in the formula (16), the parameter epsilon is more than 0 so as to avoid the situation that the denominator is zero, and p represents a pixel point in the image u; d (D) h (p) and D v (p) represents the total variation of the window in the horizontal direction and the vertical direction, respectively:r (p) is a variation region centered on p pixels, q is a pixel in the region, g p,q Is a Gaussian weighting function with standard deviation sigma, i.e.>p i And p j The position coordinates in the horizontal direction and the vertical direction of the p point are respectively, q i And q j Similarly; l (L) h (p) and L v (p) is called window inherent degradation: />Equation (16) can be solved by converting some relaxation operations into a system of linear equations;
equation (15) can be solved by fast fourier transform, then there is equation (17):
in estimating the fuzzy kernel k t+1 Setting the negative element of the fuzzy core to 0, and performing the next normalization treatment;
step 5, up-sampling the blur kernel of the original clear image obtained by solving in the step 4, transmitting the up-sampled blur kernel to the next layer of scale in the image pyramid in the step 1, and circularly and alternately executing the step 3 and the step 4 according to the set times until the final blur kernel is estimated on the finest scale, wherein in the embodiment, the set times are circulated for 5 times; according to the invention, the fuzzy core of the original clear image is solved through the step 4 and the step 5, and the final fuzzy core is estimated on the finest scale, so that the accuracy of fuzzy core estimation is improved;
step 6, performing deblurring treatment by adopting the following non-blind deblurring method according to the blurred image acquired in the step 1 and the blur kernel estimated in the step 4:
6.1 firstly, estimating an original clear image I by using a Laplacian prior method according to the blurred image and the blur kernel estimated in the step 5 l ；
6.2 estimating the distinct image using equation (18) as I 0 ：
6.3 calculation of I l And I 0 A difference image between the two original clear images is obtained, and double-sided filtering is used for removing artifacts;
6.4 finally, at I l And subtracting the filtered difference image to obtain a final clear image.
The invention is further tested and verified:
at 2.30GHz Intel (R) Core TM The i7-10875H CPU Windows 10 system uses version 9.9.0 (R2020 b) Matlab software to make test experiments, selects a gray image data set containing 80 gray blurred images, and is obtained by convolving 8 blur kernels (13×13, 15×15, 17×17, 19×19, 21×21, 23×23, 27×27) with 10 gray images (242×242) respectively, and the storage format is png, and uses peak signal-to-noise ratio PSNR and structural similarity SSIM value as evaluation indexes to test the existing method such as L 0 The restoration performance of the regularization model, the DCP model, the LMG model and the method provided by the invention for deblurring the blind image is shown in the table 1:
TABLE 1
As can be seen from Table 1, the average peak signal-to-noise ratio PSNR value and the average structural similarity SSIM value of the invention on the whole data set are obviously higher than those of other prior art, the average peak signal-to-noise ratio PSRN value obtained by the LMG model is different by 1.01dB, and the structural similarity SSIM value is different by 0.02, because the invention enhances the obvious edges in the image, is beneficial to the estimation of the fuzzy core, and improves the restoration performance of the model to the fuzzy image.
Referring to fig. 2, the average peak signal-to-noise ratio PSNR values of the deblurred images obtained from the 4 blurred images and the average peak signal-to-noise ratio PSNR values of the 48 test images by different methods can be clearly observed from the figure, and compared with other methods, the average peak signal-to-noise ratio of the recovered image of the invention is obviously improved, that is, the recovered image of the invention has higher definition.
Referring to FIG. 3, which shows the visual contrast and corresponding peak signal to noise ratio PSNR values on a color image dataset plot "Im02_ker010", two layout details in a timepiece image restored by four methods and an original sharp image are magnified, marked with red and green boxes, respectively, and the magnified result is placed under the image, wherein FIG. 3 (a) is a blurred image and FIG. 3 (b) is L 0 The regularized model restored image, fig. 3 (c) is a DCP model restored image, fig. 3 (d) is an LMG model restored image, fig. 3 (e) is an inventive restored image, and fig. 3 (f) is a true clear image, which can be obtained by comparing the other three restored images in fig. 3 and the inventive restored image with the true clear image, respectively: the result of the recovery of the invention is that the peak signal to noise ratio PSNR value is higher than other three methods, the visual effect is also the best, and the recovered image is the most clear.
Claims (8)
1. An image blind deblurring method based on edge enhancement, which is characterized by comprising the following steps:
step 1, acquiring an original blurred image, converting the blurred image into a gray image if the blurred image is a color image, and layering the pyramid by downsampling to construct an image pyramid; if the blurred image is a black-and-white image, pyramid layering is directly carried out on the blurred image through downsampling, and an image pyramid is constructed;
step 2, acquiring an original clear image, and calculating a local second order statistic HOS value on each pixel point of the original clear image so as to obtain a weight matrix of the original clear image; obtaining a regular term of the original clear image by using the weight matrix and the gradient of the original clear image, and establishing an image blind deblurring model;
step 3, taking the blurred image under each layer of scale in the image pyramid in the step 1 as an initial original clear image of the layer, calculating an initial weight matrix, and solving the original clear image under the layer according to the image blind deblurring model constructed in the step 2;
step 4, solving the original clear image under the layer obtained by solving in the step 3 to obtain a fuzzy kernel of the original clear image under the same layer scale;
step 5, up-sampling the fuzzy core of the original clear image obtained by solving in the step 4, transmitting the up-sampled fuzzy core to the next layer of scale in the image pyramid in the step 1, and taking the up-sampled fuzzy core as the initial fuzzy core of the next layer, and circularly and alternately executing the step 3 and the step 4 according to the set times until the final fuzzy core is obtained on the finest scale;
and 6, processing the original blurred image by using the final blurred image estimated in the step 5 to obtain a final clear image.
2. The image blind deblurring method based on edge enhancement according to claim 1, wherein the image blind area blur model is built in the step 2, and specifically comprises the following steps:
2.1 the regular term of the original sharp image is shown as formula (1):
in the formula (1), I represents an original clear image, H i,j Representing the local second order statistic HOS value at each pixel point of the original clear image, which is defined asR i,j A rectangular region having a size of m×n centered on the pixel (i, j) is shown, and the average pixel value in the region is: />In which H is more than or equal to 0 i,j Is less than or equal to 1; when R is i,j When the pixel is positioned in the smooth area, the pixel in the area is not changed greatly, and the corresponding H i,j Smaller; when R is i,j When the edge region is included, the pixels in R (p) change drastically, and the corresponding H i,j Larger; thus, when (i, j) is in the smooth region, H, while minimizing the objective function described in step two i,j →0，1-H i,j 1, such that->Polishing noise in the region; when (i, j) is at the edge position, H i,j →1，1-H i,j 0, such that->Enhancing the image edge effect;
2.2, establishing an image blind deblurring model according to a regular term of an original clear image, wherein the image blind deblurring model is shown as a formula (2):
in the formula (2), regularization parameters gamma, tau, lambda and beta are more than 0;representing a two-dimensional linear convolution operator; B. k and I respectively represent a blurred image, a blur kernel and an original clear image; h i,j Representing the value of the local second order statistic HOS at the (I, j) position in the original sharp image I; />The gradient of the pixel point (I, j) on the blur kernel I is represented by: />Wherein (1)>And->Representing the partial derivatives of the original clear image I in the horizontal direction and the vertical direction respectively; gradient->Is->Is defined as the sum of the absolute values of its two components: />Then there are: /> L of (2) 0 The norm refers toNumber of non-zero elements: />
3. The method for blind deblurring of an image based on edge enhancement according to claim 1, wherein the solving of the original sharp image in the step 3 is specifically:
initializing regularization parameters and blur kernels in equation (2), as shown in equation (3):
in the formula (3), I t And k t The results of the t-th iteration of the original sharp image I and the blur kernel k are shown respectively,is I t The local second order statistic HOS value of the pixel point (i, j) is solved by adopting a half-quadratic splitting method and an alternating minimization mode, and the method comprises the following steps:
3.1 introduction of auxiliary parameter eta instead ofg＝(g h ,g v ) Approximation->As shown in formula (4):
in the formula (4), the parameter alpha 1 ,α 2 ＞0，η i,j Is the (i, j) element of eta,Λ represents a modulo operation:thus (S)>
Equation (4) can be solved by alternatively minimizing the following three equations:
3.2 hypothesis I t+1,s Updating eta is known t+1,s+1 As shown in formula (8):
in the formula (8), 1 represents a matrix of all 1 elements, and the size and I t+1,s In accordance with the method, the device and the system,from I t HOS value of (i, j);
3.3 introducing a parameter J to approach I, deforming equation (7) to equation (9):
in the formula (9), the parameter alpha 3 > 0, equation (9) can be solved by alternate minimization to yield two sub-problems as equation (10)And formula (11):
in the formulas (10) and (11), the parameter α 3 ＞0；
3.4 based on η obtained in step 3.2 t+1,s+1 Suppose I t+1,s+1,d Update J is known t+1,s+1,d+1 As shown in formula (12):
3.5 hypothesis I t+1,s+1,d+1,r Update g is known t+1,s+1,d+1,r+1 As shown in formula (13):
3.6 based on J obtained in step 3.4 and step 3.5 t+1,s+1,d+1 And g t+1,s+1,d+1,r+1 Updating I by fast Fourier transform t+1,s+1,d+1,r+1 As shown in formula (14):
in formula (14), F (&) and F -1 (·) represent the fast fourier transform and its inverse,representing complex conjugate operator and having +.>
4. The blind deblurring method of an image based on edge enhancement according to claim 1, wherein in the step 4, the original clear image under the layer obtained by solving in the step 3 is solved to obtain a blur kernel of the original clear image under the same layer scale, and the specific steps are as shown in formula (15):
in the formula (15), u t+1 Representing the original sharp image I t+1 The expression of which is shown in formula (16):
in the formula (16), the parameter epsilon > 0, and p represents a pixel point in the image u; d (D) h (p) and D v (p) represents the total variation of the window in the horizontal direction and the vertical direction, respectively: r (p) represents a variation region centered on p pixels, q represents a pixel in the region, g p,q Gaussian weighting function representing standard deviation sigma, i.e.>p i And p j Respectively representing the position coordinates of the p point in the horizontal direction and the vertical direction, q i And q j Respectively representing the position coordinates of the q point in the horizontal direction and the vertical direction; l (L) h (p) and L v (p) represents inherent window degradation: />
Solving the formula (15) by fast fourier transform as shown in the formula (17):
estimating a fuzzy kernel k t+1 And setting the negative element of the fuzzy core to 0, and then performing the normalization processing of the next step.
5. The method for blind deblurring of an image based on edge enhancement according to claim 1, wherein the step 6 uses the final blur kernel estimated in the step 5 to process the original image to obtain a final clear image, and the specific steps are as follows:
6.1 estimating the original clear image by using the Laplacian prior method according to the blurred image and the final blur kernel estimated in the step 5, and recording as I L ；
6.2 estimating the distinct image using equation (18) as I 0 ：
6.3 calculation of the original clear image I L And clear image I 0 A difference image between the two images, and removing artifacts by using bilateral filtering;
6.4 in the original clear image I L And subtracting the filtered difference image to obtain a final clear image.
6. An edge-enhanced based image blind deblurring system, comprising:
the image acquisition preprocessing module is used for acquiring an original blurred image, converting a color image into a gray image, and carrying out pyramid layering on the blurred image through downsampling to construct an image pyramid; collecting an original clear image, and calculating a local second order statistic HOS value on each pixel point of the original clear image so as to obtain a weight matrix of the original clear image;
the image blind deblurring model construction module is used for obtaining a regular term of the original clear image by utilizing the weight matrix of the original clear image and the gradient of the original clear image obtained by the image acquisition preprocessing module, and establishing an image blind deblurring model;
the image blind deblurring model solving module is used for carrying out model solving on the image blind deblurring model established by the image blind deblurring model establishing module to obtain an original clear image;
the fuzzy core solving module is used for solving according to the original clear image obtained by the image blind deblurring model solving module to obtain an estimated fuzzy core;
and the clear image generation module is used for obtaining a final clear image by adopting a non-blind deblurring method according to the blurred image and the blurred kernel estimated by the blurred kernel solving module.
7. An edge-enhanced based image blind deblurring apparatus comprising:
a memory storing a computer program for an edge-enhanced image blind deblurring method according to any of claims 1-5, as a computer readable device;
a processor for implementing an edge-enhanced based image blind deblurring method according to any of claims 1-5 when executing said computer program.
8. A computer readable medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, enables an edge-enhanced based image blind deblurring method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311063076.4A CN117218016A (en) | 2023-08-23 | 2023-08-23 | Image blind deblurring method, system, equipment and medium based on edge enhancement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311063076.4A CN117218016A (en) | 2023-08-23 | 2023-08-23 | Image blind deblurring method, system, equipment and medium based on edge enhancement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117218016A true CN117218016A (en) | 2023-12-12 |
Family
ID=89037942
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311063076.4A Pending CN117218016A (en) | 2023-08-23 | 2023-08-23 | Image blind deblurring method, system, equipment and medium based on edge enhancement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218016A (en) |
-
2023
- 2023-08-23 CN CN202311063076.4A patent/CN117218016A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wu et al. | Light field reconstruction using deep convolutional network on EPI | |
| CN111028177B (en) | Edge-based deep learning image motion blur removing method | |
| JP5543605B2 (en) | Blur image correction using spatial image prior probability | |
| CN110675347B (en) | Image blind restoration method based on group sparse representation | |
| US8498498B2 (en) | Apparatus and method of obtaining high resolution image | |
| CN102326379B (en) | Method for removing blur from image | |
| Khattab et al. | Regularization-based multi-frame super-resolution: a systematic review | |
| CN109064402B (en) | Single image super-resolution reconstruction method based on enhanced non-local total variation model prior | |
| CN115345791A (en) | Infrared image deblurring algorithm based on attention mechanism residual error network model | |
| Yu et al. | Split-attention multiframe alignment network for image restoration | |
| CN117274059A (en) | Low-resolution image reconstruction method and system based on image coding-decoding | |
| Sheng et al. | Frequency-domain deep guided image denoising | |
| CN108122262B (en) | Sparse representation single-frame image super-resolution reconstruction algorithm based on main structure separation | |
| Pham | Kriging-weighted laplacian kernels for grayscale image sharpening | |
| Khattab et al. | A Hybrid Regularization-Based Multi-Frame Super-Resolution Using Bayesian Framework. | |
| Nguyen et al. | Self-supervised super-resolution for multi-exposure push-frame satellites | |
| Purkait et al. | Morphologic gain-controlled regularization for edge-preserving super-resolution image reconstruction | |
| CN111986102B (en) | Digital pathological image deblurring method | |
| CN110852947B (en) | Infrared image super-resolution method based on edge sharpening | |
| Zhang et al. | A new image filtering method: Nonlocal image guided averaging | |
| CN117218016A (en) | Image blind deblurring method, system, equipment and medium based on edge enhancement | |
| Fazlali et al. | Atmospheric turbulence removal in long-range imaging using a data-driven-based approach | |
| CN114418891A (en) | Image blind deblurring algorithm based on local binary pattern | |
| Al Ismaeil et al. | Multi-frame super-resolution by enhanced shift & add | |
| Deepak et al. | Analysis of Single Image Super-Resolution Techniques: An Evolutionary Study |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |