CN110675347B - Image blind restoration method based on group sparse representation - Google Patents

Image blind restoration method based on group sparse representation Download PDF

Info

Publication number
CN110675347B
CN110675347B CN201910938167.5A CN201910938167A CN110675347B CN 110675347 B CN110675347 B CN 110675347B CN 201910938167 A CN201910938167 A CN 201910938167A CN 110675347 B CN110675347 B CN 110675347B
Authority
CN
China
Prior art keywords
image
estimation
fuzzy
group
iteration
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910938167.5A
Other languages
Chinese (zh)
Other versions
CN110675347A (en
Inventor
禹晶
彭天奇
董醒儒
肖创柏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN201910938167.5A priority Critical patent/CN110675347B/en
Publication of CN110675347A publication Critical patent/CN110675347A/en
Application granted granted Critical
Publication of CN110675347B publication Critical patent/CN110675347B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an image blind restoration method based on group sparse representation, which is used for realizing blind restoration of a blurred image. The method comprises the following steps: constructing an image pyramid, and estimating a fuzzy kernel from coarse to fine; searching similar image blocks in the down-sampled image, combining the current block and the cross-scale similar block into a similar image block group, and establishing a group sparse representation regular term in a target function; and alternately updating the fuzzy core and the clear image by iteration, and in the clear image updating step, reconstructing the clear image estimated in the previous iteration by using the group sparse representation to be used as a reference image for updating the clear image at the next time. Because the edge of the downsampled image has stronger similarity with the clear image, the low rank of the group matrix is realized by using the association of the group sparse representation and the low rank matrix and restricting the sparsity of the representation coefficients, and the edge of the reconstructed image is forced to be close to the edge of the clear image. The method disclosed by the invention improves the robustness to noise and can process the fuzzy kernel estimation with large scale.

Description

Image blind restoration method based on group sparse representation
Technical Field
The invention relates to the field of image restoration, in particular to an image blind restoration method based on group sparse representation.
Background
In the process of image acquisition and transmission, due to an imaging system, a recording device, a transmission medium and the like, various distortions generally occur in a digital image acquired by an imaging device, resulting in a reduction in image quality, which is called image degradation. The image restoration aims at modeling the image degradation process, solving the inverse process of the degradation model and restoring the original clear image from the degraded image.
Blurring is a common image degradation phenomenon, and in the process of acquiring an image by a digital imaging device, the blurring is generated due to shaking and defocusing of the device and the motion of an object. In some application scenarios, when the acquired image has a blur phenomenon, it is costly or not feasible to acquire the image again, for example, a geo-imaging device carried by a non-geosynchronous orbit satellite can only acquire the image again for a certain geographic area when the satellite passes through the area again, and the surface condition of the area may have changed when the satellite waits for a long time and is imaged again. In addition, in the imaging process, the degree of blurring can be avoided or reduced by the aid of some special devices, for example, a tripod or other fixing devices or optical anti-shake devices are adopted, but a tripod is generally inconvenient to carry and relatively limited in application, and the optical anti-shake devices can only solve blurring caused by small shaking and cannot help larger blurring.
The single image blind restoration is a technology for effectively removing blur in an image by restoring an original clear image from a single blurred image through an image processing means under the condition that a degradation model or a degradation parameter is unknown. The process of acquiring images by an imaging system is a positive problem, and the use of degraded images to recover the underlying original image is typically the inverse of the image. Typically, the degradation matrix is singular and the problem is ill-conditioned. There are two categories according to whether the degradation model is available: if the degradation matrix is known, the method is called a non-blind restoration method; if the degradation matrix is unknown, it is called the blind recovery problem. In practical applications, a degradation model or a degradation parameter of an observed image cannot be obtained, so that blind restoration of the image is more practical.
The deblurring problem is also referred to as the deconvolution problem, since the degradation process of image blur is typically expressed as a two-dimensional convolution of the blur kernel with the sharp image. The blind deconvolution of images is a serious underdetermined problem, the number of unknown variables to be solved is larger than that of known variables, and the solution is not unique. In order to find an accurate solution, it is necessary to introduce a-priori knowledge about the image, called image prior model, which is added as a regularization constraint term to the reconstruction process, called regularization. Regularization constraints provide additional information for solving the ill-conditioned problem, constraining the space of feasible solutions. The existing blind deconvolution algorithms directly or indirectly utilize various prior knowledge, and the algorithms can be roughly divided into two types, wherein one type of algorithm utilizes a heuristic edge enhancement method, and the other type of algorithm directly establishes a prior probability distribution model of a fuzzy core or a clear image.
The blind deconvolution method based on heuristic edge enhancement assumes that there are sufficient edges in the blurred image, performs approximate estimation on the sharp image by enhancing the edges in the image, and finally estimates a blur kernel by using the corresponding relation between the approximate estimation of edge enhancement and the blurred image. Because the capability of the edge enhancement algorithm is limited, it is generally impossible to obtain a clear edge through only one enhancement process, and thus continuous iteration is required in the blind deconvolution process, i.e., edge enhancement is continuously performed on the estimated clear image every time to gradually approximate the original clear image. Because the blind deconvolution method directly uses a heuristic method to enhance the edge of a blurred image, a uniform objective function is generally absent, the algorithm complexity is low, but in the iterative solution process, in order to avoid the phenomena of edge over-enhancement and the like, parameters of the edge enhancement algorithm generally need to be continuously adjusted according to the iteration times, and thus the blind deconvolution method is sensitive to parameter setting.
Another type of blind deconvolution method establishes a prior probability distribution model for the blur kernel or sharp image, which is estimated by solving the maximum a posteriori problem. The optimization objective function includes regularization constraints on sharp images and blur kernels, which help to solve the underqualitative, limiting the space of feasible solutions of the blind deconvolution problem. How to select the proper regularization constraint is the key to study the blind deconvolution method. Previous work has mainly utilized a priori knowledge about image gradients, which represent the relationship between adjacent pixels of an image and are not sufficient to represent image structures of larger scale. In recent years, some blind restoration methods utilize a priori information about image blocks, and compared with an algorithm utilizing a gradient a priori, the blind restoration method utilizing the priori knowledge of the image blocks has greatly improved accuracy and robustness.
The sparse representation-based method utilizes the sparse representation form of the image under a certain group of bases (or dictionaries), and adds the sparsity as a regularization constraint term into the objective function. The key of the image blind restoration method based on sparse representation is to construct a dictionary representing a clear image, so that an image block has a sparse representation form under the dictionary. Sparse representation has two significant problems: 1) the dictionary learning needs a large amount of calculation; 2) the correlation between different blocks is disregarded. The K-SVD algorithm under the sparse framework is the most common self-adaptive dictionary learning method, and the dictionary learning usually needs a data set with a certain scale to construct a dictionary, and the data set is used as a dictionary learning sample. On one hand, in the construction process of the global dictionary, in order to make various different types of image blocks sparsely represented under the dictionary, a large number of samples are required to be used for dictionary learning, which results in low dictionary learning efficiency; on the other hand, when the dictionary-learned samples cannot accurately provide information similar to the low-resolution image to be processed, it is difficult for this method to ensure the effect of image reconstruction. In addition, the solving of the sparse representation coefficients of each data vector in the sparse representation under the dictionary is independent, and the problem of global structure constraint on the reconstructed image is lacked.
Although many image blind restoration methods have been proposed so far, the vast majority of cases assume a low noise level. The blind image restoration is a ill-conditioned inverse problem, and in the absence of prior knowledge of noise, the blind image restoration amplifies the noise, so that a fuzzy kernel cannot be accurately estimated. For the widely researched image blind restoration method based on image gradient sparse prior, image noise can destroy the prior prediction of gradient distribution, so that the estimation of a fuzzy kernel generates certain deviation. In addition, the blind restoration model of heuristic edge enhancement usually looks for a significant edge to estimate a blur kernel, and noise in a blurred image causes an error of edge positioning, thereby causing a deviation of estimation of the blur kernel, resulting in failure of blind restoration of the image.
Disclosure of Invention
The invention discloses an image blind restoration method based on group sparse representation, which performs integral sparsity constraint on each group of sparse representation coefficients by constraint under a specific dictionary, realizes the rank minimization of each group matrix and forces the edge of the current image to be closer to the edge of a clear image. The group sparse representation can fuse local sparse and non-local self-similar prior of the image, and rank minimization under a specific dictionary can better represent the global structural characteristics of data. The method disclosed by the invention improves the robustness to noise, can process the blurred image interfered by the noise, can estimate the large-scale blur kernel and restore the large-scale blurred image.
The technical scheme adopted by the invention is an image blind restoration method based on group sparse representation, which is characterized by comprising the following steps of:
step 1, reading in a fuzzy image and constructing an image pyramid;
reading a blurred image, setting the initial size of a blur kernel, downsampling the blurred image layer by layer to construct an image pyramid, wherein the sizes of the blur kernels corresponding to image pyramids of different levels are different, and when the image pyramid is constructed, if the size of the blur kernel corresponding to the image pyramid of the current level is smaller than 3 multiplied by 3, the construction of the image pyramid is stopped. In the estimation process, the clear image estimated on the coarse-layer image pyramid is used as the initial estimation of the clear image on the fine-layer image pyramid after image interpolation. The blind deconvolution objective function established is the same at each level of the image pyramid. The following steps are the solving process of blind deconvolution objective functions on each layer of the image pyramid.
Step 2, initializing a clear image;
and recording the number of loop iterations as k, setting the initialization to be k equal to 0, and taking the blurred image y as a clear image initial estimation if the current layer is the first layer of the image pyramid
Figure BDA0002222144890000041
Is provided with
Figure BDA0002222144890000042
Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimation of the clear image of the current layer
Figure BDA0002222144890000043
Step 3, screening image blocks and estimating a mark matrix M;
setting the mark matrix as M, wherein M is a binary image at the same time, and if a certain image in M isThe value of the element is 1, which indicates that the corresponding image block participates in fuzzy kernel estimation and group sparse representation constraint. The method only limits the group sparsity regularization constraint to the edge area of the image, so that the image is restored
Figure BDA0002222144890000051
The background smooth area of (2) is less constrained, which may result in the smooth area of the restored image containing more noise, and in order to reduce the interference of the noise to the edge estimation, the restored image is first subjected to
Figure BDA0002222144890000052
Gaussian filtering is performed and then edges are estimated for the filtered image.
Step 4, estimating a fuzzy core;
estimation of stationary current image
Figure BDA0002222144890000053
Updating the fuzzy kernel of the next iteration with the following equation
Figure BDA0002222144890000054
Figure BDA0002222144890000055
In the formula, y represents a blurred image,
Figure BDA0002222144890000056
for gradient operators of images, λhA regularization parameter, an indicates element by element multiplication,
Figure BDA0002222144890000057
which represents the fourier transform of the signal,
Figure BDA0002222144890000058
which represents the inverse of the fourier transform,
Figure BDA0002222144890000059
representing the complex conjugate of the fourier transform. Will restore the image
Figure BDA00022221448900000510
Medium gradient
Figure BDA00022221448900000511
The gradient of the pixel points smaller than a certain threshold is set to be 0. Let the gradient threshold be τ and the blur kernel size be NhThen τ is selected by: the image gradients are first divided into four groups according to their direction, and then the value of τ is set to ensure that each group remains at least
Figure BDA00022221448900000512
And each pixel point is used for estimating the fuzzy core. Because the restored image is clearer along with the increase of the iteration times, in order to enable more pixel points in the restored image to be gradually added into the process of fuzzy kernel estimation, the value of tau is reduced to 1.1 times of that of the previous iteration in each iteration.
Step 5, estimating the clear image;
given the estimation of the current image
Figure BDA00022221448900000513
Fixing the fuzzy kernel estimate for the next iteration
Figure BDA00022221448900000514
Updating the image of the next iteration
Figure BDA00022221448900000515
Step 5.1, constructing a similar image block group;
let the sharp image and its down-sampled image be respectively expressed as
Figure BDA00022221448900000516
And
Figure BDA00022221448900000517
where N is the number of pixels of the sharp imageAnd a represents a down-sampling factor,
Figure BDA0002222144890000061
is an N-dimensional column vector, and is,
Figure BDA0002222144890000062
is N/a2And (5) maintaining column vectors. From sharp image X and downsampled image XaThe extracted image blocks are respectively represented as QjX and RiXaIn which
Figure BDA0002222144890000063
And
Figure BDA0002222144890000064
the extraction matrix is used for extracting the jth and ith image blocks from a clear image and a downsampled image respectively, and the block size of the extracted image is n. For any image block Q in an imagejX in the downsampled image XaSearching for similar image blocks RiXa. Since the similarity of image blocks widely exists between different scales of images, i.e. for QjX, searching a plurality of similar image blocks in a certain-size search window in the downsampled image by using an image block matching method, and setting the similar image blocks in the XaSearch for m-1 and QjX most similar image blocks and are represented by columns
Figure BDA0002222144890000065
QjX and the non-local similar image blocks in the down-sampled image are aggregated to form a similar image block group PjExpressed as:
Figure BDA0002222144890000066
wherein n is the image block size and m is the number of similar image blocks.
Step 5.2 reconstruction of reference image z by group sparse representations；
Pairing group matrix P using singular value threshold operatorsjCarrying out reconstruction, the closed solution of which is
Figure BDA0002222144890000067
Wherein,
Figure BDA00022221448900000618
sigma taking beta not less than 0 as parameterjSoft threshold operator, defined as
Figure BDA00022221448900000619
From the above equation, the larger beta, the larger the reconstruction set matrix
Figure BDA0002222144890000068
The smaller the rank of (c). By using
Figure BDA0002222144890000069
Approximation
Figure BDA00022221448900000610
Figure BDA00022221448900000611
First column in (1) namely reconstructed image block
Figure BDA00022221448900000612
DjIs an adaptive dictionary. Order to
Figure BDA00022221448900000613
In the course of alternative iterative solutions, zsUpdating the clear image to be estimated as the next iteration
Figure BDA00022221448900000614
The reference image of (2).
Step 5.3 solving
Figure BDA00022221448900000615
Is obtained by
Figure BDA00022221448900000616
Thereafter, the pair is updated by solving the following system of linear equations
Figure BDA00022221448900000617
Estimation of (2):
Figure BDA0002222144890000071
wherein | M | is the number of non-zero elements in M, N is the number of image pixels,
Figure BDA0002222144890000072
a one-dimensional column vector of y,
Figure BDA0002222144890000073
as a function of point spread
Figure BDA0002222144890000074
The matrix form of (a), i.e. the blur matrix,
Figure BDA0002222144890000075
as gradient operators
Figure BDA0002222144890000076
In the form of a matrix. Due to the action of M, the fast solution in the frequency domain can not be realized through Fourier transform, and a bi-conjugate gradient method is adopted to solve the linear equation system as an upper line to obtain
Figure BDA0002222144890000077
Step 6, judging convergence to obtain the estimation of a fuzzy core;
performing one-time iteration solution on the objective function through the steps 3, 4 and 5 to obtain the estimation of the fuzzy core
Figure BDA0002222144890000078
And estimation of sharp images
Figure BDA0002222144890000079
Is updated into
Figure BDA00022221448900000710
If the iteration reaches the maximum iteration number or the iteration converges at the moment, the iteration is stopped. Otherwise, let k be k +1, k represent the number of iterations, and then repeat step 3, step 4, and step 5.
And 7, after the fuzzy kernel estimation, estimating the clear image by using a non-blind restoration algorithm.
Obtaining an estimation result of the blur kernel through steps 1 to 6
Figure BDA00022221448900000711
In that
Figure BDA00022221448900000712
On the basis, an effective non-blind restoration algorithm is adopted to obtain the final clear image estimation. The non-blind restoration algorithm comprises a total variation regularization method, a sparse non-blind restoration method, an EPLL algorithm and the like.
Preferably, the specific implementation manner of screening the image block and eliminating the interference of the smooth region on the estimation of the fuzzy core is as follows: and performing edge estimation on the current image estimation through an edge detection method, and estimating a fuzzy core and a clear image by using the image block corresponding to the edge pixel.
Preferably, the size n of the image block is 5 × 5.
Preferably, the singular value threshold β is 0.2.
Preferably, the number m of the similar image blocks is 19.
Preferably, the size of the search window is 25 × 25.
Preferably, the down-sampling factor a is 4/3.
Preferably, the iteration number of the loop is 14, and an iteration convergence threshold value is not set.
Preferably, the size of the blur kernel is 51 × 51.
Because image blind restoration is a serious ill-conditioned inverse problem, when the priori knowledge of noise is lacked, the noise is amplified by the image blind restoration, and the fuzzy kernel cannot be accurately estimated. The sparse representation represents each image block with as few linear combinations of dictionary elements as possible, and the solution of the sparse representation coefficients of each image block under the dictionary is independent of each other, so the sparse representation lacks the overall constraint on the data representing the coefficients under the dictionary. Since the image has self-similarity, the group sparse representation may constrain the sparsity of the group sparse representation coefficients as a whole. The invention provides an image prior constraint term expressed by group sparsity, which is used for solving the problem of blind restoration of a regularization image. Because the degree of blurring is reduced along with the down-sampling of the image, the edge of the down-sampled image has stronger similarity with a clear image, therefore, by utilizing the cross-scale self-similarity, the invention searches a plurality of similar image blocks in the down-sampled image to form a similar block image group, and because a group matrix combined by the similar image blocks has low rank, the low rank of the group matrix is realized by utilizing the association of the group sparse representation and the low rank matrix and by restricting the sparsity of the representation coefficients, the problem of constructing a group sparse representation dictionary is not required to be considered. The low-rank constraint can avoid the interference of noise on a blind restoration model, and meanwhile, the edge of the current image is forced to be clearer in iteration through the overall constraint of the cross-scale similar blocks in the down-sampled image, so that the edge of the reconstructed image is close to the edge of the clear image.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a diagram illustrating blind restoration of an image according to an embodiment of the present invention;
fig. 2 is a flowchart of a group sparse representation-based image blind restoration method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an image pyramid structure according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a group of similar image blocks according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating the similarity between a sharp image and a blurred image downsampled image according to an embodiment of the present invention;
fig. 6 is an example of a mark matrix of an edge image block according to an embodiment of the present invention;
FIG. 7 is a comparison of the average PSNR for various methods on the Koehler dataset as provided by an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The degradation process from sharp to blurred images is usually represented as a convolution model as follows:
y＝h*x+n (1)
wherein y represents a blurred image, x represents a sharp image, x represents a convolution operation, h is a blur kernel, and n is noise. Here, the blur kernel h and the noise n contain all information of the degradation model. Under the convolution model, the blind image restoration method is to study how to estimate the blur kernel h and the sharp image x from the blurred image y at the same time, as shown in fig. 1. The blind deblurring problem is also referred to as the blind deconvolution problem, since the blurring process is modeled as a form of convolution.
The blind deconvolution of the image is an underdetermined problem, the number of unknown variables to be solved is larger than the number of known variables, and the solution is not unique. In order to find an accurate solution, it is necessary to introduce a-priori knowledge about the image, called the image-prior model, denoted Ψ (x), which is added as a regularization constraint term to the objective function, a process called regularization. Regularization constraints provide additional information for solving the ill-conditioned problem, constrain the solution space of the objective function, and regularization methods are generally expressed as the following optimization problem:
Figure BDA0002222144890000101
in the formula, Ψx(x) And Ψh(h) A priori constraints of the sharp image x and the blur kernel h, respectively, constant λxAnd λhIs a lagrange multiplier, also known as a regularization parameter. The first term represents the data fidelity term, the latter two terms represent the regularization constraint term, the regularization parameter λxAnd λhThe effect of (a) is to balance the reconstruction error in the objective function against the weight of the a priori constraints.
The image prior model occupies a very important position in the image blind deconvolution problem, and how to design an effective regularization constraint term to describe the prior knowledge or additional information of the image is a key problem of the image blind deconvolution method. According to the method, priori knowledge is mined from the multiscale self-similarity of the image, additional information contained in the multiscale self-similarity structure is effectively added into a reconstructed image by constructing a regularization constraint term, and the blind deconvolution problem of the image is converted into the optimization problem which is found to meet a specific constraint condition solution.
A multi-scale self-similarity structure is widely existed in an image, and the multi-scale self-similarity can provide necessary additional information for blind restoration of the image. The self-similarity is expressed by similar image blocks with the same scale and different scales in the image, such as obvious similarity of aerial planes, ships and warships and the like, and potential similarity of house road edges, mountain river veins and the like. Let a one-dimensional column vector representation of the sharp image composition be
Figure BDA0002222144890000102
Extracting an image block therefrom and representing it as Q by columnsjX, wherein
Figure BDA0002222144890000103
For extracting matrix, searching Q in image by image block matching methodjSimilar image blocks of X are represented by columns
Figure BDA0002222144890000111
Aggregating the non-local similar image blocks into a similar image group which is recorded as
Figure BDA0002222144890000112
Wherein m is the number of similar image blocks. Similar to sparse representation, a dictionary is designed
Figure BDA0002222144890000113
Where t is the size of the dictionary, set of similar image blocks
Figure BDA0002222144890000114
By solving the minimization problem as follows:
Figure BDA0002222144890000115
in the formula, | · the luminance | |FRepresents the Frobenius norm of the matrix, card (A)j) Represents a coefficient matrix AjThe number of non-zero elements in the table, K represents the coefficient AjThe sparsity constraint of (1). Equation (3) represents minimizing the group sparse representation error under the premise of constraining the degree of representation coefficient sparsity.
Will group matrix PjUsing dictionaries DjPerforming group sparse representation, i.e.
Figure BDA0002222144890000116
Coefficients are represented for group sparseness. The representation coefficient sparsity is ensured by selecting a proper dictionary, the sparsity can be used as regularization constraint on the image and added into an objective function of blind restoration of the image, and the objective function can be expressed as follows:
Figure BDA0002222144890000117
wherein y is a blurred image, x is a sharp image, h is a blur kernel,
Figure BDA0002222144890000118
representing a convolution operation for the gradient operator of the image, | · | | | luminance2Represents a vector l2Norm, | \ | circumflectingFRepresenting the Frobenius norm, P of the matrixjRepresenting an image block QjGroup matrix of X and its similar image blocks, DjTo adapt the dictionary, AjFor groups of sparse representation coefficients, λg、λs、λhIs a regularization parameter. The first term in the formula (4) is a data fidelity term, the second term is a group sparse representation regularization constraint term, the third term is a gradient constraint term, and the fourth term is a regularization constraint term of a fuzzy kernel. The form of the data fidelity term is determined by a probability model of noise, the noise contained in the gradient image of the blurred image is modeled into common additive white Gaussian noise, and then the data fidelity term constrained by 2 norm square as shown in a formula (4) is obtained. The data fidelity item can be defined in an image domain and can also be defined in a gradient domain of the image, and because the solving speed of the objective function is accelerated through two-dimensional Fast Fourier Transform (FFT) in the invention, and the value of the gradient image at the image boundary is 0, the periodic boundary condition required by the FFT can be better met, the data fidelity item is defined in the gradient domain of the image in the invention, so that the distortion of the restored image at the boundary is reduced. For the fuzzy kernel, the invention adopts simple 2 norm constraint to simplify the solving process.
The method solves the estimation of the fuzzy core through the optimization problem shown in the formula (4), and because the formula (4) is non-convex and has no closed solution, the method solves the optimization problem shown in the formula (4) in an alternate iteration mode, namely, the estimation result of the clear image at present is fixed
Figure BDA0002222144890000121
Updating estimation results for fuzzy kernels
Figure BDA0002222144890000122
Then is fixed at
Figure BDA0002222144890000123
On the basis of the update
Figure BDA0002222144890000124
And repeating the iteration in such a loop until the estimation result converges or the iteration number reaches a certain preset threshold value.
According to the multiscale self-similarity of the image, the embodiment discloses an image blind restoration method based on group sparse representation, so as to realize blind restoration of a blurred image. Referring to fig. 2, the above method comprises at least the following steps:
step 1, reading in a fuzzy image and constructing an image pyramid;
in the invention, a common image pyramid mode of image blind deconvolution is adopted, a clear image and a fuzzy kernel are estimated layer by layer from coarse to fine, and in the estimation process, the clear image estimated on the coarse image pyramid is used as the initial estimation of the clear image on the fine image pyramid after image interpolation, as shown in fig. 3. Because the fuzzy degree of the fuzzy image is reduced by the down-sampling operation, and the fuzzy degree is smaller when the down-sampling factor is larger, the fuzzy degree of the fuzzy image at the bottom layer of the image pyramid is smaller, the clear image which is subjected to blind deconvolution estimation is more accurate, so that if the estimation result is interpolated and used as the initial estimation of the clear image in the fine image pyramid, the initial estimation of the clear image on the fine image pyramid is closer to the real clear image to be estimated, the estimation process on the fine image pyramid can be accelerated, and the accuracy of the estimation result on the fine image pyramid is improved. And constructing an image pyramid by downsampling the fuzzy observation image layer by layer, wherein fuzzy kernel sizes corresponding to image pyramids of different levels are different, and when the image pyramid is constructed, if the fuzzy kernel size corresponding to the image pyramid of the current layer is smaller than 3 multiplied by 3, the construction of the image pyramid is stopped. The blind deconvolution objective function established is the same at each level of the image pyramid. The following steps are the solving process of blind deconvolution objective functions on each layer of the image pyramid.
Step 2, initializing a clear image;
and recording the number of loop iterations as k, setting the initialization to be k equal to 0, and taking the blurred image y as a clear image initial estimation if the current layer is the first layer of the image pyramid
Figure BDA0002222144890000131
Is provided with
Figure BDA0002222144890000132
Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimation of the clear image of the current layer
Figure BDA0002222144890000133
Step 3, screening image blocks and estimating a mark matrix M;
although multi-scale self-similarity in an image exists widely, not all image blocks can provide effective additional information for image restoration, and the effect of pixels in different areas in the restored image on the estimation of a blur kernel is different, for example, if the pixel value of a certain image area is a constant, the value of the blurred pixels in the area is still the same, and a clear image and a blurred image are completely the same in the area, so that the area cannot provide effective information for the estimation of the blur kernel. Therefore, only the image edge plays a key role in the estimation of the blur kernel, and the edge is widely subject to multi-scale self-similarity, so that similar blocks are easier to search.
The invention provides three screening schemes for screening image blocks, thereby eliminating the interference of a smooth area on fuzzy kernel estimation. Firstly, calculating the variance of an image block, selecting the image block with larger variance, and corresponding to the edge with more drastic change in the image; secondly, performing edge estimation on the current image estimation through an edge detection method, and estimating a fuzzy core by using an image block corresponding to an edge pixel; thirdly, calculating image gradient, wherein the gradient is a measure of gray level change in a local area of the image, and estimating a fuzzy core by using an image block corresponding to a pixel with the gradient larger than a certain threshold value.
Setting the mark matrix as M, where M is a binary image, and if the value of a certain pixel in M is 1, as shown in fig. 5, it indicates that the corresponding image block participates in the blur kernel estimation and the group sparse representation constraint. The invention only limits the group sparsity regularization constraint to the edge region of the image, thereby restoring the image
Figure BDA0002222144890000134
The background smooth area of the invention is less restricted, thereby possibly leading the smooth area of the restored image to contain more noise, in order to reduce the interference of the noise to the edge estimation, the invention firstly restores the image
Figure BDA0002222144890000141
Gaussian filtering is performed and then edges are estimated for the filtered image.
Step 4, estimating a fuzzy core;
fixing
Figure BDA0002222144890000142
Updating
Figure BDA0002222144890000143
The objective function is simplified as:
Figure BDA0002222144890000144
in the formula, "-" indicates element-by-element multiplication. According to the paseuler's theorem, the energy of the image in the spatial domain is equivalent to the frequency domain energy of the fourier transform, and the minimization problem shown in formula (5) in the spatial domain is equivalent to the minimization problem in the frequency domain as follows:
Figure BDA0002222144890000145
formula (6) relates to
Figure BDA0002222144890000146
And thus a closed solution exists. Order the above type is to
Figure BDA0002222144890000147
Is zero, a closed solution can be obtained:
Figure BDA0002222144890000148
in the formula,
Figure BDA0002222144890000149
which represents the fourier transform of the signal,
Figure BDA00022221448900001410
which represents the inverse of the fourier transform,
Figure BDA00022221448900001411
representing the complex conjugate of the fourier transform. In the formula (7), the invention adopts a common mode to eliminate the interference of the image smooth area on the fuzzy kernel estimation, namely, the image is restored
Figure BDA00022221448900001412
Medium gradient
Figure BDA00022221448900001413
The gradient of the pixel points smaller than a certain threshold is set to be 0. Let the gradient threshold be τ and the blur kernel size be NhThen τ is selected by: the image gradients are first divided into 4 groups according to their direction, and then the value of τ is set to ensure that each group remains at least
Figure BDA00022221448900001414
And each pixel point is used for estimating the fuzzy core. Because the restored image becomes clearer along with the increase of the iteration times, in order to enable more pixel points in the restored image to be gradually added into the process of estimating the fuzzy kernel, the value of tau is reduced to 1.1 times of that of the previous iteration in each iteration.
Step 5, estimating the clear image;
given a
Figure BDA00022221448900001415
Fixing
Figure BDA00022221448900001416
Updating
Figure BDA00022221448900001417
And (3) adjusting the weight of the group sparse representation regularization constraint term according to the number of the image blocks participating in the calculation, wherein the objective function shown in the formula (4) is simplified as follows:
Figure BDA0002222144890000151
wherein | M | is the number of non-zero elements in M, and N is the number of image pixels. Is provided with
Figure BDA0002222144890000152
For a one-dimensional column vector of y, convert equation (8) to the form of a matrix-vector product:
Figure BDA0002222144890000153
in the formula,
Figure BDA0002222144890000154
as gradient operator
Figure BDA0002222144890000155
In the form of a matrix of (a),
Figure BDA0002222144890000156
as a function of point spread
Figure BDA0002222144890000157
I.e. a blur matrix.
Step 5.1, constructing a similar image block group;
according to the imageThe self multi-scale self-similarity comprises the same-scale and cross-scale self-similarity, and the invention designs a regularization constraint term of group sparse representation, namely
Figure BDA0002222144890000158
Because the image has cross-scale self-similarity, and the fuzzy degree in the down sampling is less than that of the current layer, the edge of the image is clearer and has stronger similarity with a clear image. Therefore, the invention provides that a plurality of similar image blocks are searched in the downsampled image so as to form the similar image block group, the sparsity of the similar image block group is integrally restricted, and the edge of the current image is forced to be closer to the edge of a clear image.
The present invention gives a relaxed proof of this conclusion. The two-dimensional coordinate is recorded as xi, one image block in the clear image is represented as f (xi), the fuzzy core is h (xi), under the action of the fuzzy core h (xi), the fuzzy image block corresponding to the clear image block f (xi) is represented as q (xi), and then the clear image block f (xi) has
q(ξ)＝h(ξ)*f(ξ) (10)
Since multi-scale self-similarity is ubiquitous in clear images, assuming that there is an image block similar to f (ξ) in a clear image and its scale is a multiple (a > 1) of the f (ξ) scale, the similar image block can be represented as f (ξ/a). Under the action of the blur kernel h (xi), a blurred image block corresponding to the clear image block f (xi/a) is represented as r (xi), and then
r(ξ)＝h(ξ)*f(ξ/a) (11)
If the blurred image is downsampled by a downsampling factor a, the reduced image block corresponding to the blurred image block r (ξ) can be represented as
ra(ξ)＝r(aξ)＝h(aξ)*f(ξ) (12)
As can be seen from equation (12), the image block ra(xi) can be thought of as the result of the convolution of the sharp image block f (xi) with the blur kernel h (a xi). Since the size of h (a xi) is 1/a times of the size of h (xi), h (a xi) causes a smaller degree of blurring to the image than h (xi). As can be seen from a comparison of the formulas (10) and (12), ra(xi) is less blurred than q (xi), i.e. with blurred image blocks q (xi)ξ) of the image block r in the blurred image downsampled imagea(xi) is more similar to the clear image block f (xi) and can provide more accurate additional information for restoration of f (xi). Fig. 6 visually illustrates the relationship of the image blocks in the above-described certification process.
Since similar image blocks often occur in neighboring areas, similar image blocks are searched for within a certain size search window in the down-sampled image using an image block matching method. Let the sharp image and its down-sampled image be respectively expressed as
Figure BDA0002222144890000161
And
Figure BDA0002222144890000162
where N is the size of the sharp image and a represents the down-sampling factor. From sharp image X and downsampled image XaThe extracted image blocks are respectively represented as QjX and RiXaWherein
Figure BDA0002222144890000163
And
Figure BDA0002222144890000164
the extraction matrix is used for extracting the jth and ith image blocks from a clear image and a downsampled image respectively, and the block size of the extracted image is n. For any image block Q in an imagejX in the downsampled image XaSearching for its similar image block RiXa. Since the similarity of image blocks widely exists between different scales of images, i.e. for QjX, a plurality of image blocks similar to the X can be searched in the down-sampled image and are arranged at the XaSearch for m-1 and QjX most similar image blocks and are represented by columns
Figure BDA0002222144890000165
QjX and the non-local similar image blocks in the down-sampled image are aggregated to form a similar image block group PjIt can be expressed as:
Figure BDA0002222144890000166
wherein n is the image block size and m is the number of similar image blocks.
The similarity criterion of the image blocks has various measurement criteria, such as Euclidean distance, correlation coefficient and the like, the Euclidean distance is used as the measurement basis of the similarity between the image blocks, and the similar image blocks with the Euclidean distance smaller than a set threshold delta d are searched instead of searching a fixed number of similar image blocks for each image block. For searching similar image blocks, the invention adopts an adaptive threshold method for judging similarity of the image blocks to carry out interpolation shift on an original image X to generate an image with 1/2 sub-pixel displacement
Figure BDA0002222144890000171
For each input image block QjX is in
Figure BDA0002222144890000172
Finding out image block in corresponding position
Figure BDA0002222144890000173
The threshold Δ d is calculated as:
Figure BDA0002222144890000174
wherein gamma is a control coefficient. As can be seen from equation (14), the threshold Δ d is small for smooth image blocks, and large for edge image blocks; in other words, the more the image block content changes, the larger the threshold Δ d; on the contrary, the flatter the content change of the image block is, the smaller the threshold Δ d is, and the purpose is to balance the number of similar image blocks searched by the smooth image block and the edge image block. In addition, the invention sets the lower limit L of the searching number of the similar blockslowAnd an upper limit of LhighThen the number of similar blocks Llow≤m≤Lhigh. If the number of the searched similar blocks is less than LlowThen, thenThis input image block is not used in the set of equations of the set up type (9). If the number of the searched similar blocks is more than LhighThen only the front L is selectedhighSimilar image blocks.
From the formula (13), PjFor groups of similar blocks, wherein the first column is block QjX, in order to establish a relationship with the image X in the expression, the expression (9) will be
Figure BDA0002222144890000175
Is written as
Figure BDA0002222144890000176
Can be expressed as
Figure BDA0002222144890000177
Let its derivative to X be 0, the following equation can be obtained:
Figure BDA0002222144890000181
wherein,
Figure BDA0002222144890000182
step 5.2 reconstruction of reference image z by group sparse representations；
Since equation (16) is equal sign and the unknown sparse representation coefficient α is rightjDependent on the solution of an equation
Figure BDA0002222144890000183
Thus the equation has no closed solution. For the invention
Figure BDA0002222144890000184
Approximation
Figure BDA0002222144890000185
Solving dictionary DjThe following groups of sparse representation coefficients, using the groupsSparse representation of the sharp image estimated in the last iteration
Figure BDA0002222144890000186
Rebuild as an update
Figure BDA0002222144890000187
The reference image of (2). Order to
Figure BDA0002222144890000188
In the course of alternative iterative solutions, zsCan be regarded as the next iteration to update the clear image to be estimated
Figure BDA0002222144890000189
Wherein z issIs the result of the current estimation of the sharp image
Figure BDA00022221448900001810
Similar image blocks in the downsampled image are obtained through group sparse reconstruction, the fuzzy degree of the image is reduced due to downsampling operation, the edges of the reconstructed image are forced to be clearer by carrying out overall group sparse representation on the similar image block group matrix, and z issIs smaller, with z being smallersAs a reference image, the estimation of the sharp image obtained in the next iteration can be used
Figure BDA00022221448900001811
Current estimate of
Figure BDA00022221448900001812
And the degree of blurring of the clear image estimation result is smaller and smaller as the number of iterations increases. This is why the present invention utilizes cross-scale rather than co-scale structure self-similarity as a regularization constraint. If the same scale structure self-similarity is adopted as the regularization constraint, the reference image zsIs based on
Figure BDA00022221448900001813
In (1)The similar image blocks are reconstructed, and the fuzzy degree thereof is
Figure BDA00022221448900001814
Rather, it cannot be used as a clearer reference image, and thus cannot provide effective information for a clear image to be estimated in an iterative process.
The process of solving the approximation of equation (16) will be described in detail below. For the current image
Figure BDA00022221448900001815
Under which the image is sampled
Figure BDA00022221448900001816
Similar image blocks are searched to form a group matrix. Since the group matrix combined by the similar image blocks is a low-rank matrix, the low-rank property of the group matrix is realized by constraining the sparsity of the representation coefficients by utilizing the association of the group sparse representation and the low-rank matrix, and the problem of constructing a group sparse representation dictionary is not required to be considered. The solution of the sparse representation coefficient of each data vector in the sparse representation under the dictionary is mutually independent, the overall constraint of the data representation coefficient under the dictionary is lacked, the rank minimization under the specific dictionary can better represent the global structure characteristic of the data, so that the problem that the sparse representation lacks the global structure constraint of the reconstructed image is solved, the robustness to noise can be improved, and the accuracy of fuzzy kernel estimation is further improved.
First pair matrix PjSingular value decomposition is carried out:
Figure BDA0002222144890000191
therein, sigmaj＝diag(σj，1，…，σj，r) As diagonal matrix of singular values, σj，iWhere i is 1, …, r is the matrix PjR ═ min (n, m), m and n represent the set matrix PjOf (c) is calculated. u. ofj，iAnd vj，iAre respectively UjAnd VjThe column vector of (a) is,
Figure BDA0002222144890000192
is a rank 1 matrix.
Zha et al propose an adaptive dictionary construction method as an interpretation of the association of a group sparse representation with a low rank matrix. Will be provided with
Figure BDA0002222144890000193
Arranged as column vectors by column
Figure BDA0002222144890000194
Each similar image block group PjAdaptive dictionary DjFrom r number dj，iIs composed of, i.e.
Figure BDA0002222144890000195
Wherein d isj，iAs a dictionary DjOf (2) is used. Will PjArranged as column vectors by column
Figure BDA0002222144890000196
Singular value vector representation as
Figure BDA0002222144890000197
Then, equation (17) can be written as:
Pj＝DjAj (18)
from the equation (18), the group matrix PjIs equivalent to its singular value vector ajL of0Norm, i.e. rank (P)j)＝||Aj||0，||·||0Is a vector l0And (4) norm. That is, the low rank of the set matrix is equivalent to the sparsity of the matrix singular value vectors. Thus, the sparse representation coefficient A is constrainedjIs equivalent to the constraint set matrix PjLow rank property of (1). Due to l0Norm is a non-convex function, here denoted by1Norm pair l0The norm undergoes convex relaxation due to the set matrix PjIs equivalent to its singular value vector ajL of1Norm, i.e. Pj||*＝||Aj||1Convex relaxation of the rank function of the matrix with the kernel norm of the matrix to constrain the set matrix PjLow rank property of (1). Solving a set matrix P using singular value threshold operatorsjNuclear norm Pj||*The closed solution of (1) is
Figure BDA0002222144890000198
Wherein,
Figure BDA0002222144890000199
sigma taking beta not less than 0 as parameterjSoft threshold operator, defined as
Figure BDA00022221448900002016
As can be seen from equation (20), the larger β is, the larger the reconstruction set matrix
Figure BDA0002222144890000201
The smaller the rank of (c).
Figure BDA0002222144890000202
First column in (1) namely reconstructed image block
Figure BDA0002222144890000203
Because the edges of the image blocks in the downsampling are clearer and have stronger similarity with the clear image, the edges of the current image are forced to be closer to the edges of the clear image by restricting the rank of the matrix formed by the similar image blocks.
Step 5.3 solving
Figure BDA0002222144890000204
Upon obtaining an approximate set of sparse reconstructed blocks
Figure BDA0002222144890000205
Then, the clear image estimated in the last iteration is processed
Figure BDA0002222144890000206
Rebuild as an update
Figure BDA0002222144890000207
The reference image of (2). Will update the pairs by solving a linear system of equations
Figure BDA0002222144890000208
When equation (16) can be expressed approximately as
Figure BDA0002222144890000209
Because of the effect of M, the fast solution in the frequency domain can not be realized through Fourier transform, and the invention adopts a Bi-Conjugate Gradient (BICG) method to solve the formula (21) to obtain
Figure BDA00022221448900002010
Step 6, judging convergence to obtain the estimation of a fuzzy core;
performing one-time iteration solution on the objective function through the steps 3, 4 and 5 to obtain the estimation of the fuzzy core
Figure BDA00022221448900002011
And estimation of sharp images
Figure BDA00022221448900002012
Update to
Figure BDA00022221448900002013
If the iteration reaches the maximum iteration number or the iteration converges at the moment, the iteration is stopped. Otherwise, let k be k +1, and then repeat step 3, step 4, and step 5 until the iteration reaches the maximum number of iterations or the iteration converges.
And 7, after the fuzzy kernel estimation, estimating the clear image by using a non-blind restoration algorithm.
The fuzzy kernel estimation corresponds to the optimization problem shown in the solving formula (4), and the estimation result of the fuzzy kernel is obtained
Figure BDA00022221448900002014
In that
Figure BDA00022221448900002015
On the basis, an effective non-blind restoration algorithm is adopted to obtain the final clear image estimation. The non-blind restoration algorithm comprises a total variation regularization method, a sparse non-blind restoration method, an EPLL algorithm and the like.
Preferably, the present invention sets the image block size n to 5 × 5, the singular value threshold β to 0.2, and the regularization parameter λs＝0.0008、λg0.002 and λh0.0003N, 19 similar image blocks, and a search window size of 25 × 25. The larger the down-sampling factor a is, the clearer the image blocks in the down-sampled image of the blurred image are, but the fewer the number of similar image blocks between the images with different scales is, so that the value of the down-sampling factor needs to be comprehensively considered, and the scaling factor between the pyramids is set to 4/3. Since the blind restoration does not know the true size of the blur kernel, the size of the blur kernel needs to be estimated by other means or preset during restoration. If the size of the estimated or preset fuzzy core is too large, the small-size fuzzy core is not easy to solve; if the estimated or preset size of the fuzzy core is smaller, the complete fuzzy core cannot be solved. The present invention sets the size of the blur kernel to 51 × 51. Experiments on simulation and real data show that the size of a blur kernel of most blurred images is not larger than 51 × 51, and when the real blur kernel is small, the invention can still obtain an accurate estimation result under the condition that the size of the blur kernel is set to be 51 × 51. The invention sets the iteration number of the loop as 14 and does not set the iteration convergence threshold.
The disclosed method was validated on a Koehler dataset comprising 4 images, each image having 12 blur kernel types (of which the last 5 are large size blur kernels), yielding a total of 48 blurred images. For a large-sized blur kernel, its initial size is set to 151 × 151. The Peak signal-to-noise ratio (PSNR) of 199 unblurred images captured along the camera motion trajectory was compared to the results of each deblurring, and the maximum PSNR was recorded as an indicator for quantitative evaluation. The larger PSNR between the restored image and the true value image indicates that the restored image is closer to the true value image. The methods proposed by Pan et al and Yan et al are two generally accepted blind deblurring methods with optimal performance, wherein Pan et al introduces a dark channel prior based on image block statistics in kernel estimation, and Yan et al proposes an image regularization term based on an image block bright channel prior. On the Koehler data set, fig. 7 quantitatively compares the average PSNR of the method disclosed by the present invention with that of the two methods on each image, and it can be seen that the method disclosed by the present invention is superior to the method of Yan and the like on the average PSNR of all images, and superior to the method of Pan and the like on the average PSNR of the last two images.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (6)

1. An image blind restoration method based on group sparse representation is characterized by comprising the following steps:
step 1, reading in a fuzzy image and constructing an image pyramid;
reading a fuzzy image, setting the initial size of a fuzzy kernel, downsampling the fuzzy image layer by layer to construct an image pyramid, wherein the sizes of the fuzzy kernels corresponding to image pyramids of different levels are different, and when the image pyramid is constructed, if the size of the fuzzy kernel corresponding to the image pyramid of the current layer is smaller than 3 multiplied by 3, the construction of the image pyramid is stopped;
step 2, initializing a clear image;
and recording the number of loop iterations as k, setting the initialization to be k equal to 0, and taking the blurred image y as a clear image initial estimation if the current layer is the first layer of the image pyramid
Figure FDA0003539560340000011
Is provided with
Figure FDA0003539560340000012
Otherwise, the interpolation result of the clear image estimated by the coarse layer pyramid is used as the initial estimation of the clear image of the current layer
Figure FDA0003539560340000013
Step 3, screening image blocks and estimating a mark matrix M;
setting a mark matrix as M, wherein the M is a binary image, and if the value of a certain pixel in the M is 1, indicating that the corresponding image block participates in fuzzy kernel estimation and group sparse representation constraint; for the restored image
Figure FDA0003539560340000014
Performing Gaussian filtering, and then estimating edges of the filtered image;
step 4, estimating a fuzzy core;
estimation of stationary current image
Figure FDA0003539560340000015
Updating the fuzzy kernel of the next iteration with the following equation
Figure FDA0003539560340000016
Figure FDA0003539560340000017
In the formula, y represents a blurred image,
Figure FDA0003539560340000018
for gradient operators of the image, λhA regularization parameter, an indicates element by element multiplication,
Figure FDA0003539560340000019
which represents the fourier transform of the signal,
Figure FDA00035395603400000110
which represents the inverse of the fourier transform,
Figure FDA00035395603400000111
a complex conjugate representing a fourier variation; will restore the image
Figure FDA00035395603400000112
Medium gradient
Figure FDA00035395603400000113
Setting the gradient of pixel points smaller than a certain threshold value to be 0; noting the gradient threshold as
Figure FDA00035395603400000114
Fuzzy kernel size of NhThen, then
Figure FDA00035395603400000115
The selection method comprises the following steps: the image gradients are first divided into four groups according to their direction and then set
Figure FDA00035395603400000116
Is chosen to ensure that each group remains at least
Figure FDA0003539560340000021
Each pixel point is used for estimating a fuzzy kernel; because the restored image is clearer and clearer along with the increase of the iteration times, in order to enable more pixel points in the restored image to be gradually added into the process of fuzzy kernel estimation, the restored image is subjected to iterative processing each time
Figure FDA0003539560340000022
The value of (a) is reduced to 1.1 times of that of the last iteration;
step 5, clear images are estimated;
given the estimation of the current image
Figure FDA0003539560340000023
Fixing the fuzzy kernel estimate for the next iteration
Figure FDA0003539560340000024
Updating the image of the next iteration
Figure FDA0003539560340000025
Step 6, judging convergence to obtain the estimation of a fuzzy core;
performing one-time iteration solution on the objective function through the steps 3, 4 and 5 to obtain the estimation of the fuzzy core
Figure FDA0003539560340000026
And estimation of sharp images
Figure FDA0003539560340000027
Update to
Figure FDA0003539560340000028
If the iteration reaches the maximum iteration times or the iteration converges, stopping the iteration; otherwise, let k be k +1, k represents the number of iterations, and then repeat step 3, step 4, and step 5;
step 7, after fuzzy kernel estimation, clear images are estimated by using a non-blind restoration algorithm;
obtaining an estimation result of the blur kernel through steps 1 to 6
Figure FDA0003539560340000029
In that
Figure FDA00035395603400000210
On the basis, an effective non-blind restoration algorithm is adopted to obtain the final clear image estimation; the non-blind restoration algorithm comprises a total variation regularization method, a sparse non-blind restoration method and an EPLL algorithm.
2. The blind restoration method for images based on group sparse representation according to claim 1,
step 5.1, constructing a similar image block group;
let the sharp image and its down-sampled image be respectively expressed as
Figure FDA00035395603400000211
And
Figure FDA00035395603400000212
where N is the number of pixels of the sharp image, a represents the down-sampling factor,
Figure FDA00035395603400000213
is an N-dimensional column vector, and is,
Figure FDA00035395603400000214
is N/a2A dimension column vector; from sharp image X and downsampled image XaThe extracted image blocks are respectively represented as QjX and RiXaWherein
Figure FDA00035395603400000215
And
Figure FDA00035395603400000216
extracting matrixes respectively used for extracting a jth image block and an ith image block from a clear image and a downsampled image, wherein the size of the extracted image block is n; for any image block Q in an imagejX in the downsampled image XaSearching for similar image blocks RiXa(ii) a Due to the figureThe similarity of image blocks widely exists among different scales of images, namely for QjX, searching a plurality of similar image blocks in a certain-size search window in the downsampled image by using an image block matching method, and setting the similar image blocks in the XaSearch for m-1 and QjX most similar image blocks and are represented by columns
Figure FDA0003539560340000031
QjX and the non-local similar image blocks in the down-sampled image are aggregated to form a similar image block group PjExpressed as:
Figure FDA0003539560340000032
wherein n is the size of the image block, and m is the number of similar image blocks;
step 5.2 reconstruction of reference image z by group sparse representations；
Pairing group matrix P using singular value threshold operatorjCarrying out reconstruction, the closed solution of which is
Figure FDA0003539560340000033
Wherein S isβ(∑j) Is beta to>Sigma with parameter 0jA soft threshold operator defined as
Sβ(∑j)＝soft(∑j；β)＝max(∑j-β；0)
From the above equation, the larger beta, the larger the reconstruction set matrix
Figure FDA0003539560340000034
The smaller the rank of (d); by using
Figure FDA0003539560340000035
Approximation
Figure FDA0003539560340000036
First column in (1) namely reconstructed image block
Figure FDA0003539560340000037
DjIs an adaptive dictionary; order to
Figure FDA0003539560340000038
In the course of alternative iterative solutions, zsUpdating the clear image to be estimated as the next iteration
Figure FDA0003539560340000039
The reference image of (a);
step 5.3 solving
Figure FDA00035395603400000310
Is obtained by
Figure FDA00035395603400000311
Thereafter, the pair is updated by solving the following system of linear equations
Figure FDA00035395603400000312
Estimation of (2):
Figure FDA00035395603400000313
wherein | M | is the number of non-zero elements in M, N is the number of image pixels,
Figure FDA00035395603400000314
a one-dimensional column vector of y,
Figure FDA00035395603400000315
as a function of point spread
Figure FDA00035395603400000316
The matrix form of (a), i.e. the blur matrix,
Figure FDA00035395603400000317
as gradient operators
Figure FDA0003539560340000041
In the form of a matrix; due to the action of M, the fast solution in the frequency domain can not be realized through Fourier transform, and a bi-conjugate gradient method is adopted to solve the linear equation system as an upper line to obtain
Figure FDA0003539560340000042
3. The blind restoration method for images based on group sparse representation according to claim 1,
the size n of the image block is 5 × 5.
4. The blind restoration method for images based on group sparse representation according to claim 2,
the down-sampling factor a is 4/3.
5. The blind restoration method for images based on group sparse representation according to claim 1,
the iteration number of the loop is 14, and an iteration convergence threshold value is not set.
6. The blind restoration method for images based on group sparse representation according to claim 1,
the blur kernel has a size of 51 × 51.
CN201910938167.5A 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation Active CN110675347B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910938167.5A CN110675347B (en) 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910938167.5A CN110675347B (en) 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation

Publications (2)

Publication Number Publication Date
CN110675347A CN110675347A (en) 2020-01-10
CN110675347B true CN110675347B (en) 2022-05-06

Family

ID=69080294

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910938167.5A Active CN110675347B (en) 2019-09-30 2019-09-30 Image blind restoration method based on group sparse representation

Country Status (1)

Country Link
CN (1) CN110675347B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112365420B (en) * 2020-11-12 2024-03-15 重庆邮电大学 Fuzzy image blind restoration method based on non-local window gradient
US11721001B2 (en) 2021-02-16 2023-08-08 Samsung Electronics Co., Ltd. Multiple point spread function based image reconstruction for a camera behind a display
US11722796B2 (en) 2021-02-26 2023-08-08 Samsung Electronics Co., Ltd. Self-regularizing inverse filter for image deblurring
CN113487491B (en) * 2021-05-26 2024-04-26 辽宁工程技术大学 Image restoration method based on sparsity and non-local mean self-similarity
CN113535996B (en) * 2021-05-27 2023-08-04 中国人民解放军火箭军工程大学 Road image dataset preparation method and device based on aerial image
CN113379647B (en) * 2021-07-08 2024-07-02 湘潭大学 Multi-feature image restoration method for optimizing PSF estimation
CN117501304A (en) * 2021-09-28 2024-02-02 华为技术有限公司 Image processing method and related equipment
CN114418877B (en) * 2022-01-12 2023-09-01 新乡学院 Image non-blind deblurring method and system based on gradient amplitude similarity
CN114418883B (en) * 2022-01-18 2024-03-29 北京工业大学 Blind image deblurring method based on depth priori
CN115249028B (en) * 2022-05-13 2023-06-23 哈尔滨工业大学 Blind deconvolution wireless communication signal reconstruction method based on sparse regularization constraint
CN118134804B (en) * 2024-02-02 2024-08-06 湖北师范大学 Image sharpening processing system based on ambiguity recognition

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875349A (en) * 2016-12-30 2017-06-20 无锡高新兴智能交通技术有限公司 The computational methods and blind image restoring method of fuzzy core in blind image restoring method
CN107730466A (en) * 2017-10-11 2018-02-23 北京工业大学 The method for blindly restoring image represented based on structure self-similarity and low-rank matrix
CN107742278A (en) * 2017-10-25 2018-02-27 重庆邮电大学 With reference to L0The motion blur image blind restoration method of norm and space scale information
CN109919871A (en) * 2019-03-05 2019-06-21 重庆大学 Fuzzy core estimation method based on image and fuzzy core mixed constraints

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150030252A1 (en) * 2011-12-16 2015-01-29 The Research Foundation For The State University Of New York Methods of recognizing activity in video
CN105957025A (en) * 2016-04-21 2016-09-21 天津大学 Inconsistent image blind restoration method based on sparse representation
KR102614012B1 (en) * 2016-11-08 2023-12-14 한화비전 주식회사 Aapparatus of processing image and method of providing image thereof
CN108564544B (en) * 2018-04-11 2023-04-28 南京邮电大学 Image blind deblurring combined sparse optimization method based on edge perception

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875349A (en) * 2016-12-30 2017-06-20 无锡高新兴智能交通技术有限公司 The computational methods and blind image restoring method of fuzzy core in blind image restoring method
CN107730466A (en) * 2017-10-11 2018-02-23 北京工业大学 The method for blindly restoring image represented based on structure self-similarity and low-rank matrix
CN107742278A (en) * 2017-10-25 2018-02-27 重庆邮电大学 With reference to L0The motion blur image blind restoration method of norm and space scale information
CN109919871A (en) * 2019-03-05 2019-06-21 重庆大学 Fuzzy core estimation method based on image and fuzzy core mixed constraints

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Blur kernel estimation using sparse representation and cross-scale self-similarity;Jing Yu 等;《Multimedia Tools and Applications》;20190129;第18549-18570页 *
基于组稀疏残差约束的自适应强噪声图像复原算法;高红霞 等;《华南理工大学学报(自然科学版)》;20180831;第46卷(第8期);第11-18页 *

Also Published As

Publication number Publication date
CN110675347A (en) 2020-01-10

Similar Documents

Publication Publication Date Title
CN110675347B (en) Image blind restoration method based on group sparse representation
Sroubek et al. Robust multichannel blind deconvolution via fast alternating minimization
CN106920220A (en) Based on the turbulent flow method for blindly restoring image that dark primary and alternating direction multiplier method optimize
CN111738954B (en) Single-frame turbulence degradation image distortion removal method based on double-layer cavity U-Net model
CN106251297A (en) A kind of estimation based on multiple image fuzzy core the rebuilding blind super-resolution algorithm of improvement
Rani et al. A brief review on image restoration techniques
CN108629741B (en) Fuzzy kernel estimation method based on L0 and L1 regular terms
Mafi et al. Survey on mixed impulse and Gaussian denoising filters
CN115345791A (en) Infrared image deblurring algorithm based on attention mechanism residual error network model
CN113379647B (en) Multi-feature image restoration method for optimizing PSF estimation
Cao et al. Single image motion deblurring with reduced ringing effects using variational Bayesian estimation
CN106846286B (en) Video super-resolution algorithm for reconstructing based on a variety of complementary priori
CN113487491B (en) Image restoration method based on sparsity and non-local mean self-similarity
Wang et al. Optical satellite image MTF compensation for remote-sensing data production
Chen et al. Blind restoration for nonuniform aerial images using nonlocal Retinex model and shearlet-based higher-order regularization
CN114170087A (en) Cross-scale low-rank constraint-based image blind super-resolution method
CN108681988B (en) Robust image resolution enhancement method based on multiple images
Su et al. Atmospheric turbulence degraded image restoration using a modified dilated convolutional network
Seke et al. Multi‐frame super‐resolution algorithm using common vector approach
Deledalle et al. BATUD: blind atmospheric turbulence deconvolution
Wen et al. Patch-wise blind image deblurring via Michelson channel prior
Gunturk et al. Fundamentals of image restoration
Kaur et al. Study of Image Denoising and Its Techniques
Yang et al. Non-blind image blur removal method based on a Bayesian hierarchical model with hyperparameter priors
Gunturk Super-resolution imaging

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant