CN105321156B - A kind of image recovery method based on multi-factor structure - Google Patents
A kind of image recovery method based on multi-factor structure Download PDFInfo
- Publication number
- CN105321156B CN105321156B CN201510835252.0A CN201510835252A CN105321156B CN 105321156 B CN105321156 B CN 105321156B CN 201510835252 A CN201510835252 A CN 201510835252A CN 105321156 B CN105321156 B CN 105321156B
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- matrix
- dictionary
- weight matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The present invention relates to a kind of image recovery method based on multi-factor structure, the present invention includes two stages：(1) training sample is clustered using K Mean Methods, trains to obtain the sub- dictionary of K class principal component analysis using principal component analysis method；(2) binding signal rarefaction representation, multi-factor structure is introduced in image restoration model, including bound term partial structurtes, non local structure and marginal texture as model, the optimal models that image sparse represents coefficient is established, solves and represents coefficient and by zygote dictionary reconstructed image.Beneficial effect of the present invention：The present invention is applied to image restoration model using multi-factor structure as constraint, improves the reconstructed results at restored image edge details.
Description
Technical Field
The invention belongs to the technical field of image processing, relates to an image restoration method, and more particularly relates to an image restoration method based on a multi-element structure.
Background
During the transmission, processing, recording and the like of the image, the image quality is reduced due to the influence of blurring, down sampling, noise and the like. The process of image quality degradation is called image degradation.
The image degradation model can be expressed as:
Y＝SBX+n
in the above formula, X is the original high quality image, B is the blurring operator, S is the down-sampling matrix, n is additive white gaussian noise, and Y is the degraded image. The image restoration is a process of solving the unknown high-quality image X according to the degraded image Y, and is an inverse process of the image degradation process. I represents an identity matrix, and when S = B = I, the process of solving the high-quality image X becomes an image denoising process; when S = I and B is a blurring operator, the problem is image deblurring; when S is a downsampling matrix and B is a blurring operator, the problem becomes the image super-resolution problem.
Image restoration refers to a process of reconstructing a high-quality image from a low-quality image, and the process is a pathological inverse problem, and the introduction of image prior knowledge helps to solve the problem. An important prior knowledge of an image is that the image has local self-similarity, but it only analyzes pixels in a region close to the observed pixel, ignoring the similarity of pixels in regions further away from the observed pixel. The image edge information is an important component of a high-quality image, and one of main information lost by a blurred image and a low-resolution image is an edge, so that edge details of the reconstructed image contribute to improving the image quality.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, and provides an image restoration method based on a multi-element structure, which is integrally divided into two stages: the method comprises a dictionary training stage and an image reconstruction stage, wherein the training stage is performed off line, and the reconstruction stage reconstructs a high-quality image by using multi-element structure prior information of the dictionary and the natural image obtained by training, including image local structure, non-local structure and edge structure information.
The purpose of the invention is realized by the following technical scheme. The image restoration method based on the multi-element structure comprises the following steps:
step 1: training a K-class principal component analysis sub-dictionary, and performing block processing on a training sample imageAre divided into sizes ofAre represented as x after being vectorized i ∈R p . The further steps are as follows:
step 101: screening image subblocks, and recording subblocks with standard deviation larger than or equal to delta as X = [ X ] 1 ,x 2 ,…x M ]。
Step 102: carrying out high-pass filtering on the image subblocks, extracting high-frequency information of the image subblocks and taking the high-frequency information as a training sample, and recording the obtained subblocks as training samples
Step 103: performing K-means clustering on the input M image sub-blocks, training a principal component analysis sub-dictionary for each class by adopting a principal component analysis method to obtain K class centers and sub-dictionaries which are respectively marked as mu m 、Φ m ，m＝1…K。
And 2, step: and (3) image restoration reconstruction, wherein a restoration reconstruction model of the image is represented as:
wherein W is a local weight matrix, V is a non-local weight matrix, X L Representing the original high quality image, of the same size as image X, E (-) represents an edge structure extraction operator, and γ, η, σ are constraint coefficients.
Local similarity of images indicates: image(s)Medium size isThe central pixel point of the window can be obtained by weighting and predicting the neighborhood pixel points in the window. Local weight matrix is solved for image X by adopting steering kernel regression methodThe body is as follows:
wherein det (-) represents the determinant,respectively represent window central pixel point x i And neighborhood pixel x j And T denotes transposition. Vector of neighborhood pixelsw ij Representing the center pixel point x of each window i And neighborhood pixel x j Structural similarity of (c). Q i And (3) representing a symmetric covariance matrix of the data, h is a global smoothing parameter, and u is the local density of the data sample. Weight vector of windowx j ∈χ i . The image column vector form is expressed as X epsilon R N×1 Then the local weight matrix W is equal to R N×N In whichχ s Represents x s Neighborhood of, w s Represents x s The weight vector of the window of the central pixel.
The self-similarity of the image only analyzes the pixel points in the area close to the observation pixel, and actually, the pixels in the area far away from the observation pixel have similar characteristics. The basic idea of non-local similarity is: based on an image block matching method, searching blocks similar to a target block in an image, weighting the similar blocks, and finally solving to obtain a non-local weight matrix V.
The low-quality image lacks image high-frequency information relative to the high-quality image, and the image high-frequency information formsEdges and details of the image. Introducing an edge structure constraint term, and directly adopting the difference value of the reconstructed image and the initialized high-quality imageApproximately representing the missing edge information, equation (1) is written as:
the image may be developed into a dictionary representation:
in the formula (4), R i Extracting the matrix for the subblock, the ith block X of the image X i ＝R i X, i =1,2, \ 8230and N1, N1 is the total number of image blocks. Phi denotes a set of sub-dictionaries, phi mi Represents a sub-block x i Selecting the best sub-dictionary from K sub-dictionaries according to the selection rule of x i Class center μ with minimum Euclidean distance mi 。α i Represents a sub-block x i And alpha is a sparse representation coefficient set. Equation (3), in conjunction with the sparsity of the sparse representation coefficients, can be described as:
equation (5) can be rewritten as:
order toFormula (6) can be represented as:
then the image is reconstructed as
The image restoration algorithm is as follows:
inputting: the method comprises the following steps of degrading an image Y, a fuzzy operator B, a down-sampling matrix S, additive white Gaussian noise n, a principal component analysis sub-dictionary set phi, constraint coefficients mu, gamma, eta, sigma and lambda, iteration times k =0, and maximum iteration times max.
1) Solving for an initial high quality image X from an input low quality image Y 0 . For super-resolution reconstruction of images, X 0 Interpolation results for the input low resolution image Y; for deblurring, X 0 Is the input blurred image Y. Let X L ＝X 0 。
2) According to image X 0 And solving initial values of the local weight matrix W and the non-local weight matrix V.
3) When k ≦ max:
1 updating the image by gradient descent method.
2. Extraction of matrix R using image subblocks i Partitioning the image, and solving sparse representation coefficients according to orthogonality among sub-dictionary atoms of principal component analysis
3. According to a constraint coefficient lambda, forObtaining sparse representation coefficient by adopting soft threshold method
4. Reconstructing an image
5.k＝k+1。
6. When k is an integer multiple of C, from image X k And recalculating the local weight matrix W and the non-local weight matrix V.
7. When the temperature is higher than the set temperatureWhen, the loop is ended, or k>, max ends the cycle.
And (3) outputting: high quality imagesOr whenWhen the utility model is used, the water is discharged,
the invention has the following advantages: the invention combines the multi-element structure prior information of the image, including the information of the local structure, the non-local structure and the edge structure of the image, uses the information as the constraint item of the image restoration model, combines the sparsity of the image sparse representation coefficient to solve the representation coefficient and uses the representation coefficient for image reconstruction, and improves the reconstruction quality of the restored image, particularly the reconstruction result at the edge detail of the image.
Drawings
FIG. 1 is a flow chart of the steps of the present invention.
Detailed Description
The present invention will be further described with reference to the following examples.
The image restoration method based on the multi-element structure comprises the following steps:
step (1) training a principal component analysis sub-dictionary, which specifically comprises the following steps:
the training sample image is processed by block division into the size ofOf mutually overlapping image sub-blocks x i ∈R p The overlapping pixels are 6 × 7. Firstly, screening image subblocks to obtain an image subblock with a standard deviation of more than or equal to delta =16, wherein the image subblock is marked as X = [ X ] 1 ,x 2 ,…x M ]. Secondly, the image sub-blocks are high-pass filtered,
extracting high-frequency information of the image subblocks and taking the high-frequency information as a training sample, and recording the obtained subblocks as training samples
Finally, performing K-means clustering on the input M image sub-blocks, training a principal component analysis sub-dictionary for each class by adopting a principal component analysis method to obtain K class centers and sub-dictionaries which are respectively marked as mu m 、Φ m M =1 \8230, K is 200.
Restoring and reconstructing the image in the step (2), specifically:
since the human eye is more sensitive to the luminance component Y, for a color RGB image, it is first converted into a YUV image. When the restoration model is super-resolution reconstruction, performing super-resolution reconstruction on the Y component, and amplifying the UV component by adopting bicubic interpolation; when the restoration model is deblurred, deblurring is performed on only the Y component, and the UV component is not processed. Then, converting the YUV image into an RGB image; for the gray image, super-resolution reconstruction or deblurring is directly carried out on the gray image.
Calculating local weight matrix W by using a steering kernel regression method, wherein the window size is 5 multiplied by 5, and the global smoothing parameterh is 0.25 and the local density u of the data sample is taken to be 1. When solving the non-local weight matrix V, the size of the window is 5 multiplied by 5. And searching the 10 blocks most similar to the target block in the whole graph to obtain a non-local weight matrix V. When super-resolution reconstruction, μ =5.5, μ γ =0.08, μ η =0.04, μ σ =0.001, λ =0.7, x L ＝X 0 Taking an image X of a low-resolution image Y amplified by bicubic interpolation b (ii) a When deblurring, μ =1.0, μ γ =0.002, μ η =0.018, μ σ =0.001, λ =1.28, x L ＝X 0 An initial blurred image is taken.
The image restoration algorithm is as follows:
inputting: degraded image Y, fuzzy operator B, downsampling matrix S, additive white Gaussian noise n, principal component analysis sub-dictionary set phi, constraint coefficients mu, gamma, eta, sigma, lambda, iteration times k =0, maximum iteration times max =1000, th =2 × 10 -6 。
1) Solving for an initial high quality image X from an input low quality image Y 0 For super-resolution reconstruction of images, X 0 Interpolation results for the input low resolution image Y; for deblurring, X 0 An input blurred image Y. Let X L ＝X 0 。
2) According to image X 0 And solving initial values of the local weight matrix W and the non-local weight matrix V.
3) When k ≦ max or:
1. the image is updated by the gradient descent method shown in equation (8).
2. Extraction of matrix R using image subblocks i Partitioning the image, and solving the sparse representation coefficient by using the formula (9)
3. According to a constraint coefficient lambda, forObtaining sparse representation coefficients by adopting soft threshold method
4. Reconstructing an image
5.k＝k+1。
6. When k is an integer multiple of C, X is selected from the image k And recalculating the local weight matrix W and the non-local weight matrix V, and taking C =300.
7.When, the loop is ended, or k>, max ends the cycle.
And (3) outputting: high quality imagesOr whenWhen the utility model is used, the water is discharged,
in addition to the above embodiments, the present invention may have other embodiments. All technical solutions formed by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the claims of the present invention.
Claims (2)
1. An image restoration method based on a multi-element structure is characterized in that: the method comprises the following steps:
step 1: training a K-class principal component analysis sub-dictionary, and performing block processing on a training sample image to obtain a size ofAre represented as x after being vectorized i ∈R p Further comprising:
step 101: screening image subblocks to obtain large standard deviationSubblocks equal to Δ are denoted X = [ X = 1 ,x 2 ,…x M ]；
Step 102: carrying out high-pass filtering on the image subblocks, extracting high-frequency information of the image subblocks and taking the high-frequency information as a training sample, and recording the obtained subblocks as training samples
Step 103: performing K-means clustering on the input M image sub-blocks, training a principal component analysis sub-dictionary for each class by adopting a principal component analysis method to obtain K class centers and sub-dictionaries which are respectively marked as mu m 、Φ m ，m＝1…K；
And 2, step: and image restoration reconstruction, further comprising:
step 201: establishing an image restoration reconstruction model, wherein the model is represented as:
wherein, the degraded image Y, the fuzzy operator B, the downsampling matrix S, I represent the unit matrix, W is the local weight matrix, V is the non-local weight matrix, X L Representing an initial high-quality image with the same size as the image X, E (-) represents an edge structure extraction operator, and gamma, eta and sigma are constraint coefficients;
step 202: using a steering kernel regression method to perform on the image X and the imageMedium size isSolving the local weight matrix, specifically:
wherein det (-) denotes determinant,respectively representing window central pixel points x i And neighborhood pixel x j T represents transpose; vector of neighborhood pixelsw ij Representing the center pixel point x of each window i And neighborhood pixel x j Structural similarity of (c); q i Representing a symmetric covariance matrix of the data sample, h is a global smoothing parameter, and u is the local density of the data sample; weight vector of windowx j ∈χ i (ii) a The image column vector form is expressed as X epsilon R N×1 Then the local weight matrix W is equal to R N×N In whichχ s Represents x s Neighborhood of, w s Represents x s A weight vector of a window which is a central pixel;
and step 3: introducing edge structure constraint term, and directly adopting difference value of reconstructed image and initialized high-quality imageApproximately representing the missing edge information, equation (1) is written as:
the image is developed into a dictionary representation:
wherein R is i Extracting the matrix for the subblock, the ith block X of the image X i ＝R i X, i =1,2, \ 8230, N1, N1 is the total number of image blocks; phi denotes a set of sub-dictionaries, phi mi Representing a subblock x i Selecting the best sub-dictionary from K sub-dictionaries according to the selection rule of x i Class center μ with minimum Euclidean distance mi ；α i Represents a sub-block x i Alpha is a sparse representation coefficient set; equation (3) in conjunction with the sparsity of the sparsity representation coefficients, translates into:
equation (5) is rewritten as:
order toFormula (6) can be represented as:
the reconstructed image is
2. The method for restoring an image based on a multi-component structure according to claim 1, wherein: the image restoration algorithm is as follows:
inputting: the method comprises the following steps of (1) degrading an image Y, a fuzzy operator B, a down-sampling matrix S, additive white Gaussian noise n, a principal component analysis sub-dictionary set phi, constraint coefficients mu, gamma, eta, sigma and lambda, iteration times k =0, and maximum iteration times max;
1) Low quality according to inputMeasuring image Y, solving initial high quality image X 0 (ii) a For super-resolution reconstruction, X 0 Interpolation results for the input low resolution image Y; for deblurring, X 0 For inputting a blurred image Y, let X L ＝X 0 ；
2) According to image X 0 Solving initial values of a local weight matrix W and a non-local weight matrix V;
3) When k ≦ max:
(1) Updating the image by adopting a gradient descent method;
(2) Extracting the matrix R with the image subblocks i Partitioning the image, and solving sparse coefficients according to orthogonality among sub-dictionary atoms of principal component analysis
(3) According to the constraint coefficient lambda, pairObtaining sparse representation coefficient by adopting soft threshold method
(4) Reconstructing the image
(5).k＝k+1；
(6) When k is an integer multiple of C, according to image X k Recalculating a local weight matrix W and a non-local weight matrix V;
(7).when, the loop is ended, or k&Max, finish the cycle;
and (3) outputting: high quality imagesOr
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510835252.0A CN105321156B (en) | 2015-11-26 | 2015-11-26 | A kind of image recovery method based on multi-factor structure |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510835252.0A CN105321156B (en) | 2015-11-26 | 2015-11-26 | A kind of image recovery method based on multi-factor structure |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105321156A CN105321156A (en) | 2016-02-10 |
| CN105321156B true CN105321156B (en) | 2018-05-22 |
Family
ID=55248477
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510835252.0A Active CN105321156B (en) | 2015-11-26 | 2015-11-26 | A kind of image recovery method based on multi-factor structure |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105321156B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084752B (en) * | 2019-05-06 | 2023-04-21 | 电子科技大学 | Image super-resolution reconstruction method based on edge direction and K-means clustering |
| CN111314741B (en) * | 2020-05-15 | 2021-01-05 | 腾讯科技（深圳）有限公司 | Video super-resolution processing method and device, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101872502A (en) * | 2010-05-21 | 2010-10-27 | 杭州电子科技大学 | Coin image recognition method based on sparse representation |
| CN102147915A (en) * | 2011-05-06 | 2011-08-10 | 重庆大学 | Method for restoring weighting sparse edge regularization image |
| CN102354395A (en) * | 2011-09-22 | 2012-02-15 | 西北工业大学 | Sparse representation-based blind restoration method of broad image |
| CN103116880A (en) * | 2013-01-16 | 2013-05-22 | 杭州电子科技大学 | Image super resolution rebuilding method based on sparse representation and various residual |
| CN103971335A (en) * | 2014-04-25 | 2014-08-06 | 河海大学 | Image super-resolution rebuilding method based on confidence coefficient kernel regression |
| CN104063886A (en) * | 2014-03-24 | 2014-09-24 | 杭州电子科技大学 | Nuclear magnetic resonance image reconstruction method based on sparse representation and non-local similarity |
| CN104299193A (en) * | 2014-09-29 | 2015-01-21 | 杭州电子科技大学 | Image super-resolution reconstruction method based on high-frequency information and medium-frequency information |
| CN104766272A (en) * | 2015-04-17 | 2015-07-08 | 河海大学 | Image super-resolution reestablishing method based on sub pixel displacement model |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7260269B2 (en) * | 2002-08-28 | 2007-08-21 | Seiko Epson Corporation | Image recovery using thresholding and direct linear solvers |
| US9159121B2 (en) * | 2014-02-18 | 2015-10-13 | Signal Processing, Inc. | Method for image denoising |
-
2015
- 2015-11-26 CN CN201510835252.0A patent/CN105321156B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101872502A (en) * | 2010-05-21 | 2010-10-27 | 杭州电子科技大学 | Coin image recognition method based on sparse representation |
| CN102147915A (en) * | 2011-05-06 | 2011-08-10 | 重庆大学 | Method for restoring weighting sparse edge regularization image |
| CN102354395A (en) * | 2011-09-22 | 2012-02-15 | 西北工业大学 | Sparse representation-based blind restoration method of broad image |
| CN103116880A (en) * | 2013-01-16 | 2013-05-22 | 杭州电子科技大学 | Image super resolution rebuilding method based on sparse representation and various residual |
| CN104063886A (en) * | 2014-03-24 | 2014-09-24 | 杭州电子科技大学 | Nuclear magnetic resonance image reconstruction method based on sparse representation and non-local similarity |
| CN103971335A (en) * | 2014-04-25 | 2014-08-06 | 河海大学 | Image super-resolution rebuilding method based on confidence coefficient kernel regression |
| CN104299193A (en) * | 2014-09-29 | 2015-01-21 | 杭州电子科技大学 | Image super-resolution reconstruction method based on high-frequency information and medium-frequency information |
| CN104766272A (en) * | 2015-04-17 | 2015-07-08 | 河海大学 | Image super-resolution reestablishing method based on sub pixel displacement model |
Non-Patent Citations (6)
| Title |
|---|
| EDGE PRESERVING SINGLE IMAGE SUPER RESOLUTION IN SPARSE ENVIRONMENT;Srimanta Mandal、Anil Kumar Sao;《Image Processing》;20130918;第967-971页 * |
| Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization;Weisheng Dong 等;《IMAGE PROCESSING》;20110731;第20卷(第7期);第1838-1857页 * |
| Kernel Regression for Image Processing and Reconstruction;Hiroyuki Takeda 等;《IMAGE PROCESSING》;20070228;第16卷(第2期);第349-366页 * |
| 基于结构信息提取的图像质量评价;叶盛楠 等;《电子学报》;20080531;第36卷(第5期);第856-861页 * |
| 基于非局部相似性的立体图像超分辨率技术;钱斌豪、陈华华;《杭州电子科技大学学报(自然科学版)》;20150531;第35卷(第3期);第35-39页 * |
| 基于高频和中频信息的图像超分辨率重建;陈伟超、陈华华;《杭州电子科技大学学报(自然科学版)》;20150131;第35卷(第1期);第49-52页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105321156A (en) | 2016-02-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112200750B (en) | Ultrasonic image denoising model establishing method and ultrasonic image denoising method | |
| Xu et al. | Image inpainting by patch propagation using patch sparsity | |
| Liu et al. | Image restoration using total variation with overlapping group sparsity | |
| CN108133465B (en) | Non-convex low-rank relaxation hyperspectral image recovery method based on spatial spectrum weighted TV | |
| WO2015180053A1 (en) | Method and apparatus for rapidly reconstructing super-resolution image | |
| US10198801B2 (en) | Image enhancement using self-examples and external examples | |
| CN111028163A (en) | Convolution neural network-based combined image denoising and weak light enhancement method | |
| CN111583123A (en) | Wavelet transform-based image enhancement algorithm for fusing high-frequency and low-frequency information | |
| WO2006106919A1 (en) | Image processing method | |
| WO2007114363A1 (en) | Image processing method | |
| CN112150354B (en) | Single image super-resolution method combining contour enhancement and denoising statistical prior | |
| CN111738954B (en) | Single-frame turbulence degradation image distortion removal method based on double-layer cavity U-Net model | |
| CN110992292A (en) | Enhanced low-rank sparse decomposition model medical CT image denoising method | |
| Huang et al. | Lightweight deep residue learning for joint color image demosaicking and denoising | |
| CN111127354A (en) | Single-image rain removing method based on multi-scale dictionary learning | |
| JP5256582B2 (en) | Image processing method, image processing program, and image processing apparatus | |
| CN105321156B (en) | A kind of image recovery method based on multi-factor structure | |
| CN104050644A (en) | SAR image denoising method based on non-local restriction sparse representation | |
| Sun et al. | Progressive multi-branch embedding fusion network for underwater image enhancement | |
| CN107451961B (en) | Method for recovering sharp image under multiple fuzzy noise images | |
| CN112767271A (en) | Hyperspectral image depth noise reduction method based on three-dimensional variation network | |
| CN104166961B (en) | The fuzzy core method of estimation that low-rank for blindly restoring image approaches | |
| CN113139920A (en) | Ancient book image restoration method, terminal device and storage medium | |
| Nawaz et al. | Image super resolution by sparse linear regression and iterative back projection | |
| Zhang et al. | Multi-noise and multi-channel derived prior information for grayscale image restoration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |