CN105321156A - Multi-structure-based image restoration method - Google Patents

Abstract

The invention relates to a multi-structure-based image restoration method. The multi-structure-based image restoration method includes two stages: (1) clustering training samples based on a K means algorithm, and obtaining a K type principal component analysis sub-dictionary by means of training based on a principal component analysis method; (2) based on signal sparse representation, importing a multi-structure into an image restoration model, wherein the multi-structure includes a local structure, a non-local structure, and a marginal structure, and serves as a bound term of the model, establishing an optimal model of image sparse representation coefficients, and solving the representation coefficients and reconstructing an image according to the sub-dictionary. The beneficial effects of the multi-structure-based image restoration method are that: the multi-structure is applied to the image restoration model as the bound term, so that reconstruction results of edge details of the restored image are improved.

Description

A kind of image recovery method based on multi-factor structure

Technical field

The invention belongs to technical field of image processing, relate to the method for image restoration, more specifically, it relates to a kind of image recovery method based on multi-factor structure.

Background technology

Image, in the processes such as transmission, process, record, due to the impact such as fuzzy, down-sampling, noise, can make image quality decrease.The process of image quality decrease, is referred to as image degradation.

Image degradation model can be expressed as:

Y＝SBX+n

In above formula, X is original high quality, and B is fuzzy operator, and S is down-sampling matrix, and n is additive white Gaussian noise, and Y is degraded image.Image restoration is exactly the process solving unknown high quality graphic X according to degraded image Y, is the inverse process of image degradation process.I representation unit matrix, as S=B=I, the process solving high quality graphic X just becomes image denoising process; Work as S=I, when B is fuzzy operator, the problems referred to above just become image deblurring; When S is down-sampling matrix, when B is fuzzy operator, the problems referred to above just become image super-resolution problem.

Image restoration refers to the process reconstructing high quality graphic from low-quality image, and this process is an ill indirect problem, introduces Image Priori Knowledge and contributes to solving this problem.An important priori of image is that image has local self-similarity, but it only analyzes and the pixel observing pixel proximate region, have ignored and the similarity observing pixel in pixel region apart from each other.Image edge information is the important component part of high quality graphic, and one of main information that the image after fuzzy and low-resolution image are lost is edge, and the edge details therefore reconstructing image contributes to improving picture quality.

Summary of the invention

The object of the invention is to overcome deficiency of the prior art, and a kind of image recovery method based on multi-factor structure is provided, be divided into two stages on the whole: dictionary training stage and image reconstruction stage, training stage off-line carries out, phase of regeneration utilizes the multi-factor structure prior imformation of training dictionary and the natural image obtained, comprise Local Structure of Image, non local structure and marginal texture information, reconstruct high-quality image.

The object of the invention is to be achieved through the following technical solutions.This image recovery method based on multi-factor structure, the method comprises following steps:

Step 1: the sub-dictionary of training K class principal component analysis, training sample image is carried out piecemeal process, and being divided into size is overlapped image subblock dyad after be expressed as x _i∈ R ^p.Further step is as follows:

Step 101: screen image subblock, the sub-block that acquisition standard deviation is more than or equal to Δ is designated as X=[x ₁, x ₂... x _m].

Step 102: carry out high-pass filtering to image subblock, extracts the high-frequency information of image subblock and as training sample, the sub-block of acquisition is designated as $X_h=\[x_1^h,x_2^h,\mathrm{...},x_M^h\].$

Step 103: carry out K mean cluster to M image subblock of input, employing principal component analysis method trains a sub-dictionary of principal component analysis for each class, obtains K Ge Lei center and sub-dictionary, is designated as μ respectively _m, Φ _m, m=1 ... K.

Step 2: image restoration is rebuild, and the recovery reconstruction model of image is expressed as:

$\hat{X}=\arg \underset{x}{\min }\left|\right|Y-SBX|{|}_2^2+\mathrm{\γ}|\left|(I-W)X\right|{|}_2^2+\mathrm{\η}\left|\right|(I-V)X|{|}_2^2+\mathrm{\σ}||E\left(X_L\right)-E\left(X\right)|{|}_2^2---\left(1\right)$

Wherein, W is local weight matrix, and V is non local weight matrix, X _lrepresent initial high quality graphic, have same size with image X, E () represents that marginal texture extracts operational character, and γ, η, σ are constraint factor.

The local similarity of image represents: image middle size is the central pixel point of window can be obtained by neighborhood territory pixel point weight estimation in its window.Employing turns to kernel regression method to solve local weight matrix to image X, specifically:

$w_{ij}=\frac{\sqrt{\det \left(Q_i\right)}}{2{\mathrm{\πh}}^{2}u}\exp (-\frac{{({\stackrel{\·}{x}}_i-{\stackrel{\·}{x}}_j)}^T{Q}_i({\stackrel{\·}{x}}_i-{\stackrel{\·}{x}}_j)}{2h^{2}u})---\left(2\right)$

Wherein, determinant is asked in det () expression, represent window center pixel x respectively _iwith neighborhood territory pixel point x _jposition, T represents transposition.The vector that neighborhood territory pixel is formed w _ijrepresent each window center pixel x _iwith neighborhood territory pixel point x _jstructural similarity.Q _irepresent its symmetrical covariance matrix, h is overall smoothing parameter, and u is the local density of data sample.The weight vector of window x _j∈ χ _i.Image column vector form is expressed as X ∈ R ^{n × 1}, then local weight matrix W ∈ R ^{n × N}, wherein $W\left(s,t\right)=\left\{\begin{array}{c}{w}_{st},x_t\∈{\mathrm{\χ}}_s,w_{st}\∈w_s\\ 0,x_t\∉{\mathrm{\χ}}_s\end{array}\right.,$ χ _srepresent x _sneighborhood, w _srepresent x _scentered by the weight vector of window of pixel.

Image self-similarity only analyzes and the pixel observing pixel proximate region, in fact also has similar feature to the pixel in observation pixel region apart from each other.The basic thought of non local similarity is: based on image block matching process, the block that search is similar to object block in the picture, and is weighted similar block, finally solves and obtains non local weight matrix V.

Low-quality image, relative to high quality graphic, lacked image high-frequency information, and image high-frequency information defines edge and the details of image.Introduce marginal texture bound term, directly adopt the difference of rebuilding image and initialization high quality graphic the marginal information that approximate representation is lost, formula (1) is write as:

$\hat{X}=\arg \underset{x}{\min }\left|\right|Y-SBX|{|}_2^2+\mathrm{\γ}|\left|(I-W)X\right|{|}_2^2+\mathrm{\η}\left|\right|(I-V)X|{|}_2^2+\mathrm{\σ}||X_L-X|{|}_2^2---\left(3\right)$

Image can be launched into dictionary representation:

In formula (4), R _ifor sub-block extracts matrix, i-th block x of image X _i=R _ix, i=1,2 ... N1, N1 are image block sum.Φ represents sub-dictionary set, Φ _mirepresent sub-block x _ifrom the sub-dictionary of the best selected in K sub-dictionary, selection principle selects and x _ithere is the class center μ of minimum Eustachian distance _mi.α _irepresent sub-block x _irarefaction representation coefficient vector, α is rarefaction representation coefficient sets.Openness in conjunction with rarefaction representation coefficient of formula (3), can be described as:

Formula (5) can be write as again:

Order $y=\left[\begin{array}{c}Y\\ 0\\ 0\\ \sqrt{\mathrm{\σ}}{X}_L\end{array}\right],H=\left[\begin{array}{c}SB\\ \sqrt{\mathrm{\γ}}(I-W)\\ \sqrt{\mathrm{\η}}(I-V)\\ \sqrt{\mathrm{\σ}}I\end{array}\right],$ Formula (6) can be expressed as:

Then rebuilding image is

Image Restoration Algorithm is as follows:

Input: degraded image Y, fuzzy operator B, down-sampling matrix S, additive white Gaussian noise n, the sub-dictionary set Φ of principal component analysis, constraint factor μ, γ, η, σ, λ, iterations k=0, maximum iteration time max.

1). according to the low-quality image Y of input, solve initial high quality graphic X ⁰.To image super-resolution rebuilding, X ⁰for the interpolation result of input low-resolution image Y; To deblurring, X ⁰for input blurred picture Y.Make X _l=X ⁰.

2). according to image X ⁰, solve the initial value of local weight matrix W and non local weight matrix V.

3). as k≤max:

1 adopts gradient descent method more new images.

$\begin{array}{c}{X}^{k+1/2}=X^k+\mathrm{\μ}{H}^T(y-H{X}^k)\\ =X^k+\mathrm{\μ}\left[\begin{array}{cccc}{\left(\mathrm{SB}\right)}^T& \sqrt{\mathrm{\γ}}{(I-W)}^T& \sqrt{\mathrm{\η}}{(I-V)}^T& \sqrt{\mathrm{\σ}}I\end{array}\right](\left[\begin{array}{c}y\\ 0\\ 0\\ \sqrt{\mathrm{\σ}}{X}_L\end{array}\right]-\left[\begin{array}{c}\mathrm{SB}\\ \sqrt{\mathrm{\γ}}(I-W)\\ \sqrt{\mathrm{\η}}(I-V)\\ \sqrt{\mathrm{\σ}}I\end{array}\right]X^k)\\ =X^k+\mathrm{\μ}{\left(\mathrm{SB}\right)}^{T}y-\mathrm{\μ}{\left(\mathrm{SB}\right)}^T\left(\mathrm{SB}\right)X^k-\mathrm{\μ\γ}{(I-W)}^T(I-W)X^k\\ -\mathrm{\μ\η}{(I-V)}^T(I-V)X^k-\mathrm{\μ\σ}(X^k-X_L)\end{array}0---\left(8\right)$

2. utilize image subblock to extract matrix R _ipiecemeal is carried out to image, then solves rarefaction representation coefficient according to the orthogonality between the sub-dictionary atom of principal component analysis

${\widehat{\mathrm{\α}}}^{k+1/2}=\[{\mathrm{\Φ}}_{k1}^T\left(R_1{X}^{k+1/2}\right),{\mathrm{\Φ}}_{k2}^T\left(R_2{X}^{k+1/2}\right),...,{\mathrm{\Φ}}_{kN}^T\left(R_N{X}^{k+1/2}\right)\]---\left(9\right)$

3. according to constraint factor λ, right soft thresholding is adopted to obtain rarefaction representation coefficient

4. rebuild image

5.k＝k+1。

6. when k is the integral multiple of C, according to image X ^krecalculate local weight matrix W and non local weight matrix V.

7. work as time, end loop, or k>max end loop.

Export: high quality graphic $\hat{X}=X^{\max },$ Or work as $\left|\right|X^k-X^{k-1}|{|}_2^2<Th$ Time, $\hat{X}=X^k.$

The present invention has the following advantages: the multi-factor structure prior imformation of combining image of the present invention, comprise Local Structure of Image, non local structure and marginal texture information, it can be used as the bound term of image deblur model, combining image rarefaction representation coefficient openness solves and represents coefficient and for image reconstruction, improve the reconstruction quality of restored image, particularly the reconstructed results at image edge details place.

Accompanying drawing explanation

Fig. 1 is flow chart of steps of the present invention.

Embodiment

Below in conjunction with embodiment, the present invention is described further.

This image recovery method based on multi-factor structure of the present invention, the method comprises following steps:

The sub-dictionary of step (1) training principal component analysis (PCA), specifically:

Training sample image is carried out piecemeal process, and being divided into size is overlapped image subblock x _i∈ R ^p, overlaid pixel is 6 × 7.First, screen image subblock, the image subblock that acquisition standard deviation is more than or equal to Δ=16 is designated as X=[x ₁, x ₂... x _m].Secondly, high-pass filtering is carried out to image subblock,

Extract image subblock high-frequency information and as training sample, the sub-block of acquisition is designated as

Finally, carry out K mean cluster to M image subblock of input, employing principal component analysis method trains a sub-dictionary of principal component analysis for each class, obtains K Ge Lei center and sub-dictionary, is designated as μ respectively _m, Φ _m, m=1 ... K, K get 200.

Step (2) image restoration is rebuild, specifically:

Because human eye is more responsive for luminance component Y, for the RGB image of colour, first convert YUV image to.When restoration model is super-resolution rebuilding, Y-component carries out super-resolution rebuilding, and UV component adopts bicubic interpolation to amplify; When restoration model is deblurring, only in Y-component, carry out deblurring, UV component does not process.Then YUV image is converted to RGB image again; For gray level image, directly on gray-scale map, carry out super-resolution rebuilding or deblurring.

Employing turns to kernel regression method to calculate local weight matrix W, and the size of window is 5 × 5, and overall smoothing parameter h is 0.25, and the local density u value of data sample is 1.When solving non local weight matrix V, the size of getting window is 5 × 5.In view picture figure, find 10 blocks the most similar to object block, obtain non local weight matrix V.When super-resolution rebuilding, μ=5.5, μ γ=0.08, μ η=0.04, μ σ=0.001, λ=0.7, X _l=X ₀get the image X of low-resolution image Y after bicubic interpolation amplifies _b; When deblurring, μ=1.0, μ γ=0.002, μ η=0.018, μ σ=0.001, λ=1.28, X _l=X ₀get initial blurred picture.

Image Restoration Algorithm is as follows:

Input: degraded image Y, fuzzy operator B, down-sampling matrix S, additive white Gaussian noise n, the sub-dictionary set Φ of principal component analysis, constraint factor μ, γ, η, σ, λ, iterations k=0, maximum iteration time max=1000, Th=2 × 10 ^-6.

1). according to the low-quality image Y of input, solve initial high quality graphic X ⁰to image super-resolution rebuilding, X ⁰for the interpolation result of input low-resolution image Y; To deblurring, X ⁰for input blurred picture Y.Make X _l=X ⁰.

2). according to image X ⁰, solve the initial value of local weight matrix W and non local weight matrix V.

3). when k≤max or time:

1. adopt gradient descent method more new images shown in formula (8).

2. utilize image subblock to extract matrix R _ipiecemeal is carried out to image, then uses formula (9) to solve rarefaction representation coefficient

3. according to constraint factor λ, right soft thresholding is adopted to obtain rarefaction representation coefficient

4. rebuild image

5.k＝k+1。

6. when k is the integral multiple of C, according to image X ^krecalculate local weight matrix W and non local weight matrix V, get C=300.

7. time, end loop, or k>max end loop.

Export: high quality graphic $\hat{X}=X^{\max },$ Or work as $\left|\right|X^k-X^{k-1}|{|}_2^2<Th$ Time, $\hat{X}=X^k.$

In addition to the implementation, the present invention can also have other embodiments.All employings are equal to the technical scheme of replacement or equivalent transformation formation, all drop on the protection domain of application claims.

Claims (2)

1. based on an image recovery method for multi-factor structure, it is characterized in that: the method comprises following steps:

Step 1: the sub-dictionary of training K class principal component analysis, training sample image is carried out piecemeal process, and being divided into size is overlapped image subblock dyad after be expressed as x _i∈ R ^p, comprise further:

Step 101: screen image subblock, the sub-block that acquisition standard deviation is more than or equal to Δ is designated as X=[x ₁, x ₂... x _m];

Step 102: carry out high-pass filtering to image subblock, extracts the high-frequency information of image subblock and as training sample, the sub-block of acquisition is designated as $X_h=\left[\begin{array}{cccc}{x}_1^h,& x_2^h,& ...,& x_M^h,\end{array}\right];$

Step 103: carry out K mean cluster to M image subblock of input, employing principal component analysis method trains a sub-dictionary of principal component analysis for each class, obtains K Ge Lei center and sub-dictionary, is designated as μ respectively _m, Φ _m, m=1 ... K;

Step 2: image restoration is rebuild, and comprises further:

Step 201: set up image restoration reconstruction model, described model representation is:

$\hat{X}=\arg \underset{X}{min}\left|\right|Y-SBX|{|}_2^2+\mathrm{\&gamma;}|\left|(I-W)X\right|{|}_2^2+\mathrm{\&eta;}\left|\right|(I-V)X|{|}_2^2+\mathrm{\&sigma;}||E\left(X_L\right)-E\left(X\right)|{|}_2^2---\left(1\right)$

Wherein, W is local weight matrix, and V is non local weight matrix, X _lrepresent initial high quality graphic, have same size with image X, E () represents that marginal texture extracts operational character, and γ, η, σ are constraint factor;

Step 202: adopt and turn to kernel regression method to image X, image middle size is solve local weight matrix, specifically:

$w_{ij}=\frac{\sqrt{\det \left(Q_i\right)}}{2{\mathrm{\&pi;h}}^{2}u}\exp (-\frac{{({\stackrel{\&CenterDot;}{x}}_i-{\stackrel{\&CenterDot;}{x}}_j)}^T{Q}_i({\stackrel{\&CenterDot;}{x}}_i-{\stackrel{\&CenterDot;}{x}}_j)}{2h^{2}u})---\left(2\right)$

Wherein, determinant is asked in det () expression, represent window center pixel x respectively _iwith neighborhood territory pixel point x _jposition, T represents transposition; The vector that neighborhood territory pixel is formed w _ijrepresent each window center pixel x _iwith neighborhood territory pixel point x _jstructural similarity; Q _irepresent its symmetrical covariance matrix, h is overall smoothing parameter, and u is the local density of data sample; The weight vector of window x _j∈ χ _i; Image column vector form is expressed as X ∈ R ^{n × 1}, then local weight matrix W ∈ R ^{n × N}, wherein $W(s,t)=\left\{\begin{array}{ccc}{w}_{st},& x_t\&NotElement;{\mathrm{\&chi;}}_s,& w_{st}\&Element;w_s\\ 0,& x_t\&NotElement;{\mathrm{\&chi;}}_s& \end{array}\right.,$ χ _srepresent x _sneighborhood, w _srepresent x _scentered by the weight vector of window of pixel;

Step 3: introduce marginal texture bound term, directly adopts the difference of rebuilding image and initialization high quality graphic the marginal information that approximate representation is lost, formula (1) is write as:

$\hat{X}=\arg \underset{X}{min}\left|\right|Y-SBX|{|}_2^2+\mathrm{\&gamma;}|\left|(I-W)X\right|{|}_2^2+\mathrm{\&eta;}\left|\right|(I-V)X|{|}_2^2+\mathrm{\&sigma;}||X_L-X|{|}_2^2---\left(3\right)$

Image spread becomes dictionary representation:

Wherein, R _ifor sub-block extracts matrix, i-th block x of image X _i=R _ix, i=1,2 ... N1, N1 are image block sum; Φ represents sub-dictionary set, Φ _mirepresent sub-block x _ifrom the sub-dictionary of the best selected in K sub-dictionary, selection principle selects and x _ithere is the class center μ of minimum Eustachian distance _mi; α _irepresent sub-block x _irarefaction representation coefficient vector, α is rarefaction representation coefficient sets; Openness in conjunction with rarefaction representation coefficient of formula (3), is converted into:

Formula (5) is write as again:

Order $y=\left[\begin{array}{c}Y\\ 0\\ 0\\ \sqrt{\mathrm{\&sigma;}}{X}_L\end{array}\right],H=\left[\begin{array}{c}SB\\ \sqrt{\mathrm{\&gamma;}}(I-W)\\ \sqrt{\mathrm{\&eta;}}(I-V)\\ \sqrt{\mathrm{\&sigma;}}I\end{array}\right],$ Formula (6) can be expressed as:

Rebuilding image is then

2. the image recovery method based on multi-factor structure according to claim 1, is characterized in that: Image Restoration Algorithm is as follows:

Input: degraded image Y, fuzzy operator B, down-sampling matrix S, additive white Gaussian noise n, the sub-dictionary set Φ of principal component analysis, constraint factor μ, γ, η, σ, λ, iterations k=0, maximum iteration time max;

1). according to the low-quality image Y of input, solve initial high quality graphic X ⁰; To super-resolution rebuilding, X ⁰for the interpolation result of input low-resolution image Y; To deblurring, X ⁰for input blurred picture Y, make X _l=X ⁰;

2). according to image X ⁰, solve the initial value of local weight matrix W and non local weight matrix V;

3). as k≤max:

(1). adopt gradient descent method more new images;

$\begin{array}{c}{X}^{k+1/2}=X^k+{\mathrm{\&mu;H}}^T(y-{\mathrm{HX}}^k)\\ =X^k+\mathrm{\&mu;}\left[\begin{array}{cccc}{\left(SB\right)}^T& \sqrt{\mathrm{\&gamma;}}{(I-W)}^T& \sqrt{\mathrm{\&eta;}}{(I-V)}^T& \sqrt{\mathrm{\&sigma;}}I\end{array}\right](\left[\begin{array}{c}Y\\ 0\\ 0\\ \sqrt{\mathrm{\&sigma;}}{X}_L\end{array}\right]-\left[\begin{array}{c}SB\\ \sqrt{\mathrm{\&gamma;}}(I-W)\\ \sqrt{\mathrm{\&eta;}}(I-V)\\ \sqrt{\mathrm{\&sigma;}}I\end{array}\right]X^k)\\ =X^k+\mathrm{\&mu;}{\left(SB\right)}^{T}y-\mathrm{\&mu;}{\left(SB\right)}^T\left(SB\right)X^k-\mathrm{\&mu;}\mathrm{\&gamma;}{(I-W)}^T(I-W)X^k\\ =\mathrm{\&mu;}\mathrm{\&eta;}{(I-V)}^T(I-V)X^k-\mathrm{\&mu;}\mathrm{\&sigma;}(X^k-X_L)\end{array}---\left(8\right)$

(2). utilize image subblock to extract matrix R _ipiecemeal is carried out to image, then solves sparse coefficient according to the orthogonality between the sub-dictionary atom of principal component analysis

${\widehat{\mathrm{\&alpha;}}}^{k+1/2}=\&lsqb;{\mathrm{\&Phi;}}_{k1}^T\left(R_1{X}^{k+1/2}\right),{\mathrm{\&Phi;}}_{k2}^T\left(R_2{X}^{k+1/2}\right),...,{\mathrm{\&Phi;}}_{kN}^T\left(R_N{X}^{k+1/2}\right)\&rsqb;---\left(9\right)$

(3). according to constraint factor λ, right soft thresholding is adopted to obtain rarefaction representation coefficient

(4). rebuild image

(5).k＝k+1；

(6). when k is the integral multiple of C, according to image X ^krecalculate local weight matrix W and non local weight matrix V;

(7). time, end loop, or during k>max, end loop;

Export: high quality graphic $\hat{X}=X^{max},$ Or $\left|\right|X^k-X^{k-1}|{|}_2^2<Th,\hat{X}=X^k.$