CN102800077A - Bayes non-local mean image restoration method - Google Patents

Abstract

The invention discloses a Bayes non-local image restoration method which mainly solves the problems that the search of similar blocks is inaccurate and a parameter value is determined by experience in the existing sample-based non-local mean image restoration method. The method comprises the following steps: (1) determining a to-be-restored area omega and the boundary delta thereof for a to-be-restored image I; (2) finding a restoration block with the highest priority on the boundary, and modeling by use of a Bayesian framework; (3) pre-selecting a search area by use of an adaptive threshold; (4) searching for m sample blocks the most similar to the restoration block, and taking the weighted mean of the m sample blocks as a filling block of the restoration block; and (5) updating the confidence item and the to-be-restored area, and repeating the steps (1) to (5) until all points in the to-be-restored area are restored. The method disclosed by the invention can be used for restoring the image-damaged area, restoring the image scratch and removing the text in the image.

Description

The bayesian non-local mean image repair method

Technical field

The invention belongs to technical field of image processing, relate to image repair, can be used for repairing damaged zone in the image, the removal of image scratch and image Chinese version.

Background technology

Image information with its contain much information, transmission speed is fast, operating distance is far away etc., and advantage becomes the important means that the mankind obtain the important source of information and utilize information; And the image in the reality can cause losing of image information for various reasons, at this moment will use the image repair technology.

The purpose of image repair is the information of coming recovery automatically to lose according to the existing information of image, and it can be used for recovery, video text removal and the video error concealing etc. of old photo drop-out.Existing image repair method roughly can be divided into based on the restorative procedure of structure with based on two big types of the restorative procedures of texture.Wherein the restorative procedure based on structure all is a kind of restorative procedure based on PDE in essence; Propose by people such as Bertalmio the earliest; The restorative procedure that proposes by people such as Chan subsequently based on overall variation TV model, and the curvature Driven Diffusion CDD model restorative procedure that is produced by the inspiration of TV repairing model all belongs to the restorative procedure based on structure.These methods all are that the diffusion through information realizes, are only applicable to the damaged image repair of non-texture image and small scale.

In addition; The restorative procedure based on sample that people such as Criminisi propose is a kind of restorative procedure based on texture; This method used for reference the thought in the texture synthesis method seek sample block and the coupling duplicate; Made full use of simultaneously based on the diffusion way in the restorative procedure of structure and defined the priority of repairing piece; Make that being near the reparation piece in edge with more structural information has higher reparation priority, thereby when repairing texture information, structural information is also had certain maintenance.This method adopts single sample block directly to fill the area to be repaired; Owing to be difficult to make in the reality sample block and to be repaired to reach Optimum Matching; Therefore when filling to be repaired, can have certain error, along with the carrying out of repair process, this way can cause the accumulation of error.

Afterwards; Alexander Wong and Jeff Orchar have proposed a kind of non-local mean based on sample and have repaired algorithm; Adopt the weighted mean of a plurality of sample block to synthesize the filling block that is used to fill the area to be repaired, improved defective to a certain extent based on the sample restorative procedure.But this method is owing to use an attenuation coefficient to calculate sample block and to be repaired similarity weights as the negative exponential function of constant; And the information that is comprised in different to be repaired is different; Do like this and will certainly cause the calculating of similarity weights not accurate enough, and then cause repairing the well detail textures in the connection layout picture of result.

Summary of the invention

The objective of the invention is to deficiency, propose a kind of bayesian non-local mean image repair method, make the clean mark among the image repair result, thereby improve repairing effect to above-mentioned non-local mean restorative procedure based on sample.

The technical thought that realizes the object of the invention is; On basis based on the non-local mean restorative procedure of sample, utilize Bayesian Estimation theoretical, the point in the region of search is carried out preliminary election get; And based on Bayesian Estimation belfry a new weights computing formula; Utilize it to calculate the similarity weights of repairing between piece and the sample block, can search for similar more accurately, better to be repaired the result.Implementation step comprises as follows:

(1), confirms the border δ of area to be repaired Ω and area to be repaired for the image I to be repaired of input;

(2) utilize following formula, calculate the priority P (p) that central point all on the δ of the border of area to be repaired are repaired piece:

P(p)＝C(p)·D(p)，

Wherein, D (p) is a data item, and C (p) is the degree of confidence item, the credibility of presentation video pixel, and C (p) is initialized as C (p)=0, p ∈ Ω, C (p)=1, p ∈ I-Ω;

(3) central point

with the highest reparation piece of priority

is the center; Choose size and be the neighborhood of the M * M region of search as this reparation piece, definition is that the piece

at center is a sample block with point

in should the zone;

(3.1) utilize the Bayesian frame modeling to repairing piece

with sample block Ψ

, calculate average

and the equal value difference

i.e. that calculates them

of average

and the sample block of reparation piece respectively

(3.2) according to equal value difference

Obey The characteristic of Gaussian distribution, define an adaptive threshold: t=λ σ ₀, wherein, σ ₀For

Standard deviation, λ=1.65, u ₀For

Average, For Variance;

(3.3) point of all

in the selection region of search is got new region of search, back as preliminary election;

(4) Calculate the new repair block within the search area and the sample block

similarity distance:

$d({\mathrm{\ψ}}_{\hat{p}},{\mathrm{\ψ}}_{\hat{q}})={\left|\right|{\mathrm{\ψ}}_{\hat{p}}-{\mathrm{\ψ}}_{\hat{q}}\left|\right|}_2^2,$

Wherein,

is 2 norms;

(5) according to similarity distance

Obeying degree of freedom is card side's distribution X of n ²(n) characteristic, when n>=25, quantile

Choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece

Set;

(6) according to following formula, the similarity weights of sample block

in the set of computations and reparation piece

:

$\mathrm{\ω}=\frac{1}{Z}\exp (-\frac{d({\mathrm{\ψ}}_{\hat{p}},{\mathrm{\ψ}}_{\hat{q}})}{4{\mathrm{\σ}}^{2}N}),$

Wherein, Z is a normalized parameter, σ ²For repairing piece

Variance, N is for repairing piece

The number of the point that middle pixel value is known;

(7) according to the similarity weights, with the weighted mean of the whole sample block in the set, as filling block Ψ ₀, and with this filling block to repairing piece

Fill reparation;

(8) after repairing piece

completion reparation; Upgrade the area to be repaired, and upgrade the degree of confidence C (p) of the point of having accomplished reparation with the degree of confidence of repairing piece central point

:

$C\left(p\right)=C\left(\hat{p}\right),$ $p\∈{\mathrm{\ψ}}_{\hat{p}}\∩\mathrm{\Ω},$

Wherein, for repairing the degree of confidence of piece

central point

, and ∩ representes ' with ' relation;

(9) repeating step (1) ~ (8), have a few in the area to be repaired repaired;

The present invention compared with prior art has following advantage:

(1) the present invention gets through the average preliminary election is carried out in the region of search, gives up the different point that distributes, and only keeps the identical point that distributes, and makes similar of searching more accurate, thereby improves repairing effect.

(2) the present invention is according to the distribution character of sample block with the similarity distance

of repairing piece; Set a most similar adaptive sample block number m; Make the sample block number that searches more accurate, thereby improve repairing effect.

(3) the present invention is the basis with the Bayesian Estimation theory, has constructed a new adaptive weight, and these weights can be estimated different sample block more accurately for the contribution of repairing piece, thereby improve repairing effect.

Description of drawings

Fig. 1 is a process flow diagram of the present invention;

Fig. 2 is the breakage image that the present invention tests use;

Fig. 3 is with the reparation result of the present invention to Fig. 2;

Fig. 4 is the cut image that the present invention tests use;

Fig. 5 is with the reparation result of the present invention to Fig. 4;

Fig. 6 is that the text that experiment is used among the present invention is removed image;

Fig. 7 is with the reparation result of the present invention to Fig. 6.

Embodiment

With reference to Fig. 1, performing step of the present invention is following:

Step 1 is read in image I to be repaired, Fig. 2 for example, and Fig. 4 or Fig. 6 confirm area to be repaired Ω and border δ thereof.

Step 2, the priority of all pieces of computing center's point on the δ of border:

(2.1) definition D (p) is a data item, and C (p) is the degree of confidence item, and the credibility of its presentation video pixel carries out initialization: C (p)=0 to C (p), p ∈ Ω, C (p)=1, p ∈ I-Ω;

(2.2) utilize following formula, calculate degree of confidence item C (p) and data item D (p):

$C\left(p\right)=\frac{{\mathrm{\Σ}}_{q\∈{\mathrm{\ψ}}_{\mathrm{pI}(I-\mathrm{\Ω})}}C\left(q\right)}{\left|{\mathrm{\ψ}}_p\right|},$

$D\left(p\right)=\frac{|{{\▿I}_p}^{\⊥}\·n_p|}{\mathrm{\α}},$

Wherein, q is for repairing piece Ψ _pThe middle known point of pixel value, C (q) is the degree of confidence of some q, | Ψ _p| for repairing piece Ψ _pArea, n _pBe the vector of unit length vertical with the border, area to be repaired at the p place,

Be the p point place vector of unit length vertical with gradient, i.e. isophote direction, α is a normalizing parameter, for 8 gray level image α=255;

(2.3) utilize following formula, calculate the priority P (p) of all pieces of computing center's point on the δ of border:

P(p)＝C(p)D(p)。

Step 3, the distribution character through equal value difference carries out preliminary election to the region of search and gets:

(3.1) central point

with the highest reparation piece of priority

is the center; Choose size and be the neighborhood of the M * M field of search as this reparation piece, definition is that the piece

at center is a sample block with point

in should the zone;

(3.2) utilize the Bayesian frame modeling to repairing piece

with sample block

, calculate average and the equal value difference

i.e. that calculates them

of average and the sample block of reparation piece respectively

(3.3) according to equal value difference

Obey The characteristic of Gaussian distribution, define an adaptive threshold: t=λ σ ₀, wherein, σ ₀For

Standard deviation, λ=1.65, u ₀For Average, For

Variance;

(3.4) point of all

in the selection region of search is got new region of search, back as preliminary election.

Step 4 uses the bayesian non-local method that it is repaired for

.

(4.1) Calculate the new sample block within the search area

and repair block

similarity distance:

$d({\mathrm{\ψ}}_{\hat{p}},{\mathrm{\ψ}}_{\hat{q}})={\left|\right|{\mathrm{\ψ}}_{\hat{p}}-{\mathrm{\ψ}}_{\hat{q}}\left|\right|}_2^2,$ Wherein,

Be 2 norms;

(4.2) according to similarity distance Obeying degree of freedom is card side's distribution X of n ²(n) characteristic, when n>=25, quantile

Choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece

Set;

(4.3) utilize following formula, respectively the similarity weights of sample block

and reparation piece

in the set of computations:

$\mathrm{\ω}=\frac{1}{Z}\exp (-\frac{{\left|\right|{\mathrm{\ψ}}_{\hat{p}}-{\mathrm{\ψ}}_{\hat{q}}\left|\right|}_2^2}{4{\mathrm{\σ}}^{2}N}),$

Wherein, Z is a normalized parameter, σ ²For repairing piece

Variance, N is for repairing piece

The number of the point that middle pixel value is known;

(4.4) according to the similarity weights, with the weighted mean of the whole sample block in the set, as filling block Ψ ₀, and with this filling block to repairing piece

Fill reparation.

Step 5; After repairing piece

completion reparation; Upgrade the area to be repaired, and upgrade the degree of confidence C (p) of the point of having accomplished reparation with the degree of confidence of repairing piece

central point

:

$C\left(p\right)=C\left(\hat{p}\right),$ $p\∈{\mathrm{\ψ}}_{\hat{p}}\∩\mathrm{\Ω},$

Wherein,

for repairing the degree of confidence of piece

central point

, and ∩ representes ' with ' relation; Repeat above five steps, have a few in the area to be repaired repaired.

Effect of the present invention can further confirm through following experiment:

1. experiment condition:

The Criminisi method is used in this experiment respectively, compares test based on non-local mean restorative procedure and the inventive method of sample, and the reparation block size gets 7 * 7 in the experiment, and the region of search size gets 41 * 41.This experiment is divided into three parts: experiment is repaired in the damaged zone of (1) image, and its experiment uses figure to be Fig. 2 (b), and the experimental result contrast uses figure to be Fig. 2 (a); (2) image scratch reparation experiment, its experiment uses figure to be Fig. 4 (b), and the experimental result contrast uses figure to be Fig. 4 (a); (3) text is removed experiment, and its experiment uses figure to be Fig. 6 (b), and the experimental result contrast uses figure to be Fig. 6 (a).Various control methodss all are to use the MATLAB Programming with Pascal Language to realize in this experiment.

2. experiment content and result:

Under above-mentioned experiment condition, carry out the experiment of three parts respectively.

Experiment (1): utilize the present invention and existing two kinds of methods; Repair process is carried out in damaged zone in Fig. 2 (b) image; Result such as Fig. 3; Wherein Fig. 3 (a) is for using the figure as a result of Criminisi method reparation, and Fig. 3 (b) is for using the figure as a result that repairs based on the non-local mean restorative procedure of sample, the figure as a result that Fig. 3 (c) repairs for use the present invention.

With experimental result Fig. 3 (a) of top three kinds of methods, Fig. 3 (b) and Fig. 3 (c) and original image Fig. 2 (a) compare respectively, see from visual effect, use the texture information that the inventive method can better the connection layout picture, more near former figure.

Experiment (2) utilizes the present invention and existing two kinds of methods; Cut in Fig. 4 (b) image is carried out repair process; Result such as Fig. 5; Wherein Fig. 5 (a) is for using the figure as a result of Criminisi method reparation, and Fig. 5 (b) is for using the figure as a result that repairs based on the non-local mean restorative procedure of sample, the figure as a result that Fig. 5 (c) repairs for use the present invention.

With experimental result Fig. 5 (a) of top three kinds of methods, Fig. 5 (b) and Fig. 5 (c) and original image Fig. 4 (a) compare respectively, see from visual effect, use reparation that the inventive method obtains grain details clear and natural more as a result, more near former figure.

Experiment (3) utilizes the present invention and existing two kinds of methods; Fig. 6 (b) image Chinese version is removed; Result such as Fig. 7; Wherein Fig. 7 (a) is for using the figure as a result of Criminisi method reparation, and Fig. 7 (b) is for using the figure as a result that repairs based on the non-local mean restorative procedure of sample, the figure as a result that Fig. 7 (c) repairs for use the present invention.

Respectively with experimental result Fig. 7 (a) of top three kinds of methods; Fig. 7 (b) and Fig. 7 (c) and original image Fig. 6 (a) compare; See from visual effect, use reparation that the inventive method obtains grain details clear and natural more as a result, at borderline region and texture region all more near former figure.

Calculate above-mentioned three kinds of method reparation results' Y-PSNR PSNR respectively, its result is as shown in table 1:

Table 1 uses distinct methods to repair result's PSNR value contrast

Visible from table 1, the Y-PSNR PSNR of the inventive method has raising than existing two kinds of methods.

Above experimental result shows that the present invention is superior to existing two kinds of methods on overall performance, can repair better edge and the texture information of keeping on the result, and image is clear and natural more.

Claims (2)

1. a bayesian non-local mean image repair method comprises the steps:

(1), confirms the border δ of area to be repaired Ω and area to be repaired for the image I to be repaired of input;

(2) utilize following formula, calculate the priority P (p) that central point all on the δ of the border of area to be repaired are repaired piece:

P(p)＝C(p)·D(p)，

Wherein, D (p) is a data item, and C (p) is the degree of confidence item, the credibility of presentation video pixel, and C (p) is initialized as C (p)=0, p ∈ Ω, C (p)=1, p ∈ I-Ω;

(3) central point

with the highest reparation piece of priority

is the center; Choose size and be the neighborhood of the M * M region of search as this reparation piece, definition is that the piece

at center is a sample block with point

in should the zone;

(3.1) utilize the Bayesian frame modeling to repairing piece with sample block

, calculate average and the equal value difference

i.e. that calculates them

of average

and the sample block of reparation piece respectively

(3.2) according to equal value difference

Obey

The characteristic of Gaussian distribution, define an adaptive threshold: t=λ σ ₀, wherein, σ ₀For

Standard deviation, λ=1.65, u ₀For

Average,

For

Variance;

(3.3) point of all

in the selection region of search is got new region of search, back as preliminary election;

(4) Calculate the new repair block within the search area?

and the sample block?

similarity distance:

Wherein,

is 2 norms;

(5) according to similarity distance

Obeying degree of freedom is card side's distribution X of n ²(n) characteristic, when n>=25, quantile

Choose m the most similar sample block that similarity distance is positioned at quantile β left side, as repairing piece

Set;

(6) according to following formula, the similarity weights of sample block

in the set of computations and reparation piece :

Wherein, Z is a normalized parameter, σ ²For repairing piece

Variance, N is for repairing piece The number of the point that middle pixel value is known;

(7) according to the similarity weights, with the weighted mean of the whole sample block in the set, as filling block Ψ ₀, and with this filling block to repairing piece

Fill reparation;

(8) after repairing piece

completion reparation; Upgrade the area to be repaired, and upgrade the degree of confidence C (p) of the point of having accomplished reparation with the degree of confidence of repairing piece

central point :

Wherein,

for repairing the degree of confidence of piece

central point

, and ∩ representes ' with ' relation;

(9) repeating step (1) ~ (8), have a few in the area to be repaired repaired.

2. according to claims 1 described bayesian non-local mean image repair method, it is characterized in that the computing method of step (2) described degree of confidence item C (p) and data item D (p) are:

Wherein, x is for repairing piece Ψ _pThe middle known point of pixel value, C (x) is the degree of confidence of some x, | Ψ _p| for repairing piece Ψ _pArea, n _pBe the vector of unit length vertical with the border, area to be repaired at the p place,

For with the vertical vector of unit length of gradient at p point place, i.e. the vector of unit length of p point place isophote direction, α is a normalizing parameter, for 8 gray level images, α=255.

Images