CN101847255A - Structural information synthesis-based image completion method - Google Patents

Abstract

The invention belongs to the technical field of image inpainting, and relates to a structural information synthesis-based image completion method, which comprises the following steps of: for an image, determining an inpainting border area; calculating the cost energy of nodes per se; calculating the correlation energy of the nodes; determining an integral target function; initializing and updating message values between every two adjacent nodes; and calculating the optimal solution of the nodes. The method has the advantages of achieving the effect of simultaneously reserving structural information and texture information without human intervention and greatly improving image inpainting effect.

Description

A kind of image complementing method of compages information

Technical field

The invention belongs to the image repair technical field, relate to a kind of image complementing method of compages information.

Background technology

The background technology that relates among the present invention has:

(1) TV image repair algorithm: the image repair model based on the total variational method is by Chan﹠amp the earliest; Shen sets up based on variational principle, is called the TV model, and it is one of best model that keeps structural information based on partial differential equation, promptly utilizes diffusion equation in the physics that the information of known region is propagated into and treats in the repairing area.Breakage has reasonable effect to TV image repair algorithm to small scale, can keep its structural information effectively.If but the area to be repaired area more greatly then can produce significantly fuzzyly, not too meets people's visual sensory.

(2) propose the synthetic image completion algorithm of block-based texture based on the synthetic image completion algorithm of texture: Criminisi and solved the problem that from image, removes the large tracts of land object.Utilize this algorithm, can fill the zone that object removes with a kind of reasonable manner visually.This algorithm combines the advantage based on the texture synthetic technology of sample and small scale image repair technology effectively.This algorithm has proposed the notion of confidence level, and it is similar to the information propagation pattern in the small scale image repair technology.Though this algorithm has been eliminated the blurring effect in the former algorithm, but cause matching error easily, therefore for the abundant damaged image of structural information, repairing effect is not good.

Summary of the invention

The objective of the invention is to overcome the above-mentioned deficiency of prior art, a kind of complementing method of image more efficiently is provided.Do not need human intervention to reach the effect that keeps structural information and texture information simultaneously.

For this reason, the present invention adopts following technical scheme: a kind of image complementing method of compages information comprises the following steps:

1. determine the target area of image, read in the image of the target area to be repaired excessively of mark.Judge the target area according to color value, draw mask images, represent with a logic matrix, wherein element value is that 1 this pixel of expression needs to repair in the matrix; Element value is 0 to be expressed as known pixels.Then this logic matrix and Laplace operator are done convolution, find out the border of this target area.

2. the cost energy of computing node self, E ₁(xi)=C (i) d (x _i, φ) wherein,

The expression node credibility, and when overlapping with known region, d (x _i, φ) expression piece x _iWith the matching degree of known region, it is that SSD weighs with the difference quadratic sum of color value between the pixel of overlapping region.

3. the relevance energy function of computing node, E ₂(x _i, x _j)=λ ₁E ₂ ^t(x _i, x _j)+λ ₂E ₂ ^s(x _i, x _j), E ₂ ^t(x _i, x _j)=d (x _i, x _j), wherein, d (x _i, x _j) expression adjacent block x _iAnd x _jThe SSD of overlapping region. D wherein _Gx(x _i, x _j) and d _Gy(x _i, x _j) represent adjacent block x respectively _iAnd x _jGradient difference in X and Y direction.

4. determine the overall goals function $E\left(x\right)=\underset{v_i\∈}{\mathrm{\Σ}}{E}_1\left(x_i\right)+\underset{(i,j)\∈E}{\mathrm{\Σ}}{E}_2(x_i,x_j).$

5. initialization message values between every two adjacent nodes, and upgrade.With any two node n among the bayesian network structure figure G _iAnd n _jBetween message be initially 0, promptly

Upgrade the internodal value of information.During each iteration, for separating x _i, from n _iPropagate into n _jThe computing formula of information be:

Wherein N (j) i represent n _jNon-n _iThe field collection.m _{J → i} ^t(x _i) expression n _jNode is to x _iBe n _iDegree of belief.After T iteration, match block is x _iNode n _iThe reliability computing formula be: $b_i\left(x_i\right)=E_1\left(x_i\right)+\underset{j\∈N\left(i\right)}{\mathrm{\Σ}}{m}_{j\→i}^T\left(x_i\right).$

6. the optimum solution of computing node finds to make b _i(x _i) minimized x _i ^*Be exactly node n _iBest matching blocks.That is:

$x_i^{*}=\arg \underset{x_i}{\min }\{E_1\left(x_i\right)+\underset{j\∈N\left(i\right)}{\mathrm{\Σ}}{m}_{j\→i}^T\}.$

The present invention is on the basis of traditional image completion algorithm that synthesizes based on texture and TV image repair algorithm, has proposed the image completion algorithm of compages information, has greatly improved the effect of image completion.This fundamental idea of the invention is to propose corresponding energy object function at characteristics of image, uses the reliability propagation algorithm with its optimization then, and the match block that chooses is copied to image after defect area draws reparation.This paper has proposed new energy function, and it has comprised structural energy, texture energy and correlativity energy.Combine structure and texture information just because of new energy function, make this invention remedy the deficiency of two class classic algorithm, make that can keep structural information can avoid blurring effect again, and proved superiority of the present invention from theoretical and experiment two aspects.

Description of drawings

Fig. 1 is based on the image complementing method overview flow chart of compages information.

Fig. 2 chooses target.

The initial pictures that Fig. 3 is to be repaired.

Fig. 4 has marked the image that need remove object.

Fig. 5 (a) and (b), (c) are respectively the image completion arithmetic result figure of the compages information that proposes with the figure as a result of the figure as a result of TV image repair algorithm reparation, Criminisi method, the present invention.

Embodiment

The present invention proposes piece disappearance image repair algorithm frame based on structural information, this algorithm utilizes Bayesian Network will be applicable to the TV algorithm of repair structure information and is applicable to the texture composition algorithm combination of repairing texture information, its essence is to propose corresponding energy object function at characteristics of image, then with its optimization, the coupling texture block that will choose then copies to defect area, draws the image after the reparation.This invention does not need human intervention to reach the effect that keeps structural information and texture information simultaneously.

The present invention is based on the image complementing method of compages information, and Fig. 1 is an overview flow chart, specifically may further comprise the steps:

1. determine the target area of image.

As shown in Figure 4, read in the image of the target area to be repaired excessively of color mark.Judge the target area according to color value, draw mask images, represent with a logic matrix, wherein element value is that 1 this pixel of expression needs to repair in the matrix; Element value is 0 to be expressed as known pixels.Then this logic matrix and Laplace operator are done convolution, find out the border of this target area.Judge boundary value, if be non-0, expression also has needs the pixel of repairing, then down execution; If 0, expression reparation is finished, and withdraws from.

2. the cost energy of computing node self

If I represents whole image, Ω is target area (promptly treating the completion zone).The same with the texture synthetic technology, need elder generation to determine the size of sample window Ψ, default value is 9 * 9 pixels.Fig. 2 shows the algorithm pattern that mates energy function.In order to define the node in the algorithm, the image lattice is by horizontal space pixel gap _xWith vertical space pixel gap _yForm.Known observation label is made up of all w * h piece of regional φ, is designated as X={x ₁, x ₂... x _k.And

N block of pixels in the expression target area is node.

According to the cost energy of each node self in formula (1) calculating chart 2 target areas, the i.e. rehabilitation cost of node.Wherein, C (i) represents node credibility, and its computing formula is (2), and C (i) value that the outer field more pixel in target area contains is big more, promptly needs more early to fill; And it is littler in C (i) value of the pixel at the center of target area.In the formula | Ψ _i| be piece | Ψ _i| area.

E ₁(x _i)＝C(i)d(x _i，φ) (1)

$C\left(i\right)=\frac{{\mathrm{\Σ}}_{q\∈\mathrm{\Ψi}\∩\stackrel{\‾}{\mathrm{\Ω}}}C\left(q\right)}{\left|{\mathrm{\Ψ}}_i\right|}---\left(2\right)$

And when overlapping with known region, d (x _i, φ) expression piece x _iWith the matching degree of known region, it is that SSD weighs with the difference quadratic sum of color value between the pixel of overlapping region.Work as x _iAnd there is not when overlapping E between known region ₁(x _i)=0.As shown in Figure 2, E ₁(x _i) then be that the P piece is put into x _iThe SSD value of time domain 1 (red line mark) on the position.

3. the relevance energy function of computing node

As publicity (3), E ₂(x _i, x _j) expression (x _i, x _j) the consistance cost.Wherein, E ₂ ^t(x _i, x _j) the texture consistance cost of adjacent block in the presentation video, E ₂ ^s(x _i, x _j) expression adjacent block the structural integrity cost.λ ₁And λ ₂Weight coefficient for balance texture cost and infrastructure cost.When containing significant structural information in the target area, λ ₂Then bigger; When texture information in the target area relatively enriches, λ ₁Then bigger.

E ₂(x _i，x _j)＝λ ₁E ₂ ^t(x _i，x _j)+λ ₂E ₂ ^s(x _i，x _j) (3)

E ₂ ^t(x _i, x _j) be defined as:

E ₂ ^t(x _i，x _j)＝d(x _i，x _j) (4)

Wherein, d (x _i, x _j) expression adjacent block x _iAnd x _jThe SSD of overlapping region.As shown in Figure 2: E ₂ ^t(x _i, x _j) be that piece P in any known region and P ' are put into x respectively _iAnd x _jThe time, lap is the SSD value in Green Marker zone.

E ₂ ^s(x _i, x _j) be defined as:

${E_2}^s(x_i,x_j)=\sqrt{d_{\mathrm{gx}}^2(x_i,x_j)+d_{\mathrm{gy}}^2(x_i,x_j)}---\left(5\right)$

D wherein _Gx(x _i, x _j) and d _Gy(x _i, x _j) represent adjacent block x respectively _iAnd x _jGradient difference in X and Y direction.

4. determine the overall goals function.

The overall goals function definition is (6).The overall goals function has been represented the Global Information in the entire image, for example: structure, texture, whole consistance etc.The overall goals function be all nodes of target area energy and.In this algorithm, the energy function of node comprises the cost ENERGY E of self ₁With with the relevance energy function E of field node ₂Determining of overall goals function is the key of whole algorithm, and it is related to whether Global Information can keep effectively in the repair process.

$E\left(x\right)=\underset{v_i\∈v}{\mathrm{\Σ}}{E}_1\left(x_i\right)+\underset{(i,j)\∈E}{\mathrm{\Σ}}{E}_2(x_i,x_j)---\left(6\right)$

Wherein, E ₁(x _i) expression node rehabilitation cost, E ₂(x _i, x _j) expression (x _i, x _j) the consistance cost.

5. initialization message values between every two adjacent nodes, and upgrade.

With any two node n among the bayesian network structure figure G _iAnd n _jBetween message be initially 0, promptly

Upgrade the internodal value of information

During each iteration, for separating x _i, from n _iPropagate into n _jThe computing formula of information be:

$m_{j\→i}^t\left(x\right)=\min [E_2(x_i,x_j)+E_1\left(x_j\right)+\underset{s\∈N\left(j\right)\backslash i}{\mathrm{\Σ}}{m}_{s\→j}^{t-1}\left(x_j\right)]---\left(7\right)$

Wherein N (j) i represent n _jNon-n _iThe field collection.m _{J → i} ^t(x _i) expression n _jNode is to x _iBe n _iDegree of belief.After T iteration, match block is x _iNode n _iThe reliability computing formula be:

$b_i\left(x_i\right)=E_1\left(x_i\right)+\underset{j\∈N\left(i\right)}{\mathrm{\Σ}}{m}_{j\→i}^T\left(x_i\right)---\left(8\right)$

6. the optimum solution of computing node.

Find and make b _i(x _i) minimized x _i ^*Be exactly node n _iBest matching blocks.That is:

$x_i^{*}=\arg \underset{x_i}{\min }\{E_1\left(x_i\right)+\underset{j\∈N\left(i\right)}{\mathrm{\Σ}}{m}_{j\→i}^T\}---\left(9\right)$

Fig. 3 is an original image, Fig. 4 is for marking the image that need remove object, and Fig. 5 (a) and (b), (c) are respectively the image completion arithmetic result figure of the compages information that proposes with the figure as a result of the figure as a result of TV image repair algorithm reparation, Criminisi method, the present invention.Difference to three width of cloth figure is carried out mark with red rectangle, as can be seen: (a) occur large-area bluring among the figure; (b) more serious matching error has appearred, the roof line in the red rectangle of mark and discontinuous among the figure; (d) figure has remedied the defective among preceding two width of cloth figure significantly.The color on roof is very complete in two rectangles, and lake surface completion effect is fine.

Claims (1)

1. the image complementing method of a compages information comprises the following steps:

(1) determine the target area of image: read in the image that is labeled, judge the target area according to color value, draw mask images, represent with a logic matrix, wherein element value is that 1 this pixel of expression needs to repair in the matrix; Element value is 0 to be expressed as known pixels, then this logic matrix and Laplace operator is done convolution, finds out the border of this target area;

(2) the cost energy of computing node self, E ₁(x _i)=C (i) d (x _i, φ) wherein,

The expression node credibility, and when overlapping with known region, d (x _i, φ) expression piece x _iWith the matching degree of known region, it is that SSD weighs with the difference quadratic sum of color value between the pixel of overlapping region;

(3) the relevance energy function E of computing node ₂(x _i, x _j)=λ ₁E ₂ ^t(x _i, x _j)+λ ₂E ₂ ^s(x _i, x _j), E ₂ ^t(x _i, x _j)=d (x _i, x _j), wherein, d (x _i, x _j) expression adjacent block x _iAnd x _jThe SSD of overlapping region;

Wherein, d _Gx(x _i, x _j) and d _Gy(x _i, x _j) represent adjacent block x respectively _iAnd x _jGradient difference in X and Y direction;

(4) determine the overall goals function

(5) beginningization message values between every two adjacent nodes, and upgrade: with any two node n among the bayesian network structure figure G _iAnd n _jBetween message be initially 0, promptly

Upgrade the internodal value of information, during each iteration, for separating x _i, from n _iPropagate into n _jThe computing formula of information be:

Wherein, N (j) i represent n _jNon-n _iThe field collection;

Expression n _jNode is to x _iBe n _iDegree of belief; After T iteration, match block is x _iNode n _iThe reliability computing formula be:

(6) optimum solution of computing node finds to make b _i(x _i) minimized x _i ^*Be exactly node n _iBest matching blocks, that is:

$x_i^{*}=\arg \underset{x_i}{\min }\{E_1(x_i+\underset{j\∈N\left(i\right)}{\mathrm{\Σ}}{m}_{j->i}^T).$

Images