CN103700093A - Criminisi image restoration method based on textures and edge features - Google Patents

Abstract

The invention discloses a criminisi image restoration method based on textures and edge features. A difference factor which enhances the discrimination capability for a structural part is introduced in a priority model to optimize priority calculation through analyzing texture structure features and edge structure features. The Criminisi image restoration method disclosed by the invention can effectively restrain priority repair of texture parts, and prevent the texture part from extending excessively to an edge part to result in image linear structural failure.

Description

A kind of Criminisi image repair method based on texture and edge feature

Technical field

The invention belongs to Digital Image Inpainting field, be specifically related to a kind of Criminisi image repair method based on texture and edge feature.

Background technology

Present image recovery technique is divided into two large classes: a class is to repair (inpainting) technology for repairing the digital picture of small scale, as BSCB model, and the reparation algorithm based on total variational and the algorithm based on Curvature-driven diffusion model that the people such as Chan propose.Such algorithm has good repairing effect when repairing the breakage image of small scale, but when repairing the larger image in damaged area, tends to produce fuzzy phenomenon.Another kind of is image completion (image completion) technology for blank map picture bulk drop-out---the image repair technology based on texture synthetic (texture synthesis).The image repair algorithm based on sample that wherein people such as Criminisi proposes, the setting by right of priority is preferentially repaired the marginal portion of lost regions, has obtained good repairing effect.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, a kind of Criminisi image repair method based on texture and edge feature is provided.

Find in actual applications, the right of priority model of Criminisi algorithm can not be distinguished out by marginal portion completely effectively, sometimes easily thinks some texture part by mistake marginal portion, has affected final repairing effect.The priority of bringing for the difference between texture structure and marginal texture is obscured and the automatic problem such as calculating of right of priority, the invention provides a kind of right of priority account form based on texture and edge feature, which can strengthen the resolving ability to structure division in right of priority model, thereby improves the image repair quality of Criminisi algorithm.

The present invention takes following technical scheme: by the analysis to texture and structural characteristic and marginal texture feature, in right of priority model, introduce and strengthen the calculating of the discrimination factor of structure division resolving ability being optimized to right of priority, it carries out as follows:

The first step: by image block ψ to be repaired _palong n _pdirection is cut apart, and makes ψ _pbe split into two pieces of uniform size, then respectively the known pixels point of two pieces averaged and (is designated as E ₁and E (p) ₂(p)), variance (is designated as F ₁and F (p) ₂(p) equal value difference δ E (p)=(E), and after normalization ₁(p)-E ₂(p))/α and variance value of delta F (p)=(F ₁(p)-F ₂(p))/α.Wherein, α is normalized parameter (getting 255 in gray level image).Average difference after normalization and variance difference can effectively judge that whether damaged area is in edge, texture or smooth region.

Second step: introduce discrimination factor.In order to overcome in Criminisi algorithm, structure degree of confidence D (p) can not distinguish the whether shortcoming in texture or structural region of damaged area, place, and the present invention introduces the discrimination factor that can effectively identify texture and structure:

$E\left(p\right)=\left\{\begin{array}{cc}\frac{|E_1\left(p\right)-E_2\left(p\right)|}{\mathrm{\&alpha;}},& \mathrm{\&delta;E}\left(p\right)\&GreaterEqual;\mathrm{\&beta;}\\ \frac{|F_1\left(p\right)-F_2\left(p\right)|}{0.1\&times;{\mathrm{\&alpha;}}^2},& \mathrm{\&delta;E}\left(p\right)<\mathrm{\&beta;}\end{array}---\left(1\right)\right.$

Wherein, α is normalized parameter (gray level image gets 255), and β is empirical constant, generally gets 0.02.

The 3rd step: the structure degree of confidence D (p) of discrimination factor E (p) and former algorithm is combined, and right of priority model can be revised as:

$P\left(p\right)=C\left(p\right)\&CenterDot;((1-\mathrm{\&lambda;}\left(p\right))\&CenterDot;D\left(p\right)+\mathrm{\&lambda;}\left(p\right)\&CenterDot;E\left(p\right))---\left(2\right)$

Wherein, the value of λ (p) depends on the difference reparation part of image to be repaired.λ (p) gets higher value, weakens the impact of texture part edge part, prevents that texture part is to the extension of marginal portion.λ (p) gets smaller value, reduces the impact of E (p) on non-marginal portion priority, to repair smooth.λ (p) is defined as follows:

$\mathrm{\&lambda;}\left(p\right)=\left\{\begin{array}{ccc}0.7~0.9& E\left(p\right)\&GreaterEqual;3\&times;\mathrm{mean}\left(E\left(p\right)\right)& \\ 0.4~0.6& E\left(p\right)<3\&times;\mathrm{mean}\left(E\left(p\right)\right)\mathrm{and}& E\left(P\right)\&GreaterEqual;\mathrm{mean}\left(E\left(p\right)\right)---\left(3\right)\\ 0.1~0.3& \mathrm{else}& \end{array}\right.$

Wherein, mean (E (p)) is the average of discrimination factor E (p).

The 4th step: in order to improve matching efficiency, the present invention is on the basis of Criminisi algorithm global search, an additional search condition: if the central point of the match block searching distance is excessive with the central point distance of multiblock to be repaired, suspend the reparation to this reparation piece, the reparation piece that reparation target is transferred to time priority, if still there is same case, continue to shift and repair target.For avoiding the impact of this condition on reparation order, the present invention will be limited on transfer number 3 times,, when shifting after 3 times, ignores this condition in search procedure.

The 5th step: according to improved way of search and right of priority model, adopt the method for image block coupling, start progressively to complete from outside to inside image repair work from the reparation border of image.

Beneficial effect of the present invention:

1, can effectively suppress the preferential reparation of texture part, prevent texture part undue extension and linear structural failure of image of causing to marginal portion.

2, can remove existing flaw point in repair process, obtain better repairing effect.

Accompanying drawing explanation

Fig. 1 is process flow diagram of the present invention.

Fig. 2 is multiblock ψ to be repaired _pcut apart schematic diagram.

Fig. 3 for respectively in edge, the different damaged region of texture or smooth region.

Fig. 4 is for adopting respectively method of the present invention, the repairing effect comparison to Great Wall image of Criminisi method method and close method, (a) original image in figure; (b) image to be repaired; (c) Criminisi algorithm (PSNR=31.2211dB); (d) close algorithm (PSNR=31.9971dB); (e) the present invention (PSNR=35.1019dB).

Fig. 5 is for adopting the repairing effect comparison of said method to barb image, (a) original image in figure; (b) image to be repaired; (c) Criminisi algorithm (PSNR=38.2422dB); (d) close algorithm (PSNR=39.9149dB); (e) the present invention (PSNR=40.0111dB); (f) only change way of search (PSNR=38.3515dB); (g) only change right of priority (PSNR=39.3067dB).

Fig. 6 is for adopting the repairing effect comparison of said method to Baboon image, (a) original image in figure; (b) image to be repaired; (c) Criminisi algorithm (PSNR=39.8729dB); (d) close algorithm (PSNR=40.1276dB); (e) the present invention (PSNR=40.5594dB).

Fig. 7 is the comparison of effect after adopting said method to remove bungee image object, (a) original image in figure; (b) remove region; (c) Criminisi algorithm; (d) close algorithm; (e) the present invention;

Embodiment

At the present embodiment process flow diagram as shown in Figure 1.

The first step: by image block ψ to be repaired _palong n _pdirection is cut apart, and makes ψ _pbe split into two pieces of uniform size, then respectively the known pixels point of two pieces averaged and (is designated as E ₁and E (p) ₂(p)), variance (is designated as F ₁and F (p) ₂(p) equal value difference δ E (p)=(E), and after normalization ₁(p)-E ₂(p))/α and variance value of delta F (p)=(F ₁(p)-F ₂(p))/α.Wherein, α is normalized parameter (getting 255 in gray level image).Average difference after normalization and variance difference can effectively judge that whether damaged area is in edge, texture or smooth region.As shown in Figure 2, first multiblock to be repaired is divided into of uniform size 1 and piece 2, respectively the known pixels point of two pieces is averaged and variance, then through type (1), obtains average difference and variance difference.

Second step: introduce discrimination factor.In order to overcome in Criminisi algorithm, structure degree of confidence D (p) can not distinguish the whether shortcoming in texture or structural region of damaged area, place, and the present invention introduces the discrimination factor that can effectively identify texture and structure:

$E\left(p\right)=\left\{\begin{array}{cc}\frac{|E_1\left(p\right)-E_2\left(p\right)|}{\mathrm{\&alpha;}},& \mathrm{\&delta;E}\left(p\right)\&GreaterEqual;\mathrm{\&beta;}\\ \frac{|F_1\left(p\right)-F_2\left(p\right)|}{0.1\&times;{\mathrm{\&alpha;}}^2},& \mathrm{\&delta;E}\left(p\right)<\mathrm{\&beta;}\end{array}---\left(1\right)\right.$

Wherein, α is normalized parameter (gray level image gets 255), and β is empirical constant, generally gets 0.02.

8 zoness of different in Fig. 3 of take are example, table 1 has provided the average difference in 8 regions, no matter can find out 1st district, 2nd district, 5th district and 6th district, or in 3rd district, the region such as 4th district, 7th district and 8th district, the average difference at edge and the average difference of texture differ larger, this explanation, and average difference has characterized edge and the non-marginal texture feature in 3rd district, 4th district, 7th district and 8th district preferably, judged exactly the attribute of damaged area, whether damaged area belongs to edge, texture or smooth region.And table 2 has provided respectively the variance difference after 8 region normalization in Fig. 3, to compare and can see with table 2, the judgement of variance difference edge region and non-fringe region is consistent with average difference, has judged exactly the attribute of damaged area.

Table 1

Table 2

The 3rd step: the structure degree of confidence D (p) of discrimination factor E (p) and former algorithm is combined, and right of priority model can be revised as:

$P\left(p\right)=C\left(p\right)\&CenterDot;((1-\mathrm{\&lambda;}\left(p\right))\&CenterDot;D\left(p\right)+\mathrm{\&lambda;}\left(p\right)\&CenterDot;E\left(p\right))---\left(2\right)$

Wherein, the value of λ (p) depends on the difference reparation part of image to be repaired.λ (p) gets higher value, weakens the impact of texture part edge part, prevents that texture part is to the extension of marginal portion.λ (p) gets smaller value, reduces the impact of E (p) on non-marginal portion priority, to repair smooth.λ (p) is defined as follows:

$\mathrm{\&lambda;}\left(p\right)=\left\{\begin{array}{ccc}0.7~0.9& E\left(p\right)\&GreaterEqual;3\&times;\mathrm{mean}& \\ 0.4~0.6& E\left(p\right)<3\&times;\mathrm{mean}\left(E\left(p\right)\right)\mathrm{and}& E\left(P\right)\&GreaterEqual;\mathrm{mean}\left(E\left(p\right)\right)---\left(3\right)\\ 0.1~0.3& \mathrm{else}& \end{array}\right.$

Wherein, mean (E (p)) is the average of discrimination factor E (p).

The 4th step: in order to improve matching efficiency, the present invention is on the basis of Criminisi algorithm global search, an additional search condition: if the central point of the match block searching distance is excessive with the central point distance of multiblock to be repaired, suspend the reparation to this reparation piece, the reparation piece that reparation target is transferred to time priority, if still there is same case, continue to shift and repair target.For avoiding the impact of this condition on reparation order, the present invention will be limited on transfer number 3 times,, when shifting after 3 times, ignores this condition in search procedure.

The 5th step: according to improved way of search and right of priority model, adopt the method for image block coupling, start progressively to complete from outside to inside image repair work from the reparation border of image.

In order to verify validity of the present invention, carried out on computers emulation experiment.In emulation experiment, take MATLAB7.0 as emulation platform, on the PC of Intel Celeron dual core processor (2.5GHz), 2G internal memory, realize.When to image repair effect assessment, except adopting subjective assessment, also adopt Y-PSNR (PSNR) to carry out objective evaluation simultaneously.

Fig. 4 has provided respectively the present invention, Criminisi algorithm and the repairing effect of close algorithm to Great Wall breakage image.In figure, can see, Criminisi algorithm is when repairing a region, b region, having produced certain texture extends, and when repairing b region, c region, produced flaw point, and close algorithm is when repairing a region, also produce certain extension, when repairing b, c region, do not avoided the generation of flaw point.The present invention has effectively avoided the extension of texture, and has eliminated flaw point, has good visual effect.

Fig. 5 has provided respectively the repairing effect of above-mentioned algorithm to barb breakage image, has also provided the repairing effect to barb breakage image when only changing in the present invention way of search and only changing right of priority simultaneously.Can find out, Criminisi algorithm has produced flaw point in repair process at shoulder place, at wrist place, texture has produced extension, destroyed the linear structure of wrist, although close algorithm has successfully been eliminated flaw point, but to the reparation at wrist edge, not still very desirable, do not solve the existing texture extension problems of Criminisi algorithm.The present invention, when keeping wrist linearity, has also eliminated flaw point, has good visual effect.In addition, from Fig. 5 (f), (g), can find out, if while only changing way of search in the present invention, texture extension problems that can not fine solution wrist place; Equally only change right of priority, also cannot eliminate the flaw point at shoulder place.

Fig. 6 has provided respectively the repairing effect of above-mentioned algorithm to Baboon.Can find out, Criminisi algorithm, when nose edge is repaired, does not keep the linear structure of nose, and has flaw near nose and face, and close algorithm keeps better the linear structure of nose, but does not eliminate the flaw of face.And the present invention has overcome above-mentioned two shortcomings, there is good repairing effect.

In emulation experiment, also object removal has been carried out to emulation.Fig. 7 has provided respectively the effect that above-mentioned algorithm removes object.Can find out, Criminisi algorithm, when dykes and dams are repaired, has destroyed the linear structure of dykes and dams, and the texture on riverbank is extended out, and does not meet vision.Close algorithm is not fine to the reparation of dykes and dams the latter half, and the present invention repairs better to dykes and dams, has roughly kept the linear structure of dykes and dams, has obtained good repairing effect.

Above the preferred embodiments of the present invention and principle are had been described in detail, for those of ordinary skill in the art, according to thought provided by the invention, in embodiment, will change, and these changes also should be considered as protection scope of the present invention.

Claims (1)

1. the Criminisi image repair method based on texture and edge feature, is characterized in that:

Step 1: calculate average difference and the variance difference of divided image block, specifically:

By image block ψ to be repaired _palong n _pdirection is cut apart, and makes ψ _pbe split into two pieces of uniform size, then respectively the known pixels point of two pieces averaged, be designated as E ₁and E (p) ₂(p); Variance, is designated as F ₁and F (p) ₂(p) equal value difference δ E (p)=(E), and after normalization ₁(p)-E ₂(p))/α and variance value of delta F (p)=(F ₁(p)-F ₂(p))/α; Wherein, α is normalized parameter; Average difference after normalization and variance difference can effectively judge that whether damaged area is in edge, texture or smooth region;

Step 2: introduce discrimination factor E (p), specifically:

$E\left(p\right)=\left\{\begin{array}{cc}\frac{|E_1\left(p\right)-E_2\left(p\right)|}{\mathrm{\&alpha;}},& \mathrm{\&delta;E}\left(p\right)\&GreaterEqual;\mathrm{\&beta;}\\ \frac{|F_1\left(p\right)-F_2\left(p\right)|}{0.1\&times;{\mathrm{\&alpha;}}^2},& \mathrm{\&delta;E}\left(p\right)<\mathrm{\&beta;}\end{array}---\left(1\right)\right.$

Wherein, α is normalized parameter, and β is constant;

Step 3: discrimination factor and structure degree of confidence are combined, revise right of priority model P (p), specifically:

$\mathrm{\&lambda;}\left(p\right)=\left\{\begin{array}{ccc}0.7~0.9& E\left(p\right)\&GreaterEqual;3\&times;\mathrm{mean}\left(E\left(p\right)\right)& \\ 0.4~0.6& E\left(p\right)<3\&times;\mathrm{mean}\left(E\left(p\right)\right)\mathrm{and}& E\left(P\right)\&GreaterEqual;\mathrm{mean}\left(E\left(p\right)\right)---\left(3\right)\\ 0.1~0.3& \mathrm{else}& \end{array}\right.$

Wherein, mean (E (p)) is the average of discrimination factor E (p);

Step 4: improve match search mode, specifically:

On the basis of Criminisi algorithm global search, an additional search condition: if the central point of the match block searching distance is excessive with the central point distance of multiblock to be repaired, suspend the reparation to this reparation piece, the reparation piece that reparation target is transferred to time priority, if still there is same case, continue to shift

$P\left(p\right)=C\left(p\right)\&CenterDot;((1-\mathrm{\&lambda;}\left(p\right))\&CenterDot;D\left(p\right)+\mathrm{\&lambda;}\left(p\right)\&CenterDot;E\left(p\right))---\left(2\right)$

Wherein, the value of λ (p) depends on the difference reparation part of image to be repaired; λ (p) gets higher value, weakens the impact of texture part edge part, prevents that texture part is to the extension of marginal portion; λ (p) gets smaller value, reduces the impact of E (p) on non-marginal portion priority, to repair smooth; λ (p) is defined as follows: repair target, be limited to 3 times on transfer number,, when shifting after 3 times, ignore this condition in search procedure;

Step 5: according to improved way of search and right of priority model, complete image repair work, specifically: according to improved way of search and right of priority model, adopt the method for image block coupling, from the reparation border of image, start progressively to complete from outside to inside image repair work.

Images