CN104766283A - Digitized repairing method for grave mural image - Google Patents

Abstract

The invention discloses a digitized repairing method for a grave mural image, and belongs to the field of digital image repairing. According to the method, a total variation mathematical model is utilized to decompose the grave mural image to obtain a cartoon image; a structure factor term is added in priority calculation, and the cartoon image is utilized to drive the priority calculation of edge points to be repaired; the point with the maximum priority serves as a center to construct a repairing block, and a sample block of which the Euclidean distance with the repairing block is smallest is searched in a known area; the average pixel difference quadratic sum of the repairing block and the sample block is calculated and compared with a set threshold value, the size of the repairing block is self-adaptively adjusted until the copy condition is met, and sample copying and edge updating are performed; lastly, an area to be repaired is checked, and if the area to be repaired is not null, the above steps are iterated until repairing is completed. The method overcomes the problems such as incoherence and overstretching generated in repairing of an existing method and is mainly used in (but not limited to) virtual repairing of the grave mural image.

Description

The Digital repair method of a kind of grave mural painting image

Technical field

The invention belongs to digital picture and repair field, be specially a kind of Digital repair method eliminating the grave mural painting image of the phenomenons such as the discontinuous and mistake extension produced in repair process.

Background technology

Digital Image Inpainting refers to and utilizes algorithm to carry out automatic filling to the loss of data in image local area or damage, recovers the technology of its vision continuity.Along with the development of computer technology, Digital Image Inpainting is intelligent with it, quick, the advantage of low cost, has been widely applied to the fields such as historical relic's protection, photo reparation, video display special effect making, barrier removal.

Grave mural painting is the cultural heritage of China's preciousness, and cultural morphology is unique, and artistic value is high.But, it is badly damaged that the weathering of environment and artificial destruction cause these mural paintings to receive, entity reparation expends greatly, difficulty is high, Digital repair not only can realize the work such as the virtual display of mural painting, and the peripheral doses process that can be mural painting entity provides sufficient scientific basis and test environment, is down to minimum by the danger of historical relic repair.

Grave mural painting is once damage can lose number of colors and structural information, and the restore design in recent years for grave mural painting image mainly contains three classes, the reparation based on partial differential equation, the reparation based on rarefaction representation and the recovery technique based on sample.First two recovery technique is better for the repairing effect of mural painting crackle, but in large area repair, all easily produce blooming, recovery technique based on sample then can overcome above-mentioned defect, sample restore design the most classical is the Criminisi algorithm proposed in 2004 by people such as Criminisi, this algorithm principle is simple, and texture repairing effect is better.But due to the impact of noise and tiny texture, the isophote discriminating direction of data item is inaccurate, and does not take into full account the preferential reparation of structural region, for the mural painting image that structural failure is more serious, in repair process, easily produce the discontinuous of structural information; And fix owing to repairing block size, for structure and the comparatively complicated region of texture, texture information often excessively extends, and causes repairing effect not good.

Summary of the invention

The present invention is the discontinuous of generation during solution grave mural painting is repaired and crosses the problems such as extension, provide the Digital repair method of a kind of grave mural painting image, the method is in conjunction with the obvious feature of grave mural painting picture structure, guided the calculating of priority by cartoon driving and structure factor, ensure that the continuity that image structure information is repaired; Repair block size by self-adaptation, inhibit the excessive extension of texture information, obtain good repairing effect.

The present invention adopts following technical scheme to realize: the Digital repair method of a kind of grave mural painting image, comprises the following steps:

S1: fixing digital camera, makes camera lens vertical with grave mural painting, gathers grave mural painting image;

S2: the damaged portion choosing grave mural painting image, is painted with white, as area to be repaired, remainder is as known region;

S3: construct full variation mathematical model and the image that S2 obtains is decomposed, obtain the cartoon image of filtering noise and tiny texture;

S4: the cartoon image utilizing S3 to obtain carries out the calculating driving priority, detailed process is as follows: calculate the confidence item of edge, cartoon image area to be repaired each point, data item and the structure factor item of rectangular block centered by edge each point, according to the priority of priority computing formula edge calculation each point, and obtain the coordinate of priority maximum point;

S5: according to the coordinate in S4, the point finding priority maximum in the image that S2 obtains, is designated as and with point centered by construction size be the multiblock to be repaired of m × m

S6: carry out global search in image known region, find with the block that color Euclidean distance is minimum, as the most similar sample block, is designated as Ψ _q;

S7: calculate Ψ _qwith multiblock to be repaired average pixel difference quadratic sum, be designated as ASSD ( Ψ _q), and with threshold value T _sSDcompare, detailed process is as follows: if ASSD ( Ψ _q) <T _sSD, then think that this sample block meets copy condition, by block Ψ _qimage information copy to area to be repaired; Otherwise adjustment is repaired block and is of a size of (m-1) × (m-1), return and perform step S6, until meet copy condition, carry out block and copy;

S8: add in known region by new filler pixels, upgrades and repairs border; Then upgrade the degree of confidence of filler pixels, complete and once repair, degree of confidence upgrades principle and is: $C\left(p\right)=C\left(\hat{p}\right)e^{-{\left(\mathrm{ASSD}\right)}^2},$ for degree of confidence penalty factor;

S9: check whether area to be repaired is empty, if not empty, then returns and performs S3; Otherwise, stop iteration, finally will repair result and preserve or export.

The present invention compared with prior art has the following advantages:

1. the present invention utilizes full variation mathematical model to be cartoon image by grave mural painting picture breakdown, and the calculating of area to be repaired marginal point priority is driven by cartoon image, eliminate tiny texture and noise to the interference of isophote discriminating direction, take full advantage of the structural information of image, it is more accurate that data item calculates.

2. the present invention adds structure factor item in priority calculates, and fully in conjunction with grave mural painting color clear, the obvious feature of structure, the strong structural region of image is preferentially repaired, ensure that the continuity of structural information.

3. the present invention by ASSD ( Ψ _q) with the magnitude relationship of setting threshold value, adaptively reparation block size is adjusted, for structure and the comparatively complicated restoring area of texture, chooses comparatively light maintenance multiblock, inhibit the excessive extension of texture information.

Accompanying drawing explanation

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is sign flag schematic diagram;

Fig. 3 is the mural painting image collected;

Fig. 4 is the image marking damaged area;

Fig. 5 is cartoon image;

Fig. 6 is for repairing image;

Fig. 7 is that innovative point of the present invention probes into case diagram, and in figure, (a) manually marks figure for damaged area; B () is repair process figure of the present invention; C () is the repairing effect figure adopting separately priority of the present invention; D () is the repairing effect figure adopting separately Exemplar Matching of the present invention; E () is for combining the repairing effect figure of above two aspects (i.e. the inventive method);

Fig. 8 adopts Criminisi method respectively, to improve one's methods and the inventive method advertises the repairing effect comparison diagram of figure to grave mural painting Nan Bi in the Northern Qi Dynasty, in figure, (a) is mural painting image, b (), for marking the image of area to be repaired, (c) is the repairing effect figure adopting Criminisi method; D () is for adopting the repairing effect figure improved one's methods; E () is for adopting the inventive method repairing effect figure;

Fig. 9 be adopt Criminisi method, improve one's methods and the inventive method to the repairing effect comparison diagram of grave mural painting in Northern Qi Dynasty east wall pommel horse flags, weapons, etc. carried by a guard of honor, in figure, (a) is mural painting image, b (), for marking the image of area to be repaired, (c) is the repairing effect figure adopting Criminisi method; D () is for adopting the repairing effect figure improved one's methods; E () is for adopting the inventive method repairing effect figure.

Embodiment

First, according to Fig. 2, some symbols are defined.Note I is image to be repaired, and Ω is area to be repaired, and its border is Φ is the known region of image to be repaired, Ψ _pfor the multiblock to be repaired centered by a p, n _pthe borderline unit normal vector of a p, represent p point isophote direction and intensity.

With reference to the process flow diagram of Fig. 1, the grave mural painting in the Northern Qi Dynasty provided with museum of Shanxi Province is research object, and test, concrete implementation step is as follows:

S1: utilize high definition camera to gather grave mural painting image, during shooting, camera lens is vertical with mural painting to be placed, Fig. 3 is the sub-picture example gathered;

S2: the damaged portion choosing grave mural painting image, is painted with white, as area to be repaired, remainder is as known region, and Fig. 4 is the image marking damaged area;

S3: utilize full variation mathematical model to decompose the image that S2 obtains, obtain the cartoon image of filtering noise and tiny texture, full variation mathematical model is such as formula (1):

$E=\arg \underset{s}{\min }\underset{e}{\mathrm{\&Sigma;}}\{\frac{1}{2\mathrm{\&lambda;}}{(S_e-I_e)}^2+|\&dtri;S_e\left|\right\}---\left(1\right)$

Wherein, E represents energy functional, I _efor input picture, e is image pixel index, S _efor the cartoon image exported, λ is regularization parameter, and span is [0,1], and this example gets λ=0.05, for fidelity item, control image departure degree, for regular terms, with S _egradient modulus value characterize, be used for restraint speckle and tiny texture, obtain cartoon image S by solving above formula energy functional minimum value, Fig. 5 is cartoon image;

S4: add structure factor item in priority calculates, and calculate border, cartoon image area to be repaired each point priority, obtain the coordinate of priority maximum point, concrete steps are as follows:

S41: the data item calculating each frontier point according to formula (2), wherein, for pixel is along the illumination change amount in isophote direction;

$D\left(p\right)=\frac{|{\&dtri;I}_p^{\&perp;}\&CenterDot;n_p|}{255}---\left(2\right)$

S42: according to the confidence item of formula (3) computation bound point, time initial, c (i)=0; c (i)=1, | Ψ _p| be multiblock Ψ to be repaired _parea;

$C\left(p\right)=\frac{\underset{i\&Element;{\mathrm{\&Psi;}}_p\&cap;(1-\mathrm{\&Omega;})}{\mathrm{\&Sigma;}}C\left(i\right)}{\left|{\mathrm{\&Psi;}}_p\right|}---\left(3\right)$

S43: the local variance calculating 3 × 3 pieces centered by frontier point, the exponential form of local variance is defined as the structure factor item that priority calculates, local variance V (p) represents, its computing formula is such as formula (4):

$V\left(p\right)=\frac{1}{3}\underset{R,G,B}{\mathrm{\&Sigma;}}{\mathrm{\&sigma;}}^2(x,y)---\left(4\right)$

X and y is the transverse and longitudinal coordinate of each pixel in region, σ ²(x, y) is 3 × 3 region single channel variance yields;

S44: calculate each marginal point priority, and control confidence item, data item and the structure factor item three effect in priority calculates by coefficient value, priority computing formula is such as formula (5), through experimental verification, the span of α and β is [0,1], this example gets α=β=0.5;

P(p)＝αC(p)+β(D(p)+e ^V(p)) s.t.α+β＝1 (5)

S45: select the marginal point that priority is maximum, and obtain its coordinate in cartoon image;

S5: utilize the coordinate obtained, find the point that priority is maximum in the diagram, be designated as a little with point centered by construction size be the multiblock to be repaired of m × m (embodiment gets m=6)

S6: search at known region Φ, finds and multiblock to be repaired the sample block that color Euclidean distance is minimum, as the most similar sample block, is designated as Ψ _q;

S7: calculate Ψ _qwith multiblock to be repaired average pixel difference quadratic sum, be designated as ASSD ( Ψ _q), and with threshold value T _sSDcompare, if ASSD ( Ψ _q) <T _sSD, then think that this sample block meets copy condition, by Ψ _qimage information copy to middle area to be repaired; Otherwise, adjustment be of a size of (m-1) × (m-1), return and perform S6, until meet copy condition, carry out block and copy, through experimental verification, T _sSDspan be [0,1], this example gets T _sSD=0.6, ASSD ( Ψ _q) calculate such as formula (6):

$\mathrm{ASSD}({\mathrm{\&Psi;}}_{\hat{p}},{\mathrm{\&Psi;}}_q)=\frac{\mathrm{SSD}({\mathrm{\&Psi;}}_{\hat{p}},{\mathrm{\&Psi;}}_q)}{\mathrm{\&epsiv;}\&CenterDot;N_{\mathrm{count}}}---\left(6\right)$

Wherein, SSD ( Ψ _q) be with Ψ _qthe minimum value of known pixels color Euclidean distance, ε is normalized factor, gets ε=255, N _countfor multiblock to be repaired the number of middle known pixels;

S8: new fill area is added in known region Φ, and upgrades border then upgrade the degree of confidence C (p) of filler pixels, complete and once repair.The degree of confidence replacement criteria of new filler pixels is: wherein, for degree of confidence penalty factor, ASSD ( Ψ _q) larger, less, the degree of confidence of filler pixels is less; Otherwise the degree of confidence of filler pixels is larger, the going down of error message can be controlled well by penalty factor;

S9: check whether area to be repaired Ω is empty, if not empty, then returns and performs S3; Otherwise stop iteration, finally will repair result and preserve or export, Fig. 6 is final reparation result figure.

Can find out that the inventive method is preferentially repaired the strong structural region of Incomplete image (rectangle frame region) by Fig. 7 (b), repair result and meet vision continuity; Fig. 7 (c) shows that priority computing method of the present invention ensure that the continuity that structural information is repaired, but there is extension problems; Fig. 7 (d) shows that self-adaptation is chosen reparation block size and detail structure and texture are better repaired; Fig. 7 (e) shows that the inventive method preferably resolves the reparation problem of structural breakage, and texture region repairing effect is also more satisfactory.

Can find out that the inventive method has for grave mural painting destroyed area repairing effect by Fig. 8 and Fig. 9 more obviously to improve, the structural region such as figure clothes position, eastern wall pommel horse flags, weapons, etc. carried by a guard of honor figure horse rope position is advertised for Nan Bi, it is more level and smooth that the inventive method repairs lines, and structural continuity is better; For the texture region at eastern wall pommel horse flags, weapons, etc. carried by a guard of honor figure personage's arm position, the inventive method, without excessive extension, comparatively meets vision and appreciates.Table 1 is the index contrast of two groups of image repairing effects, the present invention structural similarity (Structural Similarity, and Y-PSNR (Peak Signal to NoiseRatio SSIM), PSNR) evaluation map is carried out as repairing effect, can find out that the inventive method is compared with Criminisi method, structural similarity at least improves 9.64%, and signal to noise ratio (S/N ratio) improves 5.73%.

Claims (1)

1. a Digital repair method for grave mural painting image, is characterized in that comprising the following steps:

S1: fixing digital camera, makes camera lens vertical with grave mural painting, gathers grave mural painting image;

S2: the damaged portion choosing grave mural painting image, is painted with white, as area to be repaired, remainder is as known region;

S3: construct full variation mathematical model and the image that S2 obtains is decomposed, obtain the cartoon image of filtering noise and tiny texture;

S4: the cartoon image utilizing S3 to obtain carries out the calculating driving priority, detailed process is as follows: calculate the confidence item of edge, cartoon image area to be repaired each point, data item and structure factor item of rectangular block centered by it, according to the priority of priority computing formula edge calculation each point, and obtain the coordinate of priority maximum point;

S5: according to the coordinate in S4, the point finding priority maximum in the image that S2 obtains, is designated as and with point centered by construction size be the multiblock to be repaired of m × m

S6: carry out global search in image known region, find with the block that color Euclidean distance is minimum, as the most similar sample block, is designated as Ψ _q;

S7: calculate Ψ _qwith multiblock to be repaired average pixel difference quadratic sum, be designated as and with threshold value T _sSDcompare, detailed process is as follows: if then think that this sample block meets copy condition, by block Ψ _qimage information copy to area to be repaired; Otherwise adjustment is repaired block and is of a size of (m-1) × (m-1), return and perform step S6, until meet copy condition, carry out block and copy;

S8: add in known region by new filler pixels, upgrades and repairs border; Then upgrade the degree of confidence C (p) of filler pixels, complete and once repair, degree of confidence upgrades principle and is: $C\left(p\right)=C\left(\hat{p}\right)e^{-{\left(\mathrm{ASSD}\right)}^2,}$ for degree of confidence penalty factor;

S9: check whether area to be repaired is empty, if not empty, then returns and performs S3; Otherwise, stop iteration, finally will repair result and preserve or export.