CN108154485B - Ancient painting restoration method based on layering and stroke direction analysis - Google Patents
Ancient painting restoration method based on layering and stroke direction analysis Download PDFInfo
- Publication number
- CN108154485B CN108154485B CN201711397822.8A CN201711397822A CN108154485B CN 108154485 B CN108154485 B CN 108154485B CN 201711397822 A CN201711397822 A CN 201711397822A CN 108154485 B CN108154485 B CN 108154485B
- Authority
- CN
- China
- Prior art keywords
- repaired
- repairing
- layer
- canvas
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010422 painting Methods 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000004458 analytical method Methods 0.000 title claims abstract description 13
- 230000009466 transformation Effects 0.000 claims abstract description 11
- 230000003993 interaction Effects 0.000 claims abstract description 10
- 230000008439 repair process Effects 0.000 claims description 29
- 238000010586 diagram Methods 0.000 claims description 15
- 230000004927 fusion Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000005452 bending Methods 0.000 claims description 2
- 239000004744 fabric Substances 0.000 claims description 2
- 230000000877 morphologic effect Effects 0.000 claims description 2
- 230000002093 peripheral effect Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 15
- 238000001914 filtration Methods 0.000 abstract 1
- 238000002474 experimental method Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000004800 variational method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/77—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
- G06T7/41—Analysis of texture based on statistical description of texture
Abstract
The invention relates to an ancient painting restoration method based on layering and stroke direction analysis. Firstly, separating a canvas layer and a content layer of the ancient painting by searching a maximum difference channel, setting a threshold value and filtering a small-area connected domain. And then, respectively repairing the canvas layer and the content layer. And for the stroke part of the content layer, decomposing and iteratively repairing in each direction through user interaction, curvelet transformation and inverse curvelet transformation. For the canvas layer or the area type content, the improved method based on the sample is adopted for repairing, so that the problem of a pseudo structure in the traditional method can be effectively avoided. And then fusing the repaired canvas layer and the repaired stroke layer to obtain a repaired image. Compared with the existing method, the method disclosed by the invention has a more smooth and natural repairing effect.
Description
Technical Field
The invention belongs to the field of digital image processing, and relates to an ancient painting restoration method based on layering and stroke direction analysis.
Background
The ancient Chinese painting is a treasure reflecting the ancient culture and art of China. The ancient painting is improperly stored, so that damage occurs in many places, the content of the ancient painting is lost, and even the canvas is damaged. The ancient painting restoration has important functions and significance in the fields of cultural relic protection, cultural transmission and historical research. The traditional physical repair method is easy to cause secondary damage to the cultural relics. With the continuous development of photographic technology, we can obtain high-definition ancient painting digital images. The method can repair the digital image of the ancient painting, can try for many times without generating the problem of secondary damage, and can play a role in guiding and trying to practice physical repair.
There are many existing digital image restoration methods, but so far, there are many limitations in ancient painting restoration applications. For example, "Non-texture Inpainting by customer-drive differences", published by Chan et al in Journal of Visual Communication and Image reproduction in 2001, fills in by progressively diffusing the data of known portions around missing portions in an Image. The method is large in calculation amount and time-consuming, is only suitable for repairing narrow areas such as scratches and stains, and generates an obvious fuzzy phenomenon when a real part of a large area is processed. "Object removal by exemplar-based embedding" published by Criminisi et al in the proceedings of Computer Vision and Pattern Recognition in 2003, by finding the best matching square to synchronize the filling of textures and structures, the structure filling effect is poor, and pseudo structures are easily generated. This article also employs interactively guided structural repairs, but only simple structures such as windows, fences, etc. The ancient Chinese painting is mostly composed of lines and has a complex structure, and the method is difficult to achieve good effect when being directly used for repairing. Deep Learning-based repair, such as "Context Encoders: Feature Learning by interpolation" and "High-Resolution Image interpolation using Multi-Scale Neural Path Systhesis", requires a large amount of training data sets, mostly takes a common definition photo as an object, and the repair algorithm is not yet mature. The ancient Chinese paintings are of various types, and the styles of the ancient Chinese paintings are different from generation to generation and even from time to time, so that the same type of materials for training are scarce. And the ancient painting requires beautiful and smooth repairing effect, and the existing repairing method is difficult to meet the requirement.
Disclosure of Invention
In view of the limitation of various algorithms at present, the invention provides an ancient painting restoration method based on layering and stroke direction analysis, which takes the beautiful appearance and smoothness of ancient painting restoration effect as the comprehensive and primary target.
In order to achieve the above object, the technical solution of the present invention is:
an ancient painting restoration method based on layering and stroke direction analysis comprises the following steps:
step 1, firstly, carrying out layered pretreatment on the repaired ancient painting, selecting a channel with the maximum difference color, and setting a threshold value on the channel to divide the image into a content layer and a canvas layer;
step 2, respectively repairing the canvas and the content on the basis of the known canvas and the content
For the canvas layer, because the texture of the whole picture is more uniform, the improved sample-based repair algorithm is adopted for repairing; and decomposing the lines of the content layer, matching with simple user interaction, and repairing and filling the lines in different directions. There is also a content of color block type, and this part is repaired by the same sample-based method as the texture.
And 3, effectively fusing the repaired canvas layer and the content layer.
Compared with the prior art, the invention has the following advantages: 1) the method and the device for repairing the ancient painting layer the ancient painting, and avoid the condition that the repairing effect is poor due to mutual interference between the canvas and the content during repairing. 2) The method repairs the texture and the content of the region block on the basis of layering, and has better repairing effect compared with the traditional method without layering. 3) The method and the device iteratively repair the content of the ancient painting line by analyzing the line direction and fusing the interactive information of the user, thereby ensuring the smoothness of the stroke repairing effect. Experiments prove that: compared with the prior art, the method can ensure that the ancient painting repair is not interfered by the texture of the canvas and lines in other directions, and the repair effect is beautiful and smooth.
Drawings
FIG. 1 is a flow chart of the present method;
fig. 2 is a graph of a content layer extraction and repair experiment in accordance with the method of the present disclosure, wherein,
fig. 2(a) the original, (b) the content layer, fig. 2(c) -2 (j) the directional diagrams of fig. 2(b), fig. 2(k) the interaction on fig. 2(b), fig. 2(l) the line to be repaired selected according to the interaction clue, fig. 2(m) -2 (o) the repair result of each iteration, fig. 2(p) the image after fusion, and fig. 2(q) the image after sample-based repair by Criminisi et al.
Fig. 3 is a texture repair experiment of the method, in which fig. 3(a) is an original image with texture flaws, fig. 3(b) is a texture area to be repaired marked by a user, and fig. 3(c) is a repair effect diagram of the invention.
Detailed Description
The invention is further described with reference to the following figures and detailed description.
The process of the invention is shown in fig. 1, and specifically comprises the following steps:
step one, layering the ancient paintings
The idea of distinguishing and separately processing different types of objects is also widely adopted in natural image restoration. At present, the work mostly adopts a variational method to perform sparse representation, such as: the "Simultaneous Cartooon and texture image inpainting using structural component analysis" method proposed by Elad et al separates structural and texture parts in an image. After separation, the two fractions were separately repaired and combined. Bertalmio et al separate the structure and texture by a total variation method in the "Simultaneous structure and texture inpainting", and superimpose the repaired structure layer and texture layer after the structure layer is transmitted as a final result. Compared with an integral processing method, the structure and the texture can be respectively processed to obviously improve the repairing effect. The layered repairing thought has a certain inspiring effect on the thought, however, the work mainly aims at natural images and is difficult to be used for repairing the ancient paintings.
The algorithm of the invention is based on the characteristics of the ancient painting: the color of the canvas and the painting content is obviously distinguished, and the painting content and the canvas are layered by taking the color as a clue. By comparing R, G, B channels of the canvas and the painting content, the channel with the largest difference between the canvas and the painting is selected, and the canvas and the content are separated from the image by setting a threshold value on the channel so as to better repair the canvas and the content. Because the ancient painting canvas also has dark small particles, the ancient painting canvas is easily divided into stroke parts by mistake. Therefore, the present invention filters the divided painting to remove small-area connected components (the experiment of the present invention is set to remove connected components smaller than 3 pixels) and classifies the connected components into the canvas part.
Step two, iterative ancient painting content layer restoration based on line direction analysis
In the step, in view of the characteristic that the ancient painting lines are very complex, the invention provides an iterative content layer repairing algorithm based on line direction analysis.
The specific operation is as follows:
1) given a content layer image of an ancient painting to be repaired (as shown in the specification, figure 2(b)), a user simply and interactively scribes to specify an area to be repaired (see the specification, figure 2 (k)). And determining the line to be repaired through the subsequent steps 2 and 3.
2) And performing curvelet transformation on the ancient picture image. The direction of set curvelet transform is divided into 8 directions, for each direction, the parameters after curvelet change in the direction are set as the original parameters, the parameters in the other directions are set as 0, and then the directional diagrams corresponding to the original ancient painting in 8 directions are generated through reverse curvelet transform (see the attached figure 2(c-j) of the specification).
3) According to the direction of user interaction, determining the direction of the line to be repaired on which directional diagram the line to be repaired falls, performing certain morphological expansion on the line interacted by the user, extracting the part on the directional diagram covered by the line to be repaired, and obtaining the corresponding line area to be repaired on the directional diagram (see the attached figure 2(l) of the specification).
4) And repairing the wire to be repaired according to the characteristic that the broken wire in the direction has certain extension according to the directional diagram after the reverse bending wave transformation. Firstly, calculating an average gray value on a directional diagram of the area, and filling the areas when the gray value of the pixels in the area is greater than zero and is less than the average gray value Tg (Tg is 1.3 in the experiment of the drawing).
5) The two lines at the two ends of the area to be repaired extend and fill independently, simultaneously and oppositely, and the lines at the two ends cannot be smoothly connected easily in the repairing process. Aiming at the situation, the algorithm provides that after each restoration, the actual growth direction is jointly calculated according to the extension direction of the current line and the traction of the opposite end point.
6) And projecting the repairing result of the current stage on a line interactively input by a user, and judging whether repairing is finished or not. If the minimum distance between the point on the projection and the point is less than Tc pixels (Tc is 10 in the experiment of the drawing), the repair is considered to be completed. Otherwise, continuing the step 2) until the repair is completed.
Step three, repairing the ancient painting canvas layer based on layered prior
The method adopts the sample-based method mentioned in Object removal by exemplar-based inpainting to repair the texture. But in contrast, the present invention performs texture repair on a layered basis. After layering, the specific properties of each pixel are known, i.e., whether it belongs to the canvas or to the content. When the texture is repaired, a user interactively marks a region to be repaired; the area is progressively repaired using a sample-based framework, except that when a patch is selected for padding, all candidates containing content are excluded. The existing sample-based repairing method is easy to introduce a pseudo structure into a filled area. The method carries out texture repair based on the sample after layering, and effectively avoids the problem. Although the content layer of the Chinese ancient painting is mostly composed of lines, the content layer also has the content of a color block type, and for the content, the method adopts the same mode as the texture to repair.
Step four, fusing the drawing cloth layer and the drawing layer
The step is mainly to fuse the repaired content layer and the canvas layer. The color statistics of the content layer for the peripheral undamaged portion resulted in the average of the three channels of content layer R, G, B. And carrying out certain random disturbance on the mean values, and assigning the mean values as color values to the lines of the repair area in the third step. Then, lines and canvas textures are fused in a certain proportion (the fusion proportion is 0.5), so that the canvas penetrates out of the content layer, and the reality of the repairing effect is improved.
Example 1:
examples of applications of the present invention are given below.
In this experiment, the pictures in fig. 2(a) and 3(a) were used to perform the experiment.
Fig. 2(a) through the preprocessing step, a content layer region in the image can be obtained as shown in fig. 2 (b). Fig. 2(c) to 2(j) are diagrams of respective directions after the curvelet transformation and the inverse curvelet transformation. Fig. 2(k) is the user interaction on the content layer (thin grey lines are user interaction inputs). Fig. 2(l) shows a line to be repaired selected by interaction. Fig. 2(m) -fig. 2(o) are three iterations of repairing a line. Fig. 2(p) shows the repairing effect after fusing with the canvas layer, i.e. the repairing result of the present invention. FIG. 2(q) is a sample-based repair result of Criminisi et al.
Fig. 3 is a repair experiment on a canvas layer, in which fig. 3(a) is an original image, fig. 3(b) is a region to be repaired indicated by user interaction, and fig. 3(c) is a texture repair result of fig. 3(a) according to the present invention.
Through experiments and comparison, the ancient painting has attractive repairing effect and smooth lines.
Claims (3)
1. An ancient painting restoration method based on layering and stroke direction analysis is characterized by comprising the following steps:
step 1, firstly, carrying out layered pretreatment on the repaired ancient painting, selecting a channel with the maximum difference color, and setting a threshold value on the channel to divide the image into a content layer and a canvas layer;
step 2, respectively repairing the canvas and the content;
step 3, the repaired canvas layer and the content layer are effectively fused;
the step 2 specifically comprises the following steps:
step 2.1, iterative ancient painting content layer repairing based on line direction analysis
The specific operation is as follows:
1) giving a content layer image of the ancient painting to be repaired, and simply and interactively marking out a line by a user for specifying an area to be repaired;
2) performing curvelet transformation on the ancient picture image: setting the direction of curvelet transformation into 8 directions, setting the parameters of curvelet transformation in the direction as original parameters and the parameters in the other directions as 0 for each direction, and then generating directional diagrams corresponding to the original ancient painting in 8 directions through reverse curvelet transformation;
3) determining a directional diagram in which the line to be repaired falls according to the interaction direction of the user, performing morphological expansion on the line interacted by the user, extracting a part on the directional diagram covered by the line to be repaired, and obtaining a corresponding line area to be repaired on the directional diagram;
4) according to the directional diagram after the reverse bending wave transformation, certain extension is provided for the broken lines in the direction, and aiming at the characteristic, the lines to be repaired are repaired: firstly, calculating an average gray value on a directional diagram of the area, and filling the areas when the gray value of pixels in the area is greater than zero and is less than the average gray value Tg;
5) in the repairing process, lines at two ends cannot be smoothly connected, and for the condition, after each repairing, the actual line direction is jointly calculated according to the current line extending direction and the traction of the opposite end point;
6) projecting the repairing result of the current stage on a line interactively input by a user, judging whether repairing is finished or not, and if the minimum distance between a point and a point on the projection is less than Tc pixels, determining that repairing is finished; otherwise, continuing the step 2) until the repair is completed;
step 2.2, repairing ancient painting canvas layer based on layered prior
When the texture is repaired, a user interactively marks a region to be repaired; the area is progressively repaired using a sample-based framework, except that when a patch is selected for padding, all candidates containing content are excluded.
2. The ancient painting restoration method based on layering and stroke direction analysis as claimed in claim 1, wherein the step 1 is specifically as follows: the method comprises the steps of firstly carrying out layered pretreatment on the repaired ancient painting, selecting a channel with the maximum difference color, and setting a threshold value on the channel to divide an image into a content layer and a painting cloth layer.
3. The ancient painting restoration method based on layering and stroke direction analysis as claimed in claim 1, wherein step 3 is specifically: carrying out color statistics on the content layer of the peripheral undamaged part to obtain the average value of three channels of the content layer R, G, B; and randomly disturbing the mean values, assigning the mean values as color values to lines of the repair area in the step 2.2, and fusing the lines and the canvas textures according to the fusion proportion of 0.5 to enable the canvas to penetrate out of the content layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711397822.8A CN108154485B (en) | 2017-12-21 | 2017-12-21 | Ancient painting restoration method based on layering and stroke direction analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711397822.8A CN108154485B (en) | 2017-12-21 | 2017-12-21 | Ancient painting restoration method based on layering and stroke direction analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108154485A CN108154485A (en) | 2018-06-12 |
| CN108154485B true CN108154485B (en) | 2021-07-16 |
Family
ID=62464188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711397822.8A Active CN108154485B (en) | 2017-12-21 | 2017-12-21 | Ancient painting restoration method based on layering and stroke direction analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108154485B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348807B (en) * | 2020-11-27 | 2022-11-18 | 安徽大学 | Endoscope highlight point repairing method and system based on contour pixel statistics |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102305769A (en) * | 2011-06-09 | 2012-01-04 | 天津大学 | Multispectral sectional drawing method applied to Chinese ancient painting repair |
| CN102968764A (en) * | 2012-10-26 | 2013-03-13 | 北京航空航天大学 | Chinese character image inpainting method based on strokes |
| CN104766283A (en) * | 2015-04-15 | 2015-07-08 | 中北大学 | Digitized repairing method for grave mural image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9886746B2 (en) * | 2015-07-20 | 2018-02-06 | Tata Consultancy Services Limited | System and method for image inpainting |
-
2017
- 2017-12-21 CN CN201711397822.8A patent/CN108154485B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102305769A (en) * | 2011-06-09 | 2012-01-04 | 天津大学 | Multispectral sectional drawing method applied to Chinese ancient painting repair |
| CN102968764A (en) * | 2012-10-26 | 2013-03-13 | 北京航空航天大学 | Chinese character image inpainting method based on strokes |
| CN104766283A (en) * | 2015-04-15 | 2015-07-08 | 中北大学 | Digitized repairing method for grave mural image |
Non-Patent Citations (2)
| Title |
|---|
| 基于Curvelet方向特征的样本块图像修复算法;李志丹;《电子学报》;20160116;第44卷(第1期);第151-153页 * |
| 基于图像分解的敦煌壁画图像修复方法;黄伟;《山东大学学报》;20100416;第40卷(第2期);第25-27页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108154485A (en) | 2018-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108537239B (en) | Method for detecting image saliency target | |
| CN107066916B (en) | Scene semantic segmentation method based on deconvolution neural network | |
| CN109785236B (en) | Image super-resolution method based on super-pixel and convolutional neural network | |
| CN108447068B (en) | Ternary diagram automatic generation method and foreground extraction method using ternary diagram | |
| CN110544300B (en) | Method for automatically generating three-dimensional model based on two-dimensional hand-drawn image characteristics | |
| CN104299263A (en) | Method for modeling cloud scene based on single image | |
| CN103020917A (en) | Method for restoring ancient Chinese calligraphy and painting images on basis of conspicuousness detection | |
| CN111080574A (en) | Fabric defect detection method based on information entropy and visual attention mechanism | |
| CN105093289A (en) | Seismic horizon identification method based on image processing | |
| CN103955942A (en) | SVM-based depth map extraction method of 2D image | |
| CN104637036A (en) | Chinese ancient painting enhancing method | |
| CN107507263B (en) | Texture generation method and system based on image | |
| CN105957027B (en) | A kind of MRF sample block image repair methods based on required direction structure characteristic statistics | |
| CN108154485B (en) | Ancient painting restoration method based on layering and stroke direction analysis | |
| Fu et al. | Fast accurate and automatic brushstroke extraction | |
| He | Application of local color simulation method of landscape painting based on deep learning generative adversarial networks | |
| CN115965844B (en) | Multi-focus image fusion method based on visual saliency priori knowledge | |
| Kong et al. | Effective 3d face depth estimation from a single 2d face image | |
| Zhang et al. | Automatic genaration of sketch-like pencil drawing from image | |
| CN111462084B (en) | Image vectorization printing bleeding point prediction system and method based on random forest | |
| CN115170387A (en) | Processing method of pixel stylized cartoon image based on artistic creation | |
| CN108171771A (en) | The line drawing in a kind of combination external margin information and interior polymeric road draws generating algorithm | |
| Srivastava et al. | An approach to pseudocoloring of grey scale image using deep learning technique | |
| Bhuvaneswari et al. | An efficient method for digital imaging of ancient stone inscriptions | |
| CN111127587A (en) | Non-reference image quality map generation method based on countermeasure generation network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |