CN111754427B - Automatic repair method for mural crack diseases based on self-organizing mapping neural network - Google Patents
Automatic repair method for mural crack diseases based on self-organizing mapping neural network Download PDFInfo
- Publication number
- CN111754427B CN111754427B CN202010521666.7A CN202010521666A CN111754427B CN 111754427 B CN111754427 B CN 111754427B CN 202010521666 A CN202010521666 A CN 202010521666A CN 111754427 B CN111754427 B CN 111754427B
- Authority
- CN
- China
- Prior art keywords
- image
- mural
- crack
- self
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000008439 repair process Effects 0.000 title claims abstract description 55
- 201000010099 disease Diseases 0.000 title claims abstract description 39
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 39
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 24
- 238000013507 mapping Methods 0.000 title claims abstract description 16
- 230000008569 process Effects 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims abstract description 8
- 230000000877 morphologic effect Effects 0.000 claims description 21
- 210000002569 neuron Anatomy 0.000 claims description 14
- 230000010339 dilation Effects 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 claims description 6
- 230000003628 erosive effect Effects 0.000 claims description 6
- 238000003708 edge detection Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 4
- 239000002131 composite material Substances 0.000 claims description 3
- 230000007797 corrosion Effects 0.000 claims description 2
- 238000005260 corrosion Methods 0.000 claims description 2
- 239000003973 paint Substances 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 abstract description 3
- 238000000926 separation method Methods 0.000 abstract description 2
- 239000010410 layer Substances 0.000 description 18
- 238000004458 analytical method Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000010422 painting Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 244000298715 Actinidia chinensis Species 0.000 description 2
- 235000009434 Actinidia chinensis Nutrition 0.000 description 2
- 235000009436 Actinidia deliciosa Nutrition 0.000 description 2
- 241000282553 Macaca Species 0.000 description 2
- 241000282454 Ursus arctos Species 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 235000019994 cava Nutrition 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000004299 exfoliation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000000051 modifying effect Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 239000000049 pigment Substances 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention relates to a mural crack disease automatic repairing method based on a self-organizing mapping neural network, which comprises the following steps: collecting a digital mural crack image, and marking mural crack diseases through mathematical morphology; clustering the original data by adopting a self-organizing mapping neural network algorithm, completing the layering work of the image data, and then repairing the clustered layered data. The method fully considers the linear structure characteristics of the mural crack, uses a self-adaptive threshold segmentation algorithm in the automatic crack marking process, realizes the separation of the target pixel and the background pixel, and improves the crack identification precision; and the accurate clustering characteristic of the self-organizing mapping neural network algorithm is fully utilized to cluster and layer each channel pixel, so that the quick repair work of the layered image is realized, the mural image is clearer and more natural, and the integral visual effect with certain similar precision to the original mural image is kept.
Description
Technical Field
The invention belongs to the technical field of digital image processing, and particularly relates to an automatic repairing method for mural crack diseases based on a self-organizing mapping neural network.
Background
The ancient mural painting is a treasure house of the classical art in China, is witness of history and culture in China, is also a valuable historical cultural heritage in China, and has immeasurable historical research value. However, under the influence of external environment and human intrinsic factors, most mural groups such as Dunhuang Mogao holes, West Qianfo holes and the like have various diseases such as cracks, falling off, shortenings, mildewing and the like with different degrees. In order to protect the precious Chinese civilizations for a long time, repair damaged parts in the murals and restore the cultural charm of the murals, the method is urgent and has great cultural inheritance.
Aiming at various diseases encountered by ancient murals, two different protection methods are mainly adopted at present: one is the traditional mural repair method, which is a manual repair. Due to the shortage of professional technicians, the method has the defects of single repairing means, long time consumption, low repairing efficiency and the like, so that the mural protection situation is severe. The other is a modern digital repairing method, the repairing of the mural is to use advanced artificial intelligence technology to digitally collect the mural, so that the mural is separated from the cultural relic main body and exists, and the diseases are marked and repaired by using a computer and other new technical means, so that the real surface of the mural is restored under the condition of not damaging the mural. The digital repairing method can avoid secondary damage to the mural to a great extent, and meanwhile, the accuracy of the repairing effect enables the repairing effect to have irreplaceable effects on analysis and development of historical cultural heritage.
The final aim of the digital image restoration technology is to realize the restoration function of the ancient architecture murals, namely, the restoration of the murals is completed by using the advanced artificial intelligence technology. The mural image crack disease is one of mural diseases, and aiming at the special mural image crack disease, the existing repairing method does not consider the linear structure characteristic of the mural crack, so that the modifying effect is poor, and the mural image crack disease is difficult to effectively recover.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides an automatic repairing method of a mural crack disease based on a self-organizing mapping neural network, and solves the problem that the mural crack disease cannot be effectively recovered.
The technical problem to be solved by the invention is realized by adopting the following technical scheme:
a mural crack disease automatic repairing method based on a self-organizing mapping neural network comprises the following steps:
and 2, clustering the original data by adopting a self-organizing mapping neural network algorithm to finish the layering work of the image data, and then repairing the clustered layered data.
And the step 3 is followed by a step of repairing the paint layer of the actual mural crack disease according to the digital image repairing result.
Moreover, the specific implementation method of the step 1 comprises the following steps:
the method comprises the steps of obtaining a digital mural crack image, and converting the collected color digital mural crack image into a gray level image;
secondly, applying multi-scale morphological edge gradient detection to the obtained gray level image, and extracting the image structure contour edge by using an image edge detection operator formed by corrosion expansion composite operation;
thirdly, dividing pixels in the mural crack image into a target type and a background type by adopting a maximum type variance method self-adaptive threshold segmentation technology, and further extracting the target pixels to obtain a binary image with only two gray values;
and fourthly, measuring the binary image by adopting the connected domain mark, and removing residual background noise to obtain a mural crack marking image.
Moreover, the specific implementation method of the step 1 is as follows: acquiring a color digital mural crack image by using a high-resolution digital camera, carrying out weighted average on three components of r, g and b of a color image (r, g and b) by different weights, and calculating a weighted formula according to the following formula:
V＝0.299×r+0.578×g+0.114×b
wherein V represents the gray scale value of the weighted gray scale image at (x, y); r, g, b represent the r, g, b three-component gray scale values at (x, y), respectively.
And, the mathematical expression of the multi-scale morphological edge gradient in the step two is:
wherein f (x, y) is a gray scale image, b (x, y) represents a structural element, k is a scale parameter,
the morphological erosion operator is represented by a graphical representation,
representing a morphological dilation operator.
Moreover, the specific implementation method of the step 2 comprises the following steps:
determining the optimal clustering of mural crack images to obtain the number of optimal clustering neurons;
performing parallel layered restoration on the plurality of clustering layer images;
thirdly, combining the plurality of repaired clustering layers to complete the repair of the mural crack image.
Moreover, the concrete implementation method of the step is as follows:
taking the crack image as a three-dimensional vector, designing an SOM network of three input units, wherein the input vector of each unit is x ═ x 1 ,x 2 ,…,x n )；
② calculating a weight vector w by the following formula j Euclidean distance from x, thereby obtaining a competition winning neuron:
wherein x is (x) 1 ,x 2 ,…,x n ) Is an n-dimensional vector; w is a j A weight vector of a jth neuron node of a competition layer;
③ updating the weight vector according to:
w j (t+1)＝w j (t)+η(t,N)[x(t)-w j (t)]
wherein x (t) and w j (t) representing the input mode and the weight vector at time t, respectively; η (t, N) is a topological relation function between the runtime t and the winning neighborhood N;
and fourthly, outputting the image clustering result.
The concrete implementation method of the steps is as follows: assuming that a damaged pixel in the image is p, 1 represents an undamaged pixel, and 0 represents a damaged pixel; firstly, traversing all damaged pixels, finding out a damaged pixel p with an adjacent pixel being 1, marking the damaged pixel p, determining a layer to which the pixel belongs, and then iteratively calculating the value of the damaged pixel, thereby completing the repair of the damaged pixel; traversing all the rest damaged pixels, searching the damaged pixels p' of which the adjacent pixels are not all 0, and sequentially circulating to finish the repair of all the crack damaged pixels.
In the layered repairing process, the mean value of each pixel is calculated by iteration instead of directly copying the pixels from the undamaged area of the image for repairing; and after clustering and layering are carried out on the images, the rapid repairing function of the layered images is realized in a parallelization mode.
The invention has the advantages and positive effects that:
1. the method fully considers the linear structure characteristics of the mural crack, uses a self-adaptive threshold segmentation algorithm in the automatic crack marking process, realizes the separation of the target pixel and the background pixel, and improves the crack identification precision; and the precise clustering characteristic of a Self-Organizing mapping neural network (SOM) algorithm is fully utilized, the improved SOM is introduced into the ancient building mural image repairing process, each channel pixel is clustered and layered firstly, the value of a damaged pixel is calculated in an iteration mode in a single layer, the layered image is rapidly repaired, the mural image is clearer and more natural, the integral visual effect with certain similar precision to the original mural image is kept, and the method and the device can be widely applied to the ancient building mural crack repairing process.
Drawings
FIG. 1 is a flow chart of a repair method of the present invention;
FIG. 2 is a schematic diagram of the single-scale morphological edge gradient detection of the present invention;
FIG. 3 is a labeled schematic view of a single fracture of the present invention;
FIG. 4 is a schematic diagram of SOM self-organizing neural network clustering in accordance with the present invention;
FIG. 5 is a graph of the single crack fresco "Kiwi picture" repair results of the present invention;
FIG. 6 is a graph of the single crack wall painting "monk picture" repair results of the present invention;
FIG. 7 is a graph of the multi-crack fresco "bear brown image" repair result of the present invention;
FIG. 8 is a graph of the repair result of the multi-crack fresco "color pattern image" of the present invention;
FIG. 9 is a diagram of the complex interactive crack fresco "Buddha image" repair result of the present invention;
FIG. 10 is a diagram of the complex interactive crack wall painting "face map" repair result of the present invention;
FIG. 11a is a mural peeling repair assessment index analysis chart (RMES index) of the present invention and other algorithms
FIG. 11b is a graph of analysis of mural drop repair assessment indicators (PSNR indicators) in accordance with the present invention and other algorithms;
FIG. 11c is an analysis graph of mural peeling repair assessment indicators (FSIM indicators) of the present invention and other algorithms;
FIG. 11d is a graph of mural drop repair assessment indicator analysis (SR-SIM indicator) for the present invention and other algorithms;
FIG. 11e is a graph of mural exfoliation repair assessment index analysis (VSI index) for the present invention and other algorithms.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
The design idea of the invention is as follows: firstly, a mural crack disease image is marked through mathematical morphology, and the marked mural crack disease image is mapped into a two-dimensional gray image so as to improve the time-space correlation analysis capability among data. Then, the original data are clustered by adopting an artificial intelligence SOM neural network algorithm, the layering work of the image data is completed, finally, the clustered layered data are repaired, and a repairing result is obtained, wherein the flow chart of the algorithm is shown in figure 1.
Based on the design idea, the invention provides a mural crack disease automatic repairing method based on a self-organizing mapping neural network, which comprises the following steps:
the method includes the steps of acquiring and obtaining digital mural crack images and conducting graying processing on the acquired mural crack images.
In the step, a high-resolution digital camera is used for acquiring and obtaining a digital mural crack image, and the acquired mural crack image is processed and converted into a gray image with the advantages of high segmentation speed, less calculation amount, full chromaticity and brightness display and the like. In order to obtain a reasonable gray image, in the step, three components of r, g and b of the color image (r, g and b) are weighted and averaged by different weights, and a weighting formula is calculated according to an equation (1):
V＝0.299×r+0.578×g+0.114×b (1)
wherein V represents the gray value of the weighted gray image at (x, y); r, g, b represent the r, g, b three-component gray scale values at (x, y), respectively.
And secondly, applying multi-scale morphological edge gradient detection to the obtained gray level image f (x, y) to extract the image structure contour edge by using an image edge detection operator formed by corrosion-expansion composite operation.
In the step, the acquired characteristics of the mural crack disease image are combined, and the local mutation information and the advantages in the background noise interference resistance are detected by means of the multi-scale morphological gradient.
The gradient in the image is mainly used for describing a region with intense gray change of a target boundary or an edge image and measuring the change rate of the gray of the image, and for the digital image, the gradient value and the gradient direction at a pixel point (x, y) are respectively as follows:
wherein G is x (x, y) is the gradient in the x-direction; g y (x, y) is a y-direction gradient;
morphological Gradient (Morphological Gradient) is defined by Morphological dilation and erosion, and the dilation or erosion is combined with the difference of the gray level image to realize the sharper transition of the gray level in the image, thereby highlighting the highlight region.
Let f (x, y) be the original mural image, b (x, y) represent the structural elements, and the image edge detection operator formed by corrosion-expansion complex operation, i.e. the single-scale morphological gradient, the edge detection process is shown in fig. 2, and the expression is:
wherein f (x, y) represents a gray image, b (x, y) is a structural element,
the morphological erosion operator is represented by a graphical representation,
representing a morphological dilation operator.
The mathematical expression for the multi-scale morphological gradient is:
wherein f (x, y) represents a gray image, b i (x, y) is a structural element, k is a scale parameter,
the morphological erosion operator is represented as a function of time,
representing a morphological dilation operator.
A adaptive threshold segmentation
In the process of extracting the crack region, the characteristics of the crack need to be fully considered, and the content of the extracted crack cannot be adaptively adjusted in a typical fixed threshold method.
The method adopts a maximum class variance method self-adaptive threshold segmentation technology to divide pixels in the mural crack image into a target class and a background class, and further extracts the target pixels, so that the segmentation purpose is achieved, and a binary image with only two gray values is obtained. Is represented as follows:
in the formula, f (x, y) and g (x, y) respectively represent the gray scale values of pixels of the image before and after processing at (x, y) coordinates, t represents a gray scale threshold, 1 represents a target pixel, and 0 represents a background pixel.
And fourthly, measuring the binary image by adopting the connected domain mark, and removing residual background noise to obtain a mural crack marking image.
The mural crack has certain linear characteristics and certain continuity and connectivity in space, but a plurality of false objects, namely noise, still exist in the image after the threshold value is adaptively adjusted, and the noise is discontinuous, disordered and isolated in spatial distribution. Therefore, the invention adopts a connected domain marking method to remove noise and adopts morphological basic operation to connect the fractured cracks. In the invention, the area is selected as a communication rule of a target area for measurement, and the area calculation formula is that if the pixel value of the target area of the image is 1px
Where s is the connected component of the desired metric and f (x, y) is the pixel value.
And removing the crack area of the false target through connected domain marking to obtain a marked image, wherein the marked image is marked by taking a 428 th Kiwi picture of Mogao caves in the northern week period of a single crack as an example, and the obtained marking result is shown in fig. 3.
And 2, clustering the original data by adopting a self-organizing mapping neural network algorithm, and then repairing the clustered hierarchical data. The specific implementation method of the step comprises the following steps:
the method includes determining the optimal clustering of mural crack images.
Algorithm initialization
Considering the crack image as a three-dimensional vector, designing an SOM network of three input units, wherein the input vector of each unit is x (x ═ x) 1 ,x 2 ,…,x n )。
In an image RGB three-dimensional color space, setting a current input mode x ═ x (x) of an ad hoc network 1 ,x 2 ,…,x n ) Is an n-dimensional vector, the output layer is a two-dimensional network with m x n nodes, w j Is the weight vector of the jth neuron node of the competition layer. Establishing an initial winning neighborhood N i* (0) And an initial value is given to the learning rate eta.
② searching winning neurons
Inputting a random input mode x into the network, calculating a weight vector w j Euclidean distance from x, finding the winning node w with the smallest distance j* And is marked as competition-winning neuron.
Wherein x is (x) 1 ,x 2 ,…,x n ) Is an n-dimensional vector; w is a j The weight vector of the jth neuron node of the competition layer.
And thirdly, adjusting the weight.
Only the winning neuron has the right to adjust its weight vector w j For the winning neighborhood N i* (t) all neurons in the set adjust weights
w j (t+1)＝w j (t)+η(t,N)[x(t)-w j (t)] (7)
Wherein, x (t) and w j (t) representing the input mode and the weight vector at time t, respectively; η (t, N) is the topological relation between the runtime t and the winning neighborhood NIs a function of.
Fourthly, outputting the image clustering result
Based on the above clustering process, the clustering result is output in the form of a hierarchical image, as shown in fig. 4. For the three color channel R, G, B, if the pixels of the multi-color channel are the same or similar, then the probability that the multi-color channel pixels will be clustered into the same cluster is greater when they are clustered.
And (3) performing layered restoration on one image according to a clustering layered thinking, and substituting the number of clustering neurons into the step II.
And image parallel layered restoration.
For layering of a color RGB image, there are usually several categories of main objects or main components in the image, which are divided into at least several layers. And obtaining the layering number of the image after obtaining the optimal number of clustering neurons in the repairing process aiming at each clustered class. In the figure, a broken pixel is assumed to be p, 1 represents an unbroken pixel, and 0 represents a broken pixel. Firstly, traversing all damaged pixels, finding out the damaged pixel p with the adjacent pixel being 1, marking the damaged pixel p and determining the layer to which the pixel belongs, and then iteratively calculating the value of the damaged pixel, thereby completing the repair of the damaged pixel. And similarly, traversing all the residual damaged pixels, searching the damaged pixels p' of which the adjacent pixels are not all 0, and sequentially circulating to finish the repair of all the damaged pixels.
In the parallelization layered repair, the method of the invention improves the following two points: the mean value of each pixel is calculated by iteration instead of directly copying the pixels from the undamaged area of the image for repair; after clustering and layering are carried out on the images, quick restoration of the layered images is realized in a parallelization mode.
Thirdly, combining the plurality of cluster layers after repairing to obtain a fused image, namely completing repairing of the damaged image
After the parallelization layering restoration of the image is completed, for the multi-layer combination of one color RGB image, due to the existence of the parallelization layering, each pixel has a brightness value in the corresponding layer, and the pixel is displayed as a blank in other layers. In order to facilitate the superposition of the layer matrixes and avoid the phenomenon that the pixel values overflow the value range, the white pixels in all the layer matrixes are assigned to be 0. And then, superposing all the layer matrixes to obtain a fused image, namely completing the repair of the damaged image.
And 3, repairing the mural crack pigment layer according to the repairing result of the digital image.
In order to verify the effectiveness of the mural crack disease automatic repairing method based on the improved self-organizing mapping neural network, multiple types of mural crack diseases such as single cracks, multiple cracks, complex interaction cracks and the like are selected as cases, the method is applied to carry out data repairing effect analysis, and meanwhile, the method is compared and analyzed with the existing common mural repairing TV algorithm, the K-SVD algorithm, the traditional SOM algorithm and the PConv algorithm.
The method selects 3 types of crack diseases of single crack, multiple cracks and complex interactive cracks commonly existing in the mural to carry out a disease repair experiment, and selects 6 murals as mural crack test patterns.
Single crack disease: the cases included fractures in the 428 st Gr "macaque map" in the Mogao Grottoes in the North Wednk and the 158 th Grotto "monk" in the Mogao Grottish in the Tang period. In both images, the cracks developed through earlier stages and were distributed in a unidirectional elongated stripe structure. The repair results using different repair algorithms are shown in fig. 5 and 6.
Multiple crack diseases: the case contains cracks in the 249 th and 156 th brown bear pictures in the Mogao Grottoes in the Chong Wei period and the colorful texture pictures in the 156 th Grottoes period. Through specific analysis of two cases, the picture structure of the brown bear picture is rough, the year is long and the picture belongs to a later crack structure, and the color pattern picture belongs to an earlier crack and is distributed in a rectangular shape. The repair results with different repair algorithms are shown in fig. 7 and 8.
Complex interactive crack disease: the case contains cracks of 220 th cave of Morgan in the first Tang period "Buddha image" and 14 th cave of people's face image "in the late Tang period. Through specific analysis of two cases, the 'Buddha image' and the 'color pattern' belong to the same early stage cracks but have brighter colors, and the 'face image' is in random late stage cracks. The repair results using different repair algorithms are shown in fig. 9 and 10.
The invention selects a plurality of evaluation indexes to evaluate the quality of the data restoration effect, and comprises the following steps: root Mean Square Error (RMSE), Peak Signal to Noise Ratio (PSNR), Feature Similarity (FSIM), Spectral information Similarity (SR-SIM), and Visual Significance Index (VSI) were used as Similarity evaluation indicators in this section to evaluate image quality.
Wherein, the Root Mean Square Error (RMSE) reflects the discrete degree of the repair result, the peak signal-to-noise ratio (PSNR) reflects the variation degree of the distortion data, and the similarity between the images before and after repair is measured, which is the most widely applied evaluation index in the image repair research, and the calculation formulas are respectively:
in the formula: i is 1 ,I 2 The two images to be compared respectively refer to an original image and a repaired image; m and n respectively represent the sizes of the two images;
by utilizing the automatic repair method for the mural crack diseases based on the improved self-organizing mapping neural network, the marked mural cracks are repaired respectively, and each repair effect evaluation index is calculated. Comparing the method of the present invention with the TV algorithm, the K-SVD algorithm, the conventional SOM algorithm, and the PConv algorithm, as shown in fig. 11a, 11b, 11c, 11d, and 11e, meanwhile, the repair efficiency ratio of different algorithms is shown in table 1.
TABLE 1 mural crack image repair efficiency assessment index
The abscissa in fig. 11a, 11b, 11c, 11d, and 11e is the similarity evaluation index. The analysis is here performed using a single crack fresco "macaque picture". The PSNR value of the improved SOM algorithm is 41.720dB, while the PSNR values of the comparative algorithms TV, K-SVD, traditional SOM and PConv are 39.910dB, 36.609dB, 40.249dB and 40.269dB respectively, and compared with the comparative algorithms, the improved SOM algorithm provided by the invention has the advantages that the PSNR is maximally improved by 13.96%, the FSIM value is maximally increased by 1.40%, the SR-SIM value is maximally increased by 0.79% and the VSI value is maximally increased by 0.44%. In the text, various repairing algorithms are operated by a single machine, and the repairing time of the six mural paintings repaired by different algorithms is given in table 1. According to experimental results, the improved SOM algorithm provided by the invention has the advantages that the repair time is shorter, the average repair time is shortened by 40.34%, and the efficiency is higher. Therefore, compared with TV, K-SVD, traditional SOM and PConv repair algorithms, the SOM repair algorithm provided by the invention has lower repair error and higher repair speed.
It should be emphasized that the embodiments described herein are illustrative rather than restrictive, and thus the present invention is not limited to the embodiments described in the detailed description, but also includes other embodiments that can be derived from the technical solutions of the present invention by those skilled in the art.
Claims (7)
1. A mural crack disease automatic repairing method based on a self-organizing mapping neural network is characterized by comprising the following steps:
step 1, collecting a digital mural crack image, and marking mural crack diseases through mathematical morphology;
step 2, clustering the original data by adopting a self-organizing mapping neural network algorithm to finish the layering work of the image data, and then repairing the clustered layered data;
the specific implementation method of the step 2 comprises the following steps:
determining the optimal clustering of mural crack images to obtain the number of optimal clustering neurons;
performing parallel layered restoration on the plurality of clustering layer images;
combining the plurality of repaired clustering layers to complete the repair of the mural crack image;
in the layered repairing process, the mean value of each pixel is calculated by iteration instead of directly copying the pixels from the undamaged region of the image for repairing; and after clustering and layering are carried out on the images, the rapid repairing function of the layered images is realized in a parallelization mode.
2. The mural crack disease automatic repair method based on the self-organizing map neural network according to claim 1, characterized in that: and step 2, repairing the paint layer of the actual mural crack diseases according to the digital image repairing result.
3. The mural crack disease automatic repair method based on the self-organizing map neural network according to claim 1 or 2, characterized in that: the specific implementation method of the step 1 comprises the following steps:
the method comprises the steps of obtaining a digital mural crack image and converting the collected color digital mural crack image into a gray level image;
secondly, applying multi-scale morphological edge gradient detection to the obtained gray level image, and extracting the image structure contour edge by using an image edge detection operator formed by corrosion expansion composite operation;
thirdly, dividing pixels in the mural crack image into a target type and a background type by adopting a maximum type variance method self-adaptive threshold segmentation technology, and further extracting the target pixels to obtain a binary image with only two gray values;
and fourthly, measuring the binary image by adopting the connected domain mark, and removing residual background noise to obtain a mural crack marking image.
4. The mural crack disease automatic repair method based on the self-organizing map neural network according to claim 3, characterized in that: the specific implementation method for obtaining the digital mural crack image and converting the collected color digital mural crack image into the gray level image comprises the following steps: acquiring a color digital mural crack image by using a high-resolution digital camera, carrying out weighted average on three components of r, g and b of a color image (r, g and b) by different weights, and calculating a weighted formula according to the following formula:
V＝0.299×r+0.578×g+0.114×b
wherein V represents the gray value of the weighted gray image at (x, y); r, g, b represent the r, g, b three-component gray scale values at (x, y), respectively.
5. The mural crack disease automatic repair method based on the self-organizing map neural network according to claim 3, characterized in that: the mathematical expression of the multi-scale morphological edge gradient is:
wherein f (x, y) is a gray scale image, b (x, y) represents a structural element, k is a scale parameter,
the morphological erosion operator is represented by a graphical representation,
representing a morphological dilation operator.
6. The mural crack disease automatic repair method based on the self-organizing map neural network according to claim 1, characterized in that: the concrete implementation method of the step comprises the following steps:
taking the crack image as a three-dimensional vector, designing an SOM network of three input units, wherein the input vector of each unit is x ═ x 1 ,x 2 ,…,x n )；
② calculating a weight vector w by the following formula j Euclidean distance from x, thereby obtaining a competition winning neuron:
wherein x is (x) 1 ,x 2 ,…,x n ) Is an n-dimensional vector; w is a j A weight vector of a jth neuron node of a competition layer;
③ updating the weight vector according to:
w j (t+1)＝w j (t)+η(t,N)[x(t)-w j (t)]
wherein x (t) and w j (t) representing the input mode and the weight vector at time t, respectively; η (t, N) is a topological relation function between the runtime t and the winning neighborhood N;
and fourthly, outputting the image clustering result.
7. The mural crack disease automatic repair method based on the self-organizing map neural network according to claim 1, characterized in that: the concrete implementation method of the steps comprises the following steps: assuming that a damaged pixel in the image is p, 1 represents an undamaged pixel, and 0 represents a damaged pixel; firstly, traversing all damaged pixels, finding out a damaged pixel p of which the adjacent pixels are 1, marking the damaged pixel p, determining the layer to which the pixel belongs, and then iteratively calculating the value of the damaged pixel, thereby completing the repair of the damaged pixel; traversing all the rest damaged pixels, searching the damaged pixels p' of which the adjacent pixels are not all 0, and sequentially circulating to finish the repair of all the crack damaged pixels.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010521666.7A CN111754427B (en) | 2020-06-10 | 2020-06-10 | Automatic repair method for mural crack diseases based on self-organizing mapping neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010521666.7A CN111754427B (en) | 2020-06-10 | 2020-06-10 | Automatic repair method for mural crack diseases based on self-organizing mapping neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111754427A CN111754427A (en) | 2020-10-09 |
| CN111754427B true CN111754427B (en) | 2022-09-16 |
Family
ID=72675023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010521666.7A Active CN111754427B (en) | 2020-06-10 | 2020-06-10 | Automatic repair method for mural crack diseases based on self-organizing mapping neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111754427B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103227963A (en) * | 2013-03-20 | 2013-07-31 | 西交利物浦大学 | Static surveillance video abstraction method based on video moving target detection and tracing |
| CN103295199A (en) * | 2013-05-29 | 2013-09-11 | 西安建筑科技大学 | Intelligent repair assistance system for cracks of ancient wall murals |
| CN106204424A (en) * | 2016-06-30 | 2016-12-07 | 厦门美图之家科技有限公司 | Image removes water mark method, applies and calculating equipment |
| CN110189273A (en) * | 2019-05-27 | 2019-08-30 | 广西师范大学 | A kind of hidden image restorative procedure based on double sampling |
-
2020
- 2020-06-10 CN CN202010521666.7A patent/CN111754427B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103227963A (en) * | 2013-03-20 | 2013-07-31 | 西交利物浦大学 | Static surveillance video abstraction method based on video moving target detection and tracing |
| CN103295199A (en) * | 2013-05-29 | 2013-09-11 | 西安建筑科技大学 | Intelligent repair assistance system for cracks of ancient wall murals |
| CN106204424A (en) * | 2016-06-30 | 2016-12-07 | 厦门美图之家科技有限公司 | Image removes water mark method, applies and calculating equipment |
| CN110189273A (en) * | 2019-05-27 | 2019-08-30 | 广西师范大学 | A kind of hidden image restorative procedure based on double sampling |
Non-Patent Citations (4)
| Title |
|---|
| Research on hierarchical image restoration of Chinese painting;Song Wei;《2014 9th International Conference on Computer Science & Education》;20141016;全文 * |
| 基于自组织神经网络SOM和K-means聚类算法的图像修复;孙震 等;《科学技术与工程》;20120318;第12卷(第8期);摘要、正文第1-3章 * |
| 多尺度唐墓室壁画病害标记及修复技术研究;吴萌 等;《计算机工程与应用》;20150313;第52卷(第11期);摘要、正文第2-4章 * |
| 多尺度形态梯度算法及其在图像分割中的应用;卢官明 等;《信号处理》;20010228;第17卷(第1期);正文第2章 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111754427A (en) | 2020-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108038853B (en) | Ceramic tile surface defect identification method based on convolutional neural network and active learning | |
| CN109446992B (en) | Remote sensing image building extraction method and system based on deep learning, storage medium and electronic equipment | |
| CN105678332B (en) | Converter steelmaking end point judgment method and system based on flame image CNN recognition modeling | |
| CN109118479B (en) | Capsule network-based insulator defect identification and positioning device and method | |
| CN107784661B (en) | Transformer substation equipment infrared image classification and identification method based on region growing method | |
| CN110544251B (en) | Dam crack detection method based on multi-migration learning model fusion | |
| CN111833306B (en) | Defect detection method and model training method for defect detection | |
| CN106023185A (en) | Power transmission equipment fault diagnosis method | |
| CN111985499B (en) | High-precision bridge apparent disease identification method based on computer vision | |
| CN109741328A (en) | A kind of automobile apparent mass detection method based on production confrontation network | |
| CN112906550B (en) | Static gesture recognition method based on watershed transformation | |
| CN108053412B (en) | Method for determining optimal segmentation result of remote sensing image by fusing multiple quantitative evaluation indexes | |
| Ni et al. | An improved Faster R-CNN for defect recognition of key components of transmission line | |
| CN115082744B (en) | Artificial intelligence-based solar heat collection efficiency analysis method and system | |
| CN114091606A (en) | Tunnel blasting blast hole half-hole mark identification and damage flatness evaluation classification method | |
| CN110135248A (en) | A kind of natural scene Method for text detection based on deep learning | |
| CN112991271A (en) | Aluminum profile surface defect visual detection method based on improved yolov3 | |
| Musicco et al. | Automatic point cloud segmentation for the detection of alterations on historical buildings through an unsupervised and clustering-based Machine Learning approach | |
| CN114140485A (en) | Method and system for generating cutting track of main root of panax notoginseng | |
| CN112232249B (en) | Remote sensing image change detection method and device based on depth characteristics | |
| CN112132798B (en) | Method for detecting complex background PCB mark point image based on Mini ARU-Net network | |
| CN111339974B (en) | Method for identifying modern ceramics and ancient ceramics | |
| CN111754427B (en) | Automatic repair method for mural crack diseases based on self-organizing mapping neural network | |
| CN115849202B (en) | Intelligent crane operation target identification method based on digital twin technology | |
| CN116051808A (en) | YOLOv 5-based lightweight part identification and positioning method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |