CN104091319A - Shredded paper picture splicing method for establishing energy function based on Monte Carlo algorithm - Google Patents

Abstract

The invention provides a mosaic method of shredded paper pictures based on the Monte Carlo algorithm to construct an energy function, which mainly involves the mosaic and restoration of double-sided printed documents. Usually, due to the large number of pictures and the large amount of information, it is usually a non-linear optimization. The problem is that it is difficult to accurately establish a model, and it is difficult to solve it, and the error may be large. Therefore, the present invention regards the image as a whole, and uses a Monte Carlo algorithm based on random thinking to select and fill. Consider how a given piece of shredder shredded paper from the same page of printed text is spliced together, including: only slits, both slits and cross-cuts, double-sided printing documents and slits with cross-cuts, etc. Scraps of paper, possibly including Chinese or English. Through the picture splicing algorithm, the present invention can automatically splice shredded paper to obtain picture splicing and restoration effects, reduce the consumption of manpower and material resources, and improve splicing and restoration efficiency.

Description

Shredded paper image mosaic method based on Monte Carlo algorithm to construct energy function

technical field

The invention belongs to the field of information technology, and relates to a mosaic method for shredded paper pictures based on a Monte Carlo algorithm to construct an energy function. Spend.

Background technique

The splicing of broken files has important applications in the fields of document restoration, judicial evidence restoration and identification, historical document restoration, and military intelligence acquisition. Many fragment splicing problems can be attributed or approximated to the splicing of two-dimensional fragments. Shredded paper stitching is a typical problem of 2D fragmented image stitching. Traditionally, splicing and restoration work has to be done manually, with high accuracy but low efficiency. However, when the number of fragments is huge and consumes a lot of manpower and material resources, it is difficult for manual splicing to complete the task quickly and accurately in a short period of time, and it may also cause certain damage to the object.

With the development of computer technology, using computer programming technology and image splicing algorithm, the automatic splicing of shredded paper can be carried out to obtain image splicing and restoration, reduce the consumption of manpower and material resources, and improve the efficiency of splicing and restoration.

Contents of the invention

The purpose of the present invention is to provide a shredded paper image mosaic method based on Monte Carlo algorithm to construct an energy function, aiming to solve the problems of NP-difficulty in image mosaic and low mosaic restoration efficiency in the prior art.

The present invention is achieved in this way, a method for mosaicing shredded paper pictures based on a Monte Carlo algorithm to construct an energy function, comprising the following steps:

S1. Scanning the scraps of paper into two-dimensional grayscale pictures to obtain (m×n) pieces, and using Matlab to read the picture information into matrix information;

S2. Using the Matlab random function randperm to generate a random, non-repetitive two-dimensional combination sequence of m×n image fragments for the above-mentioned generated pictures based on the Monte Carlo algorithm;

S3. For the two-dimensional combination sequence of m×n fragments as the file picture to be generated, based on the root mean square error RMSE, calculate the energy function of each image in the picture and the adjacent pictures, that is, up, down, left, and right, and then find out The sum of all image energy functions;

S4. Perform 10,000 cycles of step S2, and compare the size of each energy function to obtain the minimum value of the energy function;

S5. The fragment position corresponding to the obtained minimum value is the optimal arrangement mode;

S6. Through a large number of iterations, the minimum value gradually converges to obtain the optimal value, that is, the best puzzle effect.

The present invention overcomes the deficiencies of the prior art and provides a mosaic method for shredded paper pictures based on the energy function constructed by the Monte Carlo algorithm, which mainly involves the mosaic and restoration of double-sided printed documents. Therefore, it is usually a nonlinear optimization problem, and it is difficult to accurately establish a model, and it is difficult to solve it, and at the same time, the error may be large. Therefore, the present invention regards the image as a whole, and uses a Monte Carlo algorithm based on random thinking to select and fill. Consider how a given piece of shredder shredded paper from the same page of printed text is spliced together, including: only slits, both slits and cross-cuts, double-sided printing documents and slits with cross-cuts, etc. Scraps of paper, possibly including Chinese or English. The specific analysis of the three different situations is as follows: 1) For files that are only longitudinally cut, the left and right constraint association conditions between each fragment in the file are considered at the same time; 2) For files that include cross-cut and longitudinal 4 constraint association conditions of upper, lower, left, and right among fragments; 3) For double-sided files, consider the 8 constraint association conditions of each fragment in the file at the same time, and then establish an energy function , usually the smallest energy function means the best splicing effect. Find the minimum value through the random nature of Monte Carlo, and finally use Matlab programming to obtain the optimal solution and verify it.

Description of drawings

Fig. 1 is the flow chart of the steps of the shredded paper picture splicing method based on the Monte Carlo algorithm to construct the energy function of the present invention;

Fig. 2 is a schematic diagram of energy variation along with 1000 iterations in the embodiment of the present invention;

Fig. 3 is a schematic diagram of energy variation along with 10000 iterations in the embodiment of the present invention;

Fig. 4 is a schematic diagram of the relationship between top, bottom, left and right of splicing fragments in the embodiment of the present invention.

Fig. 5 is the histogram of the 5.000.bmp image in the embodiment of the present invention;

Fig. 6 is a horizontal line diagram in the embodiment of the present invention;

Fig. 7 is a point diagram in the embodiment of the present invention;

Fig. 8 is a vertical view of an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example

A kind of mosaic method of shredded paper picture based on Monte Carlo algorithm constructing energy function, as shown in Figure 1, comprises the following steps:

S1. Scan the shredded paper into a two-dimensional grayscale picture form (m×n), and use Matlab to read the picture information into matrix information;

In step S1, more specifically include:

1) Image preprocessing

Ordinary images cannot be used directly, because there are different information such as noise and grayscale, and direct use will cause errors, resulting in incorrect results or incorrect splicing, so the image needs to be preprocessed:

a) Denoising processing

The given image may have noise problems due to imaging, camera or computer reasons, and noise will easily cause data processing errors. For example, if a pixel in the last column is affected by noise and 0 becomes 1, then the final statistics will cause A certain error, so the commonly used Gaussian filter is used to denoise the image:

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;\&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

b) Binarization processing

The image given in this article is a grayscale image, ranging from 0 to 255. However, the grayscale image is usually 8 bits and requires a large amount of calculation. Therefore, it is usually binarized to reduce the amount of calculation and speed up the execution.

First of all, the commonly given printing paper image is a grayscale image (0-255), and it is found that the image information value is not only 0 and 255, but includes many values between 0-255, But there should be a greater probability at 0, 255. The imhist function in Matlab is used. Since there are more values of 0 and 255 points and less other points, the line shape is thickened by 5.0 times, as shown in Figure 2. It can be seen from Figure 2 that the gray value Mainly concentrated between 0, 255 and 0 ~ 255, so the threshold value is 180 according to experience. That is, if the gray value is between 0 and 180, the gray value is 0 after binarization processing; if the gray value is greater than 180 and less than or equal to 255, the gray value is 1 after binarization processing. Right now:

2) Image feature analysis

It can be seen from the analysis that in image stitching, the presence of text in the middle of the image has no effect on the stitching of the image, so only the text features at the left and right borders of the image need to be considered. The n gray value matrix generated after importing the image into Matlab software only needs to use the elements in the first column and the last column for data similarity analysis, and the elements in other columns can be ignored.

The feature at the boundary is collected, and the grayscale-based image binarization fixed threshold method and the mean square error statistical matching method are adopted. For the possible errors, the verification is as follows:

Common strokes in Chinese are: horizontal (一), vertical (丨), left (丿), dot (丿), right down (乀), folding (乛), etc., which can be divided into two categories:

a) Adjacent point or certain points, such as: stroke (1) is vertically cut off from the middle arrow, forming:

1. Horizontal usually 3. point (with cast aside, the same structure), as shown in Figure 3;

2. The stroke ( ) is cut off vertically from the middle arrow, as shown in Figure 4.

b) Adjacent multiple points

3. The stroke (丨) is vertically truncated from the middle arrow, as shown in Figure 5.

In summary, by counting the common strokes, it is found that if a character is disconnected from the middle, the grayscale of adjacent picture pixels is usually similar, so the method of mean square error is theoretically feasible. After that, it is feasible to choose to use Matlab software and select the features at the acquisition boundary to perform image binarization based on grayscale, fixed threshold method and mean square error statistical matching method. The error should be small and more correct results can be obtained. .

S2. Using the Matlab random function randperm based on the Monte Carlo algorithm to generate non-repetitive m×n image fragments;

S3. For the m×n fragments as the file picture to be generated, calculate the energy function of the upper, lower, left and right of each image in the picture, and find the sum of the energy functions of all pictures;

In step S3, more specifically include:

1) Grayscale-based mean square error statistical matching algorithm

Because the presence or absence of text in the middle of the picture has nothing to do with picture splicing, the features at the left, right, upper and lower boundaries of the picture are selected for matching. However, there may be a phenomenon that the gray value at the border is similar, but the picture does not match. Therefore, the feature at the acquisition boundary is selected for statistical matching based on the mean square error of the gray level. Common measures of judgment error include:

a) standard deviation

The standard deviation reflects the dispersion of the image grayscale relative to the grayscale average value, which is defined as follows:

$\mathrm{<span\; class="notranslate">\; Std</span>}<span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

in, is the mean value of the image F, defined as:

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Standard deviation can also be used to evaluate the size of image contrast. If the standard deviation is large, the gray level distribution of the image is dispersed, and the contrast of the image is large, so more information can be seen. The standard deviation is small, the image contrast is small, the contrast is not large, the tone is single and uniform, and there is not much information to be seen.

b) root mean square error RMSE

The root mean square error between the fused image F and the standard reference image R is defined as:

$\mathrm{<span\; class="notranslate">\; RMSE</span>}<span\; class="notranslate">\; =</span>\sqrt{\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Among them, M and N are the number of rows and columns of the image, respectively. Here, considering the real-time problem, it is better to use root mean square error.

2) specific algorithm

The mean square error of the corresponding difference between the first (last) column of any matrix and the last (first) column of other matrices is obtained by operation. When the probability of 0 elements appearing in the difference is greater, the gray The smaller the difference of the degree value is, the higher the similarity degree is at the boundary of the matrix corresponding to the picture, so the possibility of splicing the corresponding picture is greater. However, this method may have the same probability of 0 elements appearing, the gray value is different, and the picture does not match. Therefore, I chose to abandon the method of subtraction and use the mean square error for comparison. The matrix with the smallest variance corresponds to the image with the smallest gray value difference and the highest image matching degree. Therefore, the two images can be spliced and restored.

Usually it is more difficult to give double-sided Chinese and English words, so the double-sided is used as a specific analysis, and the other forms are simplified forms. For double-sided printing files, there are usually 2×m×n images in total, and the height of each image is M pixels.

Here, the idea of constructing a data-based energy function (cost function) is adopted, the Monte Carlo algorithm based on random thought is adopted, and Matlab is used to solve it, and finally analyzed and verified.

a) Algorithm Analysis

Assuming that the printing paper page is divided into two sides, there are a total of m×n fragments, it can be considered that if the matching is correct, the matching of the top, bottom, left and right of each fragment should be very good, and the corresponding mean square error should be the smallest , so that the mean square error between all fragments should be relatively small or minimal (considering the existence of possible errors), based on this idea, an energy function is established, and the energy function is based on the upper, lower, left, and right distance between the connected pictures. The mean square deviation matching relationship of , as shown in Figure 6, includes two situations, negative and positive, so there are a total of 8 constraints.

b) Energy function establishment

Based on the idea of image matching and the above analysis, the energy function based on the relationship between up, down, left and right of each image is constructed. Assuming that the pictures are pixel sets X _up , X _center , X _down , X _left , and X _right , then for The energy function of a picture can be obtained,

f(x ₁ ,x ₂ ,x ₃ ,x ₄ ,x ₅ )＝Ψ(x ₁ ,x ₂ )+Ψ(x ₁ ,x ₃ )+Ψ(x ₁ ,x ₄ )+Ψ(x ₁ , x ₅ )

Similarly, the global energy function can be obtained

F(x ₁ ,x ₂ ,x ₃ ,x ₄ ,x ₅ )＝Σ(Ψ(x ₁ ,x ₂ )+Ψ(x ₁ ,x ₃ )+Ψ(x ₁ ,x ₄ )+Ψ(x ₁ , x ₅ ))

Therefore, for the global energy function of the entire image

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; \&Psi;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\text{<span class="notranslate"> q</span>}}<span\; class="notranslate">\; )</span>$

where Ψ( ) is the mean square error measure between adjacent pictures $\mathrm{\&Psi;}(x_i,x_j)=\sqrt{x_i(:,1)-x_j(:,\mathrm{end})},$ N is the neighborhood of q.

S4. Perform 10,000 cycles of step S2, and compare the size of each energy function, and take the minimum value;

S5. The fragment position corresponding to the obtained minimum value is the optimal arrangement mode;

In step S5, when there is no match or disorder, the energy function is relatively large, and when the match is good, the energy function is relatively small, so that the matching problem is converted into a problem with the smallest energy, then it should be a search for the optimal solution at this time question. Right now:

${\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; .</span>$

S6. Through a large number of iterations, the minimum value gradually converges to obtain the optimal value, that is, the best puzzle effect.

Take a picture as an example for restoration stitching, as shown in Figures 7 and 8. From the figure, we can see that as the iteration ends, the initial value of the lowest energy is 341.1. As the iteration progresses, the lowest energy function is run 1000 times The value is 332.2; the lowest energy function value is 328.3 after running 10,000 times, and it is gradually converging from the image, converging on a stable value, that is, the optimal solution is obtained.

Compared with the shortcomings and deficiencies of the prior art, the present invention has the following beneficial effects: through the image splicing algorithm, the shredded paper can be spliced automatically to obtain the effect of splicing and restoration of pictures, reduce the consumption of manpower and material resources, and improve the efficiency of splicing and restoration .

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (1)

1. A shredded paper picture splicing method based on Monte Carlo algorithm construction energy function, it is characterized in that comprising the following steps: S1. Scanning the scraps of paper into two-dimensional grayscale pictures to obtain (m×n) pieces, and using Matlab to read the picture information into matrix information; S2. Using the Matlab random function randperm to generate a random, non-repetitive two-dimensional combination sequence of m×n image fragments for the above-mentioned generated pictures based on the Monte Carlo algorithm; S3. For the two-dimensional combination sequence of m×n fragments as the file picture to be generated, based on the root mean square error RMSE, calculate the energy function of each image in the picture and the adjacent pictures, that is, up, down, left, and right, and then find out The sum of all image energy functions; S4. Perform 10,000 cycles of step S2, and compare the size of each energy function to obtain the minimum value of the energy function; S5. The fragment position corresponding to the obtained minimum value is the optimal arrangement mode; S6. Through a large number of iterations, the minimum value gradually converges to obtain the optimal value, that is, the best puzzle effect.