JP2015022605A - Image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a versatile image processing system capable of handling even in the case that the cause of image quality deterioration of an input image is various in an image processing system for performing image correction.SOLUTION: The image processing system for correcting an inputted image includes input means 11 for inputting an image, histogram generation means 14 for generating a histogram on the basis of an input image being the inputted image, modeling means 16 for modeling the generated histogram by using a probability distribution, correction means 17 for correcting the input image on the basis of a statistical model modeled by the modeling means 16, and output means 19 for outputting the image from the correction means 17 as an output image.

Description

  The present invention relates to an image processing system for correcting an image.

  Images such as printed materials and historically famous paintings may change significantly from the original image due to changes in the paint over time or the adhesion of foreign objects, etc. Even with the difference, the state changes greatly with respect to the original image, and such a change in state greatly reduces the quality (image quality) of the image.

  As a technique for returning an image whose image quality has been lowered in this way to a state close to the original image, an input unit that inputs an image, a correction unit that corrects the input image that is the input image, and a correction unit An image processing system including an output unit that outputs an image as an output image is known (for example, see Patent Document 1).

JP 2011-166638 A

  In the image processing system of the above document, the correction means performs image contrast correction processing and noise reduction processing. If the correction content corresponds to the cause of the image quality degradation of the input image, the correction means While the correction effect can be expected, for example, when the correction content does not correspond to the cause of the image quality deterioration, such as when the input image is partially missing from the original image, the correction effect by the correction means is Almost no expectation and low versatility.

  An object of the present invention is to provide a highly versatile image processing system that can cope with various causes of image quality degradation of an input image in an image processing system that performs image correction.

  In order to solve the above problems, first, an image processing system for correcting an input image, which generates an histogram based on an input means 11 for inputting an image and an input image which is the input image. Histogram generation means 14, modeling means 16 for modeling the generated histogram using a probability distribution, and correction for correcting the input image based on the statistical model modeled by the modeling means 16 It is characterized by comprising means 17 and output means 19 for outputting the image from correction means 17 as an output image.

  Second, the modeling means 16 performs the above modeling using a mixed Gaussian distribution.

  Third, a parameter determining means for determining parameters included in the mixed Gaussian distribution using a variational Bayes method is provided.

  Fourth, the image processing apparatus includes a dividing unit 13 that divides a color image that is an input image into a plurality of predetermined reference colors, and a combining unit 18 that combines the plurality of reference color images divided by the dividing unit. The histogram generation means 14 generates a histogram of each reference color image, the modeling means 16 performs the above modeling of the histogram for each reference color image, and the correction means 17 performs the above correction for each reference color image. The synthesis unit 18 synthesizes the reference color images corrected by the correction unit 17.

  Fifth, the reference color is three colors of red, green and blue.

  The histogram generation means generates a histogram based on the input image, models the histogram by the modeling means using the probability distribution, and corrects the input image based on the modeled statistical model. . At this time, the statistical model greatly contributes to the histogram estimation of the original image and can cope with various causes of image quality deterioration, so that versatility is improved.

(A) An image of a photograph of a photographed two-dimensional color barcode, and (B) is an image obtained by correcting distortion of the photograph of (A). It is a block diagram which shows the structure of the image processing system to which this invention is applied. It is a block diagram which shows the content of the image process by a control part. It is a histogram in one reference color of the target image and a statistical model of the histogram. It is explanatory drawing which shows the structure of a two-dimensional color barcode by a specific example. It is an abstract conceptual diagram of a two-dimensional color barcode. It is a conceptual diagram which shows the specific structure of a two-dimensional color barcode. It is a conceptual diagram of the update of the parameter and hyperparameter using variational Bayes.

  FIG. 1A is an acquired image of a photograph of a photographed two-dimensional color barcode, and FIG. 1B is a corrected image obtained by performing distortion correction on the photograph of FIG. Two-dimensional barcodes typified by QR code (registered trademark) and the like are classified into matrix codes and stack codes, but the example shown is a matrix code, and is divided into a matrix in a long direction or a square. Information is embedded in a gray scale or color (color in this example) dot pattern drawn in the region.

  For such a two-dimensional color barcode image, first, information to be embedded is selected in a computer, two-dimensional barcode image data (original image) in which this information is embedded is generated, and this is converted into paper or the like. Print on other print media. Then, the user captures an image (acquisition target image) printed on the print medium with a camera or the like, or reads it with a scanner.

  In this way, the image data (acquired image) 1A acquired as digital data is distorted due to the shooting angle or the unevenness of the print medium. Therefore, distortion correction is performed, and the image quality of the image after the distortion correction is improved. Image data (target image) 1B to be subjected to image processing.

  In this series of steps, the image quality of the target image 1B is changed to the original image by the difference in the color scheme of the printer used for printing on the print medium, the change of the paint after printing, the environment at the time of photographing, and the distortion correction. Deteriorated compared to. For this reason, if the information embedded in the two-dimensional barcode is extracted using the target image 1B as it is, the information embedded in the two-dimensional barcode may not be extracted well due to the deterioration of the image quality. .

  An image processing system to which the present invention is applied is used as a tool for preventing such a situation. Specifically, image processing is performed to make the target image 1B closer to the image quality of the original image, and the embedded information is extracted from the corrected two-dimensional barcode image.

  FIG. 2 is a block diagram showing a configuration of an image processing system to which the present invention is applied. The image processing system 2 is mounted on a computer (image processing apparatus).

  The image processing system 2 includes an image acquisition device 3 that includes a camera that captures an acquisition target image or a scanner that reads the image, a control unit 4 that includes a CPU 4a, a RAM 4b, and the like, and that performs various processes including image processing. It consists of a storage device 6 such as an HDD or SSD for storing data, a communication means 7 for communicating with other computers via a communication network such as the Internet or a private network, and a mouse, keyboard or touch panel for performing various operations. An operation unit 8 and a monitor 9 for displaying various information including the acquired image 1A and the target image 1B on the screen are provided.

  FIG. 3 is a block diagram showing the contents of image processing by the control unit, and FIG. 4 is a histogram of one reference color of the target image and a statistical model of the histogram. The image processing performed by the control unit 4 includes an image input unit (input unit) 11 that receives the acquired image 1A as an input image, a distortion correction unit 12 that receives the input image and corrects the distortion of the input image, and A reference color dividing means (dividing means) 13 for receiving an image whose distortion has been corrected by the distortion correcting means 12 and dividing the image into a plurality of predetermined reference colors (red, green and red in the illustrated example); Histogram generating means 14, modeling means 16 and correcting means 17 provided for each reference color, combining means 18 for combining a plurality of corrected reference colors, and corrected image combined by combining means 18. And an image output means (output means) 19 for outputting as an output image.

  The image input unit 11 acquires the acquired image 1A from the image acquisition device 3, acquires the acquired image 1A from the state stored in the storage device 6, or acquires it from another computer via the communication unit 7. Image 1A is acquired.

  The distortion correction means 12 is as described above, and performs distortion correction so that an image as shown in FIG. 1A becomes an image as shown in FIG. Then, the image is transferred to the reference color dividing unit 13 as the above-described target image 1B.

  The reference color dividing unit 13 includes a reference color extracting unit 21 provided for each of a plurality of reference colors. Each reference color extraction unit 21 extracts a reference color that it is in charge of, and passes the extracted image as a reference color image to the histogram generation unit 14 that is in charge of the reference color.

  As described above, the histogram generating means 14 is provided for each reference color. Each histogram generation unit 14 generates a histogram of the reference color image in the reference color based on the reference color image passed from the reference color extraction unit 21 in charge of the same reference color. In the reference color image, each pixel is expressed by luminance (brightness and darkness) of a plurality of steps of the reference color (usually 256 steps from 0 to 255), and the horizontal axis of the histogram indicates each step (each gradation) of the reference color. The vertical axis represents the number of pixels included in the reference color image in a certain gradation, and a histogram showing the total number of images in each gradation is a histogram.

  For example, in the example illustrated in FIG. 4, the target image 1 </ b> B includes a pixel having a low luminance of the reference color constituting the reference color image and a pixel having a high luminance and a low number of medium luminance pixels. It has become.

  Then, when each histogram generation unit 14 generates a histogram of the reference color image, the histogram generation unit 14 passes the histogram to the modeling unit 16 in charge of the same reference color.

  The modeling means 16 is provided for each reference color as described above. Each modeling unit 16 models the histogram passed from the histogram generation unit 14 using a probability distribution, and generates a statistical model. The probability distribution used in this case is most preferably a confusion Gaussian distribution, and modeling is performed by regarding the number of pixels in each gradation as the existence probability in each gradation.

  The mixed Gaussian distribution has a plurality of parameters, but these parameters are estimated from actual histogram information (actual data) using Bayes' theorem (specifically, calculated using variational Bayes). This generates a statistical model. That is, the modeling unit 16 also functions as a parameter determination unit that determines the parameters of the statistical model.

  Then, each modeling unit 16 generates the statistical model as described above, and passes the statistical model to the correction unit 17 in charge of the same reference color.

  The correction means 17 is provided for each reference color as described above. In addition to receiving the statistical model from the modeling unit 16, each correction unit 17 receives the reference color image and its histogram from the reference color extraction unit 21 and the histogram generation unit 14 in charge of the same reference color.

  Then, each correcting unit 17 corrects the received reference color image based on the received histogram and statistical model, and passes it to the synthesizing unit 18. Specifically, correction (image processing) of the reference color image is performed so as to approximate the statistical model generated by the histogram of the reference color image as much as possible.

  The synthesizing unit 18 receives the corrected images from the respective correcting units 17 and synthesizes them, and the synthesized image is used as an image processed by the image processing system 1 (processed image). Pass to 19.

  The image output means 19 outputs the processed image as an output image to the monitor 9, stores it in the storage device 6, or sends it to another computer via the communication means 7.

  Next, the configuration of the two-dimensional color barcode will be described with reference to FIGS.

  FIG. 5 is an explanatory diagram showing the configuration of the two-dimensional color barcode by a specific example. It is explanatory drawing which shows the structure of a two-dimensional color barcode concretely. The two-dimensional color barcode (two-dimensional barcode) 22 drawn in the target image 1B has, for example, nine cells partitioned in a matrix of 3 rows and 3 columns, as shown in FIG. Painted in colors (white, blue, yellow, green). In this way, the two-dimensional color barcode 22 that is separately colored with the color of 2 to the power of k (k is an integer equal to or larger than 2 and k = 2 in the figure) is binarized into k two-dimensional two-dimensional color codes. It can be decomposed into a valued barcode (two-dimensional barcode) 23.

  For example, the two-dimensional color barcode 22 shown in the figure has a matrix of 3 rows and 3 columns, because each of the nine cells divided into a matrix of 3 rows and 3 columns is colored with four colors. It can be converted into a two-layer multilayered two-dimensional barcode 24 in which each layer is constituted by a two-dimensional binarized barcode 23 which is partitioned and each cell is represented by one of two colors.

  The information of the multi-layered two-dimensional barcode 24 describes the color map M stored in the storage device 6 or another computer. Based on the information of the color map M and the color information of each cell of the two-dimensional color barcode 22. The color information of each cell constituting each two-dimensional binarized barcode 23 is acquired.

  For example, according to the color map M in the figure, when the color of the cell of the two-dimensional color barcode 22 is white, the cell at that portion of the two-dimensional binarized barcode 23 is in both the first and second layers. When the color of the cell of the two-dimensional color barcode 22 is blue and the cell at that portion of the two-dimensional binarized barcode 24 is blue, the first layer is blue and the second layer is white, When the color of the cell of the two-dimensional color barcode 22 is yellow, the cell at that portion of the two-dimensional binarized barcode 23 becomes white in the first layer and yellow in the second layer. When the color of the cell of the code 22 is green, information indicating that the cell at that portion of the two-dimensional binarized barcode 23 is blue in the first layer and yellow in the second layer is stored.

  The same applies when the conversion from the two-dimensional color barcode 22 to the multilayered two-dimensional barcode 24 is performed and when the conversion from the multilayered two-dimensional barcode 24 to the two-dimensional color barcode 22 is performed. By using the color map M, correct information can be extracted from the two-dimensional color barcode 22.

  Specifically, information embedded in the two-dimensional color barcode 22 is obtained by extracting information from each two-dimensional binarized barcode and combining the extracted plurality of information based on a predetermined rule. It can be taken out.

  FIG. 6 is an abstract conceptual diagram of a two-dimensional color barcode. In the two-dimensional color barcode 22 of the present invention, generally, each of l × m cells divided in a matrix of l rows and m columns is represented by one of 2k color colors ( l and m are integers of 2 or more, and may be different from each other or the same value).

  Therefore, each layer of the two-dimensional color barcode 22 is constituted by a two-dimensional binarized barcode 23 in which each cell partitioned in a matrix of l rows and m columns is represented by one of two colors. Conversion into k-layered two-dimensional barcode 24 is possible.

  FIG. 7 is a conceptual diagram showing a specific configuration of the two-dimensional color barcode. As shown in the figure, the generated two-dimensional color barcode 22 has a square shape having four sides, and a belt-like region 26 is formed along the four sides. The belt-like region 26 is formed in a rectangular frame shape so as to surround the four sides from the outside in order to easily specify the region at the time of capturing an image.

  It consists of an upper side portion 26a along the upper side of the four sides, a lower side portion 26b along the lower side of the four sides, a left side portion 26c along the left side of the four sides, and a right side portion 26d along the right side of the four sides. In addition, the inner areas of the four sides are arranged in predetermined predetermined positions and displayed in a rectangular cell (three in the illustrated example) 27 finder patterns 27 and 2 k-colored colors. The information storage area 28 is composed of the dot patterns shown in FIGS.

  In the belt-like region 26, a plurality of reference colors are arranged. The k two-dimensional binarized barcodes 23 constituting the multi-layered two-dimensional barcode 24 are painted in two colors of white and a predetermined color that differs depending on each layer, and the reference color is the predetermined color. Shows the color. That is, there are k reference colors, and the image capture accuracy of the two-dimensional color barcode 22 is improved.

  In addition, the k reference colors are arranged on any one of the upper side 26a, the lower side 26b, the left side 26c, and the right side 26d, and which reference color is arranged on which side 26a, 26b, 26c, 26d. The color arrangement information is stored in the storage device 6 or another computer. Incidentally, when k = 4, the basic colors of the side portions 26a, 26b, 26c, and 26d are matched as much as possible, for example, one basic color is arranged on each side portion 26a, 26b, 26c, and 26d. It is preferable.

  The direction of the two-dimensional color barcode 22 can be specified based on the color arrangement information by identifying to which side 26a, 26b, 26c, 26d the predetermined reference color is arranged. Become.

  The finder pattern 27 may be omitted, but includes an identifier related to the above-described two-dimensional color barcode 22, and information such as the number of rows and columns constituting the cell of the two-dimensional color barcode 22.

  Next, a specific method for generating a statistical model performed by the modeling unit 16 will be described in detail.

  The mixed Gaussian distribution used for modeling each of the histograms can be considered as a superposition of K Gaussian distributions, and is expressed by the following equation.

  N is an individual Gaussian distribution, K is the number of superposed Gaussian distributions, actual pixel data of a two-dimensional barcode, μ is an average vector, Λ is an example of covariance row, and π is It is a mixing ratio and follows the following formula.

  In the statistical model using the mixed Gaussian distribution, when the mixture ratio, mean vector, and covariance matrix are set as the latent parameter Z, the Bayes' theorem is to maximize the posterior distribution using the prior distribution and likelihood. Think. Specifically, the natural logarithm of p (X) described above is expressed by the following expression using a new distribution q using the variational Bayes variation approximation method.

  However, L (q) is a variational lower bound, and KL (q‖p) is a KL distance, each represented by the following formula.

  Here, the parameter update is performed by alternately repeating the minimization of the KL distance and the recalculation of L (q). Incidentally, the KL distance is minimized, and the burden rate of each of the plurality of Gaussian distributions to be superimposed is updated using the limited q (Z) shown below.

However, r _nk is given by the following equation using ρ constituting each component of the covariance matrix.

  A conjugate prior distribution is used as the prior distribution necessary for the calculation of the KL distance minimization. Specifically, the mixing ratio π is obtained by the following formula using a directory distribution. Incidentally, this makes it possible to calculate a new distribution q of the mixing ratio.

  In addition, the mean vector and the covariance matrix can be obtained by the following formula by introducing a Gauss-Wishart prior distribution. However, the function of W shows the Wishart distribution. Incidentally, this makes it possible to calculate a new distribution q of the mean vector and the covariance matrix.

  From the above, the following five hyper parameters are found.

  Each of these hyper parameters and each parameter of the statistical model of the above-mentioned mixed Gaussian distribution is sequentially updated with the update of the burden rate of each of the K number of Gaussian distributions if the initial value of each hyper parameter is determined. . FIG. 8 shows a conceptual diagram of updating parameters and hyperparameters using variational Bayes. For this reason, it is necessary to select an initial value. However, m is fixed with the initial state being a zero vector, W is fixed with the initial state being a unit matrix, and initial values are appropriately determined for the remaining three hyperparameters.

  The inventors of the present application have determined initial values of the three hyperparameters as follows as a result of keen examination. Incidentally, if there is an unnecessary Gaussian distribution, its ratio is automatically lowered. Therefore, K indicating the number of Gaussian distributions to be superimposed may be arbitrarily determined.

  By setting the initial values, it was confirmed that each parameter converged appropriately with repeated calculation.

  According to the image processing system configured as described above, it is possible to cope with a case where a part of an image is missing as well as fading, a difference in a color scheme pattern of a printer, and the image quality close to the original image. Therefore, it is easy to extract correct information even from a deteriorated two-dimensional color barcode.

  In addition, although the case where the image of the target image 1B is a two-dimensional color barcode image has been described, the case where the image of the target image 1B is a two-dimensional grayscale barcode or a two-dimensional monochrome barcode is also described. Applicable.

  Specifically, these images are regarded as grayscale grayscale images. In FIG. 3, the reference color dividing unit 13 and the combining unit 17 are omitted, and the histogram teacher unit 14, the modeling unit 16 and the correction unit 17 are replaced. If the target image 1B from the distortion correction unit 12 is directly output to the histogram generation unit 14 and the image from the correction unit 17 is directly passed to the image output unit 19 as a processed image. Good.

  The target image 1B is not limited to a two-dimensional barcode, and may be a painting or other general color image or grayscale image.

2 Image processing system 11 Image input means (input means)
13 Standard color dividing means (dividing means)
14 Histogram generation means 16 Modeling means 17 Correction means 19 Image output means (output means)
18 Synthesis means

Claims (5)

An image processing system for correcting an input image,
An input means for inputting an image;
Histogram generating means for generating a histogram based on the input image which is the input image;
Modeling means for modeling the generated histogram using a probability distribution;
Correction means for correcting the input image based on the statistical model modeled by the modeling means;
An image processing system comprising: output means for outputting an image from the correction means as an output image. The image processing system according to claim 1, wherein the modeling unit performs the modeling using a mixed Gaussian distribution. The image processing system according to claim 2, further comprising parameter determination means for determining a parameter included in the mixed Gaussian distribution using a variational Bayes method. A dividing means for dividing a color image as an input image into a plurality of predetermined reference colors;
Combining means for combining a plurality of reference color images divided by the dividing means,
The histogram generation means generates a histogram of each reference color image,
The modeling means performs the modeling of the histogram for each reference color image,
The correction means performs the correction for each reference color image,
The image processing system according to claim 1, wherein the combining unit combines the reference color images corrected by the correcting unit. The image processing system according to claim 4, wherein the reference colors are three colors of red, green, and blue.

Images