JP2015156539A - Image restoration device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to remove unnecessary information included in an image.

SOLUTION: An input interface unit 20 acquires, as an observation image, a surface image obtained by reading one surface of a document. A deteriorated image information determination unit 62 calculates a variance value on the basis of pixel values of respective pixels in the observation image and determines, for each pixel in the observation image, whether a set-off is mixed in the pixel or not on the basis of a threshold calculated from the variance value. An estimation unit 65 estimates, on the basis of a calculated state amount and calculated observation amount for each processing region n in the observation image, an optimum value of a state amount of the processing region n as an original image of the processing region n, by using a prediction method based on a state space model.

COPYRIGHT: (C)2015,JPO&INPIT

Description

  The present invention relates to an image restoration apparatus, method, and program.

  2. Description of the Related Art Conventionally, in the field of image engineering, much research and development has been made on a technique for restoring an original image from a deteriorated image. In other words, a technique has been proposed in which original image information (desired information) is mixed with deterioration information (blur, noise, show-through, etc.) and the original image is restored by removing the deterioration image information from the deterioration image information (acquired information). ing. Among them, when a double-sided printed document is read using a scanner or the like and converted into a digital image, there is a phenomenon in which show-through occurs because information on the back side is reflected on the front side. This show-through problem is a problem that attracts attention as the database of information in recent years progresses, and a technique for solving this problem has been proposed.

  For example, in order to remove the show-through (deterioration information) when scanning a document with images printed on both sides, a correction method has been proposed that uses gamma correction to adjust the density of the entire image and remove the show-through. (For example, refer nonpatent literature 1).

  In addition, in order to remove show-through (deterioration information) when a document is scanned, an image reading device that obtains digitized image information by reading the front and back sides of a double-sided document, and the image reading device An image processing apparatus comprising: image storage means for temporarily storing image information; and image processing means for performing image processing on the image information stored in the image storage means and outputting image information after image processing In addition, according to the image information of the document to be read set as parameters in advance and the brightness information of the back side of the document read by the image reading device, the show-through information due to the back side appearing on the front side of the read document is removed. There has been proposed an image processing apparatus including a show-through information removing unit.

Kenji Sugiyama, “Practical Video Signal Processing”, 2008, Corona

JP 2009-224873 A

  However, according to the technique described in Non-Patent Document 1, correction processing is performed even on a light color of the surface image information among the image information on both sides, so that clear show-through removal is difficult for a color image or the like. It is. In addition, there is a problem that it is difficult to uniquely determine a parameter that affects the correction amount.

  Further, in the technique described in Patent Document 1, since the amount of change in luminance value due to edge extraction is used as a parameter, it is difficult to process show-through information over a wide range where edge extraction is difficult. Further, there is a problem that it is difficult to optimally determine the threshold value used in the binarization process.

  The present invention has been made in view of the above circumstances, and provides an image restoration apparatus, method, and program capable of removing degraded image information including blur, noise, and show-through image information included in image information. For the purpose.

  In order to achieve the above object, an image restoration apparatus according to a first aspect of the present invention provides a prediction based on a state space model including the original image information as a drive source for observed image information in which blur and noise are mixed in the original image information. An image restoration device that estimates the original image information from the observed image information using image processing, and obtains surface image information obtained by reading a document as the observed image information, and the observation Determining means for calculating a variance value based on a pixel value of each pixel of image information, and determining whether blur and noise are mixed in each pixel based on a threshold value calculated from the variance value; The state quantity in the processing area n at time n of the observed image information is the state quantity in the processing area (n-1) at time (n-1), and the state from the processing area (n-1) to the processing area n. Transition, and said A state quantity calculating means for calculating using the drive source including an element of the state quantity of the processing area n at the time n not included in the state quantity of the physical area (n−1), and a state quantity of the processing area n at the time n Based on the above, for the pixel value of each pixel in the processing region n determined to have blur and noise mixed by the determination means, observe the process of adding blur and noise to the original image information to become observation image information The state space model based on the observation amount calculation means for calculating the amount, the processing region n at time n based on the state amount calculated by the state amount calculation means, and the observation amount calculated by the observation amount calculation means. And an estimation means for estimating an optimum value of the state quantity of the processing region n at time n as original image information by a prediction method based on the prediction method.

  An image processing method according to a second invention uses a prediction method based on a state space model including the original image information as a drive source for the observed image information in which blur and noise are mixed in the original image information, and the observed image information An image restoration apparatus for estimating the original image information from the image restoration method in the image restoration apparatus including an image acquisition means, a determination means, a state quantity calculation means, an observation amount calculation means, and an estimation means, The image acquisition unit acquires surface image information obtained by reading a document as the observation image information, and the determination unit calculates a variance value based on a pixel value of each pixel of the observation image information. Calculating, based on a threshold value calculated from the variance value, determining whether blur and noise are mixed in each pixel; and the state quantity calculating means includes the observation image The state quantity in the processing area n at time n in the report is the state quantity in the processing area (n-1) at time (n-1), the state transition from the processing area (n-1) to the processing area n, And the step of calculating using the drive source including an element of the state quantity of the processing region n at time n not included in the state quantity of the processing region (n−1), and the observation amount calculating means at the time n Based on the state quantity of the processing region n, for the pixel value of each pixel of the processing region n determined to be mixed by the determination means, blur and noise are added to the original image information and the observed image A step of calculating an information process as an observation amount; and the estimation unit sets the processing region n at time n to the state amount calculated by the state amount calculation unit and the observation amount calculated by the observation amount calculation unit. On the basis of Wherein the prediction method based on the state space model, and the optimal value of the state quantity of the processing region n at time n is configured to include the steps of estimating the original image information.

  According to the image restoration apparatus according to the first invention, the image processing method according to the second invention, and the program, blur and noise included in the image can be removed.

  To achieve the above object, an image restoration apparatus according to a third aspect of the present invention includes show-through image information on the other side, which is unnecessary information on the original image information on one side of a document having images on both sides. An image restoration apparatus for estimating the original image information from the observed image information using a prediction method based on a state space model including the original image information as a drive source for the observed image information in which blur and noise are mixed, Image acquisition means for acquiring surface image information obtained by reading one side of the document as the observed image information, and calculating a variance value based on a pixel value of each pixel of the observed image information, and the variance Show-through determination means for determining whether or not deteriorated image information including blur-out image information and noise including the show-through image information is mixed in each pixel of the observed image information based on a threshold calculated from the value; The state quantity of the processing area n at time n in the image information is the state quantity of the processing area (n-1) at time (n-1), the state transition from the processing area (n-1) to the processing area n. And a state quantity calculating means for calculating using the drive source including an element of the state quantity of the processing area n not included in the state quantity of the processing area (n−1), and the show-through of the processing area n Based on the variance value calculated by the determination unit and the state quantity of the processing region n, the calculation for each pixel of the processing region n determined by the show-through determination unit that the deteriorated image information is mixed. Calculated by the state quantity calculation means and the state quantity calculation means for calculating, as the observation amount, a process in which the noise is added to the original image information to form observation image information, using the standard deviation of the distributed value as the noise. State quantity, and Estimating means for estimating an optimum value of the state quantity of the processing area n as original image information of the processing area n by a prediction method based on the state space model based on the observation quantity calculated by the observation quantity calculating means; , Including.

  An image processing method according to a fourth aspect of the present invention is an observation image in which blur and noise are included including show-through image information on the other side, which is unnecessary information on the original image information on one side of a document having images on both sides. An image restoration apparatus for estimating the original image information from the observed image information using a prediction method based on a state space model including the original image information as a drive source for information, and an image acquisition means, show-through An image restoration method in an image restoration apparatus including a determination unit, a state quantity calculation unit, an observation amount calculation unit, and an estimation unit, wherein the image acquisition unit reads a surface of the document and reads the surface image Acquiring the information as the observed image information; and the show-through determining unit calculates a variance value based on a pixel value of each pixel of the observed image information, and based on a threshold value calculated from the variance value. Determining whether or not each pixel of the observed image information is mixed with blurred image information including blurred image information and degraded image information including noise, and the state quantity calculating means includes a time in the observed image information. the state quantity of the processing area n at n, the state quantity of the processing area (n-1) at time (n-1), the state transition from the processing area (n-1) to the processing area n, and the processing A step of calculating using the drive source including an element of the state quantity of the processing area n not included in the state quantity of the area (n−1), and the observation amount calculating means Based on the variance value calculated by the determination unit and the state quantity of the processing region n, each pixel of the processing region n determined by the show-through determination unit to be mixed with the degraded image information is calculated. Standard deviation of the distributed variance A step of calculating as an observed amount a process in which the noise is added to the original image information to become observed image information, and the estimating means includes the state quantity calculated by the state quantity calculating means; and Estimating an optimum value of the state quantity of the processing region n as original image information of the processing region n by a prediction method based on the state space model based on the observation amount calculated by the observation amount calculating means. It is configured to include.

  According to the image restoration device according to the third invention, the image processing method according to the fourth invention, and the program, the variance value is calculated based on the pixel value of each pixel of the observed image information, and is calculated from the variance value. Based on the measured threshold value, it is determined whether each pixel of the observed image information is mixed with blurred image information including blurred image information and degraded image information including noise, and the state of the processing region n at time n in the observed image information The amount is included in the state amount of the processing region (n-1) at time (n-1), the state transition from the processing region (n-1) to the processing region n, and the state amount of the processing region (n-1). A processing region that is calculated using a drive source that includes an element of the state quantity of the processing region n that has not been determined, and that the deteriorated image information is determined to be mixed based on the calculated dispersion value and the state amount of the processing region n For each pixel of n, the standard deviation of the calculated dispersion value is defined as noise. Used to calculate the observed image information by adding noise to the original image information as an observed amount, and based on the calculated state amount and the calculated observed amount, the prediction method based on the state space model, By estimating the optimum value of the state quantity of the processing region n as the original image information of the processing region n, it is possible to remove show-through from only the surface image.

  An image restoration apparatus according to a fifth aspect of the present invention provides a state space model that includes the original image information as a drive source for the observed image information in which the original image information includes a blurred image including the background color of the document and degraded image information including noise. An image restoration device that estimates the original image information from the observed image information using a prediction method based on the image, the image obtaining means for obtaining the surface image information obtained by reading a document as the observed image information, and the observation A variance value is calculated based on a pixel value of each pixel of image information, and it is determined whether or not the degraded image information is mixed in each pixel of the observed image information based on a threshold value calculated from the variance value. The color determination means and the state quantity of the processing area n at the time n of the observation image information are changed from the state quantity of the processing area (n-1) at the time (n-1), the processing area (n-1) to the processing area n. State to And a state quantity calculation means for calculating using the drive source including an element of the state quantity of the processing area n not included in the state quantity of the processing area (n-1), information on the processing area n at time n, and Based on the state quantity, the pixel value of each pixel in the processing area n determined by the ground color determination means to be mixed is used as the deteriorated image information, and the deteriorated image is used as the original image information. Based on the observation amount calculation means for calculating the process that becomes the observation image information by adding information as an observation amount, the state amount calculated by the state amount calculation means, and the observation amount calculated by the observation amount calculation means, Estimation means for estimating an optimum value of the state quantity of the processing region n as original image information by a prediction method based on the state space model.

  An image processing method according to a sixth aspect of the present invention provides a state space model including the original image information as a drive source for the observed image information in which the original image information includes a blurred image including the background color of the document and degraded image information including noise. An image restoration device for estimating the original image information from the observed image information using a prediction method based on the image, and an image acquisition unit, a ground color determination unit, a state quantity calculation unit, an observation amount calculation unit, and an estimation An image restoration method in an image restoration apparatus including: a step in which the image acquisition unit acquires surface image information obtained by reading a document as the observation image information; and the ground color determination unit includes the observation image information. A variance value is calculated based on the pixel value of each pixel, and it is determined whether or not the degraded image information is mixed in each pixel of the observed image information based on a threshold value calculated from the variance value. And the state quantity calculation means determines the state quantity of the processing area n at time n of the observation image information as the state quantity of the processing area (n−1) at time (n−1), and the processing area (n−1). ) To the processing region n, and the step of calculating using the driving source including the state quantity element of the processing region n not included in the processing region (n−1) state quantity, and the observation amount calculation The means determines the pixel value of each pixel of the processing region n determined by the ground color determination unit as a mixture of the deteriorated image information based on the information and state quantity of the processing region n at time n. A step of calculating, as an observation amount, a process in which the degraded image information is added to the original image information to become observation image information, and the estimation means includes the state quantity calculated by the state quantity calculation means, and Observation amount calculation Based on the observation amount calculated by the step, the prediction method based on the state space model, the optimum value of the state quantity of the processing region n is configured to include the steps of estimating the original image information.

  According to the image restoration device according to the fifth invention, the image processing method according to the sixth invention, and the program, the variance value is calculated based on the pixel value of each pixel of the observed image information, and is calculated from the variance value. Based on the threshold value, it is determined whether or not the degraded image information is mixed in each pixel of the observed image information, and the state quantity of the processing region n at the time n of the observed image information is determined as the processing region at the time (n−1) ( n-1) state quantity, state transition from the processing area (n-1) to the processing area n, and driving including state quantity elements not included in the processing area (n-1) state quantity. The pixel value of each pixel in the processing region n that is determined to be mixed based on the information and state quantity of the processing region n at time n is used as the deteriorated image information. Deteriorated image information is added to image information, and observation image information Process as an observed quantity, and based on the calculated state quantity and the calculated observed quantity, an optimal value of the state quantity in the processing region n is estimated as original image information by a prediction method based on the state space model. Thus, the ground color can be removed.

  An image restoration apparatus according to a seventh aspect of the present invention includes an observation image in which blur and noise are mixed, including read-out image information on one side of a document having images on both sides and show-through information on the other side as degraded image information. An image restoration apparatus for estimating the original image information from the observed image information by using a prediction method based on a state space model including the original image information as a drive source for information, and reading both surfaces of the document respectively Information and back surface image information, image acquisition means using the front surface image information as the observed image information, and a state quantity in the processing region n at time n of the observed image information is processed at time (n−1). State quantity of region (n-1), state transition from processing region (n-1) to processing region n, and element of state amount of processing region n not included in state amount of processing region (n-1) Including the drive source And calculating the state quantity at the time n in the image information obtained by inverting the back side image information in the horizontal direction based on the state quantity of the processing region n at the time n of the observation image information and the back side image information. An observation amount calculation that uses, as the deteriorated image information, a value calculated based on information on a region corresponding to the processing region n, and calculates a process in which the deteriorated image information is added to the original image information to become observation image information as an observation amount And a prediction based on the state space model based on the state quantity calculated by the state quantity calculation means and the observation quantity calculated by the observation quantity calculation means for the processing region n at time n of the surface image information And estimating means for estimating an optimum value of the state quantity of the processing region n as original image information of the processing region n.

  An image processing method according to an eighth aspect of the present invention is an observation image in which blur and noise are mixed, wherein the original image information on one side of a document having images on both sides includes the show-through information on the other side as degraded image information. An image restoration apparatus for estimating the original image information from the observed image information using a prediction method based on a state space model including the original image information as a drive source for information, and an image acquisition unit and a state quantity calculation An image restoration method in an image restoration apparatus including a means, an observation amount calculation means, and an estimation means, wherein the image acquisition means reads both sides of the document to obtain front surface image information and back surface image information, and The step of setting the information as the observed image information, and the state quantity calculating means determine the state quantity in the processing area n at time n of the observed image information as the processing area (n-1) at time (n−1). 1) the state quantity, the state transition from the processing area (n−1) to the processing area n, and the driving including the state quantity elements not included in the processing area (n−1) state quantity. A step of calculating using a source, and the observation amount calculation means inverts the back surface image information in the horizontal direction based on the state quantity of the processing region n at time n of the observation image information and the back surface image information. Using the value calculated based on the information of the region corresponding to the processing region n at time n in the obtained image information as the deteriorated image information, and observing the process in which the deteriorated image information is added to the original image information to become the observed image information A step of calculating as a quantity; and the estimating means is calculated by the state quantity calculated by the state quantity calculating means and the observation quantity calculating means for the processing region n at time n of the surface image information. Based on the observed quantity, the prediction method based on the state space model is configured to include the steps of: estimating the optimum value of the state quantity of the processing region n as an original image information of the processing region n.

  According to the image restoration apparatus according to the seventh invention, the image processing method and the program according to the eighth invention, both sides of the document are respectively read to acquire the front surface image information and the back surface image information, and the front surface image information is converted into the observed image information. And the state quantity in the processing area n at the time n of the observed image information is the state quantity in the processing area (n-1) at the time (n-1) and the state from the processing area (n-1) to the processing area n. The state quantity of the processing region n at the time n of the observation image information, calculated using a drive source including the transition and the state quantity element of the processing region n not included in the state quantity of the processing region (n−1), and Based on the backside image information, the image information obtained by inverting the backside image information in the horizontal direction uses the value calculated based on the information of the region corresponding to the processing region n at time n as the deteriorated image information, and deteriorates to the original image information. Image information added The process that becomes the observation image information is calculated as an observation amount, and the processing region n at the time n of the surface image information is calculated based on the calculated state amount and the calculated observation amount by a prediction method based on the state space model. By estimating the optimum value of the state quantity of the processing region n as the original image information of the processing region n, the show-through can be accurately removed.

  The image restoration apparatus according to the seventh invention and the image processing method according to the eighth invention perform alignment between the front surface image information and the back surface image information acquired by the image acquisition means, and It is possible to further include position change amount setting means for setting the region of the processing region n at time n in the observation image information with respect to the back surface image information.

  The show-through determination means in the image restoration device according to the third invention and the image processing method according to the fourth invention calculates a variance value based on each pixel value of the partial region of the observation image information, and Based on the threshold value calculated from the variance value, it is determined whether or not the deteriorated image information is mixed in each pixel of the partial area, and the observation amount calculating means determines that the processing area n at time n of the observed image information is Based on the variance value calculated for the partial region including the processing region n and the state quantity of the processing region n, the show-through determination unit determines that the deteriorated image information is mixed. In addition, each pixel in the processing area n uses the standard deviation of the variance value calculated for the partial area as the noise, and observes the process in which the noise is added to the original image information to become observation image information. It can be calculated as the amount.

  The ground color determination means in the image restoration device according to the fifth invention and the image processing method according to the sixth invention calculates a variance value based on each pixel value of each partial region of the observed image information, Based on a threshold value calculated from the variance value, it is determined whether or not the ground color is mixed in each pixel of the partial area, and the observation amount calculating means determines the ground color in the processing area n of the observation image. Based on the variance value calculated for the partial region including the processing region n by the means and the state quantity of the processing region n, the processing color n of the processing region n determined by the ground color determination unit to be mixed For each pixel, using the standard deviation of the variance value calculated for the partial region as the noise, a process in which the noise is added to the original image information to become observation image information is calculated as an observation amount. Kill.

  In each of the above inventions, the processing of each means can be performed for each partial region L set for the observation image information.

  The processing area n may be set so as to partially overlap the processing area (n−1).

  The program according to the present invention is a program for causing a computer to function as each unit of the image restoration apparatus.

  As described above, according to the image restoration apparatus, method, and program of the present invention, there is an effect that unnecessary information included in an image can be removed.

It is a block diagram which shows schematic structure of the image restoration apparatus which concerns on 1st, 2nd and 3rd Embodiment. It is a figure for demonstrating the partial area | region and processing area which are set with respect to the image. It is a block diagram which shows schematic structure of the image restoration process part in 1st Embodiment. It is a figure for demonstrating the degradation process of an image. It is a figure for demonstrating the degradation principle of an image. It is a flowchart which shows the content of the first half part of the image restoration process in 1st Embodiment. It is a flowchart which shows the content of the second half part of the image restoration process in 1st Embodiment. It is a figure for demonstrating the area | region which has the same state quantity. It is a flowchart which shows the content of the state quantity calculation process in this Embodiment. It is a figure for demonstrating the structure of the state vector of the state n-1 and the state vector of the state n in 1st Embodiment. It is an image figure which shows the structure of the state equation in 1st Embodiment. It is an image figure which shows the structure of the observation equation in 1st Embodiment. It is a flowchart which shows the content of the prediction process based on the state space in this Embodiment. It is a figure which shows the image used for the simulation of the method 1 of invention. It is a figure which shows an example of the simulation result (visual evaluation) of the method 1 of invention. FIG. 16 is a partially enlarged view of FIG. 15. It is a block diagram which shows schematic structure of the image restoration process part in 2nd Embodiment. It is a flowchart which shows the content of the first half part of the image restoration process in 2nd Embodiment. It is a figure which shows the image used for the simulation of the method 2 of invention. It is a figure which shows an example of the simulation result (visual evaluation) of the method 2 of invention. FIG. 21 is a partially enlarged view of FIG. 20. It is a figure which shows the other example of the simulation result (visual evaluation) of the method 2 of invention. FIG. 23 is a partially enlarged view of FIG. 22. It is a block diagram which shows schematic structure of the main memory part in 3rd Embodiment. It is a block diagram which shows schematic structure of the image restoration process part in 3rd Embodiment. It is a flowchart which shows the content of the first half part of the image restoration process in 3rd Embodiment. It is an image figure which shows the structure of the state equation in 3rd Embodiment. It is a figure for demonstrating the structure of the state vector of the state n in a front surface image, and the state vector of the state n in a back surface image in 3rd Embodiment. It is an image figure which shows the structure of the observation equation in 3rd Embodiment. It is a figure which shows an example of the image used for the simulation of the method 3 of invention. It is a figure which shows an example of the simulation result (visual evaluation) of the method 3 of an invention. FIG. 32 is a partially enlarged view of FIG. 31. It is a figure which shows the other example of the simulation result (visual evaluation) of the method 3 of invention. It is a partially enlarged view of FIG. It is a figure which shows the simulation result (subjective evaluation) of the method 3 of an invention. It is a figure which shows the other example of the image used for the simulation of the method 3 of invention. It is a figure which shows the simulation result (visual evaluation) of the method 3 of invention. FIG. 38 is a partially enlarged view of FIG. 37. It is a figure which shows the simulation result (visual evaluation) of the method 3 of invention. FIG. 40 is a partially enlarged view of FIG. 39. FIG. 40 is a partially enlarged view of FIG. 39.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
First, the first embodiment will be described. The image restoration apparatus according to the first embodiment is an observation in which the original image information on one side of a document having images on both sides includes blurring and noise including the show-through image information on the other side, which is degraded image information. The original image information is estimated from the observed image information by using a prediction method based on a state space model in an image information system and including a state space model including the original image information as a drive source.

  In the first embodiment, a document having images on both sides is targeted, and the original image information on one side of the document includes blurring and noise including the show-through image information on the other side as degraded image information. A case where the present invention is applied to an image restoration apparatus that estimates original image information from image information (surface image information) will be described.

  As shown in FIG. 1, the image restoration apparatus 10 according to the first embodiment includes a computer including a CPU, a ROM, a RAM, and an HDD, and includes an input interface unit 20, an operation unit 30, and a storage unit 40. The image restoration processing unit 60 and the output interface unit 70 are provided. The input interface unit 20 is an example of an image acquisition unit.

  The input interface unit 20 acquires surface image information obtained by reading one side of the document as observation image information. That is, the input interface unit 20 performs input processing such as converting image information provided from the image input device into a data format that can be processed by a computer. The input interface unit 20 is an example of an image acquisition unit. As an example of a method for reading a document, as shown in FIG. 2, the document is read using a partial area L in the horizontal width N × longitudinal direction M of the read document as a reading unit, and image information obtained for each partial area. (L + (L + 1) +... + (L + n)) is combined to obtain surface image information of the entire read document.

  The image input device is an input device for inputting image information (observation image information) to be processed as digital data to a computer, and images each piece of image information of each successive frame constituting a moving image. Input to the restoration device 10. For example, a camera, a recording medium, a modem, or the like can be used as the image input device. A camera means any device having an imaging function. For example, in addition to a digital video camera, a mobile phone equipped with a camera function, a security camera (surveillance camera), a medical device for performing image diagnosis (internal view) Mirror, X-ray, echo, CT, MRI, etc.). Recording media widely mean recording media capable of recording image information. For example, magnetic media (HDD, FD, etc.), optical discs (CD, DVD, BD, etc.), magneto-optical discs (MO), flash memory (memory) Card, USB memory, etc.). The modem is a device for connecting to an external communication network (for example, a telephone line, a LAN, the Internet, etc.). Note that the type of the image input device may be selected according to the use of the image restoration device 10. In the present embodiment, a scanner is assumed as the image input device.

  Although not shown, the input interface unit 20 is provided separately and independently according to the type of the image input device. For example, the recording medium input interface unit 20 is also called a drive, and various types of drives can be used depending on the type of the recording medium. The drive is a device that reads and writes recording media. As far as recording media are concerned, the input interface unit 20 and the output interface unit 70 are usually integrated. In general, a modem can function as both an image input device and an image output device. Therefore, the input interface unit 20 and the output interface unit 70 are usually integrated with respect to the modem. The input interface unit 20 may be a built-in card (board) stored inside the computer main body, or may be an externally installed device connected via an internal interface unit (not shown).

  When the image input apparatus outputs image information as analog data, the corresponding input interface unit 20 includes a sampling unit and an A / D conversion unit (both not shown). The sampling unit samples the input analog signal at a predetermined sampling frequency and outputs the sampled analog signal to the A / D conversion unit. The sampling frequency can be changed according to the type of image to be restored. The A / D converter performs A / D conversion processing on the amplitude value of the sampled signal with a predetermined resolution.

  The operation unit 30 is, for example, a keyboard, a mouse, a touch panel, or the like, but a voice recognition device or the like may be used. A user can operate the apparatus using the operation unit 30. The operation unit 30 includes a parameter setting unit 31, a region specifying unit 32, and an image information setting unit 33. The parameter setting unit 31 sets various parameter values necessary for image processing according to the present embodiment by a user input operation. The area designating unit 32 designates a processing area (a specific range of an image) to be subjected to image processing with respect to the input image information by a user input operation. The image information setting unit 33 sets whether the input image is a monochrome image or an RGB image.

  The storage unit 40 is an element constituting the main body of the computer, and includes a main storage unit 41, an auxiliary storage unit 47, and a display memory 48. The main storage unit 41 includes a captured image memory 42 in which image data captured via the input interface unit 20 is stored, and a variance value memory 43 in which a variance value calculated by a later-described deteriorated image information determination unit 62 is stored. A reference value memory 44 that stores a reference value calculated by a degraded image information determination unit 62 described later, a subject area memory 45 that stores a processing result estimated by an estimation unit 65 described later, and an image described later. An image processing program for executing processing, and a processing parameter memory 46 in which various parameters are stored.

  The display memory 48 is an area in which image information to be displayed on an image output device (not shown) configured by a display or the like is stored. In order to display the captured image information (observed image information), the display memory 48 is a pre-process. Captured image memory 49 in which image information (captured image information) is stored, a processing memory 50 that is a temporary storage area for image processing, and an image after processing to display the processed image information And a restored image memory 51 in which information is stored.

  The auxiliary storage unit 47 compensates for the capacity shortage of the main storage unit 41. The auxiliary storage unit 47 may be, for example, a hard disk (HD), a portable device such as a CD-ROM, DVD, SSD (Solid State Drive), flash memory, etc. A combination thereof may also be used.

  In this embodiment, the case where an image processing program is stored in advance in the processing parameter memory 46 will be described. However, the image processing program can be installed in the storage unit 40 from the recording medium via the input interface unit 20. Alternatively, it may be downloaded to the storage unit 40 from the outside via the modem and input interface unit 20.

  The image restoration processing unit 60 is a processing unit for executing image processing to be described later. Functionally, as shown in FIG. 3, an area setting unit 61, a deteriorated image information determination unit 62, and a state amount calculation are performed. It can be expressed by a configuration including a unit 63, an observation amount calculation unit 64, an estimation unit 65, and a restored image output unit 66.

  The region setting unit 61 sets a processing region n having a size of a peripheral pixel region p × q surrounding the pixel region of interest at time n for observation image information that is image information stored in the captured image memory 42. The target pixel region may be one pixel, or may be two-dimensionally enlarged with a plurality of pixel regions. By making the pixel region of interest a plurality of pixel regions, it is possible to restore with high processing efficiency. The region setting unit 61 sets the size of the target pixel region and the size of the peripheral pixel region. In the captured image memory 49, the processing area at time n is set to partially overlap with time (n + i) so as to include the target pixel area as time changes.

  The deteriorated image information determination unit 62 calculates a variance value based on the pixel value of each pixel in the partial region L of the observation image information that is the image information stored in the captured image memory 42, and calculates the variance value from the variance value. Based on the reference value (threshold value) of the partial area L, it is determined whether or not the show-through image information is mixed in each pixel of the partial area L.

  The state quantity calculation unit 63 changes the state quantity in the processing area n at the time n of the partial area L from the processing area (n−1) state quantity at the time (n−1) and the processing area (n−1) to the processing area. Calculation is performed using a colored drive source including a state transition to n and an element of a state quantity of the processing region n that is not included in the state quantity of the processing region (n−1).

  The observation amount calculation unit 64 calculates the variance value calculated for the partial region L including the processing region by the deteriorated image information determination unit 62 and the state amount of the processing region n at the time n for the processing region n of the partial region L at time n. For the pixels in the processing region n determined by the deteriorated image information determination unit 62 to include the deteriorated image information, noise is added to the original image information using the standard deviation of the variance value as noise. The process that becomes observation image information is calculated as an observation amount.

  The estimation unit 65 predicts a processing region n at time n of the partial region L from the state quantity calculated by the state quantity calculation unit 63 and the observation amount calculated by the observation amount calculation unit 64 based on the state space model. Is used to estimate and restore the optimum value of the state quantity of the processing region n as the original image information of the processing region n.

  The restored image output unit 66 restores the target pixel region of the processing region n at time n from the result obtained by the estimation unit 65.

  The processing of each unit of the image restoration processing unit 60 will be described in detail in an image restoration processing routine described later.

  The output interface unit 70 converts the image information (restored image information) restored by the image restoration processing unit 60 and stored in the restored image memory 51 into a data format that can be output to an image output device configured with a display or the like. And perform output processing.

  The output form of the processed image is not limited to the display on the image output apparatus, and may be output to another image output apparatus. For example, a printer, a recording medium, a modem, or the like can be used as the image output device. The recording medium and the modem may be shared with the recording medium and the modem as the image input device, respectively. Note that the type of the image output apparatus may be selected according to the use of the image restoration apparatus 10.

  Although not shown, the output interface unit 70 is provided separately and independently according to the type of the image output apparatus. As described above, regarding the recording medium and the modem, the input interface unit 20 and the output interface unit 70 are usually integrated. Similarly to the input interface unit 20, the output interface unit 70 may be a built-in card (board) stored inside the computer main body, or an externally installed device connected via the internal interface unit. Also good.

  Next, an observation image model and a state space model, which are the theory of image restoration in the first embodiment, will be described.

  The observation image information is generally obtained by a model obtained by adding noise to convolution of original image information and a blur point spread function (PSF: Point Spread Function).

  FIG. 4 is a diagram for explaining the process of image degradation. For example, when a document having images printed on both sides is scanned, the image information of the back image is shown through to the original image information of the front image, and becomes the deteriorated image information shown at the right end of FIG. As shown in FIG. 5, the show-through is an image in which the image information obtained by scanning the original image (front image) is superimposed with the image information in which the back image is transmitted and blurring and noise are generated. Note that the transmittance through which the back image is transmitted depends on the thickness of the print medium, the density of the image, the illumination intensity during scanning, and the like. Therefore, the first embodiment is to remove the deteriorated image information.

  In the first embodiment, a state quantity is calculated using only original image information (original image information), and degraded image information including observation image information, original image information, blur, noise, and show-through image information is calculated. A state space model for which the amount of observation is calculated by using a prediction method based on a state space model including a colored drive source, and obtaining an optimum estimate of the amount of state, thereby restoring restored image information from the observed image information. Generate.

The state space model is represented by a state equation of the following equation (1) and an observation equation of the following equation (2). However, in the state equation, the state vector x ^f _p1 (n) at time n is a vector composed of state quantities using the surface image as original image information, and the matrix state transition matrix Φ _p1 is from the processing region (n−1) to the processing region n. The drive source vector δ ^f _p1 (n) that represents the transition to is composed of original image information. In the observation equation, the observation vector y ^f _p1 (n) is a vector composed of observation image information, the observation matrix M ^f is a matrix representing blur occurring in the surface image information, and the observation noise vector ε ^f _p1 (n) is setback information. A vector consisting of Further, the subscript p1 indicates that it relates to the first embodiment, and the subscript f indicates that it is a surface image.

The state equation (1) describes the system to be observed by a state space model, and is an internal state, that is, a state variable (here, state vectors x ^f _p1 (n−1), x ^f _p1 (n)). ) Represents the generation process for the state. The observation equation (2) describes the process of observation through some kind of observation device, and the observation vector y ^f _p1 (n)) is generated depending on the state vector x ^f _p1 (n). It shows a state.

  Next, the operation of the image restoration apparatus 10 according to the first embodiment will be described with reference to FIGS. 6 and 7 is stored as a control program in the storage unit 40, and is executed by a CPU (not shown).

  In step 100, the image information of the surface image read from the document having the images printed on both sides thereof by the scanner as the image input device is acquired via the input interface unit 20, and the captured image memory 42 of the main storage unit 41 is captured. To remember. Since the image information of the surface image includes color information of the surface image, the image information setting unit 33 sets whether the input image is a monochrome image or an RGB image.

  Next, in step 102, the area setting unit 61 sets the pixel size M × N of the partial area L that is a processing unit for obtaining the variance value of the show-through image information and the processing area n that is the processing unit of image restoration. A pixel size p × q is set. The partial region L and the processing region n are set as shown in FIG. 2 for the surface image information, for example. The partial area M × N and the processing area p × q may be set to values specified by the user from the area specifying unit 32 of the operation unit 30.

  Here, in step 102, it is assumed that the pixel size of the processing region n is set to p × q = 3 × 3 pixels. In this case, as shown in FIG. 8, the image information is moved by one pixel from the left to the right, and the processing area (n-1) at time (n-1) and the processing area n at time n are processed.

Next, in step 104, the state quantity calculation unit 63 configures the state transition matrix Φ _p1 based on the positional relationship between the processing region n at time n and the processing region (n−1) at time (n−1). To do.

Specifically, first, it is assumed that the pixel is moved horizontally from left to right from the processing region (n−1) at time (n−1) to the processing region n at time n. In this case, in each processing region n, there is a region having the same state quantity as shown in FIG. The region having the same state quantity is a pixel region shaded in FIG. As shown in FIG. 8, the state quantities x _1,2 , x _2,2 , x _3,2 , x _1,3 , x _2,3 , x of the processing region (n-1) at time (n-1). x _3,3 and the state quantity x _1,1 , x _2,1 , x _3,1 , x _1,2 , x _2,2 , x _3,2 of the processing region n at time n are the same state quantity It becomes an area with.

Then, the state transition matrix Φ _{p1 in which the} regions having the same state quantity described above are matched by the change from the state vector of the processing region (n−1) at the time (n−1) to the state vector of the processing region n at the time n. Configure.

  Next, in step 106, in the image information of the surface image stored in step 100 above, when the pixel size M × N of the partial region L is changed, the pixel size of the processing region n is changed to p ′ × q ′. It can be set, and when the number of pixels in the target pixel area is reset, it is changed.

  Next, in step 108, when the image is set as an RGB image in step 100, a specific color is set as a processing target among the R, G, and B colors in the stored image information of the surface image.

  Next, in step 110, the degraded image information determination unit 62 uses the pixel value of the pixel size M × N of the partial region L set in step 102 in the image information of the surface image stored in the captured image memory 42. The variance value is calculated based on Further, the deteriorated image information determination unit 62 calculates a reference value (threshold value) for determining whether or not show-through information is included in each pixel of the surface image based on the calculated dispersion value.

Specifically, the deteriorated image information determination unit 62 calculates the average pixel value ￣y of the color set in step 108 in the set pixel size M × N of the partial region L. Then, using the calculated average pixel value ￣y, the color dispersion value σ ² _y set in step 108 is calculated. That is, the degraded image information determination unit 62 calculates the variance value σ ² _y according to the color set in step 108 as shown in the following equation (3).

Here, ym _{, n} represents the pixel value of the color set in step 108 in the surface image. M and N represent the pixel size M × N of the partial region L as shown in FIG.

Then, the degraded image information determination unit 62 determines whether or not the show-through image information is included in the pixels in the surface image based on the average pixel value ￣y and the variance value σ ² _y. y _th is calculated as shown in the following equation (4).

  Next, in step 112, it is determined whether or not the set partial area L is an initial value. That is, it is determined whether or not the set partial area L is positioned at the uppermost stage in FIG. If the set partial area L is an initial value, the process proceeds to step 116. On the other hand, if the set partial area L is not the initial value, the process proceeds to step 114.

Next, in step 114, the variance value σ ² _y calculated in step 110 is calculated in step 110 in consideration of the variance value in the partial region (L-1) calculated in the previous step 110. The variance value σ ² _y is corrected. For example, after once calculating the variance value in the partial region L, the average value of the calculated variance value in the partial region L and the variance value in the partial region (L-1) is reset with the variance value in the partial region L. can do.

  Next, if the area setting unit 61 resets in step 106 in step 116, the pixel size of the processing area n is set to p ′ × q ′, and if not reset, the processing area set in step 102 The pixel size remains n.

Next, in step 118, the deteriorated image information determination unit 62 determines, for each pixel included in the processing region n set in step 116, the pixel based on the reference value y _th calculated in step 110. It is determined whether or not show-through image information is mixed.

Specifically, for each pixel included in the pixel size p × q of the processing region n, the deteriorated image information determination unit 62 determines the pixel value y _{m, n of} the color set in step 108 of the pixel, and the The gray value y _{gray of the} pixel is compared with the reference value y _th . When the following expression (5) is satisfied, it is determined that show-through is mixed in the pixel value ym _{, n} .

  Also,

The above (Equation 5) represents the pixel value of each color of R, G, B of the pixel. In the case where the image in step 100 is set to black and white images, the pixel value of each pixel is greater than the reference value y _th, it is determined that the show-through image information to the pixel are mixed.

  Next, in step 120, the state quantity calculation unit 63 executes a state quantity calculation process. Here, the state quantity calculation processing routine will be described with reference to FIG.

  In step 1200, the state quantity calculation unit 63 determines the state vector of the processing area (n−1) having the state quantity of each pixel of the processing area (n−1) at time (n−1) as an element, and the time n. A state vector of the processing region n having a state quantity of each pixel of the processing region n as an element is derived. Hereinafter, a method for deriving the state vector of each processing region will be specifically described.

First, as shown in FIG. 10, the state vector of the processing area (n−1) is configured by combining the state quantities of the pixels of the processing area (n−1) having the pixel size p × q. That is, the state vector x ^f _p1 (n−1) of the processing area (n−1) is a nine-dimensional vector x ^f _p1 (n−1) = [x ^f _1,1 (n−1) x ^f _2,1 (N−1)... X ^f _3,3 (n−1)] ^T Next, the state vector x ^f _p1 (n) of the processing region n is configured by combining the state quantities of each pixel of the processing region n having the pixel size p × q, as shown in FIG. That is, the state vector x ^f _p1 (n) of the processing region n is a 9-dimensional vector x ^f _p1 (n) = [x ^f _1,1 (n) x ^f _2,1 (n)... X ^f _{3, 3} (n)] ^T. However, T represents transposition of a vector. Further, the size (number of dimensions) of the state vectors x ^f _p1 (n−1) and x ^f _p1 (n) varies depending on the pixel size p × q of the set processing region.

Next, in step 1202, the state quantity calculation unit 63 configures a drive source vector. As shown in FIG. 11A, since the state quantities x _1,3 , x _2,3 , x _3,3 of the processing area n cannot be configured from the information of the processing area (n−1), In the present embodiment, by using the state quantities x _1,3 , x _2,3 , x _3,3 in a region called a drive source, as shown in FIG. 11B, the drive source vector δ ^f _p1 ( n). This means that when moving in the frame, the state quantity of the area that cannot be represented by the state quantity of the processing area (n−1) is composed of the state quantity of the processing area n. Therefore, the drive source vector δ ^f _p1 (n) is a colored drive source.

Next, in step 1204, the state quantity calculation unit 63 determines the state vectors x ^f _p1 (n−1) and x ^f _p1 (n) of the state (n−1) and the state n derived in step 1200, and the above steps. The state quantity is calculated based on the state transition matrix Φ _p1 configured in 105 and the drive source vector δ ^f _p1 (n) configured in step 1202. As shown in FIG. 11B, the state equation in the present embodiment is a state vector x ^f _p1 (n) composed of image information, a state transition matrix Φ _p1 composed of 0 and 1, and a colored signal. Since it is composed of the drive source vector δ ^f _p1 (n) composed of image information, it is composed only of a desired state quantity, that is, image information.

When the state quantity calculation process routine ends, the process returns to the image restoration process routine and proceeds to step 122, where the observation quantity calculation unit 64 is based on the observed image model shown in FIG. 4 as shown in FIG. Then, the process of adding the show-through image information of the back image to the original image information of the front image to obtain the observation image information is calculated as an observation amount (observation equation). The observation vector y ^f _p1 (n) is a nine-dimensional vector having the pixel information of each pixel in the processing region n of the surface image information as an observation amount including the observation image information. The observation matrix M ^f is a matrix for satisfying the observation equation. The vector x ^f _p1 (n) is a vector whose element is the state quantity of each pixel in the processing region n in the original image information of the surface image. Further, ε ^f _p1 (n) is an observation noise vector. The observation noise vector includes a transmittance corresponding to the show-through image information and a value corresponding to the blur point spread function (PSF) in the image degradation model. As described above, the transmittance is determined according to the thickness of the document, the density of the image, the intensity of illumination during scanning, and the like. The blur is an amount representing the degree of blur that occurs when the image information of the back surface image is transmitted to the front surface image information.

Here, each value of the observation noise vector ε ^f _p1 (n) is set according to the determination result in step 118. Specifically, the pixel value y _m in step _118, if _n is it is determined that the show-through image information are mixed, the elements of the observation noise vector corresponding to the pixel value y _{m, n} v _{ym, n} Is substituted with the standard deviation of the variance σ ² _y calculated in step 110. On the other hand, the pixel value y _m in step _118, if the show-through image information is determined not to be mixed in _n, the pixel value y _m, the element v _ym of measurement noise vector corresponding to _{n, n} is a constant value Is substituted.

  Next, in step 124, the estimation unit 65 includes the following colored drive sources based on the state quantity (state equation) calculated in step 120 and the observation quantity (observation equation) calculated in step 122. A prediction method based on the state space model is derived.

  The above algorithm is roughly divided into an initialization process [Initialization] and an iterative process [Iteration]. In the iterative process, steps 1 to 5 are sequentially repeated. The details of the prediction algorithm based on the state space model including the colored drive source will be described below with reference to FIG.

  In step 1240, it is determined whether or not the algorithm processing is the first time. If it is the first time, the process proceeds to step 1242, and if it is the second time or later, the process proceeds to step 1244.

In step 1242, initial setting is performed. Specifically, an initial value x ^ ^f _p1 (0 | 0) of an optimum estimated value of a state vector, that is, a desired signal (original image signal) vector that is a state quantity (hereinafter referred to as an “optimum estimated value vector of a desired signal”). ), The initial value P _p1 (0 | 0) of the correlation matrix of the estimation error of the desired signal vector (hereinafter referred to as “estimation error vector of the desired signal”), and the covariance value R _δp1 (n) [i, j] and the covariance value R _εp1 (n) [i, j] of the observed noise vector are set to the initial state shown in the above-described initial setting process [Initialization].

However, the vector 0 _J is a J-dimensional zero vector, and the matrix I _J is a J-order unit matrix. J is the number of pixels in the processing area. In the present embodiment, the pixel size p × q = 3 × 3 = 9 of the processing region n. Also, E [x ^ ^f _p1 (n), x ^ ^f _p1 ^T (n)] is an expected value of the autocorrelation matrix of the drive source vector x ^ ^f _p1 (n). E [ε ^f _p1 (n) ε ^f _p1 ^T (n)] [i, j] is the expected value of the autocorrelation matrix of the observed noise vector ε _p1 , and here the variance σ ² _ε Correlation value. When the show-through image information is a white signal, E [ε ^f _p1 (n) ε ^f _p1 ^T (n)] has only a diagonal element. Similarly, when the show-through image information is a colored signal, E [ε ^f _p1 (n) ε ^f _p1 ^T (n)] has a band matrix that does not have zero as an element. When the show-through image information is constant, for example, the background color of paper without change, E [ε ^f _p1 (n) ε ^f _p1 ^T (n)] has a constant value for all elements of the matrix. . In addition, when showing the element of a vector and a matrix, the i-th element of vector a (n) is described as a (n) [i], and the element of i row j column of matrix A (n) is A (n). It shall be written as [i, j].

Next, in Step 1244, the state transition matrix Φ _p1 in the state space model defined in Step 104 above, and the initial value P _p1 (0 | 0) (n of the correlation matrix of the estimation error vector of the desired signal set in Step 1242 above. = 2), or the correlation matrix P _p1 (n | n) (when n> 2) updated in step 1250 in the processing region (n−1), and the covariance value R _{δp1 of the} driving source vector ( n) Using [i, j], a correlation matrix P _p1 (n | n−1), which is an error when the state vector of the processing region n is estimated from information up to the processing region (n−1), is calculated. (Procedure 1 of the above iteration process [Iteration]).

Next, in step 1246, the correlation matrix P _p1 (n | n−1) calculated in step 1244, the observation matrix M ^f in the state space model defined in step 110, and the covariance value R of the observation noise vector _A weighting coefficient matrix K _p1 (n) is calculated using _εp1 (n) [i, j] (same procedure 2).

Next, in step 1248, the state transition matrix Φ _p1 and the initial value x ^ ^f _p1 (0 | 0) (when n = 2) of the optimum estimated value vector of the desired signal set in step 1242, or the processing region In (n−1), information up to the processing region (n−1) is obtained using the optimum estimated value vector x ^ ^f _p1 (n | n) (when n> 2) of the desired signal obtained in this step. The optimal estimated value vector x ^ ^f _p1 (n | n-1) of the desired signal in the processing region n is calculated (same procedure 3). Then, the calculated optimal estimated value vector x ^ ^f _p1 (n | n−1) of the desired signal, the weight coefficient matrix K _p1 (n) calculated in step 1246, the observation vector y ^f _p1 (n), and the observation matrix Using M ^f , the optimum estimated value vector x ^ ^f _p1 (n | n) of the desired signal in the information up to the processing region n is calculated (same procedure 4).

Next, in step 1250, the processing region n is calculated using the unit matrix I, the weighting coefficient matrix K _p1 (n), the observation matrix M, and the correlation matrix P _p1 (n | n−1) calculated in step 1244. The correlation matrix P _p1 (n | n) in the information up to is updated (same procedure 5). Next, in step 1252, the optimum estimated value vector x ^ ^f _p1 (n | n) of the desired signal calculated in step 1248 is temporarily stored in the subject area memory 45 as the processing result of the currently set processing area. To do.

  When the algorithm of the prediction method based on the state space model including the colored drive source is completed, the process returns to the image processing routine, proceeds to step 126, and image restoration is performed for all the processing areas of the partial area L set in step 102. It is determined whether or not the process has ended. If the processing has not been completed for all the processing areas of the partial area L, the process proceeds to step 128, the processing area n is incremented by 1, and the process returns to step 116 to set and process the next processing area (n + 1). repeat. When processing is completed for all processing regions of all partial regions L, the process proceeds to step 130. When the image is set as an RGB image in step 100, processing is completed for all colors (RGB). Determine whether or not. If the processing has not been completed for all colors, the process returns to step 108 to repeat the processing for the next color. When the processing is completed for all colors, the process proceeds to step 132. If the image is set as a black and white image in step 100, the process proceeds to step 132 without performing step 130.

  In step 132, the image information of the partial area L is restored from the processing result of the processing area stored in the subject area memory 45 in step 1252, and is stored in the subject area memory 45, and the restored image of the partial area is restored to the restored image. Output to the output unit 66.

  Next, in step 134, it is determined whether or not the image restoration has been completed for all partial areas in the surface image captured in step 100. If the processing has not been completed for all partial areas, the process proceeds to step 136, L is incremented by 1, and the process returns to step 106 to reset the processing area n in the next partial area (L + 1) and perform processing. repeat. When the processing is completed for all partial areas, the process proceeds to step 138.

  Next, in step 138, the restored image output unit 66 stores the processing result of the processing area of each partial area stored in the subject area memory 45 in step 1252 in the subject area memory 45.

  In step 140, the image restoration processing unit 60 outputs the restored image information of the surface image stored in the subject area memory 45 via the output interface unit 70, and displays it on the image output device for image restoration. The processing routine ends.

  As described above, according to the image restoration apparatus 10 of the first embodiment, the variance value of each partial region is calculated based on the pixel value of each pixel of the surface image, and the reference calculated from the variance value Based on the value, it is determined whether or not show-through image information is mixed in each pixel of the surface image, and the state quantity in the processing area n is set as the state quantity in the processing area (n−1) and the processing area (n−1). Calculation using a chromaticity driving source that includes an element of the state quantity of the processing region n not included in the state transition of the processing region (n−1) and the state transition from the processing region n to the processing region n, and show-through image information is mixed For the pixel determined to be, the standard deviation of the variance value is used as noise, and the observed amount of the processing region n in the observed image information is calculated. Based on the calculated state amount and the calculated observed amount, Using a prediction method based on a spatial model, the state quantity of the processing region n Restoring image data from the appropriate value, it is possible to remove show-through image information from only the surface image information.

  Further, sequential processing is possible by processing each partial area. Furthermore, if the size of the processing area is reset, the amount of calculation can be suppressed and execution can be performed at high speed. Also, by setting the partial area L, it is possible to use a dispersion value suitable for each processing area, it is possible to start processing without waiting for one image to be read, and parallel processing can be applied to achieve high speed. Image processing can be performed.

<Simulation 1>
Next, a simulation for verifying the effect of the image restoration apparatus 10 according to the first embodiment will be described. In this simulation, the image shown in FIG. 14 was used as the observation image. The observation image (surface image) is a color image with a gradation of 24 bits, and the resolution is 3508 × 2480 pixels. The original image (image without show-through) of the observed image is an image in which a small character string is printed at the upper left and “surface 210 mm × 297 mm A4” is printed at the center. On the other hand, the show-through image (back side image) is an image in which a small character string is printed on the upper left and “back side 210 mm × 297 mm A4” is printed at the center. On the observed image, a show-through image appears in a state of being reversed left and right. For the above image, under the same simulation conditions, the method in the first embodiment (invention method 1: Prop. 1), the method according to non-patent document 1 (conventional method 1: Conv. 1), The method according to Patent Document 1 (conventional method 2: Conv. 2) was compared by visual evaluation. Note that the image restoration processing was performed for each of R, G, and B colors.

(1) Visual evaluation FIGS. 15 and 16 show the simulation results for the observed image. FIG. 16 is an enlarged view of a part of FIG.

  As shown in FIG. 15, the conventional methods 1 and 2 cannot remove show-through and cannot restore original image information. On the other hand, the invention method 1 can remove the show-through image information and can restore the original image information as compared with the conventional methods 1 and 2. Further, as shown in FIG. 16, the conventional methods 1 and 2 are lighter than the surface image information. On the other hand, the color of the inventive method 1 can be clearly restored as compared with the conventional methods 1 and 2. Thus, the effectiveness of Invention Method 1 can be confirmed.

  From the above simulation results, it can be seen that the technique (invention technique 1) in the image restoration apparatus 10 according to the first embodiment exhibits higher image restoration ability than the conventional techniques 1 and 2.

<Second Embodiment>
Next, a second embodiment will be described. In the second embodiment, a case will be described in which the present invention is applied to an image restoration apparatus that removes a ground color from an image including a ground color on a document sheet.
Specifically, in the second embodiment, a document having an image including a ground color of one side of document paper and a document having an image on both sides are blurred, and the original image information on one side of the document is blurred. The present invention is applied to an image restoration apparatus that estimates original image information from deteriorated image information including show-through image information including noise and ground color.

  That is, the image restoration apparatus according to the second embodiment is a state space model using observed image information in which original image information is blurred and deteriorated image information composed of show-through image information including noise and ground color is mixed. The original image information is estimated from the observed image information using a prediction method based on a state space model including the original image information as a drive source.

  In the present embodiment, the background color of the document paper surface is the color of the background of the printing paper or the color of the filled area existing in the background of the image. Since this ground color makes it difficult to identify characters, it is often unnecessary when printing.

  The configuration of the image restoration apparatus 210 according to the second embodiment will be described with respect to parts that are different from the image restoration apparatus 10 according to the first embodiment, and the other parts are denoted by the same reference numerals as those in the first embodiment. Detailed description will be omitted.

  As illustrated in FIG. 17, the image restoration processing unit 260 in the second embodiment includes an area setting unit 61, a ground color determination unit 262, a state quantity calculation unit 63, an observation amount calculation unit 64, and an estimation unit. 65 and a restored image output unit 66.

  The ground color determination unit 262 calculates a variance value based on the pixel value of each pixel in the partial region L of the observation image information, and based on the reference value (threshold value) of the partial region calculated from the variance value, It is determined whether or not ground color information is mixed in each pixel in the region L.

  Next, with reference to FIG. 18 and FIG. 7, the operation of the image restoration apparatus 210 according to the second embodiment will be described. The image restoration processing routine shown in FIG. 18 and FIG. 7 is stored as a control program in the storage unit 40 and is executed by a CPU (not shown).

  In step 210, the ground color determination unit 262 can reproduce and set p ′ × q ′ when changing the pixel size of the processing region n in the pixel size M × N of the partial region L set in step 106, If the number of pixels in the target pixel area is also reset, it is changed and set.

In step 218, the ground color determination unit 262 has the ground color information for each pixel constituting the processing region n set in step 116 based on the reference value y _th calculated in step 110. It is determined whether they are mixed or not.

Specifically, the ground color determination unit 262 determines the pixel value y _{m, n of} the color set in step 108 for each pixel constituting the processing region n, the gray value y _{gray of the} pixel, and the reference value y. Compare _th . If the above equation (5) is satisfied, it is determined that show-through image information is mixed in the pixel value ym _{, n} .

  Note that other configurations and operations of the image restoration apparatus according to the second embodiment are the same as those of the first embodiment, and thus the description thereof is omitted.

As described above, according to the image restoration apparatus 210 of the second embodiment, the variance value is calculated based on the pixel value of each pixel of the observed image information, and based on the reference value calculated from the variance value. Thus, it is determined whether or not ground color information is mixed in each pixel of the observed image information, and the state vector x ^f _p1 (n) in the processing region n of the observed image information is a state quantity whose surface image is the original image information. A state transition matrix Φ _p1 is a vector x ^f _p1 (n) consisting of state quantities (of the processing region (n−1)), a matrix representing a transition from the processing region (n−1) to the processing region n, and The calculation is performed using a chromaticity driving source including an element of the state quantity of the processing area n that is not included in the state quantity of the processing area (n−1), and the dispersion is performed on the pixels that are determined to include ground color information. Processing of observed image information using standard deviation of values as noise The observed amount of the region n is calculated, and the optimum value of the state amount of the processing region n is calculated based on the calculated state amount and the calculated observed amount by the prediction method based on the state space model. Estimate as information and remove ground color information.

<Simulation 2>
Next, a simulation for verifying the effect of the image restoration apparatus 210 according to the second embodiment will be described. In this simulation, two images shown in FIG. 19 were used as observation images. The two images shown in FIG. 19 have an image on one side and a document image including a ground color (Text1 image), and an image on a document surface having both sides and a ground color (Text2 image). ). Each of the two images shown in FIG. 19 is a color image having a gradation of 24 bits, and the resolution is 3508 × 2480 pixels. The above image was subjected to a visual evaluation by performing a simulation of the method (Invention Method 2: Prop. 2) in the second embodiment. Note that the image restoration processing was performed for each of R, G, and B colors.

(1) Visual Evaluation FIGS. 20 to 23 show simulation results for the observed image. 21 is an enlarged view of a part of FIG. 20, and FIG. 23 is an enlarged view of a part of FIG.

  As shown in FIG. 20, Invention Method 2 can remove ground color information. Further, as shown in FIG. 21, Invention Method 2 can remove ground color information while clearly restoring the character portion of the image on the document surface and the curved pattern portion. Thus, the effectiveness of Invention Method 2 can be confirmed.

  Further, as shown in FIG. 22, it can be seen that the ground color information can be removed even from the Text2 image which is an image of the document surface including the ground color on both sides. In the enlarged view shown in FIG. 21, Invention Method 2 can remove not only the ground color information but also the show-through image information while clearly restoring the character portion of the surface image information. Thus, the effectiveness of Invention Method 2 can be confirmed.

  From the above simulation results, it can be seen that the technique (invention technique 2) in the image restoration apparatus 210 according to the second embodiment exhibits a high image restoration capability.

<Third Embodiment>
Next, a third embodiment will be described. In the third embodiment, for a document having images printed on both sides, a front image and a back image are scanned simultaneously from both sides. Based on the front image information and the back image information, the front image information is blurred and noisy. A case in which the present invention is applied to an image restoration apparatus that removes degraded image information including show-through image information including ground color will be described.

  Specifically, the image restoration apparatus according to the third embodiment is deteriorated in the original image information on one side of a document having images on both sides, and deteriorated image information including show-through image information including noise and ground color. The original image information is estimated from the observed image information using a prediction method based on a state space model of observed image information in which the image data is mixed and includes the original image information as a drive source.

  The configuration of the image restoration apparatus 310 according to the third embodiment will be described with respect to parts different from those of the image restoration apparatus 10 according to the first embodiment, and the other parts are denoted by the same reference numerals as those in the first embodiment. Detailed description will be omitted.

  The input interface unit 320 according to the third embodiment acquires front surface image information and back surface image information obtained by simultaneously reading both sides of a document, and uses the front surface image information as observation image information.

  In addition, as shown in FIG. 24, the main storage unit 341 in the third embodiment includes a capture table image memory 342, a capture back image memory 343, a subject area memory 45, and a processing parameter memory 46. It can be expressed in a configuration that includes it.

  The capture table image memory 342 stores surface image information among the image information captured via the input interface unit 20. Further, the back side image information of the image information captured via the input interface unit 20 is stored in the captured back image memory 343.

  As shown in FIG. 25, the image restoration processing unit 360 according to the third embodiment includes a scan information receiving unit 361, a region setting unit 61, a position change amount setting unit 362, a state amount calculation unit 363, and an observation. It can be expressed by a configuration including an amount calculation unit 364, an estimation unit 365, and a restored image output unit 66.

  The scan information receiving unit 361 receives scan information indicating whether the image information acquired via the input interface unit 20 is acquired by double-sided scanning or single-sided scanning.

The position change amount setting unit 362 receives the show-through image information acquired by the input interface unit 20 when the scan information receiving unit 361 receives scan information indicating whether the scan is acquired by single-sided scanning. block matching Tsu packaging method alignment between the surface image information and the back surface image information including, or performed using existing techniques such Lucas-Kanade method, the position information of the processing area n _F set in the observed image information Correspondingly, the processing area n _B for the back image information is set. The subscript F represents the front surface image information, and the subscript B represents the back surface image information. In the case of double-sided scanning, the processing area n _B of the back surface image information corresponding to the processing area n of the observation image information is determined in advance.

State quantity calculating unit 363, a state quantity in the processing area _{n F} of the observation image information processing area _(n-1) state quantity _F, the processing region _(n-1) state transition matrix from _F to the processing region _{n F} calculated using the [Phi _p1, and processing region (n-1) colored driving source including a state of elements of the processing region n _F which is not included in the state of _F.

Observation amount calculating unit 364, state quantity of the processing region n _F of the observed image information, and based on the back surface image information, based on the region corresponding to the processing region n _B in the image information obtained by inverting the back surface image information in the horizontal direction Using the calculated value as degraded image information including blur and noise and show-through image information, the process of adding the degraded image information to the original image information to become observed image information is calculated as an observation amount.

The estimation unit 365 is based on the state space model including the state quantity calculated by the state quantity calculation unit 363 and the observation quantity calculated by the observation quantity calculation unit 364 for the processing region n _F of the observation image information of the surface image information. the prediction method, estimated to restore the optimum value of the state quantity of the processing region n _F as an original image information processing regions n _F.

  Next, a state space model that is the theory of image restoration in the third embodiment will be described.

  For example, when a document with images printed on both sides is scanned, the image information of the back image is shown through to the original image information of the front image, resulting in a deteriorated image shown at the right end of FIG. As shown in FIG. 4, show-through is image information in which a back image is transmitted through image information obtained by scanning an original image (front image) and deteriorated image information including blur, noise, and show-through image information is superimposed. Note that the transmittance through which the back image is transmitted depends on the thickness of the document paper surface, the density of the image, the illumination intensity during scanning, and the like. Therefore, the restoration of the deteriorated image information in the third embodiment is to remove the show-through back image information from the deteriorated image information, as in the first embodiment.

  In the third embodiment, the state quantity is calculated using only the original image information, and the observation quantity is calculated using the degraded image information including the observation image information, the original image information, blur, noise, and show-through image information. Reconstructed image information is generated from the observed image information by obtaining an optimum estimated value of the state quantity by a prediction method based on the calculated state space model and including a colored drive source.

The state space model is represented by the following equation (6) and the following equation (7). However, in the state equation, the vector x ^f _p3 (n) is a state vector (a vector composed of a state quantity having surface image information as original image information), and the state transition matrix Φ _p3 is a processing region n−1) to the processing region n. The drive source vector δ ^f _p3 (n) representing the transition of the original image information. Further, in the observation equation, the vector y ^f _p3 (n) is an observation vector (observation image information), the observation matrix M is a matrix representing a blur occurring in the surface image information, s is a transmittance, H is a blur PSF, and a vector y ^br _p3 (n) is a state vector (a state quantity when the reverse side image information is inverted in the horizontal direction). The subscript p3 represents that it relates to the third embodiment. Further, the subscript f represents the front surface image information, the subscript b represents the back surface image information, and the subscript r represents the inversion in the horizontal direction. That is, the subscript br represents that the reverse side image information is inverted in the horizontal direction.

The state equation (6) describes the state space model to be observed, and state variables (here, state vectors x ^f _p3 (n−1), x ^f _p3 (n)) Represents. The observation equation (7) describes the process of observation through some kind of observation device, and the observation result (here, the observation vector y ^f _p3 (n)) is the observed quantity, that is, the input (here) , State vector x ^f _p3 (n)).

  Here, a method for deriving the observation equation shown in the equation (7) will be described. As shown in FIG. 4, show-through is image information in which the original image (front image) is scanned with image information in which the back image is blurred and deteriorated image information including noise and show-through image information is superimposed. is there. Therefore, in principle, the observation equation can be expressed by the following equation (8).

In the above equation (8), the transmittance s and the matrix H that is a blurred PSF are superimposed on the state vector x ^b _p3 (n) corresponding to the back surface image information.

On the other hand, when an observation equation of the back surface image information is calculated based on the back surface image information, it can be expressed by the following equation (9). In the following observation equation (9), the vector y ^b (n) is the observation vector (observation image information) of the back surface image information, the matrix M is the observation matrix, s is the transmittance, H is the blurred PSF, and the vector x ^b (N) is a state vector (a state quantity having the back image information as the original image information), and a vector x ^f (n) is a state vector (a state quantity having the front image information as the original image information). The subscript b represents the back surface image information, and the subscript f represents the front surface image information.

  In the above formula (9), the transmittance s is small, so the following formula (10) is satisfied.

  Therefore, the above formula (9) becomes the following formula (11) in consideration of the above formula (10).

  When the above (11) is modified, the following equation (12) is obtained.

  Then, considering that the back surface image information is inverted in the horizontal direction and added to the front surface image information, the above equation (12) is substituted into the above equation (8) to obtain the above equation (7).

  Next, with reference to FIG. 26 and FIG. 7, the operation of the image restoration apparatus 310 according to the third embodiment will be described. The image restoration processing routine shown in FIG. 26 and FIG. 7 is stored as a control program in the storage unit 40 and executed by a CPU (not shown).

  First, in step 300, image information of a front image and a back image read from a document having images printed on both sides thereof by a scanner serving as an image input device is acquired via the input interface unit 20, and stored in the main storage unit 41. Store in the captured image memory 42. Since the image information of the surface image includes color information of the surface image, the image information setting unit 33 may set whether the input image is a monochrome image or an RGB image. .

  Next, in step 301, whether the image information acquired by the scan information receiving unit 361 via the input interface unit 20 is acquired by double-sided scanning or acquired by single-sided scanning. Scan information indicating is received.

Next, in step 302, the region setting unit 61 sets the pixel size of the partial region L and the pixel size of the processing region n that is a unit of image restoration processing.
Note that the pixel size of the processing region n may be set by a value designated by the user from the region designation unit 32 of the operation unit 30.

  Next, in step 303, based on the scan information received in step 301, it is determined whether the captured image information has been acquired by double-sided scanning or single-sided scanning. To do. If the captured image information is acquired by double-sided scanning, the process proceeds to step 104. On the other hand, if the captured image information has been acquired by single-sided scanning, the process proceeds to step 304.

  Next, in step 304, the position change amount setting unit 362 reads the front surface image information stored in the capture table image memory 342 and the back surface image information stored in the capture back image memory 343, Position alignment is performed between the front surface image information including the show-through image information and the back surface image information, and each region of the processing region n is set for the back surface image information. Existing methods such as block matching and the Lucas-Kanade method can be used for the alignment of image information. In this embodiment, a case where block matching is used will be described.

Block matching is a method of searching for similar areas in the back image information within the search range set in the front image information. The pixel values of the small area extracted from the back image information and the pixel values of the small area extracted from the back image information This is a method of finding a region where the sum of errors with the minimum is found and searching for a corresponding position. As means for calculating the sum of error of pixel values, an SSD (Sum of Squared Distance) that calculates the sum of square distances of the respective pixel values shown in the following formula (13) or an absolute value of each pixel value shown in the following formula (14) SAD (Sum of Absolute Distance) that takes the sum of the value distances is mentioned (reference patent literature “Shinya Fujiwara, Ryo Taguchi,“ A method for improving frame rate based on block matching motion estimation ”, IEICE Technical Report SIS2006-15, Jun. 2006 ”). Note that x _{i, j} in the equations (13) and (14) is the pixel value of the pixel (i, j).

  Next, in step 320, the state quantity calculation unit 363 executes state quantity calculation processing. Here, the state quantity calculation processing routine will be described with reference to FIG.

  In step 1200, the state quantity calculation unit 263 derives a state vector whose element is the state quantity of each pixel in the processing area (n−1) and a state vector whose element is the state quantity of each pixel in the processing area n. To do. Hereinafter, a method for deriving the state vector will be specifically described.

First, the state vector shown in FIG. 27 is configured by combining the state quantities of the respective pixels in the processing area (n−1) having the pixel size p × q. That is, the state vector is a 9-dimensional vector x ^f _p3 (n−1) = [x ^f _1,1 (n−1) x ^f _2,1 (n−1)... X ^f _3,3 (n− 1)] ^T Next, the state quantities of the respective pixels in the processing area n having the pixel size p × q are combined. That is, the state vector of state n is a nine-dimensional vector x ^f _p3 (n) = [x ^f _1,1 (n) x ^f _2,1 (n)... X ^f _3,3 (n)] ^T Become. However, T represents transposition of a vector. In addition, the size (number of dimensions) of the state vectors x ^f _p3 (n−1) and x ^f _p3 (n) varies depending on the pixel size p × q of the set processing region n.

Next, in step 1202, the state quantity calculation unit 263 configures a drive source vector. As shown in FIG. 27, the state quantities x _1,3 (n), x _2,3 (n), x _3,3 (n) can be configured from information in the processing area (n−1). In this embodiment, since the state quantities x _1,3 (n-1), x _2,3 (n-1), x _3,3 (n-1) are used as the drive source vector, the driving is performed. A source vector δ ^f _p3 (n) is constructed. This means that when the processing area changes, the state quantity that cannot be represented by the state quantity of the processing area (n−1) is constituted by the state quantity of the processing area n, and the original image of the surface image information Since the information is included, the drive source vector δ ^f _p3 (n) is a colored drive source.

Next, in step 1204, the state quantity calculation unit 363 uses the state vectors x ^f _p3 (n−1) and x ^f _p3 (n) derived in step 1200, the state transition matrix Φ _p3 configured in step 105, and step The state quantity is calculated based on the drive source vector δ ^f _p3 (n) configured at 1202. As shown in FIG. 27, the state equation in the present embodiment includes a state vector x ^f _p3 (n) composed of original image information, a state transition matrix Φ _p3 composed of 0 and 1, and an original image that is a colored signal. Since it is composed of the drive source vector δ ^f _p3 (n) consisting of information, the desired state quantity is composed only of the original image information (restored image information).

When the state quantity calculation process routine ends, the process returns to the image restoration process routine and proceeds to step 322, where the observation quantity calculation unit 364 performs the process shown in FIGS. 28 and 29 based on the observation image model shown in FIG. As shown, a process in which the show-through image information of the back image is added to the original image information of the front image and becomes observation image information is calculated as an observation amount (observation equation). The observation vector y ^f _p3 (n) is a nine-dimensional vector whose element is the pixel value of each pixel in the processing region n of the surface image information as an observation amount including the observation image information. The observation matrix M is a unit matrix for satisfying the observation equation, and M ⁻¹ is a generalized inverse matrix of the observation matrix M. The vector x ^f _p3 (n) is a vector whose element is the state quantity of each pixel in the processing region n in the original image information of the surface image. Further, s is a transmittance relating to the show-through, and H is a matrix having elements corresponding to the blur point spread function (PSF) in the image degradation model. The vector y ^br _p3 (n) is a vector whose element is the state quantity of each pixel in the processing area n in the image information obtained by inverting the observed image information of the back image in the horizontal direction.

Next, in step 124, the estimation unit 65 uses the state quantity (state equation) calculated in step 320 and the observation quantity (observation equation) calculated in step 322 as in the first embodiment. A prediction method based on a state space model including a colored drive source is derived. In the prediction method based on the state space model, the observed noise vector ε ^f _p1 (n) in the first embodiment corresponds to sHM ⁻¹ y ^br _p3 (n) in the above equation (7).

  Thereafter, steps 126 to 140 are executed, and the image restoration processing routine is terminated.

  Note that other configurations and operations of the image restoration apparatus according to the third embodiment are the same as those of the first embodiment, and thus description thereof is omitted.

  As described above, according to the image restoration apparatus 310 of the third embodiment, the front side image information and the back side image information obtained by reading both sides of a document are acquired, and the front side image information is used as observation image information. , The state quantity in the processing area n of the observed image information, the state quantity in the processing area (n−1), the state transition from the processing area (n−1) to the processing area n, and the state of the processing area (n−1). Calculated using a chromaticity driving source including an element of the state quantity of the processing area n not included in the quantity, and for the processing area n of the observation image information, information about the state quantity of the processing area n in the observation image information and the back image Based on this, the value calculated in the processing area n in the image information obtained by inverting the reverse image information in the horizontal direction is used as degraded image information including noise and show-through image information, and the degraded image information is added to the original image information. View The process that becomes the image information is calculated as an observation amount, and based on the calculated state amount and the calculated observation amount, the image information is obtained from the optimum value of the state amount of the processing region n by a prediction method based on the state space model. It is possible to restore and remove the degraded image information.

<Simulation 3>

  Next, a simulation for verifying the effect of the image restoration apparatus 310 according to the third embodiment will be described. In this simulation, (1) visual evaluation, (2) objective evaluation, and (3) subjective evaluation were performed in order to evaluate the image restoration capability of the image restoration apparatus 10 according to the third embodiment. The visual evaluation is an evaluation in which an original image and a restored image are compared visually. Objective evaluation is numerical evaluation. Subjective evaluation is an interview survey. Hereinafter, these will be described in order.

  First, simulation conditions will be described. In this simulation, three images (“Flowchart”, “Lenna”, and “Word”) shown in FIG. 30 were used as original images.

  Here, the image “Flowchart” is used as the surface image 1. The image “Flowchart” is a grayscale image having a gradation of 8 bits and has a resolution of 256 × 256 pixels. The image “Lenna” is used as the surface image 2. The image “Lenna” is a color image having a gradation of 24 bits and has a resolution of 256 × 256 pixels. The image “Word” is used as a back image. The image “Word” is a grayscale image having a gradation of 8 bits and has a resolution of 256 × 256 pixels.

  Under the same simulation conditions, the method in the third embodiment (invention method 3: Prop. 1), the method according to Non-Patent Document 1 (conventional method 1: Conv. 1), and Patent Document 1 The method according to the above (conventional method 2: Conv. 2) was compared.

(1) Visual Evaluation FIGS. 31 and 32 show simulation results for the image “Flowchart”, and FIGS. 33 and 34 show simulation results for the image “Lenna”. 32 is an enlarged view of a part of FIG. 31, and FIG. 34 is an enlarged view of a part of FIG.

  From the simulation result for the image “Flowchart”, as shown in FIG. 32, it can be seen that in the conventional method 1, the show-through has not completely disappeared and is incomplete. Further, it can be seen that in the conventional method 2, the show-through has disappeared, but the information of the surface image has deteriorated. On the other hand, Invention Method 3 can clearly remove the show-through while maintaining the information of the surface image.

  From the simulation result for the image “Lenna”, as shown in FIG. 34, it can be seen that the conventional method 1 is white as a whole and the show-through removal is incomplete. Further, in the conventional method 2, the show-through is not completely removed, and the result is unclear. On the other hand, Invention Method 3 can remove the show-through and has a clear image.

Thus, the effectiveness of Invention Method 3 can be confirmed.

(2) Objective evaluation (numerical evaluation)
Table 1 below shows the simulation results (objective evaluation) for the original image.

  Here, in order to evaluate the image restoration capability of the conventional method 1, the conventional method 2, and the invention method 3 by numerical values, the image restoration capability was evaluated using PSNR [dB] expressed by the following equation (15). PSNR (Peak Signal-to-Noise Ratio) is the ratio of noise to image signal, and the larger the value, the less degradation (blur, noise, etc.), and the better the image.

  As shown in Table 1, it can be confirmed that the invention method 3 has a larger PSNR value than the conventional method 1 and the conventional method 2 for both “Flowchart” and “Lenna” images. Thus, it can be seen that Invention Method 3 has higher image restoration capability than Objective Method 1 and Conventional Method 2 in terms of objective evaluation.

(3) Subjective evaluation (interview survey)
FIG. 35 shows a simulation result (subjective evaluation) for the original image.

  Here, in order to evaluate the image restoration ability of the conventional method 1, the conventional method 2, and the inventive method 3, subjective evaluation was performed by interview survey. As shown in Table 2 below, the performance evaluation of image restoration was performed by an interview survey using evaluation values in five stages based on MOS (Mean Opinion Score) evaluation criteria. 50 listeners (25 men and 25 women) evaluated images (see FIGS. 18 to 21) obtained by image restoration. Each listener determines an evaluation value “1” to “5”. The evaluation value “5” is the best.

  As shown in FIG. 35, it can be confirmed that the invention method 3 obtains a higher MOS evaluation value than the conventional method 1 and the conventional method 2 for both “Flowchart” and “Lenna” images. Thus, it can be seen that Invention Method 3 has higher image restoration capability than Subject Method 1 and Conventional Method 2 also in terms of subjective evaluation.

  From the above experimental results, it can be seen that the technique (invention technique 3) in the image restoration apparatus 310 according to the third embodiment exhibits a higher image restoration capability than the conventional technique 1 and the conventional technique 2.

<Simulation 4>

  Next, a simulation for demonstrating the effect of the image restoration apparatus 310 according to the third embodiment as compared with the image restoration apparatus 10 according to the first embodiment will be described. In this simulation, visual evaluation was performed in order to evaluate the image restoration capability of the image restoration apparatus 10 according to the third embodiment.

  First, simulation conditions will be described. In this simulation, two images (“Front” and “Back”) shown in FIG. 36 were used as original images.

  Here, the image “Front” is used as a front image, and the image “Back” is used as a back image. Each image is a color image having a gradation of 24 bits, and the resolution is 3508 × 2480 pixels.

  Under the same simulation conditions, the method in the first embodiment (invention method 1: Prop. 1), the method in the third embodiment (invention method 3: Prop. 3), non- The method according to Patent Document 1 (Conventional Method 1: Conv. 1) and the method according to Patent Document 1 (Conventional Method 2: Conv. 2) were compared.

(1) Visual evaluation The method in the third embodiment (invention method 3: Prop. 3), the method according to Non-Patent Document 1 (conventional method 1: Conv. 1), and the method according to Patent Document 1 (conventional method 2) : Conv.2) and the simulation results are shown in FIGS. FIG. 38 is an enlarged view of a part of FIG.
39 to 41 show simulation results for the technique in the first embodiment (invention technique 1: Prop. 1) and the technique in the third embodiment (invention technique 3: Prop. 3). Show. 40 and 41 are enlarged views of a part of FIG.

  The method according to the third embodiment (invention method 3: Prop. 3), the method according to Non-Patent Document 1 (conventional method 1: Conv. 1), and the method according to Patent Document 1 (conventional method 2: Conv. 2) As shown in FIG. 38, the simulation results for and show that in the conventional method 1 and the conventional method 2, the show-through has not completely disappeared and is incomplete. On the other hand, Invention Method 3 can clearly remove the show-through while maintaining the information of the surface image.

  In addition, simulation results for the method in the first embodiment (invention method 1: Prop. 1) and the method in the third embodiment (invention method 3: Prop. 3) are as shown in FIG. In the invention method 1, the show-through can be removed from the background, but it can be seen that the show-through is mixed in the letter “A”. On the other hand, it can be seen that, in Invention Method 3, the show-through of the background is removed and the show-through is also removed in the letter “A”.

Thus, the effectiveness of Invention Method 3 can be confirmed.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, in the above embodiment, the case where each process is performed for each partial region has been described as an example. However, the present invention is not limited to this, and the entire observation image may be processed.

  In the first and second embodiments, the case where the dispersion value is calculated based on the pixel value of each pixel in the partial area has been described as an example. However, the present invention is not limited to this. Instead, the variance value may be calculated for the entire observation image based on the pixel value of each pixel in the observation image.

  In the above-described embodiment, the case where the processing of each unit is executed by executing a program by the CPU has been described, but each unit may be configured by hardware.

10, 210, 310 Image restoration device 20, 320 Input interface unit 30 Operation unit 31 Parameter setting unit 32 Area specifying unit 34 Image information setting unit 40 Storage unit 41, 341 Main storage unit 42 Captured image memory 45 Subject area memory 46 Processing Parameter memory 47 Auxiliary storage unit 48 Display memory 49 Captured image memory 50 Processing memory 51 Restored image memory 60, 260, 360 Image restoration processing unit 61 Area setting unit 62 Degraded image information determination unit 63, 263, 363 State quantity calculation unit 64 , 364 Observation amount calculation unit 65, 365 Estimation unit 66 Restored image output unit 70 Output interface unit 262 Ground color determination unit 342 Capture table image memory 343 Captured back image memory 361 Scan information reception unit 362 Position change amount setting unit

Claims (14)

An image restoration device that estimates the original image information from the observed image information by using a prediction method based on a state space model including the original image information as a drive source for the observed image information in which blur and noise are mixed in the original image information Because
Image acquisition means for acquiring surface image information obtained by reading a document as the observation image information;
Determining means for calculating a variance value based on a pixel value of each pixel of the observed image information, and determining whether blur and noise are mixed in each pixel based on a threshold value calculated from the variance value; ,
The state quantity in the processing area n at the time n of the observation image information is the state quantity in the processing area (n-1) at the time (n-1), and the state quantity from the processing area (n-1) to the processing area n. State quantity calculating means for calculating using the drive source including state transition and an element of the state quantity of the processing area n at time n not included in the state quantity of the processing area (n-1);
Based on the state quantity of the processing area n at time n, the blur and noise are added to the original image information for the pixel value of each pixel of the processing area n determined by the determination means to be mixed. Observation amount calculating means for calculating the process that becomes observation image information as an observation amount;
Based on the state quantity calculated by the state quantity calculation means and the observation quantity calculated by the observation quantity calculation means, the processing area n at the time n is predicted by the prediction method based on the state space model. estimating means for estimating the optimum value of the state quantity of n as original image information;
An image restoration apparatus including: The original image information is used as a driving source for the observed image information including blur and noise including the show-through image information on the other side, which is unnecessary information on the original image information on one side of the document having images on both sides. An image restoration device that estimates the original image information from the observed image information using a prediction method based on a state space model including:
Image acquisition means for acquiring surface image information obtained by reading one side of the document as the observation image information;
A variance value is calculated based on a pixel value of each pixel of the observed image information, and a deterioration including blur and noise including show-through image information in each pixel of the observed image information based on a threshold value calculated from the variance value Show-through determination means for determining whether image information is mixed,
The state quantity of the processing area n at the time n in the observation image information is the state quantity of the processing area (n-1) at the time (n-1) and the processing area (n-1) to the processing area n. State quantity calculation means for calculating using the drive source including state transition and an element of the state quantity of the processing region n not included in the state quantity of the processing region (n-1);
The processing region in which the deteriorated image information is determined to be mixed by the show-through determination unit based on the variance value calculated by the show-through determination unit and the state quantity of the processing region n. For each pixel of n, an observation amount calculation means for calculating, as an observation amount, a process in which the noise is added to the original image information to become observation image information, using the standard deviation of the calculated dispersion value as the noise;
Based on the state quantity calculated by the state quantity calculation means and the observation quantity calculated by the observation quantity calculation means, the optimal value of the state quantity of the processing region n is calculated by the prediction method based on the state space model. Estimating means for estimating the original image information of the processing area n;
An image restoration apparatus including: Using the prediction method based on a state space model including the original image information as a driving source for the observed image information in which the original image information includes a blurred image including the background color of the document and deteriorated image information including noise, the observed image is used. An image restoration device for estimating the original image information from information,
Image acquisition means for acquiring surface image information obtained by reading a document as the observation image information;
A variance value is calculated based on the pixel value of each pixel of the observed image information, and it is determined whether or not the degraded image information is mixed in each pixel of the observed image information based on a threshold value calculated from the variance value Ground color determination means to perform,
The state quantity of the processing area n at time n of the observed image information is represented by the state quantity of the processing area (n−1) at time (n−1), the state transition from the processing area (n−1) to the processing area n, And a state quantity calculating means for calculating using the drive source including an element of the state quantity of the processing area n not included in the state quantity of the processing area (n−1),
Based on the information and state quantity of the processing area n at time n, the pixel value of each pixel in the processing area n determined by the ground color determination means to be mixed is used as the deteriorated image information. An observation amount calculating means for calculating, as an observation amount, a process in which the degraded image information is added to the original image information to become observation image information;
Based on the state quantity calculated by the state quantity calculation means and the observation quantity calculated by the observation quantity calculation means, an optimal value of the state quantity of the processing region n is generated by a prediction method based on the state space model. Estimating means for estimating as image information;
An image restoration apparatus including: Using original image information as a drive source for observed image information with mixed blur and noise, including original image information on one side of a document having images on both sides and show-through information on the other side, which is degraded image information An image restoration apparatus that estimates the original image information from the observed image information using a prediction method based on a state space model including:
Image acquisition means for reading both sides of the document to obtain front side image information and back side image information, and using the front side image information as the observation image information;
The state quantity in the processing area n at time n of the observed image information is the state quantity in the processing area (n-1) at time (n-1), and the state transition from the processing area (n-1) to the processing area n. And a state quantity calculating means for calculating using the drive source including an element of the state quantity of the processing area n not included in the state quantity of the processing area (n−1),
Information on the region corresponding to the processing region n at time n in the image information obtained by inverting the back surface image information in the horizontal direction based on the state quantity of the processing region n at time n of the observation image information and the back surface image information. An observation amount calculation means for calculating, as the observation image information, a process in which the deterioration image information is added to the original image information to obtain observation image information.
Based on the state quantity calculated by the state quantity calculation means for the processing region n at time n of the surface image information and the observation quantity calculated by the observation quantity calculation means, by a prediction method based on the state space model, Estimating means for estimating an optimum value of the state quantity of the processing region n as original image information of the processing region n;
An image restoration apparatus including:   Alignment is performed between the front surface image information and the back surface image information acquired by the image acquisition means, and a region of the processing region n at time n in the observation image information is set for the back surface image information. The image restoration device according to claim 4, further comprising position change amount setting means for performing the operation. The show-through determination unit calculates a variance value based on each pixel value of the partial region of the observed image information, and the degraded image information is mixed in each pixel of the partial region based on a threshold value calculated from the variance value. Determine whether or not
The observation amount calculation means is based on the variance value calculated for the partial region including the processing region n and the state amount of the processing region n by the show-through determination means for the processing region n at time n of the observation image information. The standard deviation of the variance value calculated for the partial area by each pixel of the processing area n determined by the show-through determination means that the deteriorated image information is mixed is used as the noise. The image restoration apparatus according to claim 2, wherein a process in which the noise is added to the information to form observation image information is calculated as an observation amount. The ground color determination means calculates a variance value based on each pixel value of each partial area of the observed image information, and the ground color is mixed in each pixel of the partial area based on a threshold value calculated from the variance value. Determine whether or not
The observation amount calculation means is based on the variance value calculated for the partial area including the processing area n in the processing area n of the observation image information by the ground color determination means and the state quantity of the processing area n. For each pixel of the processing area n determined that the ground color is mixed by the color determination means, the standard deviation of the variance value calculated for the partial area is used as the noise, and the noise is included in the original image information. The image restoration apparatus according to claim 3, wherein a process in which observation image information is added by adding as an observation amount is calculated.   The image restoration apparatus according to any one of claims 1 to 7, wherein processing of each unit is performed for each partial region L set for the observation image information.   The image restoration device according to any one of claims 1 to 8, wherein the processing area n is set to partially overlap the processing area (n-1). An image restoration device that estimates the original image information from the observed image information by using a prediction method based on a state space model including the original image information as a drive source for the observed image information in which blur and noise are mixed in the original image information And an image restoration method in an image restoration apparatus including an image acquisition unit, a determination unit, a state amount calculation unit, an observation amount calculation unit, and an estimation unit,
The image acquisition means acquiring surface image information obtained by reading a document as the observation image information;
The determination unit calculates a variance value based on a pixel value of each pixel of the observation image information, and determines whether blur and noise are mixed in each pixel based on a threshold value calculated from the variance value. A determining step;
The state quantity calculation means determines the state quantity in the processing area n at time n of the observation image information as the state quantity in the processing area (n-1) at time (n-1), and the processing area (n-1). Calculating using the drive source including the state transition from the processing region n to the processing region n and the state quantity of the processing region n at time n not included in the processing region (n-1) state amount;
For the pixel value of each pixel in the processing region n determined by the determining unit based on the state quantity of the processing region n at time n, the observation amount calculation unit determines that the blur and noise are mixed. Calculating a process in which blur and noise are added to the information to form observation image information as an observation amount;
The estimation means uses the prediction method based on the state space model based on the state quantity calculated by the state quantity calculation means for the processing region n at time n and the observation quantity calculated by the observation quantity calculation means, Estimating an optimal value of the state quantity of the processing region n at time n as original image information;
Image restoration method including: The original image information is used as a driving source for the observed image information including blur and noise including the show-through image information on the other side, which is unnecessary information on the original image information on one side of the document having images on both sides. An image restoration apparatus that estimates the original image information from the observed image information using a prediction method based on a state space model included as an image acquisition unit, a show-through determination unit, a state amount calculation unit, and an observation amount calculation And an image restoration method in an image restoration apparatus including an estimation means,
The image acquisition means acquiring surface image information obtained by reading one side of the document as the observation image information;
The show-through determination means calculates a variance value based on a pixel value of each pixel of the observation image information, and shows-through image information to each pixel of the observation image information based on a threshold value calculated from the variance value. Determining whether or not degraded image information including blur and noise is mixed;
The state quantity calculating means calculates the state quantity of the processing area n at time n in the observation image information, the state quantity of the processing area (n-1) at time (n-1), and the processing area (n-1). A state transition from the processing region n to the processing region n, and a calculation using the driving source including an element of the state amount of the processing region n not included in the state amount of the processing region (n−1);
Based on the variance value calculated by the show-through determination unit and the state quantity of the processing region n in the observed amount calculation unit, the deteriorated image information is mixed by the show-through determination unit. Using the standard deviation of the calculated variance value as the noise, the process of adding the noise to the original image information to become observation image information is calculated as an observation amount for each pixel in the processing region n determined as And steps to
Based on the state quantity calculated by the state quantity calculation means and the observation quantity calculated by the observation quantity calculation means, the estimation means performs a state quantity of the processing region n based on a prediction method based on the state space model. Estimating the optimal value of the original image information of the processing region n;
Image restoration method including: Using the prediction method based on a state space model including the original image information as a driving source for the observed image information in which the original image information includes a blurred image including the background color of the document and deteriorated image information including noise, the observed image is used. An image restoration apparatus for estimating the original image information from information, and an image restoration method in an image restoration apparatus including an image acquisition means, a ground color determination means, a state quantity calculation means, an observation amount calculation means, and an estimation means There,
The image acquisition means acquires the surface image information obtained by reading a document as the observation image information;
The ground color determination means calculates a variance value based on a pixel value of each pixel of the observed image information, and the degraded image information is mixed in each pixel of the observed image information based on a threshold value calculated from the variance value. Determining whether or not
The state quantity calculation means processes the state quantity of the processing area n at time n of the observation image information from the state quantity of the processing area (n-1) at time (n-1), the processing area (n-1). A step of calculating using the drive source including a state transition to the region n and an element of the state amount of the processing region n not included in the state amount of the processing region (n−1);
Based on the information and state quantity of the processing region n at time n, the observation amount calculation unit determines the pixel value of each pixel in the processing region n determined by the ground color determination unit to include the degraded image information. Using as the degraded image information, calculating the process of adding the degraded image information to the original image information to become observed image information as an observation amount;
Based on the state quantity calculated by the state quantity calculation means and the observation quantity calculated by the observation quantity calculation means, the estimation means performs a state quantity of the processing region n based on a prediction method based on the state space model. Estimating an optimal value of as original image information;
Image restoration method including: Using original image information as a drive source for observed image information with mixed blur and noise, including original image information on one side of a document having images on both sides and show-through information on the other side, which is degraded image information An image restoration apparatus that estimates the original image information from the observed image information using a prediction method based on a state space model including an image acquisition unit, a state amount calculation unit, an observation amount calculation unit, and an estimation unit. An image restoration method in an image restoration apparatus including:
The image acquisition means reads both sides of the document, acquires front surface image information and back surface image information, and sets the front surface image information as the observation image information;
The state quantity calculation means calculates the state quantity in the processing area n at time n of the observation image information from the state quantity, processing area (n-1) in the processing area (n-1) at time (n-1). Calculating using the drive source including the state transition to the processing region n and the state amount element of the processing region n not included in the processing region (n−1) state amount;
The processing area at the time n in the image information obtained by inverting the back surface image information in the horizontal direction based on the state quantity of the processing area n at the time n of the observation image information and the back surface image information. using a value calculated based on information of a region corresponding to n as the degraded image information, calculating a process in which the degraded image information is added to the original image information to become observed image information as an observation amount;
The estimation means adds the processing region n at time n of the surface image information to the state space model based on the state quantity calculated by the state quantity calculation means and the observation quantity calculated by the observation quantity calculation means. Estimating an optimum value of the state quantity of the processing region n as original image information of the processing region n by a prediction method based on
Image restoration method including:   The program for functioning a computer as each means of the image restoration apparatus of any one of Claims 1-9.